The US Pentagon’s Joint Artificial Intelligence Center (JAIC) submitted a request for information (RFI) looking for input on testing and evaluation capabilities to support the “full spectrum” of the Defense Department’s emerging AI technologies – including machine learning, deep learning, and neural networks. It is seeking to augment the JAIC’s Test and Evaluation office, which develops standards and conducts algorithm testing, system testing and operational testing on the military’s AI initiatives.
“The JAIC is requesting testing tools and expertise in planning, data management, and analysis of inputs and outputs associated with those tools. The introduction of AI-enabled systems is bringing changes to the process, metrics, data, and skills necessary to produce the level of testing the military needs, and that is why the JAIC is requesting information,” Dr. Jane Pinelis, chief of test, evaluation and assessment at the JAIC, said. “Testing and Evaluation provides knowledge of system capabilities and limitations to the acquisition community and to the warfighter. The JAIC’s T&E team will make rigorous and objective assessments of systems under operational conditions and against realistic threats, so that our warfighters ultimately trust the systems they are operating and that the risks associated with operating these systems are well-known to military acquisition decision-makers.”
The solicitation states that it plans to use feedback from the solicitation to guide how it further builds out its capabilities. Specific areas of interest include:
- Conversational interface applications using voice to text;
- Speech-enabled products and services for DOD applications and systems;
- Image analysis, testing deep learning-based visual search and image classifier;
- Natural Language Processing-enabled products and services’
- Humans augmented by machines; and,
- Autonomous systems.
The Pentagon is also seeking feedback regarding evaluation services in five mission areas: dataset curation, test harness development, model output analysis, test reporting, and testing services.