How novice coders can develop AI systems for military applications | MIT News

In today’s world, artificial intelligence chatbots like ChatGPT and Claude can perform many tasks, such as composing work emails and planning travel itineraries. These chatbots are systems built around large-scale visual language models (VLMs): AI trained on a large dataset that includes books, websites, code, and images.

AI algorithms then refine large amounts of human-generated feedback to follow instructions and avoid harmful or unwanted output, and use that “knowledge” to generate text or images based on input from the user. While chatbots have clear limitations, they can be very helpful in a wide variety of jobs, including in some areas that traditionally require specialized skills, such as computer programming.

As part of the Department of Air Force-MIT AI Accelerator’s Phantom Program project, US Air Force cadet Joshua Lynch – with the help of his mentor, Laura Niss, a technical staff member at the Embedded and AI Systems Group at MIT Lincoln Laboratory – wanted to determine whether, as a complete coder, he could develop a fully functional system. He used a process called “vibe-coding,” where the user relies entirely on intuition to direct a productive AI chatbot to write and refine the code.

His motivation was to empower anyone familiar with a military problem area, regardless of their technical background, to develop their ideas for useful software applications, bypassing the time and cost limitations of the traditional military software development pipeline. Lynch intended to build his own application while Niss guarded his knowledge of the technology.

“The Phantom student wanted to see if he could make a useful app by introducing himself to vibe-coding, with no prior experience,” Niss said. “Within this project, I wanted to understand how his perspective on AI changed over time with use. We both wanted to better understand where and how AI could be used by non-technical users in the military.”

Lynch set out to see if, starting with no coding skills and no chatbots, he could create a brand-specific app for his tactical team to help reduce collateral damage while improving survivability on a broader mission. This app will provide capabilities that include AI-assisted target recognition; modular intelligence, surveillance, and reconnaissance; independent calling; and battlefield communications management.

During the project, Lynch completed several professional development courses in AI and practiced both military and non-military uses of the technology. For the basis of his coding, he used paid models of three AI chatbots: Anthropic’s Claude, OpenAI’s ChatGPT, and Google’s Gemini. Most of this work is done only with the main function of chatting chatbots in the web browser, not as an integrated program within the development environment, as is common now. The final application was produced using the Google AI Studio App, which can create applications that interact with the Gemini application programming interface and have AI integrated into the development environment.

For more than three months, Lynch worked with these models to create his application, called the Remote Operating Modular Augmentation Device (ROMAD-AI). During this time, he learned several ways to improve code output. For example, he often had difficulty with AI chatbots that lacked phase focus and fixed unrelated parts of the code. You’ve found it’s important to break down problems into smaller parts, frame questions clearly, and bring discussions back to the topic when they stray too far from the objective.

Learning to recognize the limitations of chatbots and how to work effectively took up a large portion of the project’s timeline. As Lynch gained more experience with chatbots, limitations in AI power and development time enabled him to re-evaluate the project, moving it from a system that could assist on the battlefield to one that could perform basic document processing, such as analyzing battlefield intelligence maps and generating mission planning documents through an interface with VLM’s powerful chatbot. Although the resulting prototype did not implement all of the capabilities Lynch originally intended to include (and in its current iteration was not immune to the desired use case), it demonstrated the potential and usefulness of such an application to service members.

“I was impressed with the final product, and it showed me how powerful these systems can be in drawing designs from inexperienced people,” Niss said. “I now have the idea that these can be powerful tools for non-technical professionals to communicate problems and potential solutions to technical professionals, and help communicate desired outcomes.”

Niss noted a change in Lynch’s view of AI language models during his experience. After starting with an impressive goal, Lynch gained an understanding of the current technology’s capabilities and significantly lowered his expectations by the end of the project period. The steps of his ideas for different AI systems over time and system updates were especially interesting for Lynch and Niss, with Claude showing more stability than ChatGPT in all aspects such as similarity, anthropomorphism, and visual intelligence. Lynch found the AI ​​to be a useful tutor, but noted its accuracy on topics he knew well.

This project demonstrated that an AI chatbot can empower non-technical service members to generate effective software applications for their unique problems, although it works better as a prototyping assistant than a full production tool when handling sensitive information and critical applications. Improper code testing can lead to security risks, as demonstrated by an instance where Lynch did not realize that a final application was sending input documents to the Gemini AI model for analysis, rather than extracting the documents locally on his computer. While AI can generate significant amounts of functional code, code review remains a bottleneck in this space.

“For me, this project strengthened the relationship between experts in different fields,” said Niss. “No matter how good AI gets, I think we will always need to collaborate to reach the best solutions to the most important problems.”

The research was funded by the Department of the Air Force Artificial Intelligence Accelerator and was conducted under Cooperative Agreement Number FA8750-19-2-1000.