Soldiers Control Unmanned Vehicle Swarms with Voice and Gesture Controls

Raytheon, under the Defense Advanced Research Projects Agency’s (DARPA) Offensive Swarm-Enabled Tactics (OFFSET) program, is developing technology for soldiers to control unmanned vehicle swarms with voice and gesture commands. The OFFSET program envisions future small-unit infantry forces using small unmanned aircraft systems (UASs) and/or small unmanned ground systems (UGSs) in swarms of 250 robots or more to accomplish diverse missions in complex urban environments. The BBN Technologies subsidiary of Raytheon is creating a virtual reality interface that allows a single user to control large groups of inexpensive unmanned vehicles, allowing for “drag and drop” creation and manipulation of swarm tactics, a game-based simulator to evaluate those tactics, and a physical swarm test bed to perform live tactics evaluation.

“The goal is to allow a single user to actually control, in real time, up to hundreds of air and ground vehicles that have different capabilities or are different models,” said Shane Clark, a scientist at Raytheon and principal investigator for the company’s OFFSET efforts.

Northrop Grumman Mission Systems, Lockheed Martin, Soar Technology, and Charles River Analytics have also been awarded contracts by DARPA for the OFFSET program. These Phase 1 contracts are part of the program’s swarm sprints which focus on five main areas:

  • swarm tactics;
  • swarm autonomy;
  • human-swarm teaming;
  • virtual environment; and,
  • physical test bed.

DARPA plans to solicit proposals from potential sprinters approximately every six months.

“The swarm sprints are empirical experiments designed to accelerate our understanding of what swarms can do in urban environments,” said Timothy Chung, program manager in DARPA’s Tactical Technology Office (TTO). “By having swarm sprints at regular intervals, we’re able to ensure that we’re keeping up with the latest technologies — and are in fact helping inform and advance those technologies—to better suit the needs of the OFFSET program. Given the wide range of capabilities that we’re interested in, we’re looking for wherever those innovative solutions are going to come from, whether they be small businesses, academic institutions, or large corporations.”