DARPA has selected Raytheon, BAE Systems, and Northrop Grumman to lead research teams in the development of event-based infrared (IR) camera technologies. The Fast Event-based Neuromorphic Camera and Electronics (FENCE) program will work with event-based – or neuromorphic – cameras. These cameras act as sensors, operating asynchronously and only transmit information about pixels that have changed. This increases their efficiency, as they produce significantly less data and operate with much lower latency and power.
“Neuromorphic refers to silicon circuits that mimic brain operation; they offer sparse output, low latency, and high energy efficiency,” said Dr. Whitney Mason, FENCE program lead. “Event-based cameras operate under these same principles when dealing with sparse scenes, but currently lack advanced ‘intelligence’ to perform more difficult perception and control tasks.”
FENCE’s intelligent sensors can better handle highly cluttered and dynamic scenes by developing and demonstrating a low latency, low power, event-based infrared (IR) focal plane array (FPA) and a new class of digital signal processing and learning algorithms. The teams from the three companies will work to develop an asynchronous read-out integrated circuit (ROIC) with low-latency, as well as a processing layer that integrates with the ROIC to identify relevant spatial and temporal signals. Together the ROIC and processing layer will enable an integrated FENCE sensor which will operate on less power than 1.5 Watts.
“The goal is to develop a ‘smart’ sensor that can intelligently reduce the amount of information that is transmitted from the camera, narrowing down the data for consideration to only the most relevant pixels,” said Mason.
It is expected that the FENCE technology could be used in a number of military applications, such as autonomous vehicles, robotics, and IR search and tracking.