Researchers at MIT have developed a system that can sense tiny changes in shadows on the ground that could help autonomous vehicles determine if there’s a moving object coming around the corner. Successful experiments have been conducted using an autonomous car driving around a parking garage and an autonomous wheelchair navigating hallways. When sensing and stopping for an approaching vehicle, the car-based system beats traditional LiDAR – which can only detect visible objects – by more than half a second.
The system, called “ShadowCam,” uses computer-vision techniques to detect and classify changes to shadows on the ground. A camera is used to create sequences of video frames that target a specific area. The changes in light intensity over time, from image to image, is used to detect something moving away or coming closer. ShadowCam computes that information and classifies each image as containing a stationary object or a dynamic, moving one.
To adapt ShadowCam for autonomous vehicles, the researchers developed a novel process that combines image registration and a new visual-odometry technique. Visual odometry estimates the motion of a camera in real-time by analyzing pose and geometry in sequences of images. Signal amplification was also required in order to reduce the signal-to-noise ratio.
“For applications where robots are moving around environments with other moving objects or people, our method can give the robot an early warning that somebody is coming around the corner, so the vehicle can slow down, adapt its path, and prepare in advance to avoid a collision,” adds co-author Daniela Rus, director of the Computer Science and Artificial Intelligence Laboratory (CSAIL). “The big dream is to provide ‘X-ray vision’ of sorts to vehicles moving fast on the streets.”
This work was funded by the Toyota Research Institute.