Cameras combined with AI detect pedestrians and obstacles faster
How could cars recognize pedestrians in traffic faster than current systems? Swiss researchers present a solution.
The image shows color information from the color camera and detections (blue and red dots) from the event camera generated by a running pedestrian.
(Image: Gruppe Robotik und Wahrnehmung, UZH)
Researchers at the Institute of Computer Science at the University of Zurich (UZH) have developed a system that combines a novel, biologically inspired camera with artificial intelligence. This is designed to recognize obstacles near a car or suddenly appearing pedestrians a hundred times faster and with less computing power than current systems. They have now published their research results in the scientific journal Nature.
The new system could significantly improve the safety of automotive systems and autonomous vehicles, say Daniel Gehrig and Davide Scaramuzza. They are also thinking of existing systems in conventional vehicles that can warn drivers or initiate emergency braking. These driver assistance systems are not yet sufficiently reliable. They usually record 30 to 50 images per second. An artificial neural network can be trained to recognize objects in these images. However, if something happens in the milliseconds between two snapshots, the camera may see it too late. The frame rate could now be increased, but this would generate more data that would have to be processed in real time.
Videos by heise
Event cameras, on the other hand, have "intelligent pixels". These record information every time they detect fast movements. This means there is no blind spot between the individual images. As "neuromorphic cameras", they mimic the perception of the human eye, but they can miss things that move slowly. In addition, their images cannot be easily converted into the usual data form to train AI algorithms.
Two camera types plus AI
Gehrig and Scaramuzza are now taking both camera types and combining them with AI. A standard camera takes 20 images per second; these are processed by an AI system that is trained to recognize cars or pedestrians. The data from the event camera is coupled with another type of AI system that is particularly well suited to analyzing 3D data that changes over time.
What the event camera sees is used to anticipate what the standard camera recognizes to boost its performance. "The result is a visual detector that can recognize objects just as fast as a standard camera taking 5000 images per second. But it only needs the same bandwidth as a standard camera with 50 images per second", explains Daniel Gehrig.
The method could become even more powerful if cameras are integrated with LiDAR sensors, as in self-driving cars, the Swiss researchers believe. Such hybrid systems could be decisive in enabling the necessary safety for autonomous driving without a significant increase in data and computing power.
(anw)
