TU Munich: Robot finds misplaced items on command
A robot from TU Munich can locate misplaced items faster because it understands the items and the environment.
The TU Munich robot scans its surroundings and recognizes objects.
(Image: Learning Systems and Robotics Lab / Screenshot)
A research team from the Learning Systems and Robotic Lab at the Technical University of Munich (TUM) has developed a robot that can locate misplaced items indoors on command. The robot not only recognizes objects and their significance but also understands the environment in which they might be located. This means the robot doesn't have to search in places where the items cannot possibly be.
The TUM robot essentially consists of a mobile platform with wheels, on which a mast with a 3D camera is mounted. The robot, which can be driven by electric motor, can thus move around indoors and scan its surroundings. This is achieved using a 3D camera that combines two-dimensional images with depth information. This creates a digital, centimeter-accurate spatial representation of the environment. The scanning process is constantly repeated to always have an up-to-date digital version that can be evaluated.
Semantic understanding
"We have taught the robot to understand the environment," explains Angela Schoellig, professor and head of the robotics lab in the TUM chair for Safety, Performance and Reliability for Learning Systems. As a result, the robot not only knows the objects it might be looking for but also knows what a table, a windowsill, or a shelf are, and can assess whether certain items might be placed on them. For example, a stovetop or a sink are rarely used for placing glasses, so the robot doesn't have to search there at all. However, this requires an understanding of the object and its environment.
Empfohlener redaktioneller Inhalt
Mit Ihrer Zustimmung wird hier ein externes YouTube-Video (Google Ireland Limited) geladen.
Ich bin damit einverstanden, dass mir externe Inhalte angezeigt werden. Damit können personenbezogene Daten an Drittplattformen (Google Ireland Limited) übermittelt werden. Mehr dazu in unserer Datenschutzerklärung.
An AI language model establishes the relationships between the objects and the environment, which are then interpreted by the robot. It calculates the probability that a specific object is located in a particular place in the environment. This allows the robot to search for items more efficiently by excluding improbable locations from the search. The TUM researchers found in the study „Where Did I Leave My Glasses? Open-Vocabulary Semantic Exploration in Real-World Semi-Static Environments“, which appeared as a preprint on Arxiv, that this can achieve up to 30 percent search efficiency gains.
The robot also recognizes changes in its environment with 95 percent accuracy by comparing older recordings of the room with current ones. In the areas marked as "highly probable" for finding items, the robot then searches preferentially.
Videos by heise
The TUM researchers plan to expand their robot's finding capabilities. It should also be able to locate items in drawers or hidden behind other objects. However, this requires the robot to be able to interact with its environment – for example, with arms – to open cupboards. This demands further knowledge about its environment, as the robot must know in advance how best to open a door.
(olb)
