Cobalt: Researchers develop smartphone app for remote robot training
Scientists want to collect robot training data via crowdsourcing. In principle, anyone can contribute via a smartphone with the Cobalt app.
The researchers control a robot arm through smartphone movements. This works remotely via a teleoperation server. The recorded control data is used for later robot training.
(Image: Georgia Tech)
A research team led by Georgia Tech (with participation from UC Berkeley, NYU Abu Dhabi, the University of Toronto, and Nvidia) has developed the Cobalt app for controlling robot arms that can be operated from anywhere in the world using a smartphone. In principle, this allows anyone without technical prior knowledge to operate a robot. The underlying goal is to collect and use the resulting control data for training control algorithms for various autonomous tasks of the robot.
The Cobalt app for smartphones works comparatively simply: via a secure Wi-Fi connection, it establishes an internet connection to a teleoperation server, through which a robot arm can be controlled. To control it, users simply need to move their mobile phone within the room. The internal sensors map the movements, and the robot arm imitates them. Users can immediately view the result via a low-latency live video feed. The data transfer is realized via WebRTC (Web Real-Time Communication), the researchers write in the study “COBALT: Crowdsourcing Robot Learning via Cloud-Based Teleoperation with Smartphones,” which is published as a preprint on Arxiv. The lead author, Ayush Agarwal, also presented the results at the “IEEE International Conference on Robotics and Automation” in Vienna, which ends on June 5, 2026.
The teleoperation server can be accessed from anywhere in the world, allowing for a large user base to be acquired. After all, around five billion people worldwide own a smartphone, according to the researchers.
Using the Cobalt app, users can instruct the robot arm to grasp, move, and release an object, for example. Various object manipulations are possible. This is done more intuitively via a smartphone than with other input devices.
Empfohlener redaktioneller Inhalt
Mit Ihrer Zustimmung wird hier ein externes YouTube-Video (Google Ireland Limited) geladen.
Ich bin damit einverstanden, dass mir externe Inhalte angezeigt werden. Damit können personenbezogene Daten an Drittplattformen (Google Ireland Limited) übermittelt werden. Mehr dazu in unserer Datenschutzerklärung.
The idea behind this is that even inexperienced people can work with a robot. The scientists record the control data they generate. They use this to train robot control algorithms to automate the execution of specific tasks with a robot.
Crowdsourcing Robot Training Data
The researchers envision using such a system to enable data collection for robot training through crowdsourcing. This would solve the problem of acquiring the large-scale data required for training robots for mass production. This is because training autonomously acting robots cannot achieve the required quality through simulations alone.
Videos by heise
The scientists tested their system with test subjects from nine different countries. This included people with no prior robot experience and students from Midtown High School in Atlanta. The collected data proved to be qualitatively suitable for robot training, matching data recorded with professional virtual reality headsets, controllers, keyboards, or computer mice.
(olb)
