Researchers train AI to control robotic arm based on YouTube video

Scientists from Carnegie Mellon University have created a remote control system for automated

hand. All you have to do is stand in front of the camera and perform the necessary actions. AI, based on a two-dimensional image, generates a three-dimensional model of movement and forms an algorithm for a robotic arm.

Traditionally for training and distance learningrobots were controlled using special gloves, motion sensors or calibrated systems consisting of several cameras. The solution proposed in the study does not require sophisticated equipment and greatly simplifies the process of remote control. “Our system works using a single camera. Since calibration is not required, the user can stand anywhere and still successfully control the robot,” says Deepak Pathak, one of the developers of the system.

Human and robotic hands differ significantly inthe shape, size and structure of the joints. The motion trajectories that are recorded by the camera must not only be reproduced, but also converted into a sequence of actions of the robot's structural elements. Notably, the researchers used only YouTube videos to train their system.

“We understand that classical data collection models require significant costs,” the scientists note. “At the same time, there are a lot of videos of human hand movements on the Internet.”

Initial results of the studypromising: even untrained users can remotely control the robot. “Personally, the most important part of the research for me is the use of unstructured data from the Internet for robotics,” adds Pathak. “Our remote control system does not require gloves or markers and makes robot training more accessible. We hope that it will help in the creation of autonomous mechanisms.”

Read more

Scientists have found the healthiest people on Earth: it's all about a unique lifestyle

The danger of free VPNs. Why can't they be downloaded and how to protect yourself?

Physicists have recreated the abilities of the T-1000 from the "Terminator-2" in the laboratory