Facebook has created an artificial intelligence that turns real people into manageable game characters.

A Facebook research group has created an AI called Vid2Play that can make game characters look like

based on videos of real people.

How did they do it

The team used two neural networks:Pose2Pose and Pose2Frame. First, the video is fed into Pose2Pose, which is designed to identify different types of actions like dancing, playing tennis, or fencing. The system then separates the person's figure, along with his shadow and objects in his hands, from the background and inserts them onto a black background. After this, you can control the movement of the object using a joystick or keyboard.

To train the system it took onlyseveral short videos. The artificial intelligence was able to remove other people and compensate for different camera angles. The study is reminiscent of the function in Adobe After Effects to remove unnecessary elements from a video, such as tourists or trash cans.

So far, the animation is not perfect - the figure looks like a sliding one and the range of motion is a bit limited. However, this is a great progress compared to previous attempts to extract a character from the video.

This innovation will allow game developers to create their own characters based on personal videos or add their favorite personality from YouTube. The movements will be more realistic than when creating the animation manually.