OpenAI learned how to play Minecraft by watching 70,000 hours of YouTube videos

The developers report that the AI ​​has learned basic skills such as cutting down trees, making planks and

making tables for crafting.They also observed him swimming, hunting, cooking, and "pillar jumping." Moreover, having mastered the basic skills, the system learned how to create a diamond pickaxe. To practice this skill in the game, human players need about 20 minutes. and 24 thousand actions.

For AI training, the OpenAI team usedopen videos: about 70,000 hours of gameplay footage. To cope with this volume, the company developed a new strategy: pre-training with a “semi-teacher”.

Scheme of learning.The first stage is the search for video for training, the second stage is training the IDM neural network based on the video containing information about mouse movements and keystrokes, the third stage is video markup using IDM and training the game neural network. Source: OpenAI

In the first step, the researchers collected data fromvolunteers: they recorded video games, as well as keystrokes and mouse movements. Based on this data, the developers trained the inverse dynamics model (IDM) to determine what actions the player performs based only on video data. After that, IDM independently "viewed" and marked up the recordings of the game published on YouTube. AI training in the game is carried out using already marked IDM data.

The developers note that the behavioral modelA cloner ("player") trained on an IDM-tagged online video performs tasks in Minecraft that are almost impossible to complete with traditional reinforcement learning from scratch. He learns to chop down trees to collect logs, turn those logs into planks, and then make a crafting table out of the planks. This sequence takes about 50 seconds or a thousand consecutive game actions for a person who owns Minecraft.

The developers also showed that additional fine-tuning through AI observation of the actual game process helps to quickly teach the model more complex skills.

Researchers note that Minecraft isjust one example of the possible applications of the new technology. In general, pre-training allows the AI ​​to use minimal resources to acquire various skills based on a large amount of video data.

Read more:

The space probe flew 200 km from Mercury. Look what he saw

Chinese mind-reading helmet sounds the alarm when a person sees porn content

The satellite of Jupiter was looked at in a new light: what scientists saw there