Developers from Facebook AI and 13 other universities have created the most extensive dataset of videos that were shot
855 of 74 volunteers participated in data collectioncities and 9 countries. They filmed videos with a camera and smart glasses. During the shooting, the participants were engaged in daily activities: preparing food, choosing food in the store, communicating with other people. The total length of the records was 3,025 hours.
For each individual video there is a description of whatwhat is happening in it. Also for part of the video there is other detailed data about the sound, a three-dimensional model of the room, the direction of view, as well as stereoscopic frames taken with a wearable stereo camera from one person and a set of synchronized videos made by different people in one place and at the same time, for example, board game time.
Also, using the collected data, you can learn specific skills; for this, special thematic modules have been created:
- To train episodic memory, when the algorithm must answer the question about the events that have occurred, for example, what time I had a doctor's appointment yesterday.
- Predicting future actions, for example, remembering to salt a dish or removing things from the washing machine.
- Understanding what a person is doing and how objects move in the frame.
- Tracking the conversation and understanding who said what in the conversation.
- Analysis of the interaction between people, in particular the understanding that the camera carrier is being looked at or addressed.
The authors plan to publish the dataset by the end of November 2021; it will be available after the user signs the license agreement.
Read more
Hawking was right, but sometimes wrong: the scientist's most daring ideas
Astronomers have figured out that the Earth and solar system are in a giant magnetic tunnel
Frozen mammoth and a man in the "asphalt": how nature stops time