AI was able to translate brain signals into sentences with almost no errors

Joseph Mackin of the University of California and colleagues used deep learning algorithms to

studying brain signals of four patients. All of them suffered from epilepsy, so they had already attached brain electrodes that transmitted data about seizures.

Each woman was asked to read the set out loud.proposals, at the same time, the team recorded the activity of their brain. The largest group of sentences contained 250 unique words. The team fed this brain activity into the neural network algorithm, teaching it to identify regularly occurring patterns that may be associated with repetitive aspects of speech - for example, a combination of vowels and consonants. Then these patterns were fed into the second neural network, which tried to turn them into words in order to form sentences.

Every time a person says the same thingsuggestion, brain activity will be similar, but not identical, the researchers explained. “Memorizing a person’s brain activity while reading sentences will not help, so the algorithm should instead understand what is similar in the patterns and summarize these data,” says Makin.

AI will help doctors predict the growth of patients with COVID-19 and allocate resources to them

During the tests, the best AI results containedin itself only 3% of errors. Researchers are sure that the algorithm was helped by the fact that patients read out simple sentences with a small number of unique words. But in some cases, the AI ​​was able to parse and distinguish similar in sound words only by brain activity (for example, the words Tina and Turner).

The team tried to decode brain datasignals immediately in separate sentences. But the error rate immediately rose to 38%. Researchers note that while AI can not quickly cope with this task. “Usually people know and use up to 350 thousand words, but the algorithm cannot decrypt them all. Developing its capabilities will be incredibly difficult, ”scientists say.