Analysis of sign language and non-verbal techniques will be introduced into speech recognition systems

The study itself was devoted to people's perception of information that is spoken out loud, as well as assessment

influence of gestures.Volunteers were asked to analyze videos of people speaking. The speakers' faces were hidden, but study participants could see the speakers' gestures. Most listeners paid attention to the “rhythm” gestures that people often use to highlight important places in speech and semantic units. In addition, respondents said that it was gestures that helped them concentrate on certain parts of the performance. And they were the ones who remembered them best.

In addition, the gestures affected word recognition.For example, participants perceived a syllable as stressed if at that moment the speaker made an accent with a gesture. If the speaker pronounced only a set of letters, but again made an accent, then the study participants still perceived it as a word with different vowel lengths or stress, depending on the speaker's gesture.

“People listen not only with their ears, but also with their eyes.These results show for the first time that gestures influence what sounds we hear." The researchers believe the effect may be stronger in everyday life, where speech is typically less intelligible than during the experiment in the laboratory. So in noisy environments, gestures can be useful.”

Hart Rutger Bosker, study author from the Society for Scientific Research. Max Planck

Such studies, according to scientists,improve speech recognition systems. Since non-verbal means of communication - facial expressions and gestures - affect both the perception of speech and its semantic load. Technology should not only learn to recognize the intonation or tempo of speech, but also how the speaker prepares information through gestures.

See also:

Scientists have developed a replacement for the theory of relativity. What is the essence of the "theory of everything"?

Abortion and science: what will happen to the children who will give birth

A new concept of a solar flare rocket engine has been created