The neural network creates the illusion that the actors in the film are speaking lines from the dubbing, and not from the original audio
As the authors state, their development preserves everythingnuances and emotions of the source material. TrueSync replaces the actor's face with a deepfake, taking into account the original facial expressions, so as not to lose the subtleties of the actor's performance. The creators claim that with this approach, original emotions are preserved.
The Flawless company website has published excerpts from films processed using deepfake technologies, for example, the hero of Tom Hanks from "Forrest Gump" "spoke" in Japanese.
The TrueSync software package uses neural networks to synthesize the movements of the lips and facial muscles of the actors.
Deepfakes have already been used to "resurrect"deceased musicians. Now the use of technology has come in handy in cinematography. The authors of the development note that the technology is not ideal yet. Still, in some moments, the shortcomings of the deepfake are clearly visible. In addition, the emergence of technology raises several questions. For example, how close is a film with such a “visual dubbing” to the original idea of the director, or does the deepfake devalue the acting of the actors of the original film?
There are very good reasons to be concerned aboutabout the use of artificial intelligence and tools based on neural networks for manipulating frames and videos. It is easy to fabricate situations that never actually happened, as well as statements and actions of public figures. But these tools also hold exciting potential for other purposes, such as making filmmaking more accessible.
Previously, TikTok appeared on the social networkrealistic deepfake videos of Tom Cruise. Then they confused the actor’s fans - it was almost impossible to distinguish him from the “understudy”. Also in the USA, with the help of similar technologies, they “resurrected” a teenager who died after a school shooting.
Read more
Scientists have shown how a black hole tears apart a star
Elon Musk: the first tourists to Mars will die
Scientists have conducted the largest genetic study of super-longevity