One of the problems when it comes to dubbing films into other languages is that often the phrases we hear do not correspond to the movement of the actor’s lips. A company called Flawless has found the solution so that this does not clash: replace the artist’s lips with digital ones.
This technology, quite similar to deepfakes, uses software called TrueSync that analyzes the movement of the interpreters’ mouths. In this way, it records its muscular characteristics to associate its joints with the corresponding sound of speech. After capturing the data, the software creates an exact-to-real digital mouth that moves following the sounds of each language. The result is good and, if you don’t believe it, here is the proof:
It’s been a few years in the making. We’d like to thank the entire Flawless team for getting us to our full commercial launch. Special thanks to our collaborators in science and The Max Planck Institute for Infomatics in Germany.
We’re pleased to release the Flawless showreel… pic.twitter.com/QVtwMPMLXf
— Flawless (@Flawlessai) May 3, 2021
As you have seen in the video, Flawless’s technique allows us to see Tom Hanks playing Forest Gump speaking perfect Spanish in a scene from the film. While before what the actor said did not square with what we could read on his lips, now, with the Flawless program, it can be pretended that Tom Hanks himself acted in Spanish from the beginning. We know that this is not true, but the software allows us to imagine it.
The developers of this new method explain that their software studies absolutely all the nuances of the actors’ interpretation to preserve them without any alteration whatever the language that is later used for dubbing. Can you imagine seeing your favorite foreign actor speaking Spanish as if it were his native language and in a completely credible way?
Sign up for our newsletter and receive the latest technology news in your email.