LOS ANGELES: Bad lip-syncing in dubbing and subtitles can put off audiences and hurt box office takings of foreign films.
AI may be about to change all that.
Start-up Flawless AI, co-founded by film director Scott Mann, has a tool that it says can accurately recreate lip sync in dubbing without altering the performance of the actors.
The tool studies how actors move their mouths and swaps the movements out according to the dubbed words in different languages, making it seem like Tom Hanks can speak Japanese or Jack Nicholson is fluent in French.
Mann was inspired to come up with the tool when he saw how dubbing affected the narrative cohesion of his 2015 film "Heist", starring Robert De Niro.
"I hate dubbing as it stands," he told Reuters. "You have to change so many things to try and catch sync. You're changing words filmmakers and performers have thought about so deeply. They're thrown out to find a different word that fits, but it never really does."
Mann decided to do something about this conspicuous mismatch. After some research, he discovered a white paper by Christian Theobalt of the Max Planck Institute for Informatics, laying out a new approach that uses artificial intelligence to recreate photo-real human visual effects.
The tool worked to translate a De Niro scene in "Heist".
"It's able to essentially take an 'ooh' sound from De Niro 20 minutes earlier and place that in a different moment in the film," explained Mann. "It measures at the same time and blends it so that the performance is the same but it's a different mouth movement."
Flawless AI is now working with producers and studios to integrate the technology into post-production. The first films using this tool are around a year away.
Mann believes this technology will be a game-changer, helping to eradicate foreign-language categories at award shows and creating a more inclusive film industry with a real global stage.
"From a filmmaking point of view, you're going to see the rise of a much more diverse range of stars," he said.
There might be a dissonance and a focus on the AI technology in the beginning when it is introduced, but eventually Mann hopes its use will be seamless.
"When it's a common thing, we're not going to think about it anymore and we'll just enjoy the movies," he said.
Reuters
Thu May 20 2021
![Is that Tom Hanks speaking in Japanese? No, it's just AI Is that Tom Hanks speaking in Japanese? No, it's just AI](https://resizer-awani.eco.astro.com.my/tr:w-177,h-100,q-100,f-auto/https://img.astroawani.com/2021-01/81610602869_TomHanks.jpg)
The tool studies how actors move their mouths and swaps the movements out according to the dubbed words in different languages, making it seem like Tom Hanks can speak Japanese or Jack Nicholson is fluent in French. Filepic/REUTERS
![Bapa bintang bola sepak Sepanyol, Lamine Yamal ditikam - Laporan Bapa bintang bola sepak Sepanyol, Lamine Yamal ditikam - Laporan](https://resizer-awani.eco.astro.com.my/tr:w-177,h-100,q-100,f-auto/https://img.astroawani.com/2024-08/61723677980_MounirNasraoui.jpg)
Bapa bintang bola sepak Sepanyol, Lamine Yamal ditikam - Laporan
Mounir Nasraoui, bapa kepada Lamine Yamal, ditikam pada lewat Rabu di tempat letak kereta di bandar Mataro di timur laut Sepanyol.
Putin anggap rundingan dengan Ukraine tidak berguna
Presiden Rusia Vladimir Putin menyifatkan, tindakan Ukraine yang menyerang orang awam memperlihatkan keengganan Kyiv menyelesaikan konflik peperangan dua negara dengan cara perdamaian.
Fadillah tiba di Peru, hadiri mesyuarat menteri-menteri tenaga APEC 2024
Timbalan Perdana Menteri, Datuk Seri Fadillah Yusof tiba Lima, Peru bagi menghadiri Mesyuarat Menteri-Menteri Tenaga (EMM) di bawah Kerjasama Ekonomi Asia Pasifik (APEC) yang berlangsung selama dua hari.
BNM denda Maybank RM4.32 juta, CIMB RM760,000 berikutan gangguan perkhidmatan berpanjangan
Bank Negara Malaysia mengenakan Penalti Monetari Pentadbiran (AMP) sebanyak RM4.32 juta ke atas Malayan Banking Bhd dan Maybank Islamic Bhd serta RM760,000 ke atas CIMB Bank Bhd dan CIMB Islamic Bank Bhd susulan gangguan beberapa perkhidmatan perbankan yang berpanjangan.
![Pertubuhan kebajikan New Zealand mohon maaf edar gula-gula bercampur dadah Methamphetamine Pertubuhan kebajikan New Zealand mohon maaf edar gula-gula bercampur dadah Methamphetamine](https://resizer-awani.eco.astro.com.my/tr:w-177,h-100,q-100,f-auto/https://img.astroawani.com/2024-08/41723674988_pineapplesweetsme.jpg)
Pertubuhan kebajikan New Zealand mohon maaf edar gula-gula bercampur dadah Methamphetamine
Auckland City Mission secara tidak sengaja telah mengagihkan gula-gula bercampur Methamphetamine yang boleh membawa maut melalui pek makanan.
Pesakit buah pinggang di Gaza hadapi kekurangan mesin dialisis
Pesakit buah pinggang di Gaza menghadapi kekurangan peralatan dialisis dan bekalan perubatan yang teruk di Khan Younis.
Kekejaman zionis: Hamas hilang kepercayaan A.S jadi pengantara gencatan senjata
Hamas hilang kepercayaan terhadap keupayaan Amerika Syarikat untuk menjadi pengantara gencatan senjata di Gaza.
Kekejaman zionis: Hamas tidak ambil bahagian rundingan gencatan senjata baharu
Hamas tidak akan mengambil bahagian dalam pusingan baharu rundingan gencatan senjata Gaza yang dijadualkan di Qatar, pada Khamis.
Malaysia kecam pencerobohan Israel ke atas Masjid Al-Aqsa
Wisma Putra dalam satu kenyataan pada Rabu berkata perbuatan keji ini yang dilakukan di bawah perlindungan pasukan keselamatan Israel, merupakan pencerobohan serius terhadap satu daripada tapak paling suci Islam.
![WHO isytihar mpox sebagai kecemasan kesihatan awam global WHO isytihar mpox sebagai kecemasan kesihatan awam global](https://resizer-awani.eco.astro.com.my/tr:w-177,h-100,q-100,f-auto/https://img.astroawani.com/2024-08/41723675747_Mpoxvirus.jpg)
WHO isytihar mpox sebagai kecemasan kesihatan awam global
Ini merupakan pengisytiharan kali kedua dalam tempoh dua tahun berikutan wabak yang bermula di Congo sudah merebak ke negara jiran.