Yandex added voice-over for English-language videos

The press service of Yandex announced that the function of machine translation of videos has already been added to the stable versions of the proprietary Yandex Browser on Windows, macOS, Linux and to the mobile application for Android and iOS.

Thanks to the new service, users will be able to watch English-language videos in high-quality translation on popular platforms YouTube, Vimeo, TikTok, Twitter, VKontakte and Facebook. During translation, the original audio track will be muted so that the Russian dubbing can be heard better.

At the same time, the developers clarified that the translation process is a complex multi-stage task and can take some time. Therefore, the user will have to wait a few minutes by pausing the video. As soon as they are ready, the viewer will receive a notification.

After the user turns on the video and clicks on the translation button, which will appear right in the player, artificial intelligence will begin to recognize speech in order to convert it into text. If there are subtitles loaded, the AI ​​will skip this step and immediately start editing the text, and then send it for translation.

Next, the neural network will determine the gender of the speaker (it is known that the female voice will be voiced by Alice), synthesize voices and combine audio with video. According to experts, the last task is the most difficult, since sentences in Russian are much longer than in English. Therefore, to synchronize the picture and sound, the pauses between words will be shortened and the speech rate will accelerate.

The developers expect that the new service will be appreciated by all users and will enable Yandex to further develop and improve the new machine voice-over translation technology, add other languages ​​and voices for voice acting.