On this page
- Short answer
- Translate a video with lip sync, step by step
- What it costs to translate a video with lip sync
- What "translate a video with lip sync" means on KreatorFlow
- Which languages can you translate into?
- Can the translation keep the speaker's own voice?
- Honest limits: what this workflow cannot do
- If your video cannot be rebuilt from a photo
- Tips for a translated video that looks right
- Consent and real people
- Frequently asked questions
Short answer
On KreatorFlow you do it in three steps: translate the speech with Live Interpretation (English, Mandarin, Japanese, Spanish, Portuguese, French, German, Korean, Italian, Russian and Arabic into 11 languages, 3 credits credits per minute), voice the translation with a built-in voice or your own cloned voice, then turn a still photo of the speaker into a new talking video with Lip Sync (4 credits per second at 720p, 6 at 1080p, up to 5 minutes per take). Be clear on one limit: Lip Sync animates a photo, it does not re-time the mouth in your original footage, so the result is a new video of the speaker, not your old clip with new lips.
Get started
Translate a video with lip sync, step by step
- 1
Get the speech out of your video as audio
Live Interpretation takes an audio clip (WAV, MP3 or M4A, up to 50 MB), not a video file. Export the soundtrack from your video with any editor or converter, and trim silence at the start and end. Clear speech from one speaker gives the best transcript.
- 2
Translate it with Live Interpretation
Open Live Interpretation in the Audio studio, upload the clip, leave the source on auto-detect or pick it, and choose the language to translate into. You get three results: the original transcript, the translated text and the translation read aloud by a built-in voice. It costs 3 credits credits per started minute of audio, shown before you start.
- 3
Choose the voice for the translated version
Use the spoken translation as it is, or copy the translated text and voice it yourself: in Text to Speech with a preset voice (45 voices in 14 languages, 2 credits credits per 1,000 characters), or with your own cloned voice if you have made one. Check the translated text before you voice it: fix names, numbers and product terms first.
- 4
Take a clear still of the speaker
Pick one frame from your video where the face is sharp, looking towards the camera, with the mouth uncovered. Head and shoulders or half body works best. Save it as a JPG, PNG or WebP image.
- 5
Make the talking video in Lip Sync
Open Lip Sync, add the photo, then either upload the translated voice track (MP3 or WAV, 4 seconds to 5 minutes) or paste the translated text and pick one of our voices. Add an optional direction such as "Static medium shot. He speaks calmly and nods on the key points." Choose 720p or 1080p and generate. The video is exactly as long as the audio.
- 6
Finish in the editor
For a video longer than 5 minutes, translate and lip sync it in parts, then join the parts in the KreatorFlow editor. Add captions in the new language there if you want them, and export.
What it costs to translate a video with lip sync
Studio credits for the two steps priced by time: Live Interpretation (per started minute of the original audio) and Lip Sync (per second of the translated audio). Read from the same code that charges you.
| Length | Translate | Lip Sync 720p | Lip Sync 1080p | Total 720p / 1080p |
|---|---|---|---|---|
| 30 seconds | 3 credits credits | 120 credits credits | 180 credits credits | 123 credits credits / 183 credits credits |
| 1 minute | 3 credits credits | 240 credits credits | 360 credits credits | 243 credits credits / 363 credits credits |
| 5 minutes | 15 credits credits | 1,200 credits credits | 1,800 credits credits | 1,215 credits credits / 1,815 credits credits |
One minute of translated talking video costs about 243 credits credits at 720p or 363 credits credits at 1080p, assuming the translation is about as long as the original. Lip Sync bills per started second of the translated audio, minimum 4 seconds. If you voice the text in Text to Speech instead of using the built-in spoken translation, add 2 credits credits per 1,000 characters; a voice clone is 60 credits credits once. Lip Sync needs a paid account: any plan (from $15 for the first month on Creator) or a credit pack (from $8). Failed jobs are refunded automatically.
What "translate a video with lip sync" means on KreatorFlow
AI dubbing has two halves: changing the language of the speech, and making the speaker's mouth match the new words. KreatorFlow covers both, but with separate tools, and it is worth knowing how they fit together before you start.
The translation half is the Audio studio. Live Interpretation listens to a recorded clip, writes down what was said, translates it and reads the translation aloud with a built-in voice for the target language. The lip sync half is Lip Sync, which runs OmniHuman 2.0, ByteDance's audio-driven model, through BytePlus, a ByteDance company (KreatorFlow is an official partner of BytePlus, a ByteDance company). It takes one photo and one voice track and makes a video of that person saying the words, with mouth shapes, expressions and head movement that follow the audio.
So the finished translated video is a new performance generated from a still of your speaker. That suits talking-head content well: a founder message, a course lesson, a product explainer, a social post to camera. It suits footage with a lot going on (walking, cutaways, several people talking over each other) much less, and the limits section below says what to do there.
Which languages can you translate into?
Live Interpretation translates from English, Mandarin, Japanese, Spanish, Portuguese, French, German, Korean, Italian, Russian and Arabic, or auto-detects the source, into English, Mandarin, Japanese, Spanish, Portuguese, French, German, Korean, Italian, Russian and Arabic. Each target language has a built-in voice that reads the translation.
If you voice the translated text yourself, Text to Speech has preset voices in English, Mandarin, Japanese, Spanish, Portuguese, Indonesian, French, German, Korean, Arabic, Russian, Italian, Vietnamese and Thai. Each preset voice is listed under one language, so pick a voice that belongs to your target language. For languages and accents without a preset voice, Audio Generation speaks 24 languages from a written prompt, with the voice described in words, so two runs can sound different.
Lip Sync follows whatever audio you give it. Among our own published Lip Sync examples are clips in French and Japanese, made from Text to Speech voices.
Can the translation keep the speaker's own voice?
Not automatically. The spoken translation from Live Interpretation is read by a built-in voice, not by the original speaker. To hear your own voice in the new language, clone it first in Voice Replication, then voice the translated text with that clone in Text to Speech and use the result in Lip Sync.
A clone needs 10 to 30 seconds of clean speech from one speaker and costs 60 credits credits, once. You choose the sample's language from 14: English, Mandarin, Japanese, Spanish, Portuguese, Indonesian, French, German, Korean, Italian, Thai, Vietnamese, Russian and Arabic. The cloning rules recommend recording in the language the voice will mostly speak. KreatorFlow does not promise that a clone sounds the same in a different language from its sample, so try a short line in the target language before you voice a whole video.
You may only clone your own voice, or a voice whose owner has given you explicit written permission to clone it and use it the way you intend. You tick a box confirming that before every clone, the clone cannot start without it, and KreatorFlow records the date and time. Voice names that point to a public figure, celebrity or politician are refused, and the voices of minors are never allowed.
Honest limits: what this workflow cannot do
Before you plan a project around it, these are the things KreatorFlow does not do today.
- It does not re-dub your original footage in place. Lip Sync starts from one photo and generates a new video; the original gestures, camera moves and cuts are not kept.
- It does not take a video file for translation. Live Interpretation and Lip Sync both take audio, so you export the soundtrack first.
- It does not separate a voice from background music. Clean speech gives the best transcript, and a translated voice track will not carry the original music underneath.
- It does not translate live calls or streams. Live Interpretation works on a finished recording.
- It does not keep the original voice by itself. That needs a voice clone, made with consent.
- One Lip Sync take is 4 seconds to 5 minutes long. Longer videos are made in parts and joined in the editor.
- One voice track drives one speaker. In a photo with several people, the direction has to say who speaks.
- Lip Sync is for paid accounts: any plan or any credit pack.
If your video cannot be rebuilt from a photo
Some footage does not work as a single still: a vlog where you walk around, a tutorial that cuts to your screen, an interview with two cameras. For those, the honest options are a translated voiceover or subtitles, without lip sync.
For a voiceover, translate the speech as above, then open the KreatorFlow editor, upload the video (up to 200 MB) and put the translated track on an audio track under it (audio files up to 15 MB). For subtitles, the editor's auto captions turn speech into timed captions; if you only need the translated words as text, Live Interpretation already gives you the translated transcript, and Speech to Text gives you the original transcript for 1 credit credit per 5 minutes of audio (1 credit credit for a one-minute clip).
A middle route also works well: keep your original video for the parts with action, and use a Lip Sync clip of the speaker for the parts where they talk straight to camera, joined in the editor.
Tips for a translated video that looks right
Most problems come from the inputs, not the model. Check these before you generate.
- Use a sharp still with one face, mouth and chin uncovered, and even light. A tiny face in a wide shot gives a weak result.
- Read the translated text before voicing it. Machine translation can get names, numbers, slang and product terms wrong.
- Translations often run longer or shorter than the original. Since the video is as long as the audio, that is fine for a new talking video, but keep it in mind if the clip has to fit an existing edit.
- Test with a short line at 720p (4 credits per second) before you render the whole video at 1080p (6 credits per second).
- Match the direction to the tone of the new voice: a calm voice with an excited direction looks wrong in any language.
Consent and real people
A translated talking video puts words in someone's mouth, in a language they may not speak, so the same rules apply as for any Lip Sync video. You must be the person in the photo or have their permission to make them speak, and you must have the right to use the voice. On the website you confirm this the first time you generate with a new photo.
Never use it to deceive: no impersonation, no fake statements from real people, no political deepfakes. Sexual content is never allowed, nor anything involving someone under 18, and photos with nudity or of minors are refused before anything is charged. When you share a realistic translated video of a real person, say that it is AI-generated. To report misuse, write to support@kreatorflow.ai.
Frequently asked questions
Can AI translate a video and match the lip movements?
Yes, as a new video. Translate the speech with Live Interpretation, then use Lip Sync to make a still of the speaker say the translated audio, with mouth movement matched to the new language. KreatorFlow does not edit the lips inside your original footage.
How do I translate a video into another language with my own voice?
Clone your voice in Voice Replication (10 to 30 seconds of your speech, 60 credits credits, consent required), voice the translated text with the clone in Text to Speech, then use that track in Lip Sync. Test a short line first: the clone is trained on the sample's language.
Which languages can I dub a video into?
Live Interpretation translates into English, Mandarin, Japanese, Spanish, Portuguese, French, German, Korean, Italian, Russian and Arabic. Text to Speech has preset voices in 14 languages, and Audio Generation speaks 24 languages from a prompt.
How much does AI video translation with lip sync cost per minute?
About 243 credits credits per minute at 720p: 3 credits credits to translate a minute of speech plus 240 credits credits for a minute of Lip Sync. At 1080p it is about 363 credits credits.
Can I translate a video for free?
The Audio studio runs on credits on every plan, so a new account's starter credits can cover a short translation (3 credits credits per minute). Lip Sync is not free: it needs any paid plan or a credit pack.
Can I translate a video of someone else?
Only with their permission to make them speak, and you may only clone their voice with their explicit written permission. Public figures need permission too, and nothing involving minors is allowed.
