Caption translation failed outright: in the Hindi output run, every caption stayed in English from start to finish, including the final "I don't know." line.
What was measured
Translation Accuracy
Is the translated output semantically correct, and does it preserve technical terms, tone, and meaning?
decisive for this rankingtransformation
If the translation changes meaning, tone, or technical terms, the tool fails the core job of translating the video. (3 of 3 judges)
What was given, what came back
Test input: Fitness instructor short (English → Hindi) · video · group: video-translation-voice-clone-lip-sync
Input — what we sent
Fitness instructor short (English → Hindi)
A short fitness/coaching video with energetic single-speaker English speech, used to test whether tools can translate into Hindi while preserving fast delivery, motivational tone, and original-face lip sync.
Why this input is hard
- · Fast, energetic speech transcription
- · Tone preservation for motivational fitness content
- · English-to-Hindi translation quality
- · Lip-sync accuracy on a moving face
- · End-to-end dubbing workflow automation
Output — unretouched
Also checked on this input — same tool, 2 other criteria
Lip Sync Accuracy◐ MixedThis fitness run did not receive an independent lip-sync judgment in the report, so there is no direct evidence here for how closely the dubbed mouth movement matched the source.Output Quality & Export✗ FailedThe visual compositing pass is unstable in the fitness run: the subject's body and white tank top turn a bright unnatural green for several seconds before returning to normal.
Provenance
- Observation
- ee26b601-e997-43e0-aead-de5cf0738015
- Evidence run
- ec1dd100-6af8-4f49-8f19-b78492702b14
- Study
- Translate Videos with Voice Cloning and Lip Sync Using AI
- Research task
- 86b96dfpf
- Tested at
- not recorded
- Source
- first-party
- Evidence state
- verified
- Proof shown
- input + output shown
- Cost / latency
- not captured
- Repeat run
- not captured
- Tester
- not captured
The last three rows are honest blanks, not placeholders — our capture has no field for them yet.
Query this
get_evidence({
tool: "synthesia",
scenario: "video-translation-voice-clone-lip-sync"
})MCP · mcp.aidemos.com/api/mcp
Free with attribution.
Same input, same check — 9 other tools
measured on Translation Accuracy
Akool✗ FailedBurned-in English captions such as "This is a famous" were not localized in the Hindi output, so the tool left source captions untranslated.Camb AI✗ FailedOn the fitness clip, burned-in English captions were not translated; the output keeps captions like 'This is a famous' in English instead of localizing them.D-ID✓ WorkedIt produces an acceptable Hindi translation of the source speech.Dubverse✓ WorkedIt translates common fitness terminology from English to Hindi accurately, though the report notes some fitness-specific words were not localized naturally.ElevenLabs✓ WorkedThe Hindi translation was reported as accurate for the fitness video.HeyGen✗ FailedThe tool left the burned-in fitness captions in English instead of localizing them; the review cites unchanged lines such as "This is a famous," "For every question," and "I don't know."Rask AI✗ FailedThe English→Hindi fitness export does not localize the visible on-screen text: the translated clip still shows English questions such as 'Which head of the shoulder is most active on this exercise?' and 'Can you name the three BCAAs?'.Sync Labs◐ MixedThe Hindi translation is moderately accurate, so the meaning is mostly preserved but not at a strong or flawless level.VEED⚠ StruggledThe tool does not localize baked-in quiz overlays: the shoulder question and BCAA headline remain 100% English while Hindi dialogue captions appear below them.
This evidence is published in
From the same study (page rebuilt from a later run)
Real inputs and real outputs, no retouching · every cell queryable via API & MCP · aidemos.com