The English dub is polluted by untranslated Romanized-Hindi caption text; 'KAR RAHE HAIN' and 'MAIN' appear in the output, and 'MAIN' is described as a fabricated caption that was not present in the source frame.
What was measured
Translation Accuracy
Is the translated output semantically correct, and does it preserve technical terms, tone, and meaning?
decisive for this rankingtransformation
If the translation changes meaning, tone, or technical terms, the tool fails the core job of translating the video. (3 of 3 judges)
What was given, what came back
Test input: Hindi vlog-style talking head (Hindi → English) · video · group: video-translation-voice-clone-lip-sync
Input — what we sent
Hindi vlog-style talking head (Hindi → English)
A casual Hindi talking-head/vlog-style video used to test English dubbing on informal speech, Hinglish-like phrasing, speaker personality preservation, and lip sync under natural creator-style delivery.
Why this input is hard
- · Informal Hindi speech transcription
- · Hindi-to-English translation of casual phrasing
- · Voice cloning for conversational creator tone
- · Handling of slang/Hinglish-style content
- · Lip-sync stability during expressive speech
Output — unretouched
Also checked on this input — same tool, 1 other criterion
Provenance
- Observation
- 1e1f8faf-3b07-4aa3-ab63-88da22e098c5
- Evidence run
- ec1dd100-6af8-4f49-8f19-b78492702b14
- Study
- Translate Videos with Voice Cloning and Lip Sync Using AI
- Research task
- 86b96dfpf
- Tested at
- not recorded
- Source
- first-party
- Evidence state
- verified
- Proof shown
- input + output shown
- Cost / latency
- not captured
- Repeat run
- not captured
- Tester
- not captured
The last three rows are honest blanks, not placeholders — our capture has no field for them yet.
Query this
get_evidence({
tool: "veed-io",
scenario: "video-translation-voice-clone-lip-sync"
})MCP · mcp.aidemos.com/api/mcp
Free with attribution.
Same input, same check — 8 other tools
measured on Translation Accuracy
Akool✗ Failed"JARURAT PADTI HAI" and the later word "BATAANE" remained untranslated in the English dub.Camb AI✗ FailedOn the Hindi vlog clip, a full Hindi sentence ('JARURAT PADTI HAI') and another word ('AAP') remained untranslated, showing the tool can leave source-language on-screen text unchanged in the English output.D-ID◐ MixedThe English translation is understandable, but it sounds slightly formal and less natural for casual vlog speech.Dubverse◐ MixedThe Hindi-to-English translation remains understandable and retains the overall meaning, but it degrades on slang, informal tone, and mixed-language phrasing.HeyGen✗ FailedThe tool failed to translate the word-emphasis captions in the vlog clip; "AAP" and "MAIN" stayed untranslated in the output.Rask AI✗ FailedThe Hindi→English talking-head export renders the word-emphasis captions as Roman-script Hindi ('JARURAT PADTI HAI', 'HAR', 'BHII') instead of English, while pre-existing English branding is left untouched.Sync Labs◐ MixedThe English translation is mostly correct, but it does not preserve the casual vlog tone.Synthesia◐ MixedTranslation is partial in the vlog run: pre-existing English branding text is preserved correctly, but word-emphasis captions like "HAK" and "MAIN" remain untranslated in Roman script.
This evidence is published in
From the same study (page rebuilt from a later run)
Real inputs and real outputs, no retouching · every cell queryable via API & MCP · aidemos.com