Test 3 does regenerate lip movement, with reported face-region diffs of 5-60/255 and visibly different mouth shapes at matching timestamps.
What was measured
Lip Sync Accuracy
Does the dubbed audio visually match lip movements in the original video?
decisive for this rankingtransformation
The ranking is specifically about lip sync, so visual alignment of speech to mouth movement is a core success criterion. (3 of 3 judges)
What was given, what came back
Test input: Hindi vlog-style talking head (Hindi → English) · video · group: video-translation-voice-clone-lip-sync
Input — what we sent
Hindi vlog-style talking head (Hindi → English)
A casual Hindi talking-head/vlog-style video used to test English dubbing on informal speech, Hinglish-like phrasing, speaker personality preservation, and lip sync under natural creator-style delivery.
Why this input is hard
- · Informal Hindi speech transcription
- · Hindi-to-English translation of casual phrasing
- · Voice cloning for conversational creator tone
- · Handling of slang/Hinglish-style content
- · Lip-sync stability during expressive speech
Output — unretouched
Also checked on this input — same tool, 1 other criterion
Provenance
- Observation
- bd9dc91f-e926-41d1-9fe8-1b99b27501e7
- Evidence run
- ec1dd100-6af8-4f49-8f19-b78492702b14
- Study
- Translate Videos with Voice Cloning and Lip Sync Using AI
- Research task
- 86b96dfpf
- Tested at
- not recorded
- Source
- first-party
- Evidence state
- verified
- Proof shown
- input + output shown
- Cost / latency
- not captured
- Repeat run
- not captured
- Tester
- not captured
The last three rows are honest blanks, not placeholders — our capture has no field for them yet.
Query this
get_evidence({
tool: "veed-io",
scenario: "video-translation-voice-clone-lip-sync"
})MCP · mcp.aidemos.com/api/mcp
Free with attribution.
Same input, same check — 9 other tools
measured on Lip Sync Accuracy
Akool◐ MixedMouth movement stayed active and varying across the clip, but precise frame-matched verification was not possible because timing drifted partway through.Camb AI✗ FailedOn the Hindi→English vlog clip, the output again left mouths unchanged, with only ~1.7-2.7/255 pixel difference across the 10-second clip and frame-identical mouths at t=0.0s.D-ID✗ FailedLip sync is stable on the avatar, but it does not reflect the original speaker's expressions.Dubverse✗ FailedLip sync is the weakest of the three tests, with sync inconsistencies and slightly off audio-video alignment in fast speech.ElevenLabs✗ FailedThe tool provides no built-in lip sync, so the vlog output does not visually match the speaker’s mouth movements without external re-editing.HeyGen✗ FailedAt the matched "AAP" caption, the output kept an identical open mouth and identical raised finger, so no visible lip-sync regeneration was observed.Rask AI✗ FailedThe Hindi→English talking-head export is reported as frame-identical to the source at the same timestamp, with no mouth or face modification visible.Sync Labs◐ MixedLip sync works, but it struggles during expressive facial movements and fast speech.Synthesia✓ WorkedThe report says the vlog dub's timing held up consistently with the source, so the mouth timing looks visually stable in this run.
This evidence is published in
From the same study (page rebuilt from a later run)
Real inputs and real outputs, no retouching · every cell queryable via API & MCP · aidemos.com