In the horse-riding scene, the tool can erase the reference face entirely: the report calls the identity match very weak and says the output is a completely different character with changed face shape, eyes, and eyebrows.
What was measured
Identity preservation
How closely the generated face matches the reference image across scenes.
decisive for this rankingtransformation
This is the core of the task: the tool must keep the same character looking like the reference image across scenes. (3 of 3 judges)
What was given, what came back
Test input: Full frontal portrait · image
Input — what we sent
Full frontal portrait
Full frontal portrait reference image with fair skin, curly dark hair, bindi, gold jhumka earrings, and a green stone necklace. All features are clearly visible in good natural lighting, making it the easiest identity anchor for the tools.
Why this input is hard
- · Baseline identity preservation
- · Accessory retention
- · Best-case frontal face matching
- · Consistent character reuse across varied scenes
Output — unretouched
Also checked on this input — same tool, 4 other criteria
Expression accuracy✓ WorkedThe tool can capture a prompted angry and guarded mood, with direct eye contact and an intense expression in the interrogation shot.Scene compliance✓ WorkedThe horse-riding prompt is followed strongly, including the dusty desert setting, sunset lighting, horse motion, riding costume, gloves, boots, scarf, and believable action pose.Scene compliance✓ WorkedThe interrogation-room prompt is rendered cleanly, with a plain room, metal table, overhead lighting, formal clothing, and uncluttered environment all matching the scene.Scene compliance✓ WorkedThe warm cafe prompt is followed strongly, with the cozy cafe setting, warm lighting, background blur, sweater, braided hairstyle, and natural pose all rendered correctly.
Provenance
- Observation
- af4776d3-f013-4967-a4a8-bd0db677c8e4
- Evidence run
- 1dfb8fa4-f7f0-47dc-911b-7db3af467e9a
- Study
- Generate Consistent AI Characters Across Different Scenes and Poses
- Research task
- 86b96df11
- Tested at
- not recorded
- Source
- first-party
- Evidence state
- verified
- Proof shown
- input + output shown
- Cost / latency
- not captured
- Repeat run
- not captured
- Tester
- not captured
The last three rows are honest blanks, not placeholders — our capture has no field for them yet.
Query this
get_evidence({
tool: "gemini"
})MCP · mcp.aidemos.com/api/mcp
Free with attribution.
Same input, same check — 4 other tools
measured on Identity preservation
ChatGPT⚠ StruggledIdentity drops to only a 50–70% match in an action scene, with significant facial-structure drift away from the reference even though the body styling remains plausible.ImagineArt◐ MixedThe warm-cafe output keeps the subject recognisable but not exact; the report says eye colour shifts from black to brown and facial proportions are slightly altered.Leonardo AI◐ MixedIt can keep a frontal character recognisable in a new scene, but the face is refined enough that the match is only moderate.Scenario✓ WorkedThe tool can keep a subject recognisably the same person in an action scene, with face shape, eyes, and overall look staying close to the reference.
This evidence is published in
Real inputs and real outputs, no retouching · every cell queryable via API & MCP · aidemos.com