Generates syntactically correct Manim code on the first pass.
What was measured
Code Generation Quality
Syntactically correct, runnable on first attempt.
decisive for this rankingtransformation
If the generated code is not syntactically correct and runnable, the tool has failed at producing the animation at all. (3 of 3 judges)
What was given, what came back
Test input: Search engines basics explainer · text
Input — what we sent
The exact prompt
Create an animation video explaining how search engines work.
A vague, linear educational animation prompt asking the tool to explain how search engines work with minimal guidance. Designed to stress inference, scene structuring, pacing, and coherent visual flow from an underspecified request.
Output — unretouched
No output artifact
The verdict rests on the tester's written observation alone — no file was captured for this cell.
Also checked on this input — same tool, 5 other criteria
Export Experience✓ WorkedFinishes by producing an MP4 inside the web app and surfacing the result in a built-in video player, so getting the rendered video is straightforward.Export Experience◐ MixedGetting an MP4 is straightforward, but the free output is watermarked and the UI prompts an upgrade for watermark-free export.Output Quality⚠ StruggledAt moderate diagram complexity, the tool still matches the concept, but overlapping labels, arrows, and components make the render hard to follow.Output Quality◐ MixedMatches the approved search-engine workflow concept, but the final animation stays visually basic and shows minor clutter even on the low-complexity scene.Output Quality◐ MixedFor the low-complexity search-engine explainer, the animation stays conceptually aligned with crawling, indexing, ranking, and results display, but the visuals are basic, flat, and show minor overlap or clutter.
Provenance
- Observation
- 7062524a-d54a-4a4c-98b2-dd205aff10bb
- Evidence run
- c751a76b-a105-4b3e-9ff3-6c4d6adc9f04
- Study
- Generate Code-based Animations from text inputs
- Research task
- 86b9nnzmy
- Tested at
- Jun 24, 2026
- Source
- first-party
- Evidence state
- observed
- Proof shown
- input only
- Cost / latency
- not captured
- Repeat run
- not captured
- Tester
- not captured
The last three rows are honest blanks, not placeholders — our capture has no field for them yet.
Query this
get_evidence({
tool: "animg"
})MCP · mcp.aidemos.com/api/mcp
Free with attribution.
Same input, same check — 5 other tools
measured on Code Generation Quality
ChatGPT Canvas✓ WorkedGenerated syntactically correct, runnable HTML/CSS/JavaScript on the first attempt for the search-engine explainer.Claude Artifacts✓ WorkedIt generated a runnable webpage on the first attempt, with autoplay working on load.Gemini Canvas✓ WorkedGenerates syntactically correct HTML/CSS/JavaScript on the first attempt; the page ran immediately as a self-contained animation.Grok✓ WorkedProduced syntactically correct HTML/CSS/JavaScript code on the first pass.Replit✓ WorkedProduces runnable React animation code on the first attempt.
Real inputs and real outputs, no retouching · every cell queryable via API & MCP · aidemos.com