Produces syntactically correct HTML/CSS/JavaScript on the first attempt for a basic explainer animation.
What was measured
Code Generation Quality
Syntactically correct, runnable on first attempt.
decisive for this rankingtransformation
If the generated code is not syntactically correct and runnable, the tool has failed at producing the animation at all. (3 of 3 judges)
What was given, what came back
Test input: Search engines basics explainer · text
Input — what we sent
The exact prompt
Create an animation video explaining how search engines work.
A vague, linear educational animation prompt asking the tool to explain how search engines work with minimal guidance. Designed to stress inference, scene structuring, pacing, and coherent visual flow from an underspecified request.
Output — unretouched
Gemini Canvas — Screen Recording 2026-05-04 120930.mp4
Also checked on this input — same tool, 8 other criteria
Input Handling✓ WorkedThe tool can accept a plain-language animation prompt and infer a complete implementation stack and scene structure without needing the framework to be specified.Input Handling✓ WorkedAccepted a vague animation brief and still selected a complete HTML/CSS/JavaScript implementation without the user specifying a framework, showing it can infer a workable structure from underspecified input.Input Handling✓ WorkedAccepts a minimal search-engine prompt and infers a complete 3-step structure—crawling, indexing, and ranking—without needing the framework spelled out.Input Handling✓ WorkedAccepts a plain-language animation prompt and infers the full search workflow without needing a detailed implementation spec.Output Quality◐ MixedKeeps the animation inside a macOS-style tab frame with a very small aspect ratio, which makes the result feel presentation-like rather than like a full motion-graphics video.Output Quality⚠ StruggledThe animation can be constrained to a small macOS-style tab frame, which makes it read more like a presentation than full-viewport motion graphics.Output Quality◐ MixedDoes not fully realize the requested developer-themed UI, leaving the search animation visually plain.Output Quality✓ WorkedIt can cover the full search-engine workflow—crawling, indexing, and ranking—without overlap or visual clutter.
Provenance
- Observation
- 23138cac-317e-4e2a-a9a8-f9b791df45e4
- Evidence run
- c751a76b-a105-4b3e-9ff3-6c4d6adc9f04
- Study
- Generate Code-based Animations from text inputs
- Research task
- 86b9nnzmy
- Tested at
- Jun 24, 2026
- Source
- first-party
- Evidence state
- observed
- Proof shown
- input + output shown
- Cost / latency
- not captured
- Repeat run
- not captured
- Tester
- not captured
The last three rows are honest blanks, not placeholders — our capture has no field for them yet.
Query this
get_evidence({
tool: "gemini-canvas"
})MCP · mcp.aidemos.com/api/mcp
Free with attribution.
Same input, same check — 5 other tools
measured on Code Generation Quality
AnimG✓ WorkedGenerates syntactically correct Manim code on the first pass.ChatGPT Canvas✓ WorkedGenerated syntactically correct, runnable HTML/CSS/JavaScript on the first attempt for the search-engine explainer.Claude Artifacts✓ WorkedIt generated a runnable webpage on the first attempt, with autoplay working on load.Grok✓ WorkedProduced syntactically correct HTML/CSS/JavaScript code on the first pass.Replit✓ WorkedProduces runnable React animation code on the first attempt.
Real inputs and real outputs, no retouching · every cell queryable via API & MCP · aidemos.com