For a SaaS-discovery prompt, the tool can misread the product context by choosing a generic classroom/math-teacher template instead of a relevant tech-review persona, signaling weak product mapping.
What was measured
Product understanding
Whether the tool gets the product right in features and positioning, rather than being generic or inaccurate.
decisive for this rankingtransformation
An ad has to describe the product accurately and position it correctly; generic or wrong messaging breaks the output. (3 of 3 judges)
What was given, what came back
Test input: FutureSmart AI – SaaS · mixed · group: ugc-style-video-ads
Input — what we sent
Input, verbatim
Product URL: https://aidemos.com/ Exact script: Hook: Finding the right AI tools shouldn't take hours. Script: I've been using FutureSmart AI to discover and compare AI tools in one place. It helps me find the right tool faster with real use cases, rankings, and detailed comparisons. If you regularly use AI tools for work, it's definitely worth checking out. Output requirements: - Vertical (9:16) - 20–30 seconds - UGC-style AI presenter - Natural AI voice - Captions enabled
A short-form UGC ad prompt for a SaaS product discovery/comparison platform, designed to test whether the tool understands a software product, writes a believable testimonial-style script, and produces a realistic AI presenter video with captions.
Why this input is hard
- · SaaS product understanding
- · Script quality
- · Avatar realism
- · Voice quality
- · Caption quality
- · Overall ad readiness
Output — unretouched
Also checked on this input — same tool, 4 other criteria
Avatar realism✓ WorkedGenerates a convincing photorealistic talking-head avatar, with the report noting natural skin, hair, teeth, expressive gesturing, and no visible deformities or robotic feel.B-roll / product integration✗ FailedThe tool can keep the ad as one unbroken static talking-head shot, with no product screenshot, cutaway, or other supporting footage to integrate the SaaS visually.Caption quality✗ FailedThe tool can export a clip with zero on-screen captions for the full 18.4 seconds, so captioning is not baked into the output.Platform formats◐ MixedThe export format is correct at full-bleed vertical 9:16 (1080×1920), but the clip is only 18.4 seconds long, below the requested 20–30 second range.
Provenance
- Observation
- 0bf6b441-c1cc-457b-b234-6838553eac34
- Evidence run
- 9fdbb84b-4aa0-484d-8f89-5c46608019a5
- Study
- Generate UGC-Style Video Ads With AI Avatars
- Research task
- 86baxegpg
- Tested at
- not recorded
- Source
- first-party
- Evidence state
- verified
- Proof shown
- input + output shown
- Cost / latency
- not captured
- Repeat run
- not captured
- Tester
- not captured
The last three rows are honest blanks, not placeholders — our capture has no field for them yet.
Query this
get_evidence({
tool: "joggai",
scenario: "ugc-style-video-ads"
})MCP · mcp.aidemos.com/api/mcp
Free with attribution.
Same input, same check — 3 other tools
measured on Product understanding
Creatify.ai✓ WorkedFor the SaaS input, the tool stayed on-product by framing FutureSmart AI as a place to discover and compare AI tools with real use cases, rankings, and detailed comparisons.Invideo AI⚠ StruggledThe render never surfaces any product UI, branding, or screenshot for the SaaS, so the product is not visually grounded beyond a generic talking-head testimonial.Vidnoz AI✓ WorkedUnderstands a product-discovery SaaS well enough to frame it around comparing AI tools in one place, real use cases, rankings, and detailed comparisons instead of generic software praise.
This evidence is published in
Real inputs and real outputs, no retouching · every cell queryable via API & MCP · aidemos.com