Can Studyglen turn study material into a quiz covering its core ideas?
Studyglen generated 20-question multiple-choice quizzes from both chapters, but neither met the quality requirements. The English quiz included a contradictory question, a second defensible option and a coverage gap; the Hindi quiz stayed in Hindi but keyed 0.35 A after a new branch was added, although 0.35 A was the total before it.
2 of 2 test cases failed
Every test case in this scenario has a result.
The 2 test cases
Each test case is judged on its own: Pass, Fail, Not gradable, or Untested. The scenario result above is based on these test cases.
Study material to an ordinary multiple-choice quizFailEvidence
We uploaded the ten-page English electric-circuits chapter to Studyglen's quiz builder, selected only Multiple Choice, set the count to 20 and the language to English, and generated the quiz.
The tool's response has no text to quote: it is what the evidence under Proofs shows.
| ✗ | Expected: Multiple-choice questions must be clear, answerable and source-supported, with correct keys, no defensible distractors, and core-idea coverage without omission alongside repetition.Found: Question 9 assigns 1.5 V to an unintroduced component; Question 8 has two defensible answers. Questions 8 and 12 repeat voltage; parallel lamp failure is omitted. |
Producing the requested 20-question format was not enough: the quiz also needed clear questions, unambiguous choices and core-idea coverage. Question 9 asks about an unintroduced component, Question 8 permits another defensible answer, and branch voltage repeats while parallel lamp failure is omitted.
Marked answers versus question quality. The English quiz's examined keys were correct against the chapter. Its defects concern question framing and topic coverage, not the answers marked right.
Same-language quiz from same-language materialFailEvidence
We uploaded the Hindi edition of the electric-circuits chapter, explicitly selected Hindi in the quiz-language dropdown, selected only Multiple Choice, set the maximum to 20 and generated the quiz.
Hindi quiz: Question 1 श्रेणीक्रम और समांतर क्रम में बैटरी के विभवांतर का वितरण किस प्रकार अलग होता है? A दोनों में घटकों के बीच बाँटा जाता है B श्रेणीक्रम में पूरा मिलता है, समांतर क्रम में बाँटा जाता है C श्रेणीक्रम में बाँटा जाता है, समांतर क्रम में पूरा मिलता है Correct answer D दोनों में घटकों को पूरा मिलता है Hindi quiz: Question 4 समांतर क्रम में एक लैम्प की शाखा टूटने पर दूसरी शाखा के लैम्प पर क्या प्रभाव पड़ेगा? A वह पहले से अधिक चमक से जलेगा B वह पहले जैसी चमक से जलता रहेगा Correct answer C वह बहुत मुश्किल से जलेगा D वह भी बुझ जाएगा Hindi quiz: Question 10 and explanation समांतर परिपथ की दो शाखाएँ 0.22 A और 0.13 A धारा लेती हैं। नई शाखा जोड़ने पर बैटरी की कुल धारा के बारे में क्या होगा? A कुल धारा 0.09 A होगी और नई शाखा पर घटेगी B कुल धारा 0.35 A होगी और नई शाखा पर बढ़ेगी Correct answer C कुल धारा 0.13 A होगी और नई शाखा पर रुक जाएगी D कुल धारा 0.22 A होगी और नई शाखा पर समान रहेगी पहले कुल धारा 0.22 A + 0.13 A = 0.35 A है। नई शाखा अपनी धारा लेती है और पहले की शाखाओं की धारा घटाती नहीं, इसलिए कुल धारा बढ़ती है। Hindi quiz: Question 16 समांतर परिपथ की दो शाखाओं में अमीटर क्रमशः 0.24 A और 0.16 A पढ़ते हैं। संधि के पहले कुल धारा कितनी होगी? A 0.08 A B 0.24 A C 0.16 A D 0.40 A Correct answer
Transcribed from Proof 2 · Screenshot
Transcribed from Proof 3 · Screenshot
Transcribed from Proof 4 · Screenshot
Transcribed from Proof 5 · Screenshot
| ✗ | Expected: Quiz content must stay in the specified language, with clear, answerable, supported multiple-choice questions, correct keys, unambiguous alternatives and core-idea coverage without omission alongside repetition.Found: The quiz content is Hindi. Question 10 keys 0.35 A after adding a branch; its explanation identifies 0.35 A as the total before addition and says it increases. |
Hindi content and the 20-question format met the request; the answer key did not. In Question 10, 0.22 A plus 0.13 A gives 0.35 A before an added branch. The key wrongly gives 0.35 A afterwards while saying the total increases.
Constructed numerical example. Question 10 uses 0.22 A, 0.13 A and 0.35 A, figures absent from either chapter. Studyglen constructed this numerical example rather than taking it from the study material.
Core ideas covered despite repetition. Question 4 covers parallel lamp failure and keys that the other lamp keeps its brightness. No core idea is omitted; Questions 10 and 16 repeat branch-current addition.
Configuration and setup
How this tool was set up for the run and what the test needed in place. Each row is a fact from the run's records; a fact the records do not hold is left out, not guessed.
How this scenario is graded
How we decide Pass, Fail and Not gradable. The same rules apply to every tool tested on this scenario.
Each test case gets one result per tool: Pass, Fail or Not gradable. A test case we haven't run yet shows Untested. There are no partial results.
- Pass: the tool did everything the test expected, and nothing it said contradicts the correct answer.
- Fail: at least one expected behaviour clearly didn't happen; the test case's result says which one and quotes the tool.
- Not gradable: our evidence couldn't settle the outcome (for example a record we needed is missing). It is never counted as a fail, and the test case's result says what's missing.
Where this sits in the benchmark
This page is one cell of a larger study: one tool, one scenario. Only this benchmark's frame appears here.
| Level | Name | Scope |
|---|---|---|
| Benchmark | Quiz Generation → | 15 scenarios · 10 tools |
| Capability | Quiz Generation → | |
| Scenario | The user turns study material into a quiz → | |
| Tool | Studyglen → |
Global scenario definition → · Global capability definition → · Studyglen product page →
History of this result
What has happened to this result since it was first published. Runs and grades are never overwritten: a retest or a re-grade publishes a new result and keeps the earlier one readable.
Act on this result
Nothing filed here edits the run or the grade. A challenge opens a review, and a review can produce a new run or a re-grade — which becomes the current result and leaves this one in the history.
You run the same kind of test against your own setup and get the same behaviour.
Agree →Yours behaves differently. Tell us what you got, with a screenshot if you have one.
Disagree →Something here is wrong — a reference value, a transcription, a grade.
Report an issue →Tell us what changed and we schedule a rerun of the failing test case. The old result stays as history.
Vendor notice →The same record is available as structured data through the AI Demos MCP server, with the counts, the coverage and every per-test-case reason carried as fields.
Verify the proof files
These files support this result. Open a file to inspect the original evidence.
File fingerprints (SHA-256)
A fingerprint identifies the exact file used for this result.