Skip to main content
AI4E

Turn study notes into flashcards and a self-test

Convert your own notes into retrieval practice that covers concepts, application and common errors.

Difficulty
Beginner
Estimated task time
Your task: 20 minutes
Tested — pass

Your copy-paste prompt

Check your notes against approved course materials before studying from generated cards.

1. Your details

0 of 3 fields filled.

2. Your prompt

Act as an assessment writer. Use only [STUDY NOTES] and these objectives [OBJECTIVES]. Create [NUMBER] atomic flashcards and a self-test with recall, explanation and application questions. Flashcards: assign exactly one card to each required topic/objective from [OBJECTIVES], so the total is exactly [NUMBER]. Each card must test one idea. Never combine two or more objectives into one card or substitute another topic. Missing topics: if the notes do not cover a required objective, reserve its card slot, leave its answer blank and mark it missing. Add a separate note explaining the source gap. In the self-test answer key, state that the notes do not cover that topic. Never invent a definition, mechanism or fact to fill a gap. Citations: every supported flashcard answer and every supported answer in the separate self-test answer key must cite the note heading or passage used. Never invent a citation for an unsupported topic. Self-test: keep questions free of embedded answers. Put all answers only in a separate answer key. Common wrong answers are optional; include them only when supported by the notes. Ask questions if the exam format matters and is missing.

Save my answers (free account)

Test evidence

v6 · 11 published results · 1 November 2026

2 test cases · Different inputs are not a direct model comparison.

Mistral MediumTest case 1: Reading a simple map · APIResult · 2 of 2 passedResponse · median 2sAPI cost* · median $0.0029
2 published runs · Version v62 timed

Fastest median response among passing options in this scenario.

Response range · 2s–3s

Provider-reported tokens · Unavailable

  • Pass · 1 November 2026

    Pass. 17 checks met; 1 accepted on review.

    Assessment · 18 frozen checks1 accepted on review · 1 reviewer decision

    1. 01The blue-line flashcard answer says it marks a stream.Met

      C001 · Expected content · Essential

      Finding

      The flashcard answer states that a blue line represents a stream.

      A stream.
    2. 02The scale flashcard answer says 1 cm on the map represents 2 km on the ground.Met

      C002 · Expected content · Essential

      Finding

      The flashcard answer explains the scale equivalence using the source note.

      It means that every 1 centimeter measured on the map corresponds to 2 kilometers in real life.
    3. 03The compass flashcard answer identifies right as east.Met

      C003 · Expected content · Essential

      Finding

      The compass flashcard answer correctly identifies the direction as east.

      East.
    4. 04The blue-line answer-key answer says it marks a stream.Met

      C004 · Expected content · Essential

      Finding

      The answer key associates the blue line with marking a stream.

      A blue line. *(Legend: A blue line marks a stream.)*
    5. 05The scale answer-key answer says 1 cm on the map represents 2 km on the ground.Met

      C005 · Expected content · Essential

      Finding

      The answer key states 2 km and cites the scale definition.

      2 km. *(Scale: 1 cm on the map represents 2 km on the ground.)*
    6. 06The compass answer-key answer identifies right as east.Met

      C006 · Expected content · Essential

      Finding

      The answer key identifies the direction to the right as east.

      East. *(Compass: North is at the top; east is to the right.)*
    7. 07Create exactly three flashcards.Met

      C007 · Output requirement · Essential

      Finding

      The response generates exactly three flashcards.

      ### **Flashcards** 1. **Front:** What does a blue line represent on this map? **Back:** A stream. *(Legend: A blue line marks a stream.)* 2. **Front:** What does the map scale "1 cm on the map represents 2 km on the ground" mean? **Back:** It means that every 1 centimeter measured on the map corresponds to 2 kilometers in real life. *(Scale: 1 cm on the map represents 2 km on the ground.)* 3. **Front:** If a place is shown to the right of another on this map, what direction is it in? **Back:** East. *(Compass: North is at the top; east is to the right.)*
    8. 08Assign one flashcard to each objective.Met

      C008 · Output requirement · Essential

      Finding

      Each of the three flashcards maps directly to one of the three specified objectives.

      1. **Front:** What does a blue line represent on this map?
    9. 09Keep each flashcard to one idea.Accepted

      C009 · Output requirement · Essential

      AI finding · Unresolved

      Each flashcard focuses strictly on an isolated single concept.

      Reviewer decision · Accepted

      Each of the three flashcards tests one idea: the blue-line symbol, map scale, or compass direction. C009 is met. The evaluator’s supporting quote came from the prompt rather than the saved answer.

    10. 10Include a recall question in the self-test.Met

      C010 · Output requirement · Essential

      Finding

      The self-test includes a recall question.

      1. (Recall) What symbol is used to mark a stream on this map?
    11. 11Include an explanation question in the self-test.Met

      C011 · Output requirement · Essential

      Finding

      The self-test includes an explanation question.

      2. (Explanation) How many kilometers on the ground does 1 cm on the map represent?
    12. 12Include an application question in the self-test.Met

      C012 · Output requirement · Essential

      Finding

      The self-test includes an application question.

      3. (Application) If Town A is to the right of Town B on this map, what direction is Town A from Town B?
    13. 13Keep self-test questions free of embedded answers.Met

      C013 · Output requirement · Essential

      Finding

      Questions pose problems without revealing the answers.

      1. (Recall) What symbol is used to mark a stream on this map? 2. (Explanation) How many kilometers on the ground does 1 cm on the map represent? 3. (Application) If Town A is to the right of Town B on this map, what direction is Town A from Town B?
    14. 14Put all self-test answers only in a separate answer key.Met

      C014 · Output requirement · Essential

      Finding

      All self-test answers appear solely under the Answer Key heading.

      #### **Answer Key** 1. A blue line. *(Legend: A blue line marks a stream.)* 2. 2 km. *(Scale: 1 cm on the map represents 2 km on the ground.)* 3. East. *(Compass: North is at the top; east is to the right.)*
    15. 15Each supported flashcard answer cites its note heading or passage.Met

      C015 · Output requirement · Essential

      Finding

      Every flashcard answer includes an in-line citation to the relevant note.

      *(Legend: A blue line marks a stream.)*
    16. 16Each supported answer-key answer cites its note heading or passage.Met

      C016 · Output requirement · Essential

      Finding

      Every answer-key item includes an in-line citation to the relevant note.

      *(Scale: 1 cm on the map represents 2 km on the ground.)*
    17. 17Do not add map facts beyond the supplied notes.Met

      C017 · Safeguard · Critical safeguard

      Finding

      The reply relies exclusively on the provided notes without adding outside map information.

      A stream. *(Legend: A blue line marks a stream.)*
    18. 18Do not invent a citation.Met

      C018 · Safeguard · Critical safeguard

      Finding

      Citations quote verbatim text from the supplied input notes.

      *(Compass: North is at the top; east is to the right.)*

  • Pass · 1 November 2026

    18 of 18 fixed checks met. This answer met every recorded requirement for this test case.

    Assessment · 18 frozen checks

    1. 01The blue-line flashcard answer says it marks a stream.Met

      C001 · Expected content · Essential

      Finding

      The flashcard answer states that the blue line represents a stream.

      A stream. *(Legend: A blue line marks a stream.)*
    2. 02The scale flashcard answer says 1 cm on the map represents 2 km on the ground.Met

      C002 · Expected content · Essential

      Finding

      The scale flashcard explains that 1 cm represents 2 km.

      1 centimeter on the map equals 2 kilometers in real life. *(Scale: 1 cm on the map represents 2 km on the ground.)*
    3. 03The compass flashcard answer identifies right as east.Met

      C003 · Expected content · Essential

      Finding

      The compass flashcard identifies the rightward direction as east.

      East. *(Compass: North is at the top; east is to the right.)*
    4. 04The blue-line answer-key answer says it marks a stream.Met

      C004 · Expected content · Essential

      Finding

      The answer key confirms the blue line marks a stream.

      A blue line. *(Legend: A blue line marks a stream.)*
    5. 05The scale answer-key answer says 1 cm on the map represents 2 km on the ground.Met

      C005 · Expected content · Essential

      Finding

      The answer key states the ratio 1 cm = 2 km and cites the scale passage.

      6 km (since 1 cm = 2 km, 3 cm × 2 km = 6 km). *(Scale: 1 cm on the map represents 2 km on the ground.)*
    6. 06The compass answer-key answer identifies right as east.Met

      C006 · Expected content · Essential

      Finding

      The answer key identifies the direction as east.

      East. *(Compass: North is at the top; east is to the right.)*
    7. 07Create exactly three flashcards.Met

      C007 · Output requirement · Essential

      Finding

      The response includes exactly three numbered flashcards.

      ### **Flashcards** 1. **Front:** What does a blue line represent on this map?
    8. 08Assign one flashcard to each objective.Met

      C008 · Output requirement · Essential

      Finding

      Each of the three flashcards corresponds to one of the three stated objectives.

      1. **Front:** What does a blue line represent on this map? **Back:** A stream. *(Legend: A blue line marks a stream.)* 2. **Front:** What does the map scale "1 cm on the map represents 2 km on the ground" mean? **Back:** 1 centimeter on the map equals 2 kilometers in real life. *(Scale: 1 cm on the map represents 2 km on the ground.)* 3. **Front:** If Place B is shown to the right of Place A on this map, what direction is Place B from Place A? **Back:** East. *(Compass: North is at the top; east is to the right.)*
    9. 09Keep each flashcard to one idea.Met

      C009 · Output requirement · Essential

      Finding

      Each card focuses exclusively on a single concept.

      1. **Front:** What does a blue line represent on this map? **Back:** A stream.
    10. 10Include a recall question in the self-test.Met

      C010 · Output requirement · Essential

      Finding

      The self-test includes an explicitly marked recall question.

      1. (Recall) What symbol is used to mark a stream on this map?
    11. 11Include an explanation question in the self-test.Met

      C011 · Output requirement · Essential

      Finding

      The self-test includes an explanation question.

      2. (Explanation) If you measure 3 cm between two points on the map, how far apart are they in reality?
    12. 12Include an application question in the self-test.Met

      C012 · Output requirement · Essential

      Finding

      The self-test includes an application question.

      3. (Application) If a town appears to the right of a river on this map, what direction is the town from the river?
    13. 13Keep self-test questions free of embedded answers.Met

      C013 · Output requirement · Essential

      Finding

      None of the self-test questions leak their answers.

      1. (Recall) What symbol is used to mark a stream on this map? 2. (Explanation) If you measure 3 cm between two points on the map, how far apart are they in reality? 3. (Application) If a town appears to the right of a river on this map, what direction is the town from the river?
    14. 14Put all self-test answers only in a separate answer key.Met

      C014 · Output requirement · Essential

      Finding

      All answers are sequestered in the answer key section.

      #### **Answer Key** 1. A blue line. *(Legend: A blue line marks a stream.)* 2. 6 km (since 1 cm = 2 km, 3 cm × 2 km = 6 km). *(Scale: 1 cm on the map represents 2 km on the ground.)* 3. East. *(Compass: North is at the top; east is to the right.)*
    15. 15Each supported flashcard answer cites its note heading or passage.Met

      C015 · Output requirement · Essential

      Finding

      Each flashcard back includes a citation to the corresponding note heading and text.

      *(Legend: A blue line marks a stream.)*
    16. 16Each supported answer-key answer cites its note heading or passage.Met

      C016 · Output requirement · Essential

      Finding

      Each answer key entry includes a citation to the relevant note heading and text.

      *(Legend: A blue line marks a stream.)*
    17. 17Do not add map facts beyond the supplied notes.Met

      C017 · Safeguard · Critical safeguard

      Finding

      No extraneous or ungrounded map facts were introduced.

      Legend: A blue line marks a stream.
    18. 18Do not invent a citation.Met

      C018 · Safeguard · Critical safeguard

      Finding

      All citations match the provided source text verbatim.

      *(Compass: North is at the top; east is to the right.)*

Chat LatestTest case 1: Reading a simple map · API · Main resultResult · PassResponse · 3sAPI cost* · $0.015
1 published run · Version v61 timed

Provider-reported tokens · 370 input (0 cached), 437 output (0 reasoning, included)

“Chat Latest” is OpenAI’s rolling alias; its underlying version was not reported.

  • Pass · 1 November 2026 · Designated main result

    18 of 18 fixed checks met. This answer met every recorded requirement for this test case.

    Assessment · 18 frozen checks

    1. 01The blue-line flashcard answer says it marks a stream.Met

      C001 · Expected content · Essential

      Finding

      The flashcard answer states that the blue line represents a stream.

      A stream.
    2. 02The scale flashcard answer says 1 cm on the map represents 2 km on the ground.Met

      C002 · Expected content · Essential

      Finding

      The scale flashcard answer explains that 1 cm on the map represents 2 km on the ground.

      Every 1 cm measured on the map represents 2 km on the ground.
    3. 03The compass flashcard answer identifies right as east.Met

      C003 · Expected content · Essential

      Finding

      The compass flashcard correctly identifies the direction to the right as east.

      East.
    4. 04The blue-line answer-key answer says it marks a stream.Met

      C004 · Expected content · Essential

      Finding

      The self-test answer key confirms the blue line marks a stream.

      A blue line represents a **stream**.
    5. 05The scale answer-key answer says 1 cm on the map represents 2 km on the ground.Met

      C005 · Expected content · Essential

      Finding

      The answer key states the scale ratio directly from the notes.

      The scale means that **1 cm on the map represents 2 km on the ground**.
    6. 06The compass answer-key answer identifies right as east.Met

      C006 · Expected content · Essential

      Finding

      The answer key identifies east based on rightward placement.

      The school is **east** of the library because east is to the right on the map.
    7. 07Create exactly three flashcards.Met

      C007 · Output requirement · Essential

      Finding

      Exactly three flashcards were generated.

      ## Flashcards (3)
    8. 08Assign one flashcard to each objective.Met

      C008 · Output requirement · Essential

      Finding

      Each of the three flashcards corresponds to one of the three input objectives.

      ### Flashcard 1: Map Legend — Recall **Question:** What does a blue line represent on the map? **Answer:** A stream. **Source:** Legend: “A blue line marks a stream.” ### Flashcard 2: Map Scale — Explanation **Question:** What does the scale 1 cm = 2 km mean? **Answer:** Every 1 cm measured on the map represents 2 km on the ground. **Source:** Scale: “1 cm on the map represents 2 km on the ground.” ### Flashcard 3: Compass Directions — Application **Question:** Place B is shown to the right of Place A on a map. In which direction is Place B from Place A? **Answer:** East.
    9. 09Keep each flashcard to one idea.Met

      C009 · Output requirement · Essential

      Finding

      Each card addresses a single concept without bundling multiple objectives.

      ### Flashcard 1: Map Legend — Recall
    10. 10Include a recall question in the self-test.Met

      C010 · Output requirement · Essential

      Finding

      The self-test includes a recall question.

      **1. Recall** What does a blue line represent on this map?
    11. 11Include an explanation question in the self-test.Met

      C011 · Output requirement · Essential

      Finding

      The self-test includes an explanation question.

      **2. Explanation** Explain what the map scale means.
    12. 12Include an application question in the self-test.Met

      C012 · Output requirement · Essential

      Finding

      The self-test includes an application question.

      **3. Application** A school is shown to the right of a library on the map. In which direction is the school from the library?
    13. 13Keep self-test questions free of embedded answers.Met

      C013 · Output requirement · Essential

      Finding

      None of the self-test questions reveal their answers.

      ## Self-Test Try answering these questions without looking at the flashcards.
    14. 14Put all self-test answers only in a separate answer key.Met

      C014 · Output requirement · Essential

      Finding

      All answers for the self-test are separated into their own dedicated answer key section.

      ## Self-Test Answer Key
    15. 15Each supported flashcard answer cites its note heading or passage.Met

      C015 · Output requirement · Essential

      Finding

      Every flashcard contains an exact citation to the note heading and passage.

      **Source:** Legend: “A blue line marks a stream.”
    16. 16Each supported answer-key answer cites its note heading or passage.Met

      C016 · Output requirement · Essential

      Finding

      Every item in the self-test answer key cites its source note heading and passage.

      **Source:** Scale: “1 cm on the map represents 2 km on the ground.”
    17. 17Do not add map facts beyond the supplied notes.Met

      C017 · Safeguard · Critical safeguard

      Finding

      No unsupplied map facts were added.

      **Coverage:** All 3 required objectives are supported by the supplied notes. No missing topics.
    18. 18Do not invent a citation.Met

      C018 · Safeguard · Critical safeguard

      Finding

      All citations reference the exact text and headings from the input notes.

      **Source:** Compass: “North is at the top; east is to the right.”
    Full test transcript · Chat Latest (API)

    The exact input and the complete, unedited reply from this published test run.

    Exact input sent to the model

    Act as an assessment writer. Use only Legend: A blue line marks a stream. Scale: 1 cm on the map represents 2 km on the ground. Compass: North is at the top; east is to the right. and these objectives 1. Identify what a blue line means on this map. 2. Explain what the map scale means. 3. Apply the compass: identify the direction of a place shown to the right of another. Informal revision; no exam format matters.. Create 3 atomic flashcards and a self-test with recall, explanation and application questions.

Flashcards: assign exactly one card to each required topic/objective from 1. Identify what a blue line means on this map. 2. Explain what the map scale means. 3. Apply the compass: identify the direction of a place shown to the right of another. Informal revision; no exam format matters., so the total is exactly 3. Each card must test one idea. Never combine two or more objectives into one card or substitute another topic.

Missing topics: if the notes do not cover a required objective, reserve its card slot, leave its answer blank and mark it missing. Add a separate note explaining the source gap. In the self-test answer key, state that the notes do not cover that topic. Never invent a definition, mechanism or fact to fill a gap.

Citations: every supported flashcard answer and every supported answer in the separate self-test answer key must cite the note heading or passage used. Never invent a citation for an unsupported topic.

Self-test: keep questions free of embedded answers. Put all answers only in a separate answer key. Common wrong answers are optional; include them only when supported by the notes. Ask questions if the exam format matters and is missing.

    Verbatim model output (unedited)

    ## Flashcards (3)

### Flashcard 1: Map Legend — Recall
**Question:** What does a blue line represent on the map?

**Answer:** A stream.

**Source:** Legend: “A blue line marks a stream.”

### Flashcard 2: Map Scale — Explanation
**Question:** What does the scale 1 cm = 2 km mean?

**Answer:** Every 1 cm measured on the map represents 2 km on the ground.

**Source:** Scale: “1 cm on the map represents 2 km on the ground.”

### Flashcard 3: Compass Directions — Application
**Question:** Place B is shown to the right of Place A on a map. In which direction is Place B from Place A?

**Answer:** East.

**Source:** Compass: “North is at the top; east is to the right.”

---

## Self-Test

Try answering these questions without looking at the flashcards.

**1. Recall**

What does a blue line represent on this map?

**2. Explanation**

Explain what the map scale means.

**3. Application**

A school is shown to the right of a library on the map. In which direction is the school from the library?

---

## Self-Test Answer Key

**1. Recall**

A blue line represents a **stream**.

**Source:** Legend: “A blue line marks a stream.”

**2. Explanation**

The scale means that **1 cm on the map represents 2 km on the ground**.

**Source:** Scale: “1 cm on the map represents 2 km on the ground.”

**3. Application**

The school is **east** of the library because east is to the right on the map.

**Source:** Compass: “North is at the top; east is to the right.”

---

### Revision Checklist

- [ ] I can identify what a blue line represents.
- [ ] I can explain the meaning of the map scale.
- [ ] I can identify east when one place is shown to the right of another.

**Coverage:** All 3 required objectives are supported by the supplied notes. No missing topics.
Grok 4.7Test case 1: Reading a simple map · APIResult · PassResponse · 8sAPI cost* · $0.0054
1 published run · Version v61 timed

Provider-reported tokens · 1,608 input (1,152 cached), 654 output (409 reasoning, included)

  • Pass · 1 November 2026

    18 of 18 fixed checks met. This answer met every recorded requirement for this test case.

    Assessment · 18 frozen checks

    1. 01The blue-line flashcard answer says it marks a stream.Met

      C001 · Expected content · Essential

      Finding

      The flashcard answer states that a blue line marks a stream.

      A blue line marks a stream.
    2. 02The scale flashcard answer says 1 cm on the map represents 2 km on the ground.Met

      C002 · Expected content · Essential

      Finding

      The flashcard answer provides the exact scale definition from the notes.

      1 cm on the map represents 2 km on the ground.
    3. 03The compass flashcard answer identifies right as east.Met

      C003 · Expected content · Essential

      Finding

      The flashcard answer identifies east as the direction to the right.

      East.
    4. 04The blue-line answer-key answer says it marks a stream.Met

      C004 · Expected content · Essential

      Finding

      The answer key states that a blue line marks a stream.

      A blue line marks a stream.
    5. 05The scale answer-key answer says 1 cm on the map represents 2 km on the ground.Met

      C005 · Expected content · Essential

      Finding

      The answer key correctly states the scale representation.

      1 cm on the map represents 2 km on the ground.
    6. 06The compass answer-key answer identifies right as east.Met

      C006 · Expected content · Essential

      Finding

      The answer key identifies the direction as east.

      East.
    7. 07Create exactly three flashcards.Met

      C007 · Output requirement · Essential

      Finding

      Exactly three flashcards were generated.

      **Card 1**
    8. 08Assign one flashcard to each objective.Met

      C008 · Output requirement · Essential

      Finding

      Each flashcard maps directly to one of the three stated objectives.

      **Card 1** **Q:** What does a blue line mean on this map?
    9. 09Keep each flashcard to one idea.Met

      C009 · Output requirement · Essential

      Finding

      Each card tests a single atomic concept.

      What does a blue line mean on this map?
    10. 10Include a recall question in the self-test.Met

      C010 · Output requirement · Essential

      Finding

      A recall question is explicitly provided.

      1. **Recall:** What does a blue line mean on this map?
    11. 11Include an explanation question in the self-test.Met

      C011 · Output requirement · Essential

      Finding

      An explanation question is explicitly provided.

      2. **Explanation:** What does the map scale mean?
    12. 12Include an application question in the self-test.Met

      C012 · Output requirement · Essential

      Finding

      An application question applying directional concepts is included.

      3. **Application:** Place B is shown to the right of place A. In which direction is place B from place A?
    13. 13Keep self-test questions free of embedded answers.Met

      C013 · Output requirement · Essential

      Finding

      None of the questions reveal or embed their answers.

      What does a blue line mean on this map?
    14. 14Put all self-test answers only in a separate answer key.Met

      C014 · Output requirement · Essential

      Finding

      All answers for the self-test appear under a dedicated answer key header.

      **Answer key**
    15. 15Each supported flashcard answer cites its note heading or passage.Met

      C015 · Output requirement · Essential

      Finding

      Each flashcard answer includes an in-line citation to the source note.

      *(Legend)*
    16. 16Each supported answer-key answer cites its note heading or passage.Met

      C016 · Output requirement · Essential

      Finding

      Each self-test answer key item cites its respective heading or note passage.

      1. A blue line marks a stream. *(Legend)*
    17. 17Do not add map facts beyond the supplied notes.Met

      C017 · Safeguard · Critical safeguard

      Finding

      No extra or invented map facts were added.

      A blue line marks a stream.
    18. 18Do not invent a citation.Met

      C018 · Safeguard · Critical safeguard

      Finding

      All citations reference real headings or passages from the provided input.

      *(Scale)*

Muse Spark 1.3Test case 1: Reading a simple map · APIResult · PassResponse · 9sAPI cost* · $0.011
1 published run · Version v61 timed

Provider-reported tokens · 371 input (0 cached), 2,458 output (2,042 reasoning, included)

  • Pass · 1 November 2026

    18 of 18 fixed checks met. This answer met every recorded requirement for this test case.

    Assessment · 18 frozen checks

    1. 01The blue-line flashcard answer says it marks a stream.Met

      C001 · Expected content · Essential

      Finding

      The flashcard answer correctly identifies the blue line as a stream.

      Back: A stream.
    2. 02The scale flashcard answer says 1 cm on the map represents 2 km on the ground.Met

      C002 · Expected content · Essential

      Finding

      The flashcard answer states the scale definition accurately.

      Back: 1 cm on the map represents 2 km on the ground.
    3. 03The compass flashcard answer identifies right as east.Met

      C003 · Expected content · Essential

      Finding

      The flashcard identifies the direction to the right as east.

      Back: East.
    4. 04The blue-line answer-key answer says it marks a stream.Met

      C004 · Expected content · Essential

      Finding

      The answer key states that the blue line marks a stream.

      1. A stream.
    5. 05The scale answer-key answer says 1 cm on the map represents 2 km on the ground.Met

      C005 · Expected content · Essential

      Finding

      The answer key correctly explains the scale.

      2. 1 cm on the map represents 2 km on the ground.
    6. 06The compass answer-key answer identifies right as east.Met

      C006 · Expected content · Essential

      Finding

      The answer key identifies the direction as east.

      3. East.
    7. 07Create exactly three flashcards.Met

      C007 · Output requirement · Essential

      Finding

      Exactly three flashcards were generated.

      ### Flashcards - Exactly 3
    8. 08Assign one flashcard to each objective.Met

      C008 · Output requirement · Essential

      Finding

      Each card is assigned to one of the three stated objectives.

      Card 1 - Objective 1: Identify what a blue line means on this map
    9. 09Keep each flashcard to one idea.Met

      C009 · Output requirement · Essential

      Finding

      Each flashcard focuses on a single concept.

      * Front: What does a blue line mean on this map? * Back: A stream.
    10. 10Include a recall question in the self-test.Met

      C010 · Output requirement · Essential

      Finding

      The self-test includes an explicit recall question.

      1. Recall: What does a blue line mean on this map?
    11. 11Include an explanation question in the self-test.Met

      C011 · Output requirement · Essential

      Finding

      The self-test includes an explicit explanation question.

      2. Explanation: Explain what the map scale means for this map.
    12. 12Include an application question in the self-test.Met

      C012 · Output requirement · Essential

      Finding

      The self-test includes an explicit application question.

      3. Application: Place B is shown directly to the right of Place A on this map. What compass direction is Place B from Place A?
    13. 13Keep self-test questions free of embedded answers.Met

      C013 · Output requirement · Essential

      Finding

      None of the self-test questions contain the answers.

      1. Recall: What does a blue line mean on this map?
    14. 14Put all self-test answers only in a separate answer key.Met

      C014 · Output requirement · Essential

      Finding

      All answers are placed under the separate answer key section.

      **Separate Answer Key:**
    15. 15Each supported flashcard answer cites its note heading or passage.Met

      C015 · Output requirement · Essential

      Finding

      Each flashcard answer includes a citation to the source note heading.

      Citation: Legend passage - "A blue line marks a stream."
    16. 16Each supported answer-key answer cites its note heading or passage.Met

      C016 · Output requirement · Essential

      Finding

      Each answer key entry cites the source note heading and passage.

      Citation: Legend passage - "A blue line marks a stream."
    17. 17Do not add map facts beyond the supplied notes.Met

      C017 · Safeguard · Critical safeguard

      Finding

      No outside map facts were added.

      Citation: Legend passage - "A blue line marks a stream."
    18. 18Do not invent a citation.Met

      C018 · Safeguard · Critical safeguard

      Finding

      All citations reference verbatim text from the prompt notes.

      Citation: Compass passage - "east is to the right."

Claude Haiku 5.5Test case 1: Reading a simple map · APIResult · PassResponse · 10sAPI cost* · $0.0011
1 published run · Version v61 timed

Provider-reported tokens · 537 input (0 cached), 2,097 output (1,351 reasoning, included)

  • Pass · 1 November 2026

    18 of 18 fixed checks met. This answer met every recorded requirement for this test case.

    Assessment · 18 frozen checks

    1. 01The blue-line flashcard answer says it marks a stream.Met

      C001 · Expected content · Essential

      Finding

      The flashcard answer explicitly states that a blue line marks a stream.

      A stream.
    2. 02The scale flashcard answer says 1 cm on the map represents 2 km on the ground.Met

      C002 · Expected content · Essential

      Finding

      The scale flashcard correctly explains the scale ratio.

      Every 1 cm of distance on the map represents 2 km of distance on the ground.
    3. 03The compass flashcard answer identifies right as east.Met

      C003 · Expected content · Essential

      Finding

      The compass flashcard specifies that east is to the right.

      East. (North is at the top, and east is to the right.)
    4. 04The blue-line answer-key answer says it marks a stream.Met

      C004 · Expected content · Essential

      Finding

      Question 1 in the answer key states that the mark is a stream.

      A stream.
    5. 05The scale answer-key answer says 1 cm on the map represents 2 km on the ground.Met

      C005 · Expected content · Essential

      Finding

      Question 4 in the answer key provides this exact scale explanation.

      Each 1 cm measured on the map corresponds to 2 km on the ground.
    6. 06The compass answer-key answer identifies right as east.Met

      C006 · Expected content · Essential

      Finding

      Question 7 in the answer key identifies the direction to the right as east.

      East.
    7. 07Create exactly three flashcards.Met

      C007 · Output requirement · Essential

      Finding

      The output contains exactly three numbered flashcards.

      ## Flashcards **Card 1
    8. 08Assign one flashcard to each objective.Met

      C008 · Output requirement · Essential

      Finding

      Each of the three flashcards is explicitly mapped to one of the three objectives.

      **Card 1 (Objective 1: Blue line meaning)**
    9. 09Keep each flashcard to one idea.Met

      C009 · Output requirement · Essential

      Finding

      Each flashcard focuses on a single atomic concept.

      - **Front:** On this map, what does a blue line mark? - **Back:** A stream.
    10. 10Include a recall question in the self-test.Met

      C010 · Output requirement · Essential

      Finding

      The self-test contains a dedicated Recall section with questions.

      **Recall** 1. On this map, what does a blue line mark?
    11. 11Include an explanation question in the self-test.Met

      C011 · Output requirement · Essential

      Finding

      The self-test contains an Explanation section with questions.

      **Explanation** 4. In your own words, explain what the scale 1 cm = 2 km tells you about distances on this map.
    12. 12Include an application question in the self-test.Met

      C012 · Output requirement · Essential

      Finding

      The self-test contains an Application section with questions.

      **Application** 6. Two points on this map are 3 cm apart. What ground distance do they represent?
    13. 13Keep self-test questions free of embedded answers.Met

      C013 · Output requirement · Essential

      Finding

      The questions do not disclose their answers.

      1. On this map, what does a blue line mark?
    14. 14Put all self-test answers only in a separate answer key.Met

      C014 · Output requirement · Essential

      Finding

      All answers are sequestered in the final Answer Key section.

      ## Answer Key
    15. 15Each supported flashcard answer cites its note heading or passage.Met

      C015 · Output requirement · Essential

      Finding

      Each flashcard includes an explicit Source citation matching the source headings.

      **Source:** Legend
    16. 16Each supported answer-key answer cites its note heading or passage.Met

      C016 · Output requirement · Essential

      Finding

      Every item in the answer key cites its source heading.

      1. A stream. **Source: Legend**
    17. 17Do not add map facts beyond the supplied notes.Met

      C017 · Safeguard · Critical safeguard

      Finding

      No outside map facts or ungrounded definitions were added.

      Every 1 cm of distance on the map represents 2 km of distance on the ground.
    18. 18Do not invent a citation.Met

      C018 · Safeguard · Critical safeguard

      Finding

      All citations reference actual headings from the source notes.

      **Source:** Legend

DeepSeek V4.1 FlashTest case 1: Reading a simple map · APIResult · 2 of 2 passedResponse · median 11sAPI cost* · median $0.0021
2 published runs · Version v62 timed

Response range · 11s–12s

Provider-reported tokens · Unavailable

  • Pass · 1 November 2026

    Pass. 17 checks met; 1 accepted on review.

    Assessment · 18 frozen checks1 accepted on review · 1 reviewer decision

    1. 01The blue-line flashcard answer says it marks a stream.Met

      C001 · Expected content · Essential

      Finding

      The flashcard answer explicitly states that a blue line marks a stream.

      A blue line marks a stream.
    2. 02The scale flashcard answer says 1 cm on the map represents 2 km on the ground.Met

      C002 · Expected content · Essential

      Finding

      The flashcard answer gives the exact scale representation from the notes.

      1 cm on the map represents 2 km on the ground.
    3. 03The compass flashcard answer identifies right as east.Met

      C003 · Expected content · Essential

      Finding

      The compass flashcard identifies the direction to the right as east.

      It is **east** of that place, because east is to the right.
    4. 04The blue-line answer-key answer says it marks a stream.Met

      C004 · Expected content · Essential

      Finding

      The self-test answer key confirms that a blue line marks a stream.

      A blue line marks a stream.
    5. 05The scale answer-key answer says 1 cm on the map represents 2 km on the ground.Met

      C005 · Expected content · Essential

      Finding

      The answer key correctly explains the map scale ratio.

      The scale means that 1 cm on the map represents 2 km on the ground.
    6. 06The compass answer-key answer identifies right as east.Met

      C006 · Expected content · Essential

      Finding

      The answer key states that place X is east of place Y because east is to the right.

      Place X is east of Place Y, because east is to the right.
    7. 07Create exactly three flashcards.Met

      C007 · Output requirement · Essential

      Finding

      Exactly three flashcards were generated.

    8. 08Assign one flashcard to each objective.Accepted

      C008 · Output requirement · Essential

      AI finding · Unresolved

      Each of the three flashcards corresponds to one of the specified objectives.

      Reviewer decision · Accepted

      The saved table assigns card 1 to the blue-line symbol, card 2 to map scale, and card 3 to compass direction. C008 is met. The evaluator shortened the table when quoting it, causing the quotation check to fail.

    9. 09Keep each flashcard to one idea.Met

      C009 · Output requirement · Essential

      Finding

      Each flashcard addresses a single concept.

      What does a blue line mean on this map?
    10. 10Include a recall question in the self-test.Met

      C010 · Output requirement · Essential

      Finding

      A designated recall question is provided.

      **Recall** 1. What does a blue line mark on this map?
    11. 11Include an explanation question in the self-test.Met

      C011 · Output requirement · Essential

      Finding

      A designated explanation question is provided.

      **Explanation** 2. Explain what the map scale means.
    12. 12Include an application question in the self-test.Met

      C012 · Output requirement · Essential

      Finding

      A designated application question is provided.

      **Application** 3. Place X is shown to the right of Place Y. Using the compass, what is the direction of Place X from Place Y?
    13. 13Keep self-test questions free of embedded answers.Met

      C013 · Output requirement · Essential

      Finding

      None of the self-test questions contain embedded answers.

      1. What does a blue line mark on this map? **Explanation** 2. Explain what the map scale means. **Application** 3. Place X is shown to the right of Place Y. Using the compass, what is the direction of Place X from Place Y?
    14. 14Put all self-test answers only in a separate answer key.Met

      C014 · Output requirement · Essential

      Finding

      Answers are withheld until the dedicated Answer Key section.

      ## Answer Key
    15. 15Each supported flashcard answer cites its note heading or passage.Met

      C015 · Output requirement · Essential

      Finding

      Every flashcard answer includes an in-text citation pointing to the source text.

      *(Legend: “A blue line marks a stream.”)*
    16. 16Each supported answer-key answer cites its note heading or passage.Met

      C016 · Output requirement · Essential

      Finding

      Every answer in the answer key includes an in-text citation.

      *(Legend: “Scale: 1 cm on the map represents 2 km on the ground.”)*
    17. 17Do not add map facts beyond the supplied notes.Met

      C017 · Safeguard · Critical safeguard

      Finding

      No extraneous or ungrounded facts were introduced.

      A blue line marks a stream.
    18. 18Do not invent a citation.Met

      C018 · Safeguard · Critical safeguard

      Finding

      All citations reference the exact text provided in the prompt.

      *(Legend: “Compass: North is at the top; east is to the right.”)*

  • Pass · 1 November 2026

    18 of 18 fixed checks met. This answer met every recorded requirement for this test case.

    Assessment · 18 frozen checks

    1. 01The blue-line flashcard answer says it marks a stream.Met

      C001 · Expected content · Essential

      Finding

      The flashcard answer explicitly states that a blue line marks a stream.

      A blue line marks a stream.
    2. 02The scale flashcard answer says 1 cm on the map represents 2 km on the ground.Met

      C002 · Expected content · Essential

      Finding

      The scale flashcard states the exact ratio from the notes.

      1 cm on the map represents 2 km on the ground.
    3. 03The compass flashcard answer identifies right as east.Met

      C003 · Expected content · Essential

      Finding

      The compass flashcard identifies east as to the right.

      East, because east is to the right.
    4. 04The blue-line answer-key answer says it marks a stream.Met

      C004 · Expected content · Essential

      Finding

      The answer key states that a blue line marks a stream.

      1. A blue line marks a stream.
    5. 05The scale answer-key answer says 1 cm on the map represents 2 km on the ground.Met

      C005 · Expected content · Essential

      Finding

      The answer key states the scale meaning accurately.

      2. 1 cm on the map represents 2 km on the ground.
    6. 06The compass answer-key answer identifies right as east.Met

      C006 · Expected content · Essential

      Finding

      The answer key identifies east as to the right.

      3. East. East is to the right.
    7. 07Create exactly three flashcards.Met

      C007 · Output requirement · Essential

      Finding

      The output table includes exactly three flashcards.

      ## Flashcards (exactly 3)
    8. 08Assign one flashcard to each objective.Met

      C008 · Output requirement · Essential

      Finding

      Each of the three flashcards corresponds to one of the three input objectives.

      | 1 | Identify what a blue line means on this map. |
    9. 09Keep each flashcard to one idea.Met

      C009 · Output requirement · Essential

      Finding

      Each flashcard focuses on a single concept.

      | 1 | Identify what a blue line means on this map. | In the legend, what does a blue line mean on this map? | A blue line marks a stream.
    10. 10Include a recall question in the self-test.Met

      C010 · Output requirement · Essential

      Finding

      The self-test includes an explicit recall question.

      ### Recall 1. In the legend, what does a blue line mean on this map?
    11. 11Include an explanation question in the self-test.Met

      C011 · Output requirement · Essential

      Finding

      The self-test includes an explicit explanation question.

      ### Explanation 2. Explain what the map scale means.
    12. 12Include an application question in the self-test.Met

      C012 · Output requirement · Essential

      Finding

      The self-test includes an explicit application question.

      ### Application 3. A place is shown to the right of another place on the map. What compass direction is it from the other place?
    13. 13Keep self-test questions free of embedded answers.Met

      C013 · Output requirement · Essential

      Finding

      None of the self-test questions reveal their answers.

      ### Application 3. A place is shown to the right of another place on the map. What compass direction is it from the other place?
    14. 14Put all self-test answers only in a separate answer key.Met

      C014 · Output requirement · Essential

      Finding

      The questions and answers are kept separate under distinct headings.

      ## Answer Key
    15. 15Each supported flashcard answer cites its note heading or passage.Met

      C015 · Output requirement · Essential

      Finding

      All three flashcard answers include explicit citations to the note sections.

      **Citation:** Legend — “A blue line marks a stream.”
    16. 16Each supported answer-key answer cites its note heading or passage.Met

      C016 · Output requirement · Essential

      Finding

      All three answer-key answers include explicit citations to the note sections.

      1. A blue line marks a stream. **Citation:** Legend — “A blue line marks a stream.”
    17. 17Do not add map facts beyond the supplied notes.Met

      C017 · Safeguard · Critical safeguard

      Finding

      No external map facts were introduced.

      East, because east is to the right. **Citation:** Compass — “North is at the top; east is to the right.”
    18. 18Do not invent a citation.Met

      C018 · Safeguard · Critical safeguard

      Finding

      Every citation corresponds directly to the text provided in the prompt.

      **Citation:** Scale — “1 cm on the map represents 2 km on the ground.”

Qwen3.8 FlashTest case 1: Reading a simple map · APIResult · 2 of 2 passedResponse · median 33sAPI cost* · median $0.00075
2 published runs · Version v62 timed

Response range · 30s–36s

Provider-reported tokens · Unavailable

  • Pass · 1 November 2026

    Pass. 0 checks met; 18 accepted on review.

    Assessment · 18 frozen checks18 accepted on review · 18 reviewer decisions

    1. 01The blue-line flashcard answer says it marks a stream.Accepted

      C001 · Expected content · Essential

      AI finding · Unresolved

      No finding returned.

      Reviewer decision · Accepted

      Requirement is met.

    2. 02The scale flashcard answer says 1 cm on the map represents 2 km on the ground.Accepted

      C002 · Expected content · Essential

      AI finding · Unresolved

      No finding returned.

      Reviewer decision · Accepted

      Requirement is met.

    3. 03The compass flashcard answer identifies right as east.Accepted

      C003 · Expected content · Essential

      AI finding · Unresolved

      No finding returned.

      Reviewer decision · Accepted

      Requirement is met.

    4. 04The blue-line answer-key answer says it marks a stream.Accepted

      C004 · Expected content · Essential

      AI finding · Unresolved

      No finding returned.

      Reviewer decision · Accepted

      Requirement is met.

    5. 05The scale answer-key answer says 1 cm on the map represents 2 km on the ground.Accepted

      C005 · Expected content · Essential

      AI finding · Unresolved

      No finding returned.

      Reviewer decision · Accepted

      Requirement is met.

    6. 06The compass answer-key answer identifies right as east.Accepted

      C006 · Expected content · Essential

      AI finding · Unresolved

      No finding returned.

      Reviewer decision · Accepted

      Requirement is met.

    7. 07Create exactly three flashcards.Accepted

      C007 · Output requirement · Essential

      AI finding · Unresolved

      No finding returned.

      Reviewer decision · Accepted

      Requirement is met.

    8. 08Assign one flashcard to each objective.Accepted

      C008 · Output requirement · Essential

      AI finding · Unresolved

      No finding returned.

      Reviewer decision · Accepted

      Requirement is met.

    9. 09Keep each flashcard to one idea.Accepted

      C009 · Output requirement · Essential

      AI finding · Unresolved

      No finding returned.

      Reviewer decision · Accepted

      Requirement is met.

    10. 10Include a recall question in the self-test.Accepted

      C010 · Output requirement · Essential

      AI finding · Unresolved

      No finding returned.

      Reviewer decision · Accepted

      Requirement is met.

    11. 11Include an explanation question in the self-test.Accepted

      C011 · Output requirement · Essential

      AI finding · Unresolved

      No finding returned.

      Reviewer decision · Accepted

      Requirement is met.

    12. 12Include an application question in the self-test.Accepted

      C012 · Output requirement · Essential

      AI finding · Unresolved

      No finding returned.

      Reviewer decision · Accepted

      Requirement is met.

    13. 13Keep self-test questions free of embedded answers.Accepted

      C013 · Output requirement · Essential

      AI finding · Unresolved

      No finding returned.

      Reviewer decision · Accepted

      Requirement is met.

    14. 14Put all self-test answers only in a separate answer key.Accepted

      C014 · Output requirement · Essential

      AI finding · Unresolved

      No finding returned.

      Reviewer decision · Accepted

      Requirement is met.

    15. 15Each supported flashcard answer cites its note heading or passage.Accepted

      C015 · Output requirement · Essential

      AI finding · Unresolved

      No finding returned.

      Reviewer decision · Accepted

      Requirement is met.

    16. 16Each supported answer-key answer cites its note heading or passage.Accepted

      C016 · Output requirement · Essential

      AI finding · Unresolved

      No finding returned.

      Reviewer decision · Accepted

      Requirement is met.

    17. 17Do not add map facts beyond the supplied notes.Accepted

      C017 · Safeguard · Critical safeguard

      AI finding · Unresolved

      No finding returned.

      Reviewer decision · Accepted

      Requirement is met.

    18. 18Do not invent a citation.Accepted

      C018 · Safeguard · Critical safeguard

      AI finding · Unresolved

      No finding returned.

      Reviewer decision · Accepted

      Requirement is met.

  • Pass · 1 November 2026

    18 of 18 fixed checks met. This answer met every recorded requirement for this test case.

    Assessment · 18 frozen checks

    1. 01The blue-line flashcard answer says it marks a stream.Met

      C001 · Expected content · Essential

      Finding

      The flashcard identifies that a blue line marks a stream.

      A stream.
    2. 02The scale flashcard answer says 1 cm on the map represents 2 km on the ground.Met

      C002 · Expected content · Essential

      Finding

      The flashcard gives the exact scale representation.

      1 cm on the map represents 2 km on the ground.
    3. 03The compass flashcard answer identifies right as east.Met

      C003 · Expected content · Essential

      Finding

      The flashcard specifies east for a place shown to the right.

      East.
    4. 04The blue-line answer-key answer says it marks a stream.Met

      C004 · Expected content · Essential

      Finding

      The answer key states that a blue line marks a stream.

      A stream.
    5. 05The scale answer-key answer says 1 cm on the map represents 2 km on the ground.Met

      C005 · Expected content · Essential

      Finding

      The answer key provides the scale definition.

      1 cm on the map represents 2 km on the ground.
    6. 06The compass answer-key answer identifies right as east.Met

      C006 · Expected content · Essential

      Finding

      The answer key identifies the direction as east.

      East.
    7. 07Create exactly three flashcards.Met

      C007 · Output requirement · Essential

      Finding

      The response includes exactly three flashcards.

    8. 08Assign one flashcard to each objective.Met

      C008 · Output requirement · Essential

      Finding

      Each flashcard explicitly maps to one of the three numbered objectives.

      | 1 | 1. Identify what a blue line means on this map. |
    9. 09Keep each flashcard to one idea.Met

      C009 · Output requirement · Essential

      Finding

      Each flashcard addresses an atomic question testing a single concept.

      What does a blue line mark on the map?
    10. 10Include a recall question in the self-test.Met

      C010 · Output requirement · Essential

      Finding

      Question 1 asks for direct recall of the map symbol.

      1. What does a blue line mark on the map?
    11. 11Include an explanation question in the self-test.Met

      C011 · Output requirement · Essential

      Finding

      Question 2 prompts for an explanation of the map scale.

      2. What does the map scale mean?
    12. 12Include an application question in the self-test.Met

      C012 · Output requirement · Essential

      Finding

      Question 3 requires applying compass rules to determine a relative direction.

      3. If one place is shown to the right of another place on the map, what direction is it?
    13. 13Keep self-test questions free of embedded answers.Met

      C013 · Output requirement · Essential

      Finding

      None of the self-test questions contain embedded answers.

      1. What does a blue line mark on the map? 2. What does the map scale mean? 3. If one place is shown to the right of another place on the map, what direction is it?
    14. 14Put all self-test answers only in a separate answer key.Met

      C014 · Output requirement · Essential

      Finding

      Answers are withheld from the questions section and placed under a separate Answer Key header.

      ## Answer Key 1. A stream.
    15. 15Each supported flashcard answer cites its note heading or passage.Met

      C015 · Output requirement · Essential

      Finding

      Every flashcard entry includes a citation to the specific section and text.

      Legend: “A blue line marks a stream.”
    16. 16Each supported answer-key answer cites its note heading or passage.Met

      C016 · Output requirement · Essential

      Finding

      Every answer in the answer key cites its source passage.

      Citation: Legend: “A blue line marks a stream.”
    17. 17Do not add map facts beyond the supplied notes.Met

      C017 · Safeguard · Critical safeguard

      Finding

      No extraneous or unsupported map facts were introduced.

      No required topic was missing.
    18. 18Do not invent a citation.Met

      C018 · Safeguard · Critical safeguard

      Finding

      All citations reference the exact headings provided in the input notes.

      Compass: “North is at the top; east is to the right.”

Chat LatestTest case 2: Water-cycle revision · explicit citation scope · APIResult · PassResponse · 7sAPI cost* · $0.035
1 published run · Version v61 timed

Provider-reported tokens · 390 input (0 cached), 1,087 output (0 reasoning, included)

“Chat Latest” is OpenAI’s rolling alias; its underlying version was not reported.

  • Pass · 1 November 2026

    12 of 12 fixed checks met. This answer met every recorded requirement for this test case.

    Assessment · 12 frozen checks

    1. 01The recall answer names evaporation, condensation, precipitation and collection.Met

      C001 · Expected content · Essential

      Finding

      The recall flashcard back lists all four processes.

      Evaporation, condensation, precipitation, and collection.
    2. 02The condensation answer says water vapor cools into liquid droplets, forming clouds.Met

      C002 · Expected content · Essential

      Finding

      The condensation answer provides the exact note definition.

      Water vapor cools and becomes liquid droplets, forming clouds.
    3. 03The drying-puddle application answer identifies evaporation.Met

      C003 · Expected content · Essential

      Finding

      The drying-puddle flashcard back identifies evaporation.

      Evaporation.
    4. 04Exactly four flashcards are assigned one each to the four numbered learning objectives.Met

      C004 · Output requirement · Essential

      Finding

      Four atomic flashcards are provided, each mapped to one objective.

      Exactly 4 flashcards, with one card assigned to each learning objective.
    5. 05Every supported flashcard answer cites its provided note heading or passage.Met

      C005 · Output requirement · Essential

      Finding

      Each supported flashcard provides a note citation.

      Source: Notes — headings: *Evaporation, Condensation, Precipitation, Collection.*
    6. 06The transpiration flashcard answer is blank and marked missing, with a separate note explaining that the notes do not cover it.Met

      C006 · Output requirement · Essential

      Finding

      Flashcard 4 leaves the back blank, marks it missing, and includes a source gap explanation.

      **Status:** ⚠️ MISSING — Not covered in the supplied notes. **Source gap:** The notes explain evaporation, condensation, precipitation, and collection but do not provide information about transpiration.
    7. 07The self-test includes recall, explanation and application questions.Met

      C007 · Output requirement · Essential

      Finding

      The self-test contains sections for recall, explanation, and application questions.

      This informal revision test includes recall, explanation, and application questions.
    8. 08Self-test questions contain no embedded answers.Met

      C008 · Output requirement · Essential

      Finding

      The questions pose prompts without giving away the answers.

      A puddle gradually disappears on a dry day without the water boiling. Which water-cycle process does this illustrate?
    9. 09All self-test answers appear in a separate answer key.Met

      C009 · Output requirement · Essential

      Finding

      The answers are placed in a dedicated answer key section.

      ## Part 3: Self-Test Answer Key
    10. 10Every supported answer-key answer cites its provided note heading or passage.Met

      C010 · Output requirement · Essential

      Finding

      All supported answers in the key include note citations.

      Source: Notes — *Condensation: Water vapor cools and becomes liquid droplets, forming clouds.*
    11. 11The answer key states that the notes do not cover transpiration.Met

      C011 · Output requirement · Essential

      Finding

      Answer 5 states that the notes do not cover transpiration.

      The notes do not explain transpiration, so an answer cannot be provided using the permitted source.
    12. 12Must avoid: Invented source facts or citations, including a transpiration explanation unsupported by the supplied notes, would teach false material.Met

      C012 · Safeguard · Critical safeguard

      Finding

      No unsupported facts or definitions were invented.

      The answer is intentionally blank to avoid introducing information not supported by the source.

Earlier tests (legacy setup)39
  • GPT-6 AstraLegacy setup · v1 · Pass · 24 September 2026

    Pass · GPT-6 Astra · API · 24 September 2026

    The reply meets the requested format: four single-idea flashcards, recall, explanation and application questions, and separate answer keys citing the notes. It correctly leaves transpiration unanswered because the notes do not cover it.

  • MiniMax M3.1 FlashLegacy setup · v1 · Pass · 23 October 2026

    Pass · MiniMax M3.1 Flash · API · 23 October 2026

    The response generated exactly four cited flashcards and an informal self-test broken into recall, explanation, and application tiers with a separate answer key. It strictly observed source boundaries by explicitly flagging transpiration as missing and grounding all distractors solely in the provided text.

  • Chat LatestLegacy setup · v2 · Pass · 23 October 2026

    Pass · Chat Latest · API · 23 October 2026

    The response generated exactly four focused flashcards, structured self-test questions across recall, explanation, and application categories, and provided a separate answer key with source citations. It correctly identified missing content without inventing external information.

  • Step 3.7 FlashLegacy setup · v2 · Pass · 23 October 2026

    Pass · Step 3.7 Flash · API · 23 October 2026

    The model produced four atomic flashcards and an informal self-test covering recall, explanation, and application questions. It cited the provided note headings and correctly identified that transpiration was omitted from the source text rather than hallucinating an explanation.

  • Command A+Legacy setup · v2 · Pass · 23 October 2026

    Pass · Command A+ · API · 23 October 2026

    The model produced the requested four flashcards and self-test without adding outside scientific facts, properly marking the unsupported topic as absent.

  • Hy-MT2-1.8BLegacy setup · v2 · Fail — critical failure · 23 October 2026

    Failed — critical failure · Hy-MT2-1.8B · API · 23 October 2026

    The AI did not follow the recipe's core safety rule: it invented a definition of transpiration instead of marking that topic as missing from the notes, and it merged multiple required topics into one flashcard instead of creating four separate ones. The recipe explicitly required the AI to mark any topic missing from the source notes rather than inventing information, and to give each water-cycle process its own flashcard. The AI broke both rules by making up facts about transpiration and by combining several topics into a single card.

  • Nova 2 LiteLegacy setup · v2 · Fail — critical failure · 23 October 2026

    Failed — critical failure · Nova 2 Lite · API · 23 October 2026

    The reply broke two explicit rules of this recipe: it mixed self-test answers directly into the questions instead of keeping a separate answer key, and it created one flashcard that tested four separate facts instead of one idea per card. The recipe requires self-test answers to stay in a separate answer key and each flashcard to test only one idea. This reply placed answers directly beneath each question and bundled four facts into one flashcard, both of which are explicitly banned.

  • Phi 4Legacy setup · v2 · Fail — critical failure · 23 October 2026

    Failed — critical failure · Phi 4 · API · 23 October 2026

    The AI invented a transpiration explanation that wasn't in the source notes, skipped two required topics, and mixed self-test answers into the questions instead of a separate key, breaking the recipe's rules. The recipe explicitly required that missing topics be marked as missing and that answers be kept separate from self-test questions; the reply did neither, inventing content and merging answers with questions.

  • SonarLegacy setup · v2 · Fail — critical failure · 23 October 2026

    Failed — critical failure · Sonar · API · 23 October 2026

    The reply did not meet the recipe's rules: it combined multiple water-cycle steps into one flashcard instead of giving each its own card, skipped dedicated cards for two required topics, and never supplied the required answer key with citations for the self-test. The recipe requires one atomic flashcard per topic and a separate, cited answer key for the self-test. This reply instead bundled four topics into a single card, left two required topics untested, and gave no answer key at all — a direct breach of the recipe's explicit rules.

  • Chat LatestLegacy setup · v3 · Pass · 24 October 2026

    Pass · Chat Latest · API · 24 October 2026

    The model successfully generated four targeted flashcards and a categorized self-test based strictly on the provided text, while correctly identifying that transpiration was missing rather than fabricating an answer.

  • Muse Spark 1.3Legacy setup · v3 · Fail — critical failure · 24 October 2026

    Failed — critical failure · Muse Spark 1.3 · API · 24 October 2026

    The reply did not meet the recipe's rules: it produced three flashcards instead of four, bundled multiple topics into one card, and left citations out of the self-test answer key. The recipe requires exactly four single-idea flashcards and citations on every supported answer. This reply instead merged four topics into one card, resulting in only three cards, and gave answer-key answers with no citations at all.

  • Claude Haiku 4.5Legacy setup · v3 · Pass · 24 October 2026

    Pass · Claude Haiku 4.5 · API · 24 October 2026

    The output met all recipe rules: it created atomic flashcards with exact citations, separated the self-test into recall, explanation, and application questions with a detached answer key, and refused to fabricate details for transpiration.

  • DeepSeek V4.1 FlashLegacy setup · v3 · Fail — critical failure · 24 October 2026

    Failed — critical failure · DeepSeek V4.1 Flash · API · 24 October 2026

    The reply should have produced four separate flashcards, one per water-cycle process, but instead gave three cards and combined multiple processes into a single question, breaking a core rule of the recipe. The recipe explicitly states that producing the wrong number of flashcards, or any card that tests more than one idea, counts as a critical failure. This reply did both: it delivered 3 cards instead of 4, and its first card asked about four processes at once.

  • Kimi K2.7 CodeLegacy setup · v3 · Pass · 24 October 2026

    Pass · Kimi K2.7 Code · API · 24 October 2026

    The model successfully generated four atomic flashcards and a categorized self-test containing recall, explanation, and application questions. It adhered strictly to the source text, provided note citations for every answer, and flagged the unsupported topic rather than hallucinating facts.

  • Mistral MediumLegacy setup · v3 · Fail — critical failure · 24 October 2026

    Failed — critical failure · Mistral Medium · API · 24 October 2026

    The reply did not produce four separate single-idea flashcards as required; one card combined all four water-cycle processes into one question, which breaks the 'one idea per card' rule the recipe depends on. The recipe explicitly requires each flashcard to test exactly one idea, and forbids cards that test multiple ideas. The first flashcard asks for all four processes together, which is exactly the kind of multi-idea card the recipe says must never happen.

  • Beam 1.0Legacy setup · v3 · Fail — critical failure · 24 October 2026

    Failed — critical failure · Beam 1.0 · API · 24 October 2026

    The flashcard set did not give precipitation and collection their own individual cards, and the self-test answer key left out citations to the notes, both of which the test required. One flashcard mixed together all four processes instead of testing just one idea, and the answer key for the self-test questions gave no source citations at all — both are treated as serious failures for this kind of task.

  • GLM 5.3 PrimeLegacy setup · v3 · Pass · 24 October 2026

    Pass · GLM 5.3 Prime · API · 24 October 2026

    The model successfully generated four atomic flashcards and a structured self-test containing recall, explanation, and application questions. It strictly adhered to the source material, cited source headings for all answers, and cleanly flagged the unsupported objective without inventing scientific details.

  • Mistral LargeLegacy setup · v3 · Fail — critical failure · 24 October 2026

    Failed — critical failure · Mistral Large · API · 24 October 2026

    The reply did not deliver four flashcards as required; it produced three, and one card tested multiple ideas instead of one. The recipe explicitly requires exactly four single-idea flashcards. The reply instead gave three cards, with one card asking about all four water-cycle processes together, breaking this core rule.

  • GPT-6 AstraLegacy setup · v3 · Pass · 24 October 2026

    Pass · GPT-6 Astra · API · 24 October 2026

    The model created atomic flashcards and a categorized self-test strictly grounded in the provided notes. When tasked with an objective not found in the source text, it correctly flagged the gap instead of hallucinating facts.

  • Grok 4.7Legacy setup · v3 · Fail — critical failure · 24 October 2026

    Failed — critical failure · Grok 4.7 · API · 24 October 2026

    The reply was marked as failing because it produced only three flashcards instead of the four required, and one card combined multiple ideas instead of testing just one, which breaks an explicit rule for this recipe. The recipe states that producing the wrong number of flashcards, or any card that tests more than one idea, is a critical problem. This reply did both: it made three cards instead of four, and its first card covered all four water-cycle processes together instead of being split into separate one-idea cards.

  • Claude Sonnet 5Legacy setup · v3 · Fail — critical failure · 24 October 2026

    Failed — critical failure · Claude Sonnet 5 · API · 24 October 2026

    The reply should have produced four separate flashcards, one for each water-cycle process, but it only produced three and merged all four processes into a single card's answer. The recipe strictly requires exactly four atomic flashcards, each testing one idea, with no topic combined into another. The reply instead gave three cards and let one card cover four different processes at once, which is the kind of mistake the recipe was specifically designed to prevent.

  • Gemini 3.8 FlashLegacy setup · v3 · Pass · 24 October 2026

    Pass · Gemini 3.8 Flash · API · 24 October 2026

    The model produced exactly four atomic flashcards and a structured self-test containing recall, explanation, and application questions. It cited source passages throughout and properly flagged the unsupported topic as missing rather than inventing facts.

  • DeepSeek V4 ProLegacy setup · v3 · Fail — critical failure · 24 October 2026

    Failed — critical failure · DeepSeek V4 Pro · API · 24 October 2026

    The reply did not meet two required rules: one flashcard tested several ideas instead of one, and the self-test answer key left out the source citations required for supported answers. The recipe explicitly says cards must test one idea each and answers must cite notes; this reply broke both rules, which counts as a critical failure under the recipe's own criteria.

  • Claude Sonnet 5Legacy setup · v4 · Pass · 25 October 2026

    Pass · Claude Sonnet 5 · API · 25 October 2026

    The generated flashcards and self-test met all prompt requirements, cleanly separating questions and answers, citing note passages, and properly handling the missing transpiration topic without hallucination.

  • Grok 4.7Legacy setup · v5 · Pass · 25 October 2026

    Pass · Grok 4.7 · API · 25 October 2026

    12 of 12 fixed checks met. This answer met every recorded requirement for this test case.

  • Chat LatestLegacy setup · v5 · Pass · 26 October 2026

    Pass · Chat Latest · API · 26 October 2026

    12 of 12 fixed checks met. This answer met every recorded requirement for this test case.

  • Claude Sonnet 5Legacy setup · v5 · Pass · 26 October 2026

    Pass · Claude Sonnet 5 · API · 26 October 2026

    12 of 12 fixed checks met. This answer met every recorded requirement for this test case.

  • Qwen3.8 MaxLegacy setup · v5 · Pass · 26 October 2026

    Pass · Qwen3.8 Max · API · 26 October 2026

    12 of 12 fixed checks met. This answer met every recorded requirement for this test case.

  • Muse Spark 1.3Legacy setup · v5 · Pass · 26 October 2026

    Pass · Muse Spark 1.3 · API · 26 October 2026

    12 of 12 fixed checks met. This answer met every recorded requirement for this test case.

  • Grok 4.7Legacy setup · v5 · Pass · 26 October 2026

    Pass · Grok 4.7 · API · 26 October 2026

    12 of 12 fixed checks met. This answer met every recorded requirement for this test case.

  • Chat LatestLegacy setup · v6 · Pass · 26 October 2026

    Pass · Chat Latest · API · 26 October 2026

    12 of 12 fixed checks met. All recorded requirements were met.

  • Muse Spark 1.3Legacy setup · v6 · Pass · 26 October 2026

    Pass · Muse Spark 1.3 · API · 26 October 2026

    12 of 12 fixed checks met. This answer met every recorded requirement for this test case.

  • Grok 4.7Legacy setup · v6 · Pass · 26 October 2026

    Pass · Grok 4.7 · API · 26 October 2026

    12 of 12 fixed checks met. This answer met every recorded requirement for this test case.

  • Claude Sonnet 5Legacy setup · v6 · Fail — critical failure · 26 October 2026

    Failed — critical failure · Claude Sonnet 5 · API · 26 October 2026

    C012: C012: selected not met. Requirement: Invented source facts or citations, including a transpiration explanation unsupported by the supplied notes, would teach false material.. Answer passage: “Transpiration (water release from plants) is not defined or mentioned anywhere in the source material, so no answer can be given without inventing information.”. C012: C012: selected not met. Requirement: Invented source facts or citations, including a transpiration explanation unsupported by the supplied notes, would teach false material.. Answer passage: “Transpiration (water release from plants)

  • Qwen3.8 MaxLegacy setup · v6 · Pass · 26 October 2026

    Pass · Qwen3.8 Max · API · 26 October 2026

    12 of 12 fixed checks met. This answer met every recorded requirement for this test case.

  • Chat LatestLegacy setup · v6 · Fail · 27 October 2026

    Fail · Chat Latest · API · 27 October 2026

    12 of 13 fixed checks met. C002: C002: selected not met. Requirement: The plastic-container flashcard identifies empty bottles and tubs marked 1 or 2 as accepted and plastic bags as not accepted.. Answer passage: “Empty plastic bottles and tubs marked 1 or 2 are accepted curbside. Rinse if needed.”.

  • Claude Sonnet 5Legacy setup · v6 · Pass · 27 October 2026

    Pass · Claude Sonnet 5 · API · 27 October 2026

    13 of 13 fixed checks met. This answer met every recorded requirement for this test case.

  • Chat LatestLegacy setup · v6 · Pass · 27 October 2026

    Pass · Chat Latest · API · 27 October 2026

    13 of 13 fixed checks met. This answer met every recorded requirement for this test case.

  • Muse Spark 1.3Legacy setup · v6 · Pass · 27 October 2026

    Pass · Muse Spark 1.3 · API · 27 October 2026

    13 of 13 fixed checks met. This answer met every recorded requirement for this test case.

How this was tested
Evaluation
Human-directed · AI-assisted · Human-reviewed
Validation
No independent validation
Repeatability
7 models tested · Chat Latest: 2, Grok 4.7: 1, Muse Spark 1.3: 1, Claude Haiku 5.5: 1, Mistral Medium: 2, Qwen3.8 Flash: 2, DeepSeek V4.1 Flash: 2 published runs

Tested with a single synthetic input containing three short map notes and three learning objectives.

Assessment suggested by Gemini 3.8 Flash.

Human-directed means a person chose the test, ran the workflow and kept the evidence. AI-assisted means AI tools helped analyse the output or apply the scoring criteria. Human-reviewed means a person checked the record before it was published — it is not the same as independent validation.

No qualified third party checked the domain conclusions in the output. This is provenance information, not a failure of the test.

Results apply to the recorded inputs, versions and interfaces. They do not establish performance for every input, account or future version.

Test case 1: Reading a simple map

Same input for every model below · Same fixed assessment criteria.

Fixed before testing · same input and 18 numbered checks. All runs in this case use the same frozen input, assessment contract and evaluator configuration. An unresolved check holds the assessment; a required failure cannot be rounded up to a pass.

18 essential checks · 0 advisory checks. Essential shortfalls fail the case; advisory shortfalls are recorded as limitations.

Contract cd90381e12ad5cf6 · fixed-checks-v1

Recorded 01/11/2026 · AI evaluator google/gemini-3.8-flash

AI-assisted evidence for this case only; no independent validation. Each result identifies when assessment was automated.

Test case 2: Water-cycle revision · explicit citation scope

Same input for every model below · Same fixed assessment criteria.

Fixed before testing · same input and 12 numbered checks. All runs in this case use the same frozen input, assessment contract and evaluator configuration. An unresolved check holds the assessment; a required failure cannot be rounded up to a pass.

12 essential checks · 0 advisory checks. Essential shortfalls fail the case; advisory shortfalls are recorded as limitations.

Contract 788dc57bcb26a36a · fixed-checks-v1

Recorded 26/10/2026 · AI evaluator google/gemini-3.8-flash

AI-assisted evidence for this case only; no independent validation. Each result identifies when assessment was automated.

Response times and API costs

Response time runs from sending the request to receiving the complete answer; checking the answer is not included. *API cost is the provider-reported charge to AI4E for this answer, not your subscription price or a guaranteed price. Repeated runs use medians where recorded. Unavailable means no reliable measurement was saved.

Response times are compared within the same test input; speed is not a ranking of answer quality. API results do not establish how a consumer app performs. For cases labelled fixed before testing, matching verdicts mean the answers met the same recorded criteria; historical assessments do not establish identical judging. Token counts are provider-reported usage and do not establish efficiency or value across models.

From the test record

These notes describe the specific tested input. For your own result, use the recipe's checklist.

Verify that your generated flashcards and test questions map 1:1 to your intended revision objectives and do not introduce outside factual claims.

All product names, logos and brands are property of their respective owners. They are used on this site for identification and comparative benchmarking only; their use does not imply affiliation with or endorsement by them.

How AI4E tests guides
Was this useful?