AI for Everyone

League · Archive

Round 1 pilot results

The unranked pilot that preceded the League, kept as an archive.

Latest study

Season 1: 10 real-life cases, 6 models

Published 18 Oct 2026, corrected 19 Oct 2026. Open the Fieldboard for current standings. The Round 1 pilot below is kept as an archive.

Round 1 results

One attempt per task. No blended score and no winner: five single-attempt tasks are a snapshot, not a ranking. Open any result to read the answer and every verdict.

What this round shows

5 task cases, one answer per model per case.
22 criteria applied to those answers.
GPT-6 Astra and Claude Sonnet 5 meet all published checks.
Gemini 4 Argon fails 1 standard check (≤ 150 words).
All three meet every critical check.
This sample does not establish a general ranking. 2 errors outside the checks are listed as observations.

OpenAI

GPT-6 Astra

OpenAI API · returned gpt-6-astra

Tasks fully passed: 5/5
Checks passed: 22/22
Critical checks passed: 16/16
Standard checks passed: 6/6

Anthropic

Claude Sonnet 5

Anthropic API · returned claude-sonnet-5

Tasks fully passed: 5/5
Checks passed: 22/22
Critical checks passed: 16/16
Standard checks passed: 6/6

Google

Gemini 4 Argon

Google API · returned google/gemini-4-argon

Tasks fully passed: 4/5
Checks passed: 21/22
Critical checks passed: 16/16
Standard checks passed: 5/6

Corrections and observations

Published 16 Oct 2026 · Last corrected 16 Oct 2026. The judges were given a faulty reference note for the catch-up task. It said Friday was the only deadline, but the source asks for the pricing page “before launch” (12 November). 2 answers that reported this deadline were wrongly failed. The verdicts are corrected below. 2 errors found outside the published checks are listed separately and never counted.

2 corrections · 2 observations

Scored corrections

Evaluator correction: Source-supported "before launch" deadline wrongly failed
Gemini 4 Argon · Email and chat catch-up · criterion “No invented deadlines”
Previous: Fail → Revised: Pass
The reference notes given to the judges said Jonas's Friday deadline was the only one. The source also contains Jonas's request to update the pricing page "before launch", which Sara accepts, and the launch date is locked as 12 November. Reporting the pricing-page deadline as "before launch (12 November)" is supported by the source, so "No invented deadlines" should pass. The judges followed the faulty notes. The answer was not re-run.
eval-1.0 → eval-1.1 · prepared 2026-10-16 15:45 UTC · reviewed 2026-10-16 15:57 UTC · Reviewed and applied
Evaluator correction: Source-supported "before launch" deadline wrongly failed
GPT-6 Astra · Email and chat catch-up · criterion “No invented deadlines”
Previous: Fail → Revised: Pass
The reference notes given to the judges said Jonas's Friday deadline was the only one. The source also contains Jonas's request to update the pricing page "before launch", which Sara accepts, and the launch date is locked as 12 November. Reporting the pricing-page deadline as "before launch (12 November)" is supported by the source, so "No invented deadlines" should pass. The judges followed the faulty notes. The answer was not re-run.
eval-1.0 → eval-1.1 · prepared 2026-10-16 15:45 UTC · reviewed 2026-10-16 15:57 UTC · Reviewed and applied

Observations outside the rubric (never counted)

Observation outside the rubric: Deferred-work total misstated (outside the rubric)
Gemini 4 Argon · Messy brain dump → realistic day plan
The "moved to another day" list contains the supplier contract (75 min), shared drive cleanup (60), LinkedIn post (30), expense report (30) and project tracker (20): 215 minutes, or 3h35. The answer describes them as 2h05, and its opening workload figure also differs from the stated estimates. No Round 1 criterion checks these figures, so all four checks still pass. A scheduling-arithmetic check is drafted for the next battery.
eval-1.0 → eval-1.1 · prepared 2026-10-16 15:45 UTC · reviewed 2026-10-16 15:57 UTC · Reviewed and applied
Observation outside the rubric: Pricing-page deadline omitted (outside the rubric)
Claude Sonnet 5 · Email and chat catch-up
The answer says no deadline was stated for the pricing page, although the source asks for it "before launch" (12 November). "No invented deadlines" does not test whether stated deadlines are retained, so the pass stands. Deadline retention is added as a separate criterion in the next battery.
eval-1.0 → eval-1.1 · prepared 2026-10-16 15:45 UTC · reviewed 2026-10-16 15:57 UTC · Reviewed and applied

Original answers and judge verdicts are unchanged and remain on each result page. A correction changes the evaluation, not the model's answer; no answer was re-run.

Results matrix

Task by model

A task is fully passed only when every check passes in the final, human-reviewed verdict. Battery fingerprint d2cd2b72a708.

Checks passed per task and model
Task	GPT-6 Astra	Claude Sonnet 5	Gemini 4 Argon
Messy brain dump → realistic day plan	All 4 checks passedView evidence	All 4 checks passedView evidence	All 4 checks passedView evidence
Email and chat catch-up	All 4 checks passedView evidence	All 4 checks passedView evidence	All 4 checks passedView evidence
Polite refusal under pressure	All 4 checks passedView evidence	All 4 checks passedView evidence	3/4 checksAll critical checks passedView evidence
Receipts → clean table	All 5 checks passedView evidence	All 5 checks passedView evidence	All 5 checks passedView evidence
Contradiction spotter	All 5 checks passedView evidence	All 5 checks passedView evidence	All 5 checks passedView evidence

Messy brain dump → realistic day plan
- GPT-6 Astra
  All 4 checks passedEvidence →
- Claude Sonnet 5
  All 4 checks passedEvidence →
- Gemini 4 Argon
  All 4 checks passedEvidence →
Email and chat catch-up
- GPT-6 Astra
  All 4 checks passedEvidence →
- Claude Sonnet 5
  All 4 checks passedEvidence →
- Gemini 4 Argon
  All 4 checks passedEvidence →
Polite refusal under pressure
- GPT-6 Astra
  All 4 checks passedEvidence →
- Claude Sonnet 5
  All 4 checks passedEvidence →
- Gemini 4 Argon
  3/4 checksAll critical checks passedEvidence →
Receipts → clean table
- GPT-6 Astra
  All 5 checks passedEvidence →
- Claude Sonnet 5
  All 5 checks passedEvidence →
- Gemini 4 Argon
  All 5 checks passedEvidence →
Contradiction spotter
- GPT-6 Astra
  All 5 checks passedEvidence →
- Claude Sonnet 5
  All 5 checks passedEvidence →
- Gemini 4 Argon
  All 5 checks passedEvidence →

Failures captured by the published checks

Counts of failed criteria in the final verdicts. Several failed criteria can come from one answer. Zero means no published check failed, not that the answers contain no errors. Observations outside the rubric are not counted here.

Failed criteria per model and failure category
Model	Broke a hard rule	Failed criteria	Answers affected
GPT-6 Astra	0	0	0/5
Claude Sonnet 5	0	0	0/5
Gemini 4 Argon	1	1	1/5

Test coverage · not results

What this battery tests

The 22 checks grouped by the kind of mistake each one is designed to catch. Each check belongs to one group.

Invented or overstated: 4
Broke a hard rule: 8
Missed something: 8
Got the numbers wrong: 2

All product names, logos and brands are property of their respective owners. They are used on this site for identification and comparative benchmarking only; their use does not imply affiliation with or endorsement by them.

League guide

How to read the League

A short email guide: how rounds are scored, what the points and arrows mean, and how to check any result yourself.

Round 1 results

What this round shows

GPT-6 Astra

Claude Sonnet 5

Gemini 4 Argon

Task by model

Messy brain dump → realistic day plan

Email and chat catch-up

Polite refusal under pressure

Receipts → clean table

Contradiction spotter

Failures captured by the published checks

What this battery tests

How to read the League