Skip to main content
AI4E

Round 1 · Initial snapshot · Battery v1 · In preparation

The Practical AI Benchmark

No trivia, no vibes. Frontier models get the same messy input and the same strict rules. We publish where they deliver — and where they break. All answers will be published with each round.

Workplace tasks
5
Frontier models
3
Objective rule checks
22
Critical checks
16

Inaugural frontier tier

Each model answers through its maker's own API. Results appear only after a complete, human-reviewed round. There is no blended overall score: you see exactly which checks passed.

OpenAI

GPT-6 Astra

OpenAI API

Checks passed
—
Tasks fully passed
—
Critical checkspending
Format & limit checkspending

Anthropic

Claude Sonnet 5

Anthropic API

Checks passed
—
Tasks fully passed
—
Critical checkspending
Format & limit checkspending

Google

Gemini 4 Argon

Google API

Checks passed
—
Tasks fully passed
—
Critical checkspending
Format & limit checkspending

The test battery

Five tasks from ordinary working days. Each targets one way AI commonly fails. ● marks a critical check, where a wrong answer would cause real harm at work; the rest are format and limit checks. "code" means the check is verified by code, not AI.

  1. 01Everyday productivity

    Messy brain dump → realistic day plan

    Turn 18 unordered to-dos into a plan for one 8-hour day, with breaks and honest time estimates.

    Watching for: Cramming more work into the day than fits, or silently dropping tasks.

    • ● Planned work fits 09:00–17:00
    • ● Every task placed or explicitly deferred
    • Lunch and at least one break
    • ● Fixed 14:00 dentist kept
  2. 02Everyday productivity

    Email and chat catch-up

    Read 20 scattered messages and return the decisions, the action items and who owns each one.

    Watching for: Missing an action item, inventing an owner, or reporting an open idea as decided.

    • ● All 4 action items found
    • ● Owners match the source
    • ● No invented deadlines
    • ● Open proposal not reported as decided
  3. 03Communication

    Polite refusal under pressure

    Decline a demanding customer within company policy, in under 150 words.

    Watching for: Over-apologising, promising things outside policy, or running long.

    • ≤ 150 words · code
    • ● No refund or free express promised
    • ● Only offers what policy allows
    • Clear next step offered
  4. 04Operations

    Receipts → clean table

    Turn 12 messy receipts into a fixed table with VAT split out and categories assigned.

    Watching for: Wrong arithmetic, a changed table layout, or receipts that never existed.

    • Exact columns, in order · code
    • ● Exactly 12 rows, none invented · code
    • ● Net and VAT correct for every row · code
    • ● Gross total = 12 674.50 · code
    • Only the allowed categories · code
  5. 05Analysis

    Contradiction spotter

    Compare a supplier's formal quote with their sales email and flag every place they disagree.

    Watching for: Smoothing over discrepancies or quietly picking one version.

    • ● Price discrepancy flagged
    • ● Delivery-date discrepancy flagged
    • ● Installation-scope discrepancy flagged
    • ● No invented resolution
    • Both sources cited for each
22checks

What the checks catch

The checks grouped by the kind of mistake they are designed to catch. After the first round, this shows real failures.

How we keep it fair

Same tier, same input

Only flagship models are compared with each other. Every model gets the identical prompt as a single message, with no system prompt, no tools and no examples.

Code checks what code can

Word limits, table layout, row counts, VAT arithmetic and totals are checked by code, not by AI. AI judges only assess what needs interpretation.

No model grades itself

Interpretive checks are scored by the two models from the other makers. A check only passes when both agree. A human reviews every answer and can overturn any verdict, with the reason published.

All answers will be published

When a round is released, each result links to the exact prompt and the verbatim answer, so anyone can check the work or repeat the test.

Execution specification

Models
openai/gpt-6-astra (OpenAI API) · anthropic/claude-sonnet-5 (Anthropic API) · google/gemini-4-argon (Google API)
Attempts
One answer per model per task. A run is only retried after a technical failure (timeout or service error), never because of a poor answer.
Prompt
One user message, identical for every model. No system prompt, no examples, no tools or web access.
Sampling
Each provider's default temperature. Every model is used as its maker ships it.
Reasoning
GPT-6 Astra: medium reasoning effort. Claude Sonnet 5 and Gemini 4 Argon: provider defaults.
Output limit
Claude Sonnet 5: 16 000 tokens. Others: provider default. No answer in the inaugural round is expected to come close.
Judging
Code checks first. Interpretive checks go to the two judges from other makers; disagreements are flagged for human review.
Human review
The reviewer reads every answer and can overturn a verdict; any change is noted next to the answer. Answers are not anonymised during review.
Run date
Stated when the round is published. Exact model version returned by each provider is shown on its card.

What this round can't tell you

  • Five tasks with one attempt each is a small first snapshot, not a definitive ranking of these models.
  • AI judges from other makers still have their own biases; that is why arithmetic and format are checked by code.
  • Results describe these exact prompts on the run date. Models change over time.

This benchmark is separate from recipe testing. See the recipe testing methodology for how individual guides are verified.