Wells Bench

The puzzle, in one sentence

Every cell holds one digit: the number of orthogonal steps from that cell to the nearest cell holding 0.

That is the whole rule. Steps go up, down, left and right, never diagonally. A cell holding 0 is zero steps from itself. You are given some of the digits and a dot for each one you have to work out. Exactly one completed grid fits.

The genre is a distance transform — a grid where each cell records its distance to the nearest source. It is not Minesweeper, which counts neighbours (a local sum) rather than measuring a distance (a global minimum, expressed locally). It is not a nonogram, and there are no sums, regions or loops. As far as a search of published pencil-puzzle genres goes, distance-to-nearest has not been used as a puzzle rule before, which is part of the point: there is no corpus of worked examples to have memorised.

A worked example

Here is a small one — five by five. Dots are yours to fill in.

2 . 2 . 0
. 0 . . .
2 . . 2 .
. 2 . . 2
2 . . . 2

2 1 2 1 0
1 0 1 1 0
2 1 2 2 1
3 2 1 2 2
2 1 0 1 2

Three steps of the reasoning

1. Between a 2 and a 0. Look at row 1, column 4. It sits between the 2 at column 3 and the 0 at column 5. Neighbouring cells can differ by at most 1 — a step can only change your distance by one — so it is at least 1 and at most 1. It is 1.
2. Something has to be one less. Look at the 2 in the bottom right corner. A cell holding 2 must touch a cell holding 1, otherwise its shortest path to a zero could not start. Its only neighbours are the 2 above it and the blank to its left. So that blank is 1.
3. Finding a well nobody printed. That new 1 must in turn touch a 0. Its neighbours are the corner 2, the blank above it, and the blank to its left. The blank above it touches a printed 2, so it cannot be 0. Therefore the blank to its left is 0 — a well that appears nowhere in the puzzle and has to be deduced. Every instance in this benchmark hides at least one.

Those three moves — neighbours differ by at most one, every value above 0 touches one less, and a value whose only possible predecessor is a single cell forces that cell — are the complete technique set. The generator will not publish an instance that cannot be finished with them, so “no guessing required” is a property of every instance rather than a claim about most of them.

What is proved, and how

Exactly one solution — proved twice, by different mathematics

The generator counts solutions with backtracking search and stops at two. The verifier ignores that entirely and uses a counting argument with no search in it at all:

A legal grid is determined by its zeros, because the zeros are exactly the cells at distance 0. So counting grids is counting sets of zeros. Let C be every cell that no clue forbids — a zero at distance less than v from a clue of value v would drag that clue down, so every legal zero-set is a subset of C. For a subset of C the only remaining requirement is that each clue has a zero at exactly its own distance, and that requirement is upward closed: adding zeros can only help satisfy it. So the legal zero-sets form an up-set inside C, with C itself at the top. Therefore a solution exists exactly when C satisfies every clue, and it is unique exactly when, in addition, C with any single cell removed fails.

The two methods share no code and no premise beyond the rule. They are also checked against a third — exhaustive enumeration of every possible zero-set on grids small enough to enumerate — over 1,500 random clue-sets drawn so that impossible, unique and ambiguous cases all occur, and the test asserts that all three kinds occurred. A uniqueness check that always answered “unique” would fail that test rather than pass it.

Solvable without guessing — proved twice

Instances are built by removing clues from a completed grid for exactly as long as constraint propagation using only the three named techniques still finishes the grid. The verifier re-checks this with a second propagator written from scratch over a different data structure. Both reason over the digits 0–9 and nothing more — the same information the prompt gives.

No decorative clues

Carving runs to exhaustion, so every remaining clue is load-bearing: put any one back and it is redundant, take any one away and deduction stalls. The verifier re-tests this by removing each clue in turn. A padded instance would overstate how much reasoning it demands.

Grading is a single bit

Your submitted grid is the solution or it is not. There is no partial credit to argue about. The grader computes that bit twice — string equality against the answer key, and re-checking the rule from scratch — and because each instance has exactly one solution the two must agree. If they ever disagree the grader refuses to emit a score and reports the instance as broken, because a wrong answer key that silently marks correct solvers wrong is worse than no benchmark at all.

A useful consequence: the answer key is not needed to grade. A grid is correct exactly when it obeys the rule and keeps every printed digit. That is why the grader below works in your browser without the answers ever being on this page.

Why it cannot go stale

The argument is not that the answers are secret. Secrets leak, and a benchmark that depends on one dies the first time somebody pastes it into a forum. The argument is that the instances are generated:

• A set is a pure function of a name, a seed phrase and a tier layout.
• A fresh seed produces a fresh set, with the same proved properties, in well under a second.
• So a solver that has memorised the published set gains nothing on the set you make in the next thirty seconds — and you can prove your set has the same properties before you use it.

Each published set carries a SHA-256 digest of its instances, and a results file that does not name the right digest is refused rather than scored. Two numbers cannot end up on the same chart unless they came from the same puzzles.

Grade an answer, here, now

Pick an instance, copy the prompt into whatever model you use, and paste what it says back into the box. Nothing is uploaded — the check runs in your browser, and this page has no server, no account and no analytics.

Scoring a whole run

Drop a results file here to score all thirty at once. The format is described below, and public-001.answers-template.json is a ready-made blank one.

Running it properly

Three ways, none of which assume you have an API key.

1. By hand. Download public-001.prompts.txt, paste each prompt into a chat window, collect the replies into the answers template, and score the file — here in the browser, or with node bin/run.mjs --set bench/public-001.json --answers yours.json.
2. With your own harness. Download public-001.prompts.jsonl: one {id, prompt} per line. Send them however you like and write the outputs back into an answers file.
3. With your own key. node bin/run.mjs --set … --call --shape openai-chat --endpoint … --model … --key-env YOUR_VAR. Two request shapes are supported, openai-chat and anthropic-messages, which between them cover almost every hosted or local endpoint. No key is stored, assumed or required anywhere; if you do not set one, no request is made.

The prompt is a pure function of the instance — no dates, no phrasing variants, no per-model wording. Two people running the same set show their models byte-identical text, or their scores are not comparable.

The results format

So that anyone can publish a run and anyone else can check it. The raw model output is kept verbatim, which is what makes a disputed score re-checkable: a third party can run the grader over your outputs and get your number, or not.

{
  "wellsbench": 1,
  "set":    { "name": "public-001", "setDigest": "a5544f3e…" },
  "solver": { "name": "…", "vendor": "…", "version": "…", "access": "api | chat-ui | human" },
  "conditions": {
    "date": "2026-09-08",
    "attemptsPerInstance": 1,
    "toolsAllowed": false,
    "temperature": null,
    "maxOutputTokens": null,
    "systemPrompt": null,
    "notes": "anything a reader needs in order to compare this with another run"
  },
  "answers": { "public-001-t1-000": "the model's reply, verbatim", … }
}

date, attemptsPerInstance and toolsAllowed are required; a file that omits them is refused rather than scored, because a number nobody can interpret is worse than no number. Instances with no answer are scored as wrong, never skipped — a harness that drops the instances it failed to run inflates its own score.

Results

None collected. Nothing on this page claims that any model finds this hard, or easy. Measuring that needs a controlled run across several models at a stated temperature with a stated number of attempts, and that has not been done. If you run it, the format above is how to publish what you got.

One illustrative data point, which is not a finding: the model that wrote this generator solved four of the thirty instances in one session — the first instance of tiers 1, 1, 2 and 3 — and got all four right. It had just written the code, it knew the genre intimately, it had unlimited deliberation and no time limit, and it used a global counting argument rather than the step-by-step deduction the puzzle is built around. One model, one session, four instances, no controls. It tells you the harness works. It tells you nothing about any model.

The set

public-001 holds 30 instances, six in each of five tiers. Tiers are defined by grid size and by deduction depth — the share of cells that cannot be settled without the hardest of the three techniques. Whether those tiers track difficulty for any given solver is an empirical question this page does not answer.

Built from the seed phrase wells-benchmark-public-001. Instance digest a5544f3e8c173e99673a82cd99d636da8b1df4f0f64486004cd7b70dfcd0c007. Rebuilding from that phrase reproduces the set byte for byte, so nobody has to trust the file they downloaded.

Downloads

• public-001.json — the instances, with no answers in the file
• public-001.prompts.txt — every prompt, for pasting
• public-001.prompts.jsonl — one {id, prompt} per line
• public-001.answers-template.json — a blank results file

The answer key is generated alongside the set but kept in a separate file and is not published here. Not because it is secret — the seed reproduces it — but because pairing puzzles with answers in one published artefact is the exact shape a crawler learns from, and separating them costs nothing.