AI writes the code.Someone still has toknow it’s wrong.
Junior engineers used to become senior by writing the code and having it torn apart in review. That loop is gone. Redline rebuilds it from the other end: you read the pull request an agent opened, decide whether it ships, and point at the line that will wake somebody at 3am.
for (let attempt = 0; attempt < MAX; attempt++) { try { const charge = await payments.charges.create({ amount: amountCents, currency: "usd", customer: order.customerId, }); await db.order.update({ where: { id: orderId }, data: { status: "paid" }, });State corruption. A 502 means the response was lost, not that the charge failed. Retrying a create with no idempotency key bills the customer twice.
- pull requests to review
- 16pull requests to review
- real defects, planted and mined
- 28real defects, planted and mined
- lines that only look wrong
- 74lines that only look wrong
- days the longest one survived review
- 3187days the longest one survived review
One reviewer scores 42.The other scores 87.
Same pull request, same defect, and both of them found it. The first flagged nine lines to be sure. Seven were fine, and that is the whole difference — a reviewer who always finds something is one a team learns to merge past.
for (let attempt = 0; attempt < MAX; attempt++) { try { const charge = await payments.charges.create({ amount: amountCents, customer: order.customerId, }); return { ok: true, chargeId: charge.id }; } catch (err) { await sleep(2 ** attempt * 250); }}Adds bounded retries around the charge. No behaviour change on the success path.
The industry stopped producing seniors and did not replace the mechanism.
Generating code got cheap and evaluating it did not, so the scarce skill moved from writing to reading. Nothing trains reading.
- 17%
- lower comprehension scores for developers who learned a library with AI assistance, in a randomised trial of 52 engineers. The largest drop was in debugging.
- Anthropic ↗
- 8×
- increase in duplicated code blocks across 211 million changed lines, with two-week code churn roughly doubling since the pre-AI baseline.
- GitClear ↗
- −20%
- employment for software developers aged 22 to 25 since late 2022, while headcount for older engineers at the same firms grew.
- Stanford Digital Economy Lab ↗
“A junior engineer can now generate code faster than a senior engineer can critically audit it.”
Grinding algorithm puzzles trains the one skill that got automated.
There is nothing wrong with LeetCode. It is an answer to a question nobody is asking any more: you write from a blank page, guessing is free, and a model finishes the whole thing in three seconds.
Here, guessing is the expensive part. Select every line, catch every defect, and you score 42 out of 100 — the same on all fifteen.
| Puzzle grinding | Redline |
|---|---|
| You write code from a blank page. | You read a diff somebody else wrote, with a description that lies. |
| Puzzles built to have one clean answer. | Failure modes that shipped: idempotency, tenancy, timezones, authorisation. |
| Guessing costs nothing. Submit until green. | Flagging clean code costs you points, because on a team it costs you trust. |
| An agent solves the whole question in three seconds. | An agent wrote the question. Your job starts where it stopped. |
| You are measured on whether it passes. | You are measured on whether you can say why, before you see the answer. |
Five minutes per pull request.
THE LOOP · FIVE STAGES
- I
Read the ticket, then the agent's note
Every PR opens with a confident summary of what it does. It is fluent, specific, and part of it is not true. Reading the claim before the code is the habit the whole product is built around.
- II
Call it on the clock
Approve, request changes, or block — in ninety seconds. Seniors triage fast and go deep second. Getting the call right and the reasoning wrong is a different failure from getting both wrong, and we score them separately.
- III
Redline the exact lines
Click the lines that will hurt. Not the file, not the function — the lines. Vagueness is where review comments go to be ignored.
- IV
Name the failure, then explain it
Choose from ten failure modes, then write the conditions under which it fires. Your prose is scored against the concepts a senior would have named.
- V
See what you missed
The full defect list, the traps you fell for, and — the part nobody else shows you — an explanation of every alarming-looking line that was actually fine.
Ten ways agent-written code goes wrong.
Fluency here is the measurable difference between a junior and a senior reading a diff. A senior names the failure before they can describe the fix — and each mode has a tell you can learn.
Hallucinated API
A method, option, or flag that does not exist — or does not do what the name implies.
tell → The call reads beautifully and is not in the docs. Extra options are silently ignored.
State corruption
Data written twice, lost, or written wrong — with no error raised.
tell → A side effect sits inside a retry, a loop, or a code path that can run more than once.
Concurrency
Races, unawaited promises, check-then-act gaps.
tell → An async callback passed to something that does not await it. Shared mutable state.
Error masking
A failure is caught and turned into a success, a default, or a log line.
tell → Broad catch blocks, `|| fallback` on a value that should have thrown, empty catch.
Authorization gap
The caller is authenticated but never checked against the resource they asked for.
tell → An id comes from the request and goes straight into a query. Nobody asks whose it is.
Injection / traversal
Untrusted input reaches an interpreter, a shell, a path, or a template.
tell → String concatenation into SQL, a path join with a request parameter, `eval`-adjacent calls.
Performance cliff
Correct on the developer's ten rows, fatal on production's ten million.
tell → A query inside a map. Nested loops over request-sized data. No pagination.
Resource leak
Something is acquired and never released, or grows without a bound.
tell → Module-level caches with no eviction, missing timeouts, handles opened in a loop.
Spec drift
The code is clean, tested, and does not do what the ticket asked for.
tell → Read the ticket twice. The PR description restates the ticket; the code does not.
Boundary error
Off-by-one, empty input, null, float, locale, timezone, DST.
tell → Inclusive versus exclusive ranges. Defaults that depend on the machine's locale.
One of these pull requests is completely fine.
We will not tell you which. It has an AbortController that looks like a leak, a catch block that looks like it swallows errors, and a dependency array that looks incomplete. All three are correct.
A reviewer who always finds something is not careful, they are noisy — and a team learns fast to merge past them. That is exactly how the one genuinely critical comment gets ignored. So here, flagging clean code costs you points, and knowing when to approve is a scored skill.
SPECIMEN SCORECARD
const deps = [orderId, amountCents, retries]; // eslint-disable-next-line react-hooks/exhaustive-deps }, deps);Two of these do not assume the code is wrong.
A bug hunt has a shelf life we do not control. If generated code becomes reliably correct, an exercise built on finding its mistakes is an exercise about a problem that no longer exists — so the other two formats were built to make no such assumption.
Their difficulty has nothing to do with how good the generator is. It scales with how much of the system in front of you was written by someone other than you, which is a number that only goes up.
rename user.plan → user.subscriptionTier
41 call sites, type check clean, 340 tests green.
- session cache (redis)BREAKSold shape, written yesterday
- billing_plan columnMIGRATEcontainers roll before migrations
- churn dashboard queryBREAKSreturns null, the alert goes quiet
- iOS 4.2, in reviewBREAKSyou own neither end
- webhook payload v1MIGRATEpartners read the old key
- planRenderer.tsxUNAFFECTEDcontains 'plan'. unrelated.
- flags.tsUNAFFECTEDreads the id, not the field
- seed fixturesUNAFFECTEDregenerated every run
- Review the pull request
A diff, a ticket, and a confident description that is partly untrue. Find the defect, avoid the decoys, name the failure. The measurable one, and the one a good model also solves — which is why it is not the only one.
Ten of these- Blast radius
The change is correct, and we say so up front. Eight other places in a system you have never seen; which does it break, which need a migration, which are genuinely untouched. The answer is not in the diff, so a model handed the diff cannot produce it.
Open the rename- Adjudication
The review already happened. Five confident comments, two describing code that is not in the file, one marked blocker that is a follow-up — and the thing that will cost money has no comment on it at all. Call each one, then find what nobody said.
Five reviewers, one lock
An exercise where the answer is not in the diff.
We used to say an agent could not sit this for you. That was not true, and it is the first thing a buyer tests. Paste a diff challenge into a good model and it finds the defect.
The claim we can defend is narrower and more useful. On blast radius and adjudication the answer is not in the diff — it is in the cache key, the deploy order, and the comment nobody wrote — so a model handed the diff cannot produce it. What that measures is the assembling of context, which is the part a person has to do and the part an assessment can control the inputs to.
What the score is not validated to do is on the methodology page, including the two claims we mark as unvalidated.
There is no contact address yet. Set NEXT_PUBLIC_CONTACT_EMAIL and this becomes a mail button — until then saying so beats a link that goes nowhere.
- Onboarding
- New hires meet your codebase's real failure modes in week one, instead of finding them in production in month four.
- Hiring — not yet
- A timed review with a scored rubric and a transcript of the reasoning. We do not sell it as a screen: it is not validated as one, and the methodology page says so in more detail than you will want.
- Calibration
- Find out which of your reviewers block clean code and which approve the expensive stuff. Both are costing you.
- Your own diffs
- Turn a real incident into a challenge. The defect your team actually shipped teaches more than any exercise we write.
Start with the one thatfools almost everybody.
A batch job with a try/catch, an audit table, and a staging run where all twelve emails arrived. It has never sent an email in production.
90 seconds to triage · no signup · nothing to install