Documentation / Guides

Updated July 25, 2026 · About 9 min read

Test Failure Reports You Can Actually Read

A failing test should be a gift - a precise finger pointing at exactly the thing you just broke. Too often it arrives as a wall of red text, ten failures deep, stack traces from framework internals you didn't write, and no hint about which line actually matters. A readable failure report gives you that pointing finger back.

What makes a failure report readable

Set the format wars aside - every good failure report answers the same three questions, fast. If you can't answer all three within a few seconds of looking, the report is failing at its one job.

1. Which assertion failed? Not "the suite is red" - the single, specific test that didn't hold. One test, one line, one expectation. That's the atom of a useful report.
2. Expected vs. actual. The whole idea of an assertion is that reality disagreed with an expectation. A readable report shows both sides next to each other: this is what the test wanted, this is what it got. The gap between them is usually the bug.
3. The shortest path to reproduce. Which test file, which case, and the exact command to run just that case. If reproducing means re-running the whole suite and squinting, the report made you do its job.

Anything beyond those three is context, and context is optional. A stack trace helps when the assertion isn't enough, but its place is below the answer, not in front of it. The best reports lead with the conclusion and let you drill into the details only if you need to.

Expected vs. actual, side by side

Most of the time you waste reading a failure is spent reconstructing what the test wanted. A report that says "assertion failed" and stops there threw that work back at you. A report that shows both values does the reconstruction for you.

The difference is concrete. "AssertionError: false is not true" tells you nothing - you still have to open the test, find the assertion, and think backward about what should have been true. Compare that to a report that prints expected: "active" on one line and actual: "pending" on the next. Now you're not debugging the test, you're debugging the code, and you have a strong hint about where to look.

• Always show both values. Even for a plain boolean, state what was being checked so "false" gains meaning.
• Diff structured data. For objects and arrays, highlight only the fields that differ. An object with twelve keys and one wrong value shouldn't make you compare all twelve by eye.
• Keep values readable. Trim huge payloads down to the relevant part. A 500-line JSON dump with the mismatch buried on line 340 is noise disguised as a diff.

Cutting the noise

The enemy of a readable report is volume. When a run prints every passing test, every framework frame, and every failure that cascaded afterward, the one line you need is hidden in plain sight. Readability is mostly a subtraction problem: deciding what to leave out.

Three sources of noise dominate, and each has a clean fix:

• Passing tests. A green test that ran fine doesn't belong in a failure report. Summarize the passes as a count and spend the screen on what broke.
• Framework stack frames. Half of a raw stack trace is the test runner calling itself. Collapse the library internals and show the frames in your own code, where the fix actually lives.
• Cascaded failures. When one broken helper knocks over ten tests, nine of the reports are echoes. Group them, or lead with the root cause, so you don't read the same bug ten times.

The test is simple: could you hand the report to someone with no context and have them find the problem without scrolling? If not, there's more noise to cut.

Surfacing the first failure fast

When a change breaks something, the clock starts. The longer it takes to find the real failure, the more the context in your head evaporates - which line you touched, what you were trying to do, why you thought it was safe. A report that surfaces the first failure in under a minute keeps you inside the flow you were already in.

The speed here isn't about a suite that runs faster - it's about the report leading with the right thing:

• Lead with the first failure. Later red tests are often just fallout from the first. Put the earliest failure at the top, because fixing it often clears the rest.
• Fail fast, report immediately. You shouldn't have to wait for the whole suite to finish to see the first thing that broke. Stream the failure the moment it happens.
• Point straight at your code. The first frame you read should be a file you wrote, not the assertion library. Zero navigation between "test failed" and "here's the line."
• One command to re-run. The report should hand you the exact command to reproduce just that case, so verifying a fix is a single paste, not a hunt for flags.

The goal is a report where finding the problem takes less time than making a coffee - fast enough that a failure is just a short redirect, not a context switch that costs you the afternoon.

Flaky failures vs. real failures

Not every red test is a bug in your code. Some tests fail because of timing, run order, shared state, or a network hiccup - they're flaky, and treating a flaky failure as if it were real sends you off to debug code that was never broken. A readable report helps you tell the two apart before you burn an hour.

The distinguishing test is dead simple: re-run the exact same code. A real failure reproduces every time. A flaky failure passes on a retry with nothing changed. So the most useful thing a report can do is record whether the failure survived a retry.

• Reproduces on retry: a real failure. Something in the change genuinely broke behavior - go fix the code.
• Passes on retry: a flaky test. The code may be fine; the test depends on something it shouldn't. Fix the test, not the feature.

Flakiness is a signal worth acting on, not papering over. A test that passes and fails at random erodes trust in the whole suite - the moment you learn to ignore one red result, you start ignoring them all, and a real regression slips through. Quarantine the flaky test, find its hidden dependency, and make it deterministic, so every future red result means exactly what it says.

Making readable reports your default

A readable report isn't a one-off format choice - it's a property of your whole feedback loop. The goal is a workflow where every failure, on every change, arrives already distilled to the assertion, the values, and the reproduction command.

• The same report everywhere. The failure you see locally should be the failure you see at the gate, in the same shape, so nothing surprises you at deploy time.
• Lead with the answer. Conclusion first, context on demand. The top of every report is the one thing that broke, not a preamble.
• Retry-aware by default. Every failure carries a flaky-or-real verdict, so you never guess which one you're looking at.
• Feed it to an agent. A report clean enough for you to read in a minute is also clean enough for an agent to act on - read the assertion, propose a fix, re-run to confirm.

Done right, a failure stops being a wall you brace against and becomes what it was always meant to be: a precise, friendly pointer to the exact thing to fix next.

Go deeper: Keeping your test suite fast - a fast suite is what makes the retry-on-failure check cheap enough to run every time, so readable reports and flakiness detection stay effortless.

Frequently asked questions

What makes a test failure report readable?

A readable report answers three questions at a glance: which single assertion failed, what it expected versus what it actually got, and the shortest way to reproduce it. Everything else - the full stack dump, the passing tests, the framework internals - is noise you should be able to skip. If you can find the real problem in under a minute without scrolling hundreds of lines, the report is readable.

How do you tell a flaky failure from a real one?

Re-run the exact same code. A real failure fails every time; a flaky failure passes on a retry with nothing changed. Flaky tests usually depend on timing, run order, shared state, or the network, so a report that records whether a failure reproduces on retry tells you immediately whether to fix the code or the test.

Why is it so important to surface the first failure fast?

Later failures are often just fallout from the first one - a single broken function triggers a cascade of ten red tests. If the report leads with the first failure, you fix the root cause and most of the rest resolves on its own. Burying it under more noise wastes the minutes while the change is still fresh in your mind.