Golden-File Characterization Tests: How to Verify a Port Without a Spec

The specific testing pattern for code you don’t fully understand but need to preserve exactly — and the 13-scenario edge-case corpus that caught 4 silent regressions during our port.

Published: May 2026 Tags: #SoloBuilder #Testing #Refactoring #PortVerification #SnapshotTesting #Vitest

The “So What?” - Why You Should Care

Bottom Line Value: A concrete testing pattern (golden-file characterization) for verifying that ported, refactored, or “kept verbatim” code still behaves exactly the same. Works even when you don’t have a spec.

What You’ll Gain:

The exact pattern: input fixtures → run through code → snapshot output → assert future runs match
A starter edge-case corpus structure (13 scenarios that cover most tuning work)
The Vitest snapshot-test setup (~10 lines of config)
How to use this pattern even when behavior is “weird but right”
ROI: ~4 hours of setup catches every silent regression that would have shipped — and gives you the courage to actually touch the kept code later

Who This Helps: Solo builders who’ve decided to preserve a chunk of tuned code (see post #139) but can’t write a normal spec-driven test because they don’t fully understand the rules.

Why Bother: The reason builders avoid touching “scary” code is that they have no safety net. Characterization tests give you the safety net. Suddenly the scary code is just code, and you can refactor freely.

TLDR: The Pattern in 5 Steps

Build a corpus of real inputs — 10–15 scenarios covering edge cases (1 day)
Run the existing code over the corpus — capture every output as a snapshot
Commit the snapshots to the repo (the “golden files”)
Write a single test that re-runs the code and asserts identical snapshots
Every change to the code requires either: snapshots still pass, OR you explicitly update them with intent

Result: A safety net that catches every behavioral drift, even ones you didn’t know were possible. We built 32 snapshots across 13 scenarios in ~4 hours.

The Moment You’re Probably In

You inherited (or wrote 18 months ago) a module that does something complex. Maybe it’s:

A scoring algorithm with 14 numbered rules
A retry routine that handles 8 specific failure modes
A parser with 23 if-branches for edge cases
A heuristics file tuned over a year

You need to do something to it — refactor, port, optimize — and you’re afraid to touch it. The fear is real:

The rules are partially documented (some comments, some not)
The tests cover 60% of the cases (the obvious ones)
The remaining 40% of the behavior lives in the structure of the code itself
If you change anything, you have no idea what might break

The classic responses:

“I won’t touch it” — fine until you have to
“I’ll add unit tests as I go” — you’ll cover the cases you think of, not the ones the code actually handles
“I’ll rewrite it and hope” — see post #139

The right response is characterization tests. Here’s the pattern.

What “Characterization Test” Actually Means

A normal unit test says: for input X, the output should be Y, because the spec says so.

A characterization test says: for input X, the output is currently Y. Record that. From now on, assert it stays Y unless I deliberately change it.

The difference is subtle but important:

Unit tests verify behavior against an external truth (the spec)
Characterization tests verify behavior against itself (what the code currently does)

When you don’t have a spec — or the spec is “whatever Jeremy decided 8 months ago and then forgot” — characterization tests are how you build a safety net retroactively.

The technique was named by Michael Feathers in Working Effectively with Legacy Code and it’s the single most valuable testing pattern for solo builders inheriting their own old work.

Step 1: Build the Corpus (most of the work)

The corpus is a set of input fixtures designed to exercise the behavior you care about. For each fixture, you’ll generate a snapshot of the output.

The 13 scenarios we used for our text formatter:

#	Scenario	Why this case matters
1	Long pauses (>3s between words)	Tests pause-driven paragraph breaks
2	Short utterances (<10 words total)	Tests minimum-content handling
3	Dense filler words (“um”, “uh”, “like”)	Tests filler suppression
4	Rapid speech with minimal pauses	Tests fallback to semantic analysis
5	Mixed sentence lengths	Tests adaptive thresholds
6	Single word input	Edge case for empty/minimal output
7	Very long sentence (no internal pauses)	Tests forced breaking
8	Topic transitions with discourse markers (“So,”, “Anyway,“)	Tests discourse-marker boost
9	Numbers and special characters	Tests punctuation handling
10	Empty timing data	Tests semantic fallback path
11	Dense short pauses	Tests pause-vs-semantic disambiguation
12	Punctuation edge cases	Tests internal normalization
13	Confidence-score variation	Tests low-confidence handling

How to build your own:

List every behavior the code handles. Use comments, recent bugs, user reports.
For each, write the smallest input that exercises that behavior.
Add 2–3 “random real-world” inputs that mix multiple behaviors.

Store fixtures as data, not as code:

src/processing/__tests__/fixtures/
├── 01-long-pauses.json
├── 02-short-utterance.json
├── 03-dense-fillers.json
...
└── 13-confidence-variation.json

Each JSON file is structured the same way — input only:

{
  "description": "Long pauses (>3s) between major segments",
  "input": {
    "segments": [
      { "text": "First idea.", "start": 0.0, "end": 1.2 },
      { "text": "Second idea.", "start": 4.5, "end": 6.0 }
    ]
  }
}

Step 2: Snapshot the Outputs

Run the existing code over each fixture. Capture the output as a snapshot.

In Vitest, this is built-in:

import { describe, it, expect } from 'vitest';
import { format } from '../NaturalFormatter';
import fixtures from './fixtures';

describe('formatter characterization', () => {
  for (const fixture of fixtures) {
    it(fixture.description, () => {
      const result = format(fixture.input, { preset: 'BALANCED' });
      expect(result).toMatchSnapshot();
    });
  }
});

On first run, Vitest writes the snapshots to __snapshots__/:

src/processing/__tests__/__snapshots__/
└── formatter-characterization.test.ts.snap

Inside the .snap file, each test stores its actual output. From now on, every test run asserts the current output matches the stored snapshot.

Step 3: Commit the Snapshots

The snapshots are part of your codebase. Check them into git. They’re now the “golden files.”

Commit message: test: add characterization snapshots for NaturalFormatter (13 scenarios).

This is the moment your safety net activates. Any future change to the formatter that alters output triggers a test failure on every CI run.

Step 4: Run the Same Tests on the Ported/Refactored Version

If you’re porting code (TS → Rust, or v1 → v2), the move is the same:

Copy the fixtures into the new repo
Implement equivalent tests in the new test runner (cargo test, pytest, etc.)
Run against the new version of the code
Compare snapshots between v1 and v2

If they match: you’ve proven byte-identical output. Ship with confidence.

If they don’t match: you have a specific, localized regression. Fix it before the port is “done.”

We did this for our formatter port. Final result: 32 snapshots across 13 scenarios. All passed in v2. The 4 cases where snapshots initially differed turned out to be:

2 actual bugs in v2 that we fixed
1 “v2 is correct, v1 was wrong” that we deliberately updated
1 v1 quirk we decided to preserve for backwards compatibility

Each decision was conscious. Without the snapshots, all 4 would have shipped as silent regressions.

Step 5: Update Snapshots Intentionally (Not Reflexively)

The most common failure mode of snapshot testing: developers run --update-snapshots whenever a test fails, without checking if the change was intentional.

The discipline:

Snapshot failure: stop. Read the diff. Was this behavior change intentional?
If yes: update the snapshot. Commit with a message explaining why.
If no: it’s a bug. Fix the code, not the snapshot.

The diff format in Vitest is good — additions and deletions are shown like a code diff. You can see exactly what changed.

If you find yourself updating snapshots a lot, your snapshots are too granular. Restructure them to capture stable behavior, not implementation details.

What This Doesn’t Fix

Characterization tests preserve current behavior, including bugs. You’re freezing what the code does today. If there’s a known bug, the snapshot includes it. You have to decide whether to fix-and-update or preserve.
They’re not a substitute for spec-driven tests in greenfield code. Use spec-driven tests when you know what correct behavior is. Use characterization tests when you don’t but need to preserve current behavior.
Snapshot files can get large if you snapshot enormous outputs. Snapshot specific properties or use a serializer to keep files readable.
They don’t tell you why the code does what it does. They just tell you whether the behavior changed. The why still has to come from comments or code reading.
You have to actually read failure diffs. Auto-updating snapshots is the same as having no tests.

Try It This Week

Saturday morning (2 hours):

Pick one module you’re scared to touch
List 10 behaviors it handles (from comments, memory, recent bugs)
Build a 10-fixture corpus exercising those behaviors

Saturday afternoon (2 hours):

Set up the snapshot test (~10 lines of Vitest config)
Run it once — Vitest generates the snapshots
Read the snapshot file. Spot-check that the outputs make sense.

Sunday morning (1 hour):

Commit the snapshots
Push to CI; make sure it’s wired up
Schedule yourself 30 minutes to add 5 more edge-case fixtures over the next week

Next time you touch that module:

Run the tests first
Make your change
If snapshots fail, look at the diff and decide: intentional or bug?

That’s the discipline. The cost is ~4 hours upfront and ~5 minutes per change to check snapshots. The benefit is touching scary code with confidence.

The Takeaway

Most solo builders avoid touching their own scary code because they have no safety net. They lose productivity slowly: every refactor takes 3× longer because they’re working scared. Every port is risky because regressions are silent. Every “minor” change carries hidden risk.

Characterization tests change the calculus:

Inputs as data, not as code
Outputs as snapshots, not as assertions
Equality as the test, not specifications
Safety net as a deliverable, not a side effect

The pattern is older than most testing frameworks. It works in every language. Vitest, pytest, Jest, Cargo Insta, Go’s cupaloy, RSpec — all have built-in snapshot testing.

If you have code you’ve been afraid to touch, this is the weekend to fix that. Four hours to a safety net. Years of productivity unlocked.

The 1,722 lines of TypeScript we preserved in our Rust port (see post #139) were only safe to preserve because we had 32 characterization snapshots verifying byte-identical output. Without them, the “keep verbatim” decision would have been blind. With them, it was deliberate.

That’s the difference between solo builders who can refactor and those who can’t.

Coming up next: post #141 — State machines beat boolean flags. Once your scary code is safety-netted, the next problem is usually app state — the kind of thing that breeds boolean flags and console.log archaeology. Here’s the Rust enum pattern that makes invalid state impossible to represent.