How to test AI-generated code

A green test is useful only if it checks the behavior you intended. Start by deciding whether you are specifying something new, capturing a bug, or fixing the test environment. These are different jobs, even when an agent proposes the same request to add more tests.

Which problem do you have?

You can state the expected behavior before implementing it.

test-driven development. Use test-driven development. Ask for one assertion that fails for the missing behavior, then implementation and refactoring. The term page includes a runnable before-and-after example.

See the failure and fix Download teaching example

A bug report is too vague to act on.

minimal reproduction. Record the smallest input with explicit expected and actual results. The runnable reproduction reduces a disappearing list item to one value and one boundary assertion.

See the failure and fix Download teaching example

You need to trigger an external service failure reliably.

test double. Substitute only that boundary with a controlled test double, then run the real recovery handler. The runnable price-loader example checks timeout recovery, success, and unexpected errors. Keep separate coverage of the real integration.

See the failure and fix Download teaching example

Tests pass individually but fail together.

test isolation. Investigate shared state, storage, clocks, and cleanup. The runnable cart example reproduces the mismatch, then checks independent fixtures in both orders. A delay or retry may hide interference without removing it.

See the failure and fix Download teaching example

Run a test that catches the wrong behavior

An agent builds a URL slug function. The requirement is to trim surrounding spaces, lowercase the words, and join whitespace-separated words with a hyphen. Start with that observable behavior before asking for an implementation.

Download the runnable JavaScript example and run node --test tdd.mjs with Node.js 22.12 or later. It compares an unchanged input with a trimmed, lowercase slug.

import test from 'node:test';
import assert from 'node:assert/strict';

function before(text) {
  return text;
}

function after(text) {
  return text.trim().toLowerCase().replace(/\s+/g, '-');
}

test('the initial implementation misses the requirement', () => {
  assert.throws(() => {
    assert.equal(before('  Hello   world  '), 'hello-world');
  }, { code: 'ERR_ASSERTION' });
});

test('the implementation satisfies the chosen examples', () => {
  assert.equal(after('  Hello   world  '), 'hello-world');
  assert.equal(after('One'), 'one');
  assert.equal(after(''), '');
});

For the input ' Hello world ', the initial function returns the unchanged string; the revised function returns 'hello-world'. The comparison file checks that the first assertion fails and the revised examples pass. To experience the red step directly, remove the assert.throws wrapper and run that assertion before changing the function.

This demonstrates a small, specified behavior. It does not settle Unicode normalization, punctuation, uniqueness, or URL policy, and passing examples do not prove that the whole product is correct. Add each relevant rule deliberately. TDD includes refactoring after green; writing tests after the implementation is a different workflow. Read the full failure and fix.

What to ask the agent to verify

Write down the observable result and the boundary under test. Run the relevant check against the broken behavior, make the smallest necessary change, and run it again. Add integration or end-to-end coverage when the claim depends on components working together. Use the term that describes the actual uncertainty.

Keep the claim bounded

The TDD example demonstrates a string transformation. It does not measure whether a particular model writes better software, and a test double cannot establish that a live provider behaves the same way.

Sources

Updated and sources checked 2026-10-14. How we check content.