Keep a mock honest as the API changes

Most mocking tools solve the first day and then rot. The mock is written once, the real API moves, and your tests keep passing against a fiction — which is worse than having no mock, because now you trust it.

This is the guide to the part that fixes that.

Two oracles, not one

apifae diff asks two independent questions every time it runs:

• Drift — the live API no longer matches your mock.
• Violation — the live API no longer matches your spec.

They are genuinely independent, and that is the whole design. A checker that only compares against mocks gives a false all-clear the moment someone updates a mock to match production without checking whether production was allowed to change.

You will see both in the run below: one endpoint drifts without violating, and one violates without drifting.

Try it for real

The manual ships a deterministic backend so you can watch this happen rather than read about it. From a workspace built as in Getting started, download backend.js and run it:

curl -fsSL https://apifae.com/manual/fixtures/backend.js -o backend.js
node backend.js

It needs Node and nothing else — no dependencies to install. Working from a clone of the repository, node docs/manual/fixtures/backend.js is the same file.

It implements the fixture spec faithfully, with no timestamps and no random ids, so anything reported is a real difference and not noise.

Nothing wrong — the gate stays quiet

$ apifae diff http://localhost:9100
Comparing against http://localhost:9100

GET /orders
  ✓ No drift

POST /orders
  ✓ No drift

GET /orders/{id}
  [default]
    ✓ No drift
  [scenario=getOrder-404]
    ⚠ Skipped — it belongs to scenario `getOrder-404` — a state of the mock, not a variant of the live API

GET /reports/daily
  ✓ No drift

Summary: 4 checked, 4 unchanged, 1 variant skipped
EXIT: 0

/orders/{id} answers two ways, so diff probes each one and labels it. The skipped one is the response tagged scenario: getOrder-404: a scenario is a state you put the mock into, not something the live API can be asked for, so there is nothing to compare it against. Skipping it is not a failure and does not affect the exit code.

Something changed

Restart the backend with drift switched on:

DRIFT=1 node backend.js

$ apifae diff http://localhost:9100
Comparing against http://localhost:9100

GET /orders
  ✗ Violates the OpenAPI spec:
    ! body/data/0/note: expected string, got null
    ! body/data/1/note: expected string, got null
    ! body/data/2/note: expected string, got null
  ✗ Drift detected:
    ~ $.data[*].note: String → Null (3 elements)

POST /orders
  ✗ Violates the OpenAPI spec:
    ! status 200 is not declared by the spec (declared: 201)
  ✗ Drift detected:
    ~ status: 201 → 200

GET /orders/{id}
  [default]
    ✗ Violates the OpenAPI spec:
      ! body/status: "refunded" is not one of ["pending", "paid", "shipped"]
  [scenario=getOrder-404]
    ⚠ Skipped — it belongs to scenario `getOrder-404` — a state of the mock, not a variant of the live API

GET /reports/daily
  ✗ Drift detected:
    + $.errorCode (String)

Summary: 4 checked, 3 drifted, 3 violating, 1 variant skipped
EXIT: 1

Four different shapes of change, and the two oracles pulling apart:

The last two rows are the argument.

/reports/daily drifted without violating: the spec does not forbid extra fields, so a strict spec-checker would have passed this. Your mock still needs updating.

/orders/{id} violates without drifting: the mock had already been updated to match production, so mock-versus-live agrees. Only the spec knows that refunded was never a legal status. A tool with one oracle reports nothing here, and that is exactly the failure that ships a client which cannot handle refunded.

As a CI gate

The exit codes are designed for this:

Separating 1 from 2 is what makes the gate trustworthy. A network problem must not read as a passing contract, and it does not.

# .gitlab-ci.yml
contract:
  script:
    - apifae diff "$STAGING_URL"

# .github/workflows/contract.yml
      - run: apifae diff "${{ secrets.STAGING_URL }}"

Either fails the job on 1 or 2 and passes on 0, with no extra scripting.

Endpoints and response variants that could not be checked are reported but do not fail the run — pass --fail-on-skip if you would rather they did. --json gives per-endpoint results with typed diff entries, for a bot that wants to comment on the MR rather than just go red.

Intermittent drift

A canary, one bad node, a partial rollout: a single request misses it entirely, and the default is one request.

FLAKY_EVERY=5 node backend.js

$ apifae diff http://localhost:9100 --samples 20

GET /orders
  ⚠ 2 distinct response shapes over 20 requests:
    · 16 of 20 (80%) — status 200, no drift
    · 4 of 20 (20%) — status 200, 1 finding(s)
  ✗ Violates the OpenAPI spec:
    ! body/data/0/status: "refunded" is not one of ["pending", "paid", "shipped"]

Reach for --samples when a contract test has ever failed and then passed on retry. That is the signature of exactly this, and one sample cannot see it.

Applying what it found

patch writes the drift into your mocks.

What makes it safe to point at production is that it edits only the fields that actually changed, in place. Comments, quoting and key order all survive, so the resulting commit shows the drift and nothing else:

       error: "report_failed"        # stable machine-readable code
       message: "Report worker is not available"
       createdAt: "2026-08-01T09:15:00Z"
+      errorCode: RPT-500

That is a reviewable change. A rewriter that reformatted the file would bury one real line in fifty cosmetic ones, and nobody would read it.

Two behaviours worth knowing before you run it:

It refuses to write what the spec forbids.

$ apifae patch http://localhost:9100
POST /orders
  ~ status → 200
  ⚠ not applied — writing this would violate the OpenAPI spec
    (status 200 is not declared by the spec (declared: 201));
    pass --accept-violations to write it anyway

The default is the right one: if production has started doing something the contract forbids, silently teaching your mock to expect it destroys the evidence. Fix the spec, or pass --accept-violations deliberately.

Nothing is lost. --dry-run changes nothing on disk, and every applied change leaves a timestamped .bak beside the file.

The loop

apifae diff <live>     what changed?
apifae patch <live>    write the changes I accept
git diff               review them as a normal code change
apifae validate        confirm the result still agrees with the spec

Run the first step in CI and you find out when the API moves, rather than when a test fails for a reason nobody can reproduce.