# How to verify every error response matches one schema

> Prove that no endpoint can return an error body outside your problem schema, using a lint over the description and a runtime check against the running service.

Source: https://voxgig.com/howto/verify-every-error-response-matches-one-schema

- Audience: api-producer
- Level: intermediate
- Languages: typescript, javascript
- Verified: 2026-09-14
- Published: 2026-09-14
- Updated: 2026-09-24

## Short answer

Run two checks. A lint over the description fails when a declared 4XX or 5XX carries anything but `application/problem+json`, which is cheap and catches the paperwork. A runtime check calls each failing path and compares the status, the media type, and the required members. Only that half catches a status the service returns and the description never mentions.

---
## You will need

An OpenAPI 3 description with a problem schema, and Node 22 or later. The media type and the
required members come from [RFC 9457](https://www.rfc-editor.org/rfc/rfc9457.html), and the
response declarations live in the
[Responses Object](https://spec.openapis.org/oas/v3.1.0.html#responses-object).

## Approaches compared

| Approach | When it fits | What it costs you | When to pick something else |
| --- | --- | --- | --- |
| [A lint over the description](https://spec.openapis.org/oas/v3.1.0.html#response-object) | Every repository, because it runs in a second with no service running | It reads what you declared, so a status you never declared is invisible to it | The gap is between the description and the service |
| [Hand-written cases in your test suite](https://nodejs.org/api/test.html) | The failures you care most about, where the code and the text both matter | One case per failure, written and maintained by hand | You want coverage of paths nobody thought to write a case for |
| [Schemathesis](https://schemathesis.readthedocs.io/en/stable/) | You want generated requests to find the statuses you did not expect | A Python tool in the pipeline, and runs long enough to need their own job | The API is small and the failure paths are all known |
| [Spectral](https://docs.stoplight.io/docs/spectral/674b27b261c3c-overview) | You already lint the description and want this rule beside the others | A rule syntax to learn, and a second tool where a script might do | The rule needs logic the core functions cannot express |

Each of these sees a different part of the problem. The lint reads your intent. The fuzzer finds
statuses your intent never covered. Hand-written cases are the only ones that check whether the
`type` URI and the `title` say something a client can act on. Two of the three are enough, and the lint should be one of them
because it costs nothing. Order them by how cheap they are to run: the lint on every commit, the
hand-written cases with the rest of the suite, the fuzzer on a schedule that opens an issue.

## Lint what the description declares

The rule is short, and the second half of it matters more than the first.

```js title="lint.mjs"
for (const [status, response] of Object.entries(responses)) {
  if (!FAILURE.test(status)) continue
  const types = Object.keys(response.content ?? {})
  if (!types.includes('application/problem+json')) {
    findings.push(`${verb.toUpperCase()} ${path} ${status}: declares ${types.join(', ') || 'no body'}, not application/problem+json`)
  }
}
```

A `default` response is the trap.[^2] It makes every status conform on paper, because any code the
service returns is covered by something. Declare each status you actually return, and keep
`default` for the genuinely unexpected. A description whose only failure entry is `default` tells a
client generator nothing, and the generated client will have no typed errors at all.

Keep the list of statuses somewhere the service and the description both read from. A constant in
the code, checked by the lint, is enough. Two hand-maintained lists drift in the same week they are
written.

## Check what the service returns

The runtime half needs a running service and three assertions per call.

```js title="conform.mjs"
if (!declared[status]) findings.push(`${status} is not declared, only default covers it`)
if (type !== 'application/problem+json') findings.push(`content type is ${type}`)
```

Check the media type separately from the body. A route that returns the right JSON under
`application/json` passes any check that only parses the body. It still breaks a client that
switches on the media type before deciding how to read the payload.

Drive the check from a list of failing requests you maintain beside the description. That list is also the
answer when a security review asks which failures are tested.

## Check it worked

Run both halves over one description and one service.

```bash
node demo.mjs
```

```text output
-- static: what the description declares
  GET /meters/{id} 404: declares application/json, not application/problem+json
  GET /meters/{id} 503: returned by the service, declared only by default
-- runtime: what the service actually returns
  /meters?limit=none     400  conforms
  /meters?burst=yes      429  conforms
  /meters/missing        404  content type is application/json
  /meters/offline        503  503 is not declared, only default covers it; content type is text/plain; body is not JSON
```

The two halves overlap on the 404 and diverge on the 503. The lint sees the 404 because the
description is wrong about it. The runtime check sees the 503 fully, because only a real call
reveals a plain text body where a problem document was promised.[^3] Neither check alone covers both.

The two conforming rows matter as much as the failing ones. They show the checks can pass, which is
the property a gate needs before anyone will trust its failures.

```bash
node --test conform.test.mjs
```

```text output
1..6
# tests 6
# suites 0
# pass 6
# fail 0
# cancelled 0
# skipped 0
# todo 0
# duration_ms 279.939139
```

## When it goes wrong

The lint passes, but clients still see odd bodies. A framework is handling an error before your code
does, and returning its own default page. Add a catch-all that converts anything unhandled into a
problem document, and test it by throwing on purpose. A route that throws a string rather than an
Error is the usual way past a handler that checks the type first.

A gateway rewrites the body. Timeouts and rate limits answered by a proxy carry the proxy's format,
not yours. Configure the error response at that layer too, or document the difference.

Every status conforms and the fields are useless. The schema requires `type`, `title` and `status`,
and all three are present and generic. Assert on the values for the failures you care about, not
only on the shape. Check the declared types while you are there: a body of `{"type": 42, "title":
null, "status": "400"}` carries every required name, and a check that counts keys calls it
conforming. That is the exact body a generated client's error handler breaks on.

A generated client has one error class. The description hides every failure behind `default`.
Declare the statuses, and the generator will give you the types.

The checks pass in CI and fail in production. The service behind the load balancer is not the one
the tests started, and a layer between them is answering some failures itself. Run the conformance
list against a deployed environment once per release, not only against a local process.

## When not to do this

Do not fail a deploy on a fuzzer run. Generated requests find real problems and they also find
paths that are slow, flaky, or dependent on data. Run the fuzzer on a schedule, open an issue, and
keep the deploy gate on the lint and the hand-written cases.

Do not put the conformance check behind an integration environment you rarely run. A check that
runs weekly tells you which week broke it. Run it against a service started by the test process.

Do not treat a problem document as a place for a stack trace. The members are for a client, and
`detail` reaching a caller with internal paths in it is an information leak wearing a schema.

## Related how-tos

- [Return validation errors a client can map to a form](/howto/field-level-validation-errors-clients-can-map)

- [Validate responses against the OpenAPI document at runtime](/howto/validate-responses-against-the-openapi-document)

## Last verified

Verified 2026-09-14 against Node 22.22.2. Both output blocks are what the preceding command
printed.

[^1]: [RFC 9457](https://www.rfc-editor.org/rfc/rfc9457.html#section-3.1.1) lists five members a
problem details object can have, and says that when `type` is absent its value is assumed to be
`about:blank`, from the URI scheme [RFC 6694](https://www.rfc-editor.org/rfc/rfc6694.html)
describes. Used as a problem type, that URI means the problem has no semantics beyond the status
code. The document created a [registry](https://www.iana.org/assignments/http-problem-types) of
problem types and populated it with that one entry. The registry has since grown to six, two of
them from Oblivious HTTP and three about digest fields, and the one every document falls back to
means nothing in particular.

[^2]: The [Responses Object](https://spec.openapis.org/oas/v3.1.0.html#responses-object) describes
`default` as documentation of responses other than the ones declared for specific codes, to be
used to cover undeclared responses. That is the page's complaint stated as a feature. The same
section requires every status key to be enclosed in quotation marks, for compatibility between
JSON and YAML. A range is allowed only as `1XX` through `5XX`, with the wildcard in uppercase, and
an explicit code takes precedence over the range it falls in. Five ranges, one wildcard and a rule
about the case of a letter, all to say which responses a description has not described.

[^3]: [RFC 9110](https://www.rfc-editor.org/rfc/rfc9110.html#section-15.6.4) defines 503 as a
temporary overload or scheduled maintenance, which will likely be alleviated after some delay, and
lets the server send `Retry-After` to say how long. The permission is a MAY. A note in the same
section says a server becoming overloaded is under no obligation to use 503 at all, and might
refuse the connection instead. An agent retrying a plain-text 503 forever is doing what the status
invites, for as long as it is invited, and the body has no say.