# How to end-to-end test a CLI against a mock API

> Build the CLI binary, point it at a scripted mock through two environment variables, and assert on its output, its exit codes, and the requests the mock recorded.

Source: https://voxgig.com/howto/e2e-test-a-cli-against-a-mock-api

- Audience: api-producer
- Level: intermediate
- Languages: go, typescript
- Verified: 2026-11-19
- Published: 2026-11-19

## Short answer

Build the binary your users get, here the `go-cli` target that sdkgen generates, and run it as a child process against a mock API on `127.0.0.1`. Inject the base URL and the key through environment variables, as `FERRY_BASE` and `FERRY_APIKEY`. Assert on stdout, stderr, the exit code, and the requests the mock recorded. Script stubs for a second page, a 404, and a 429 with `Retry-After`, because a mock that serves one example per operation never reaches those paths.

---
## You will need

Node 22.18 or later, Go 1.24 or later, and an OpenAPI 3 document for the API. Verified 2026-11-19
against Node 22.23.5, Go 1.24.7, `@voxgig/create-sdkgen` 0.31.9, and `@voxgig/sdkgen` 4.44.3. The
document on this page, `openapi.json`, describes a ferry operator's API behind a bearer token: a
paged list of sailings, one sailing by id, and a `PATCH` that changes its seats. Normalizing a
transcript before comparing it is covered in
[snapshot-test CLI output across operating systems](/howto/snapshot-test-cli-output-across-operating-systems).

## Approaches compared

| Approach | When it fits | What it costs you | When to pick something else |
| --- | --- | --- | --- |
| [oapi-codegen strict server](https://github.com/oapi-codegen/oapi-codegen#strict-server) | A Go CLI on an oapi-codegen client, with a fake whose response types come from the document | Every stub written in Go, and no request validation until you add the middleware for it | The CLI is not Go, or nobody will maintain a fake |
| [orval MSW handlers](https://orval.dev/docs/guides/msw/) | A TypeScript CLI on an orval client, with handlers generated from the document's examples | A second package to put the handlers on a port, and no check of a credential or a body | The tests need the mock to refuse a bad request |
| [Prism](https://github.com/stoplightio/prism/blob/master/docs/guides/01-mocking.md) | Responses and request validation straight from the document, with no stub code | One example per operation unless the request carries a selector, which a user's command never does | The test is about a second page or a retry |
| [sdkgen test feature](https://github.com/voxgig/sdkgen/blob/main/docs/reference/features.md#test) | An sdkgen CLI, when the test is about its words and its printing | Nothing is sent and the mock ignores the URL and headers, so a wrong base URL or missing key passes | The defect you fear is in the HTTP layer |
| [WireMock](https://wiremock.org/docs/stubbing/) | Scripted sequences and faults for a CLI in any language, with stubs a team can share | Stub mappings you write or record, which name no document to check against, plus a JVM or container in CI | You want the document to stay the single source |

Voxgig maintains sdkgen. It is one of five options here, not the recommendation.

The rows differ in who writes the stubs and what holds them to the document. Prism writes none and
validates every request, but it serves one example per operation. WireMock serves whatever you
script, including a response the document never declared. oapi-codegen puts the compiler between
the fake and the document, and the price is a fake written in Go. The steps below script a
`node:http` stand-in, the same kind of mock as WireMock with nothing to install.

## Build the binary and hand it the mock's address

The CLI under test is the [`go-cli` target](https://github.com/voxgig/sdkgen/blob/main/docs/how-to/add-a-target.md),
a Go program that wraps the Go SDK generated beside it. The scaffold and the generation ran once,
and their output is under `generated/`.

```bash title="generate.sh"
npx --yes @voxgig/create-sdkgen@0.31.9 ferry --def openapi.json --target go --target go-cli
(cd ferry-sdk/.sdk && npm run generate)
```

The generated `main.go` reads two environment variables, `FERRY_APIKEY` and `FERRY_BASE`, and has
no flag for either. Left unset, `FERRY_BASE` falls back to the server in the document,[^1] so a
test that forgets it calls the real host. The harness builds the binary with `go build` on every
run, then starts it with a closed environment:

```js title="harness.mjs"
export function runCli(bin, args, env = {}) {
  return new Promise((resolve, reject) => {
    const systemRoot = process.env.SystemRoot ? { SystemRoot: process.env.SystemRoot } : {}
    const child = spawn(bin, args, { env: { ...systemRoot, ...env }, stdio: ['ignore', 'pipe', 'pipe'] })
    let stdout = ''
    let stderr = ''
    child.stdout.on('data', (chunk) => { stdout += chunk })
    child.stderr.on('data', (chunk) => { stderr += chunk })
    child.on('error', reject)
    child.on('close', (code, signal) => resolve({ code, signal, stdout: sortKeys(stdout), stderr }))
  })
}
```

The child gets only the variables the test passes, so a key exported in a developer's shell
cannot reach it. [`spawn`](https://nodejs.org/api/child_process.html#child_processspawncommand-args-options)
returns at once, which leaves the event loop free to run a mock in the same process.

## Script a second page, a 404, and a 429

The mock is a `node:http` server on port 0, which the operating system replaces with a free port.
It checks the bearer token, records every request, and holds three stubs. The page number decides
which rows come back:

```js title="mock-api.mjs"
// The stub for a second page: the page number decides which rows come back.
const page = Number(url.searchParams.get('page') ?? '1')
if (!Number.isInteger(page) || page < 1) return reply(400, problem('bad_request', 'page must be a positive integer'))
return reply(200, sailings.slice((page - 1) * pageSize, page * pageSize))
```

An id it does not hold gets a 404, and `mock.once(tooManyRequests(1))` queues a 429 with
`Retry-After: 1` for the next request.[^2] `e2e-demo.mjs` runs six commands and prints the log:

```bash
node e2e-demo.mjs
```

```text output
ferry-cli list sailing
  exit 0  [{id:'sl_101' route:'DUB-HOL' seats:120 vessel:'Ulysses'} {id:'sl_102' route:'ROS-FIS' seats:64 vessel:'Isle of Inishmore'}]
ferry-cli list {page:2} sailing
  exit 0  [{id:'sl_103' route:'DUB-CHE' seats:8 vessel:'W. B. Yeats'}]
ferry-cli load {id:"sl_999"} sailing
  exit 1  error: FerrySDK: load: request: 404: Not Found
ferry-cli list sailing, after the mock is told to answer 429 once
  exit 1  error: FerrySDK: list: request: 429: Too Many Requests
ferry-cli update {id:"sl_101",seats:118} sailing
  exit 0  {id:'sl_101' route:'DUB-HOL' seats:118 vessel:'Ulysses'}
ferry-cli list sailing, with FERRY_APIKEY unset
  exit 1  error: FerrySDK: list: request: 401: Unauthorized

requests the mock recorded:
  GET   /v1/sailings             Bearer sk_test_ferry     200
  GET   /v1/sailings?page=2      Bearer sk_test_ferry     200
  GET   /v1/sailings/sl_999      Bearer sk_test_ferry     404
  GET   /v1/sailings             Bearer sk_test_ferry     429, Retry-After: 1
  PATCH /v1/sailings/sl_101      Bearer sk_test_ferry     200
  GET   /v1/sailings             no authorization header  401
```

The second page went out as `?page=2` and came back as `sl_103` alone. The missing id and the 429
each exit 1 with the status on stderr. The log shows one request for the 429: the generated
`main.go` turns no retry feature on, so the binary does not wait and try again. Every failure
shares exit code 1, so a test that must tell them apart reads stderr.

## Run the same commands against a one-example mock

Prism serves the same document with no stub written. `prism-check.mjs` starts it on a free port
and repeats five of the commands:

```bash
node prism-check.mjs
```

```text output
Prism 5.14.2 serving openapi.json, with no stub written

ferry-cli list sailing
  exit 0  [{id:'sl_101' route:'DUB-HOL' seats:120 vessel:'Ulysses'} {id:'sl_102' route:'ROS-FIS' seats:64 vessel:'Isle of Inishmore'}]
ferry-cli list {page:2} sailing
  exit 0  [{id:'sl_101' route:'DUB-HOL' seats:120 vessel:'Ulysses'} {id:'sl_102' route:'ROS-FIS' seats:64 vessel:'Isle of Inishmore'}]
ferry-cli load {id:"sl_999"} sailing
  exit 0  {id:'sl_101' route:'DUB-HOL' seats:120 vessel:'Ulysses'}
ferry-cli update {id:"sl_101",seats:118} sailing
  exit 1  error: FerrySDK: update: request: 422: Unprocessable Entity
ferry-cli list sailing, with FERRY_APIKEY unset
  exit 1  error: FerrySDK: list: request: 401: Unauthorized
```

Page 2 is page 1, and the missing id exits 0 with the example record. Prism answers with the
lowest 2XX response and its example unless the request selects another, through a `Prefer` header
or a [`__code` query parameter](https://github.com/stoplightio/prism/blob/master/docs/getting-started/03-cli.md).[^3]
A test can add `{__code:404}` to the command, but then it tests a command no user types.

Prism also caught what the scripted mock let through. The document reuses `Sailing`, all four
fields required, as the `PATCH` body, and the CLI sends the id and the one field that changed. The
fix belongs in the document: give the operation a body schema with no required fields.

## Test the words in process when HTTP is not the question

The Go SDK under the CLI carries a test mode, an in-memory mock served from seed data. The built
binary has no switch for it. The way in is a Go test in the `go-cli` package that hands the CLI's
own words a client from `sdk.TestSDK`, and a regeneration left that file in place.

```go title="generated/go-cli/words_inprocess_test.go"
// The seed has no pages. The mock compares only what the route requires, and page is optional.
code, ids, _ = eval(t, client, "list {page:2} sailing")
t.Logf("list {page:2} sailing      exit %d, ids %s", code, ids)
if code != 0 || ids != "sl_101 sl_102" {
	t.Fatalf("list {page:2} sailing: exit %d, ids %q", code, ids)
}
```

```bash
go test -C generated/go-cli -run TestWordsOverTestMode -v .
```

```text output
=== RUN   TestWordsOverTestMode
    words_inprocess_test.go:48: client mode: test
    words_inprocess_test.go:54: list sailing               exit 0, ids sl_101 sl_102
    words_inprocess_test.go:61: list {page:2} sailing      exit 0, ids sl_101 sl_102
    words_inprocess_test.go:67: load {id:"sl_999"} sailing exit 1, error: FerrySDK: load: request: 404: Not found
```

It needs no port and no binary. It also answers page 2
with both seeded records. The mock transport ignores the URL and the headers the client hands it,
so a wrong base URL or a missing key passes.

## Mock the API behind an oapi-codegen or orval client

oapi-codegen generates a typed Go client and no CLI, so the binary is your own `main.go`: pass the
base URL to `NewClientWithResponses` and set the key in a `RequestEditorFn`. It generates no mock
either. With `std-http-server` and `strict-server` set to `true`, it generates a server interface
from the same document. A fake behind it returns generated response types, which exist only for
the statuses the document declares:

```go title="oapi/fake.go"
if f.throttle > 0 {
	f.throttle--
	wait := 1
	return ferryapi.ListSailings429JSONResponse{TooManyRequestsJSONResponse: ferryapi.TooManyRequestsJSONResponse{
		Body:    ferryapi.Problem{Code: "rate_limited", Message: "Slow down"},
		Headers: ferryapi.TooManyRequestsResponseHeaders{RetryAfter: &wait},
	}}, nil
}
```

[`httptest.NewServer`](https://pkg.go.dev/net/http/httptest#NewServer) puts the fake on a loopback
port and `exec.Command` runs the binary against it, as `oapi/fake_test.go` does. The strict server
decodes a request body but checks it against no schema, and checks no credential. The README names a
[validation middleware](https://github.com/oapi-codegen/oapi-codegen#requestresponse-validation-middleware)
for both.

orval's [`mock` option](https://orval.dev/docs/reference/configuration/output/) writes an MSW
handler per operation, seeded from the document's examples when `useExamples` is on.
[`setupServer`](https://mswjs.io/guides/integrations/node) mocks the process it runs in, so a CLI
in a child process never meets those handlers. `createServer` from
[`@msw/serve`](https://github.com/mswjs/serve) puts them on a port:

```js title="orval/serve.mjs"
const server = createServer(
  // A 429 with the header the generated 429 handler leaves out.
  http.get('*/sailings', () => {
    if (throttled > 0) {
      throttled--
      return HttpResponse.json(getListSailingsResponseMock429(), { status: 429, headers: { 'retry-after': '1' } })
    }
  }),
  // A second page, through the generated handler's override argument.
  getListSailingsMockHandler(({ request }) =>
    new URL(request.url).searchParams.get('page') === '2' ? PAGE_TWO : getListSailingsResponseMock()),
  // A missing record, with the body the document's 404 example gave the generated factory.
  http.get('*/sailings/sl_999', () => HttpResponse.json(getGetSailingResponseMock404(), { status: 404 })),
  ...getFerrylineAPIMock(),
)
```

MSW answers with the first handler that returns a response, so the scripted stubs sit ahead of
the generated ones. The generated handlers check no credential.

## Check it worked

Run the suite:

```bash
node --test e2e.test.mjs
```

```text output
1..14
# tests 14
# suites 0
# pass 14
# fail 0
# cancelled 0
# skipped 0
# todo 0
```

`# pass 14` is the line to read. Nine tests drive the binary against the scripted mock, and one
each covers the key sorter, Prism, the in-process Go test, the oapi-codegen fake, and the orval
handlers.

## When it goes wrong

Every test times out. The test called `spawnSync` while the mock ran in the same process, and a
blocked event loop cannot accept the child's connection. Use `spawn`, or run the mock in a process
of its own.

```bash
node pitfall.mjs
```

```text output
spawnSync, with the mock in the same process:
  status null, signal SIGTERM, error ETIMEDOUT
  requests the mock had answered when it returned: 0
spawn, with a mock in the same process:
  exit 0, ids sl_101 sl_102
  requests the mock answered: 1
```

A stdout assertion passes on one run and fails on the next. The CLI prints each record as a Go
map, and the [Go specification](https://go.dev/ref/spec#For_statements) leaves the iteration order
of a map unspecified. Sort the keys inside each record before comparing, as `sortKeys` does.

A test fails with `no such host`. `FERRY_BASE` was unset, so the binary called the server in the
document, a reserved name here and the real API in your project. Assert on the mock's request log
in every test.

## When not to do this

Do not make every CLI test end to end. Each one costs a build and a process, and argument parsing
needs no socket. Keep a dozen for what crosses the process boundary: the environment, the exit
code, the wire.

Do not let sdkgen's test mode stand in for the binary. It is the fastest check of the generated
words, but its mock ignores the URL and headers, so a dropped header passes it. The sdkgen
documentation describes no mock that listens on a port, so that one is yours to supply.

Do not hand-script a mock when a typed one is on offer. For a Go CLI on an oapi-codegen client,
the strict server fake is the better choice. A stub for a status the document lacks has no type to
return, so it fails to compile. For a TypeScript CLI on an orval client, the generated handlers
start from the document's examples, and you write only the stubs that differ.

Do not read a green run against Prism as coverage of the error paths.

## Related how-tos

- [Snapshot-test CLI output across operating systems](/howto/snapshot-test-cli-output-across-operating-systems)

- [Generate a REPL from an OpenAPI description](/howto/generated-repl-from-an-openapi-description)

- [Serve your OpenAPI examples as mock responses](/howto/serve-openapi-examples-as-mock-responses)

- [Control which stub wins when mock rules overlap](/howto/control-which-stub-wins-when-rules-overlap)

- [Run an in-memory mock of your API inside SDK tests](/howto/run-an-in-memory-mock-inside-sdk-tests)

- [Test interactive prompts and terminal behavior in a CLI](/howto/test-interactive-prompts-and-tty-behaviour)

## Last verified

Verified 2026-11-19 against Node 22.23.5, Go 1.24.7, `@voxgig/create-sdkgen` 0.31.9,
`@voxgig/sdkgen` 4.44.3, Prism 5.14.2, oapi-codegen v2.8.2, orval 8.48.0, MSW 3.0.7, and
`@msw/serve` 0.11.0. Every output block is what the command preceding it printed. sdkgen,
oapi-codegen, and orval each ran once on `openapi.json`; their output is committed, and the checks
build and drive it without generating again. The harness sorts the keys inside each printed
record. WireMock was not run; its row describes its documentation. No request left the loopback
interface.

[^1]: The server in this document is `api.ferryline.example`, a name no resolver will find.
[RFC 2606](https://www.rfc-editor.org/rfc/rfc2606.html) reserved four top-level names in June 1999
and recommends `.example` for use in documentation or as examples. A test that forgets the base
URL fails at the resolver here, with `no such host`. With a real host in the document, the same
forgetful test reaches production.

[^2]: The status code is younger than the habit of sending it. [RFC 6585](https://www.rfc-editor.org/rfc/rfc6585.html#section-4)
defined 429 in April 2012 and says a response may include a `Retry-After` header, which makes the
header optional. The same section declines to define how a server identifies the user or counts
the requests. A client that waits as told is honoring a header the server never had to send.

[^3]: `Prefer` is a standard header put to a private use. [RFC 7240](https://www.rfc-editor.org/rfc/rfc7240.html)
defined it in June 2014 for preferences such as `return=minimal` and `respond-async`, and the
[IANA registry](https://www.iana.org/assignments/http-parameters#preferences)
it created lists those with `wait`, `handling`, and later entries such as `safe`. Prism's `code`,
`example`, and `dynamic` are not in it. None of the four preferences the RFC defines chooses a status code.