Your CLI’s tests call its command functions with a stubbed client. Nothing runs the binary you release, so a base URL read from the wrong environment variable ships, and a customer’s first command goes to the wrong host. The mock in the one binary test answers every request with the same example, which leaves the second page, the 404, and the 429 unexercised.
What you get
You will end up with a suite that runs your built CLI against a scripted mock on the loopback interface and asserts its output, exit codes, and requests. This is for you if you ship a CLI that wraps an HTTP API.
Short answer
Build the binary your users get, here the go-cli target that sdkgen generates, and run it as a
child process against a mock API on 127.0.0.1. Inject the base URL and the key through
environment variables, as FERRY_BASE and FERRY_APIKEY. Assert on stdout, stderr, the exit
code, and the requests the mock recorded. Script stubs for a second page, a 404, and a 429 with
Retry-After, because a mock that serves one example per operation never reaches those paths.
You will need
Node 22.18 or later, Go 1.24 or later, and an OpenAPI 3 document for the API. Verified 2026-11-19
against Node 22.23.5, Go 1.24.7, @voxgig/create-sdkgen 0.31.9, and @voxgig/sdkgen 4.44.3. The
document on this page, openapi.json, describes a ferry operator’s API behind a bearer token: a
paged list of sailings, one sailing by id, and a PATCH that changes its seats. Normalizing a
transcript before comparing it is covered in
snapshot-test CLI output across operating systems.
Approaches compared
| Approach | When it fits | What it costs you | When to pick something else |
|---|---|---|---|
| oapi-codegen strict server | A Go CLI on an oapi-codegen client, with a fake whose response types come from the document | Every stub written in Go, and no request validation until you add the middleware for it | The CLI is not Go, or nobody will maintain a fake |
| orval MSW handlers | A TypeScript CLI on an orval client, with handlers generated from the document’s examples | A second package to put the handlers on a port, and no check of a credential or a body | The tests need the mock to refuse a bad request |
| Prism | Responses and request validation straight from the document, with no stub code | One example per operation unless the request carries a selector, which a user’s command never does | The test is about a second page or a retry |
| sdkgen test feature | An sdkgen CLI, when the test is about its words and its printing | Nothing is sent and the mock ignores the URL and headers, so a wrong base URL or missing key passes | The defect you fear is in the HTTP layer |
| WireMock | Scripted sequences and faults for a CLI in any language, with stubs a team can share | Stub mappings you write or record, which name no document to check against, plus a JVM or container in CI | You want the document to stay the single source |
Voxgig maintains sdkgen. It is one of five options here, not the recommendation.
The rows differ in who writes the stubs and what holds them to the document. Prism writes none and
validates every request, but it serves one example per operation. WireMock serves whatever you
script, including a response the document never declared. oapi-codegen puts the compiler between
the fake and the document, and the price is a fake written in Go. The steps below script a
node:http stand-in, the same kind of mock as WireMock with nothing to install.
Build the binary and hand it the mock’s address
The CLI under test is the go-cli target,
a Go program that wraps the Go SDK generated beside it. The scaffold and the generation ran once,
and their output is under generated/.
npx --yes @voxgig/create-sdkgen@0.31.9 ferry --def openapi.json --target go --target go-cli
(cd ferry-sdk/.sdk && npm run generate)
The generated main.go reads two environment variables, FERRY_APIKEY and FERRY_BASE, and has
no flag for either. Left unset, FERRY_BASE falls back to the server in the document,1 so a
test that forgets it calls the real host. The harness builds the binary with go build on every
run, then starts it with a closed environment:
export function runCli(bin, args, env = {}) {
return new Promise((resolve, reject) => {
const systemRoot = process.env.SystemRoot ? { SystemRoot: process.env.SystemRoot } : {}
const child = spawn(bin, args, { env: { ...systemRoot, ...env }, stdio: ['ignore', 'pipe', 'pipe'] })
let stdout = ''
let stderr = ''
child.stdout.on('data', (chunk) => { stdout += chunk })
child.stderr.on('data', (chunk) => { stderr += chunk })
child.on('error', reject)
child.on('close', (code, signal) => resolve({ code, signal, stdout: sortKeys(stdout), stderr }))
})
}
The child gets only the variables the test passes, so a key exported in a developer’s shell
cannot reach it. spawn
returns at once, which leaves the event loop free to run a mock in the same process.
Script a second page, a 404, and a 429
The mock is a node:http server on port 0, which the operating system replaces with a free port.
It checks the bearer token, records every request, and holds three stubs. The page number decides
which rows come back:
// The stub for a second page: the page number decides which rows come back.
const page = Number(url.searchParams.get('page') ?? '1')
if (!Number.isInteger(page) || page < 1) return reply(400, problem('bad_request', 'page must be a positive integer'))
return reply(200, sailings.slice((page - 1) * pageSize, page * pageSize))
An id it does not hold gets a 404, and mock.once(tooManyRequests(1)) queues a 429 with
Retry-After: 1 for the next request.2 e2e-demo.mjs runs six commands and prints the log:
node e2e-demo.mjs
ferry-cli list sailing
exit 0 [{id:'sl_101' route:'DUB-HOL' seats:120 vessel:'Ulysses'} {id:'sl_102' route:'ROS-FIS' seats:64 vessel:'Isle of Inishmore'}]
ferry-cli list {page:2} sailing
exit 0 [{id:'sl_103' route:'DUB-CHE' seats:8 vessel:'W. B. Yeats'}]
ferry-cli load {id:"sl_999"} sailing
exit 1 error: FerrySDK: load: request: 404: Not Found
ferry-cli list sailing, after the mock is told to answer 429 once
exit 1 error: FerrySDK: list: request: 429: Too Many Requests
ferry-cli update {id:"sl_101",seats:118} sailing
exit 0 {id:'sl_101' route:'DUB-HOL' seats:118 vessel:'Ulysses'}
ferry-cli list sailing, with FERRY_APIKEY unset
exit 1 error: FerrySDK: list: request: 401: Unauthorized
requests the mock recorded:
GET /v1/sailings Bearer sk_test_ferry 200
GET /v1/sailings?page=2 Bearer sk_test_ferry 200
GET /v1/sailings/sl_999 Bearer sk_test_ferry 404
GET /v1/sailings Bearer sk_test_ferry 429, Retry-After: 1
PATCH /v1/sailings/sl_101 Bearer sk_test_ferry 200
GET /v1/sailings no authorization header 401
The second page went out as ?page=2 and came back as sl_103 alone. The missing id and the 429
each exit 1 with the status on stderr. The log shows one request for the 429: the generated
main.go turns no retry feature on, so the binary does not wait and try again. Every failure
shares exit code 1, so a test that must tell them apart reads stderr.
Run the same commands against a one-example mock
Prism serves the same document with no stub written. prism-check.mjs starts it on a free port
and repeats five of the commands:
node prism-check.mjs
Prism 5.14.2 serving openapi.json, with no stub written
ferry-cli list sailing
exit 0 [{id:'sl_101' route:'DUB-HOL' seats:120 vessel:'Ulysses'} {id:'sl_102' route:'ROS-FIS' seats:64 vessel:'Isle of Inishmore'}]
ferry-cli list {page:2} sailing
exit 0 [{id:'sl_101' route:'DUB-HOL' seats:120 vessel:'Ulysses'} {id:'sl_102' route:'ROS-FIS' seats:64 vessel:'Isle of Inishmore'}]
ferry-cli load {id:"sl_999"} sailing
exit 0 {id:'sl_101' route:'DUB-HOL' seats:120 vessel:'Ulysses'}
ferry-cli update {id:"sl_101",seats:118} sailing
exit 1 error: FerrySDK: update: request: 422: Unprocessable Entity
ferry-cli list sailing, with FERRY_APIKEY unset
exit 1 error: FerrySDK: list: request: 401: Unauthorized
Page 2 is page 1, and the missing id exits 0 with the example record. Prism answers with the
lowest 2XX response and its example unless the request selects another, through a Prefer header
or a __code query parameter.3
A test can add {__code:404} to the command, but then it tests a command no user types.
Prism also caught what the scripted mock let through. The document reuses Sailing, all four
fields required, as the PATCH body, and the CLI sends the id and the one field that changed. The
fix belongs in the document: give the operation a body schema with no required fields.
Test the words in process when HTTP is not the question
The Go SDK under the CLI carries a test mode, an in-memory mock served from seed data. The built
binary has no switch for it. The way in is a Go test in the go-cli package that hands the CLI’s
own words a client from sdk.TestSDK, and a regeneration left that file in place.
// The seed has no pages. The mock compares only what the route requires, and page is optional.
code, ids, _ = eval(t, client, "list {page:2} sailing")
t.Logf("list {page:2} sailing exit %d, ids %s", code, ids)
if code != 0 || ids != "sl_101 sl_102" {
t.Fatalf("list {page:2} sailing: exit %d, ids %q", code, ids)
}
go test -C generated/go-cli -run TestWordsOverTestMode -v .
=== RUN TestWordsOverTestMode
words_inprocess_test.go:48: client mode: test
words_inprocess_test.go:54: list sailing exit 0, ids sl_101 sl_102
words_inprocess_test.go:61: list {page:2} sailing exit 0, ids sl_101 sl_102
words_inprocess_test.go:67: load {id:"sl_999"} sailing exit 1, error: FerrySDK: load: request: 404: Not found
It needs no port and no binary. It also answers page 2 with both seeded records. The mock transport ignores the URL and the headers the client hands it, so a wrong base URL or a missing key passes.
Mock the API behind an oapi-codegen or orval client
oapi-codegen generates a typed Go client and no CLI, so the binary is your own main.go: pass the
base URL to NewClientWithResponses and set the key in a RequestEditorFn. It generates no mock
either. With std-http-server and strict-server set to true, it generates a server interface
from the same document. A fake behind it returns generated response types, which exist only for
the statuses the document declares:
if f.throttle > 0 {
f.throttle--
wait := 1
return ferryapi.ListSailings429JSONResponse{TooManyRequestsJSONResponse: ferryapi.TooManyRequestsJSONResponse{
Body: ferryapi.Problem{Code: "rate_limited", Message: "Slow down"},
Headers: ferryapi.TooManyRequestsResponseHeaders{RetryAfter: &wait},
}}, nil
}
httptest.NewServer puts the fake on a loopback
port and exec.Command runs the binary against it, as oapi/fake_test.go does. The strict server
decodes a request body but checks it against no schema, and checks no credential. The README names a
validation middleware
for both.
orval’s mock option writes an MSW
handler per operation, seeded from the document’s examples when useExamples is on.
setupServer mocks the process it runs in, so a CLI
in a child process never meets those handlers. createServer from
@msw/serve puts them on a port:
const server = createServer(
// A 429 with the header the generated 429 handler leaves out.
http.get('*/sailings', () => {
if (throttled > 0) {
throttled--
return HttpResponse.json(getListSailingsResponseMock429(), { status: 429, headers: { 'retry-after': '1' } })
}
}),
// A second page, through the generated handler's override argument.
getListSailingsMockHandler(({ request }) =>
new URL(request.url).searchParams.get('page') === '2' ? PAGE_TWO : getListSailingsResponseMock()),
// A missing record, with the body the document's 404 example gave the generated factory.
http.get('*/sailings/sl_999', () => HttpResponse.json(getGetSailingResponseMock404(), { status: 404 })),
...getFerrylineAPIMock(),
)
MSW answers with the first handler that returns a response, so the scripted stubs sit ahead of the generated ones. The generated handlers check no credential.
Check it worked
Run the suite:
node --test e2e.test.mjs
1..14
# tests 14
# suites 0
# pass 14
# fail 0
# cancelled 0
# skipped 0
# todo 0
# pass 14 is the line to read. Nine tests drive the binary against the scripted mock, and one
each covers the key sorter, Prism, the in-process Go test, the oapi-codegen fake, and the orval
handlers.
When it goes wrong
Every test times out. The test called spawnSync while the mock ran in the same process, and a
blocked event loop cannot accept the child’s connection. Use spawn, or run the mock in a process
of its own.
node pitfall.mjs
spawnSync, with the mock in the same process:
status null, signal SIGTERM, error ETIMEDOUT
requests the mock had answered when it returned: 0
spawn, with a mock in the same process:
exit 0, ids sl_101 sl_102
requests the mock answered: 1
A stdout assertion passes on one run and fails on the next. The CLI prints each record as a Go
map, and the Go specification leaves the iteration order
of a map unspecified. Sort the keys inside each record before comparing, as sortKeys does.
A test fails with no such host. FERRY_BASE was unset, so the binary called the server in the
document, a reserved name here and the real API in your project. Assert on the mock’s request log
in every test.
When not to do this
Do not make every CLI test end to end. Each one costs a build and a process, and argument parsing needs no socket. Keep a dozen for what crosses the process boundary: the environment, the exit code, the wire.
Do not let sdkgen’s test mode stand in for the binary. It is the fastest check of the generated words, but its mock ignores the URL and headers, so a dropped header passes it. The sdkgen documentation describes no mock that listens on a port, so that one is yours to supply.
Do not hand-script a mock when a typed one is on offer. For a Go CLI on an oapi-codegen client, the strict server fake is the better choice. A stub for a status the document lacks has no type to return, so it fails to compile. For a TypeScript CLI on an orval client, the generated handlers start from the document’s examples, and you write only the stubs that differ.
Do not read a green run against Prism as coverage of the error paths.
Related how-tos
Last verified
Verified 2026-11-19 against Node 22.23.5, Go 1.24.7, @voxgig/create-sdkgen 0.31.9,
@voxgig/sdkgen 4.44.3, Prism 5.14.2, oapi-codegen v2.8.2, orval 8.48.0, MSW 3.0.7, and
@msw/serve 0.11.0. Every output block is what the command preceding it printed. sdkgen,
oapi-codegen, and orval each ran once on openapi.json; their output is committed, and the checks
build and drive it without generating again. The harness sorts the keys inside each printed
record. WireMock was not run; its row describes its documentation. No request left the loopback
interface.
Footnotes
-
The server in this document is
api.ferryline.example, a name no resolver will find. RFC 2606 reserved four top-level names in June 1999 and recommends.examplefor use in documentation or as examples. A test that forgets the base URL fails at the resolver here, withno such host. With a real host in the document, the same forgetful test reaches production. ↩︎ Back to text -
The status code is younger than the habit of sending it. RFC 6585 defined 429 in April 2012 and says a response may include a
Retry-Afterheader, which makes the header optional. The same section declines to define how a server identifies the user or counts the requests. A client that waits as told is honoring a header the server never had to send. ↩︎ Back to text -
Preferis a standard header put to a private use. RFC 7240 defined it in June 2014 for preferences such asreturn=minimalandrespond-async, and the IANA registry it created lists those withwait,handling, and later entries such assafe. Prism’scode,example, anddynamicare not in it. None of the four preferences the RFC defines chooses a status code. ↩︎ Back to text