# How to inject latency and failures into SDK tests

> Make a generated SDK fail on purpose with 429 responses, connection errors, and latency, then test a sixty-second backoff in milliseconds.

Source: https://voxgig.com/howto/inject-latency-and-failures-into-sdk-tests

- Audience: library-maintainer
- Level: intermediate
- Languages: typescript, go, python
- Verified: 2026-11-21
- Published: 2026-11-21

## Short answer

Switch on sdkgen's netsim feature in the client constructor, beside retry and timeout, and give retry an injected sleep. netsim answers with a 429, a connection error, an outage, or latency before a request leaves the client, so a test of a sixty-second Retry-After finishes in milliseconds. Build its options in one function that refuses any NODE_ENV but test, because a simulated 503 in production looks like a real one.

---
## You will need

Node 22.18 or later, and an SDK generated by sdkgen that you can rebuild. Python 3 and Go 1.24 or
later run the cross-language check. Verified 2026-11-21 against Node 22.23.5, Python 3.13.18,
Go 1.24.7, @voxgig/sdkgen 4.44.3, NSwag 14.8.0, and oapi-codegen v2.8.2. The document,
`api.json`, describes a rates API whose two operations both declare 429 and 503 responses. `server.ts` stands
in for the API on `127.0.0.1` and logs each request it receives.

## Approaches compared

| Approach | When it fits | What it costs you | When to pick something else |
| --- | --- | --- | --- |
| [NSwag](https://github.com/RicoSuter/NSwag/wiki/TypeScriptClientGenerator) | A .NET shop generating C# and TypeScript clients from one document | No fault option and no retry, so you write a faulty fetch or `DelegatingHandler` and the retry it tests | You want generated retry behavior to test rather than your own |
| [oapi-codegen](https://github.com/oapi-codegen/oapi-codegen#generating-api-clients) | Go clients that take any `HttpRequestDoer` you hand them | No retry or fault code generated, so both doers are yours to write and keep | You want faults switched on by an option instead of a type |
| [sdkgen netsim feature](https://github.com/voxgig/sdkgen/blob/main/docs/reference/features.md) | Your SDK is generated by sdkgen and you test its own retry and timeout | Proves only the client's layers, since no byte crosses a socket, and it ships in the SDK | You need to see a half-read body or a reset socket |
| [Toxiproxy](https://github.com/Shopify/toxiproxy#toxics) | Any client, any TCP dependency, faults set per test over its HTTP API | A Go server to run beside the tests, and a proxy address the client must use | Your CI cannot start a server process |
| [WireMock](https://wiremock.org/docs/simulating-faults/) | HTTP stubs with fixed delays, resets, and garbage responses | A JVM or container per suite, and faults in fixed shapes per stub | You want the fault inside the client, with no network |

Voxgig maintains sdkgen. This page is about its `netsim` feature, and the table's sdkgen row
says where a proxy or a stub server proves more.

Toxiproxy and WireMock sit on the wire, so they also prove what the socket does, at the price of
a process beside the tests. `netsim` needs no process and sees no socket. NSwag and oapi-codegen
generate neither faults nor retry, so their tests exercise code you wrote around the client.

## Switch netsim on beside retry and timeout

`generate.sh` scaffolded the project with `-t ts -t py -t go -f netsim -f retry -f timeout`. All
three features wrap the transport, and in a map the default order puts `timeout` outermost, then
`retry`, then `netsim`, so a retried call meets the fault again.[^1] One module makes every
faulty client:

```ts title="faults.ts"
// The only place netsim options are made. Outside NODE_ENV=test it throws,
// because a simulated 503 in production reads as a real outage.
export function netsimFor(env: Env, faults: Record<string, unknown>) {
  if (env.NODE_ENV !== 'test') {
    throw new Error(`netsim refused: NODE_ENV is ${env.NODE_ENV ?? 'unset'}, not test`)
  }
  return { active: true, ...faults }
}

export type FaultyOptions = {
  faults: Record<string, unknown>
  sleep: (ms: number) => void
  timeoutMs?: number
  env?: Env
}

// A client whose transport is wrapped, from the outside in, by timeout,
// retry and netsim: the default order for these three features.
export function faultyClient(base: string, options: FaultyOptions) {
  const feature: Record<string, unknown> = {
    netsim: netsimFor(options.env ?? process.env, options.faults),
    retry: { active: true, retries: 2, minDelay: 100, maxDelay: 120_000, jitter: false, sleep: options.sleep },
  }
  if (options.timeoutMs !== undefined) feature.timeout = { active: true, ms: options.timeoutMs }
  return new RatecardSDK({ base, feature })
}
```

The faults are `netsim` options: `rateLimitTimes` with `retryAfter` for 429 responses, `errorTimes` for
connection errors, `offline` for an outage, and `latency` in milliseconds. `maxDelay` is raised
to two minutes because `retry` caps every wait at it, a `Retry-After` included. `sleep` is the
injected clock: `virtualClock()` in the same file records each wait and returns at once.

## Wait out a sixty-second Retry-After in milliseconds

`demo.ts` runs four faults against the stand-in. `NODE_ENV=test` is what lets `netsimFor` build
them.

```bash
NODE_ENV=test node demo.ts
```

```text output
429, then Retry-After: 60
  loaded r1; attempts: 2; waits on the injected clock: [60000] ms
  requests that reached the API: 1, wall clock under 1 s: true
connection error twice
  loaded r1; attempts: 3; waits on the injected clock: [100, 200] ms
  requests that reached the API: 1, wall clock under 1 s: true
outage
  failed: netsim_offline; attempts: 3; waits on the injected clock: [100, 200] ms
  requests that reached the API: 0, wall clock under 1 s: true
300 ms of latency against a 50 ms timeout
  failed: timeout; attempts: 1; waits on the injected clock: [] ms
  requests that reached the API: 0, wall clock under 1 s: true
```

The minute passed on the injected clock and not on the wall. Every fault was answered inside the
client: the 429 attempt and the two failed connections never reached `server.ts`, which is the
limit of this approach. The timeout case uses real time, 50 ms of it, because the deadline is a
real timer.

## Check the same faults in the Python and Go SDKs

`same.ts` runs three faults on the TypeScript SDK and starts `same.py` and the Go program in
`gocheck/sdkgen`, which run them on the other two targets. All three use the `test` feature, so
nothing leaves the process.

```bash
node same.ts
```

```text output
scenario                   typescript                        python                            go
429, then Retry-After: 60  loaded r1 [60000]                 loaded r1 [60000]                 loaded r1 [60000]
connection error twice     loaded r1 [100, 200]              loaded r1 [100, 200]              loaded r1 [100, 200]
outage                     failed: netsim_offline [100, 200] failed: netsim_offline [100, 200] failed: netsim_offline [100, 200]
identical in all three: true
```

The options are the same keys in all three languages. The injected sleep differs only in type: a
function of `ms` in TypeScript, `list.append` in Python, and a `func(int)` in Go.

## Inject the same faults under NSwag and oapi-codegen clients

NSwag's [TypeScript client](https://github.com/RicoSuter/NSwag/wiki/TypeScriptClientGenerator)
takes `http?: { fetch(url, init) }` in its constructor, and its C# client takes an `HttpClient`,
so a [`DelegatingHandler`](https://learn.microsoft.com/en-us/dotnet/api/system.net.http.delegatinghandler)
is the .NET seam. Neither generated client retries, so the fault needs a retry layer to test.

```ts title="nswag.ts"
// Answers the first statuses.length requests itself, then passes on.
export function faultyFetch(statuses: number[], retryAfter: string, next: Fetch): Fetch {
  const script = [...statuses]
  return async (url, init) => {
    const status = script.shift()
    if (status === undefined) return next(url, init)
    return new Response('', { status, headers: { 'retry-after': retryAfter } })
  }
}
```

```bash
node nswag.ts
```

```text output
429, then Retry-After: 60, under a retrying fetch
  rate r1; waits on the injected clock: [60000] ms
  requests that reached the API: 1, wall clock under 1 s: true
the same fault with no retrying fetch
  ApiException 429, retry-after header: 60
```

oapi-codegen's client takes any `HttpRequestDoer` through `WithHTTPClient`, and its README
mentions no retry. `gocheck/oapi` puts a scripted `faultyDoer` under a `retryDoer` with an
injected sleep, and calls the generated `GetRateWithResponse`.

```bash
go -C gocheck run ./oapi
```

```text output
429, then Retry-After: 60, under a retrying doer
  status 200, rate r1; waits on the injected clock: [1m0s]
  requests that reached the API: 1, wall clock under 1 s: true
the same fault with no retrying doer
  status 429, JSON200 is nil: true
```

## Check it worked

The test that carries the page asserts the minute, the wall clock, and the API's request log:

```ts title="faults.test.ts"
test('a sixty second Retry-After is honoured on the injected clock in milliseconds', async (t) => {
  const api = await serve(t)
  const clock = virtualClock()
  const client = faultyClient(api.base, { faults: { rateLimitTimes: 1, retryAfter: 60 }, sleep: clock.sleep, env: TEST_ENV })
  const started = Date.now()
  const rate = await client.Rate().load({ id: 'r1' })
  assert.equal(rate.data().id, 'r1')
  assert.deepEqual(clock.slept, [60_000])
  assert.ok(Date.now() - started < 1000, 'no real minute went by')
  assert.equal(client._netsim.calls, 2)
  assert.deepEqual(api.requests, ['GET /v1/rates/r1'], 'the simulated 429 never reached the API')
})
```

```bash
node --test faults.test.ts
```

```text output
# tests 12
# suites 0
# pass 12
# fail 0
# cancelled 0
# skipped 0
# todo 0
```

The other eleven cover the remaining faults, the guard, all three languages agreeing, both
substitutes, and a `Retry-After` given as a date, which the hand-written retry layers decline to
guess at.

## When it goes wrong

The sixty-second test passes with `[2000]`. `retry` caps a `Retry-After` at `maxDelay`, whose
default is 2000 ms, so the client comes back 58 seconds early.[^2] Raise `maxDelay` to the
longest wait your API states, and keep the test that fails on `[2000]`.

Your SDK reports a 503 on every third call while the API's status page shows no incident.
`netsim` was left active in options a production client shares with the tests.

```bash
node pitfall.ts
```

```text output
netsim left on in shared options, failEvery: 3, six loads
  ok ok 503 ok ok 503
  requests that reached the API: 4 of 6
  the error the caller logs: RatecardSDK: load: request: 503: Simulated Failure
the same faults through netsimFor, NODE_ENV=production
  netsim refused: NODE_ENV is production, not test
```

The API never saw the failing calls, so its own logs show nothing wrong. Only the message says
`Simulated Failure`, and a dashboard that counts statuses never reads it. Build `netsim` options
in `netsimFor` alone, and let it throw outside tests.

## When not to do this

Do not rely on `netsim` to prove socket behavior. A response that stops halfway, a connection
that resets after the headers, and a slow TLS handshake never happen inside the client. Put
[Toxiproxy](https://github.com/Shopify/toxiproxy) or WireMock in front of a real endpoint for
those.[^3]

Do not add sdkgen to get fault injection for a client you did not generate with it. NSwag is the
better choice for a .NET team that already owns its HTTP handlers, since the faults go in a
handler beside the retry it tests. oapi-codegen is the better choice when the Go service already
wraps its `HttpRequestDoer`, because two small doers cost less than a second generator.

Do not ship a build where `netsim` can be switched on by configuration alone. The guard costs one
function, and the incident it prevents looks real.

## Related how-tos

- [Simulate slow and broken networks with Toxiproxy](/howto/simulate-slow-networks-with-toxiproxy)

- [Run an in-memory mock of your API inside SDK tests](/howto/run-an-in-memory-mock-inside-sdk-tests)

- [Choose between a network mock and an in-process mock](/howto/choose-between-network-and-in-process-mocks)

- [Enforce one total deadline across all retry attempts](/howto/enforce-a-total-deadline-across-retry-attempts)

- [Test a backoff schedule with a fake clock](/howto/test-a-backoff-schedule-with-a-fake-clock)

## Last verified

Verified 2026-11-21 against Node 22.23.5, Python 3.13.18 (and 3.12.3 for `same.py`), Go 1.24.7,
@voxgig/create-sdkgen 0.31.9, @voxgig/sdkgen 4.44.3, @voxgig/apidef 8.32.4, NSwag 14.8.0 on the
ASP.NET Core runtime 10.0.15, oapi-codegen v2.8.2, and `github.com/oapi-codegen/runtime` v1.8.0.
Every output block is what the command preceding it printed. The generators ran once, in
`generate.sh`, and their output is committed under `generated/` and `gocheck/ratecard/`. NSwag ran
once: its TypeScript client is committed and runs under Node in `nswag.ts`. Its C# client was
generated and committed to read its `HttpClient` constructor, and was neither compiled nor run. No
`DelegatingHandler` was written or run.
@voxgig/sdkgen 4.45.0 wrote identical `ts`, `py`, and `go` trees from the same project.

[^1]: The sdkgen [feature reference](https://github.com/voxgig/sdkgen/blob/main/docs/reference/features.md)
explains the order: in a map, `test` initializes first and the remaining features follow in
alphabetical order, and the one initialized last wraps the rest. So `timeout` ends up outside
`retry` because t follows r in the alphabet, and one deadline covers the whole retry sequence.
An array instead of a map sets the order by position.

[^2]: The example in [RFC 6585](https://www.rfc-editor.org/rfc/rfc6585.html#section-4), which defined
429 in April 2012, sends `Retry-After: 3600` with a page explaining that the site allows 50
requests per hour per logged-in user. Against the default cap that client would return
3,598 seconds early, and the RFC itself observes that a server under attack need not send a 429
at all.

[^3]: Toxiproxy's [README](https://github.com/Shopify/toxiproxy) describes it as made
specifically for testing, CI, and development environments, and says Shopify has run it in all of
its development and test environments since October 2014. The same paragraph says, in bold,
that it is the tool you need to prove with tests that a system has no single points of failure.