How-to › Use AI to do the integration

How to inject network failures into an AI-written client#

Feed an agent's retry code scripted 503 and 400 responses and dropped connections, and prove with a test that never sleeps that it retries and gives up.

Audience
API consumer
Level
advanced
Topic
Write an integration with a coding agent
Languages
TypeScript and Go
Verified

Read first: Run a coding agent in a sandbox with a mock API

The agent’s pull request adds exponential backoff, and every test passes against a mock that always answers 200. The retry loop has never run. When the API returns 503 for an hour, the client retries with no cap and the job hangs, because no test ever reached the code that gives up.

What you get

You will end up with a test that feeds the agent’s client scripted failures and asserts how many calls it made and how long it would have waited. This is for you if a coding agent wrote retry code you are about to merge.

Short answer

Inject failures beneath the agent’s retry code, at the fetch it calls. Use a scripted fetch in process, a local server that drops sockets, Toxiproxy for TCP faults, or sdkgen’s netsim feature inside a generated SDK. Pass the client a wait that records instead of sleeping. Then assert one retry on a 503, none on a 400, and a bounded give-up on a permanent outage.

You will need

Node 22 or later, a client with retry code written by a coding agent, and an OpenAPI 3 document for the API it calls. Go 1.24 or later for the Go part, which uses the standard library only. Verified 2026-11-20 against Node 22.23.5 and Go 1.24.7. The sample’s agent-written client wraps openapi-fetch, whose fetch option replaces globalThis.fetch, and that option is where the faults go.

Approaches compared

ApproachWhen it fitsWhat it costs youWhen to pick something else
openapi-typescript and openapi-fetch with a scripted fetchThe agent wrapped openapi-fetch, and the fault belongs in processA fake fetch you script per test, and no wire-level behavior unless you add a serverYou need resets and latency on a real socket
sdkgen netsim featureThe client is an sdkgen SDK with the retry featureExists only inside a generated SDK, so it tests sdkgen’s retry rather than the agent’sThe agent wrote the retry code itself
Swagger Codegen fetch argumentA typescript-fetch client, where the constructor’s third argument is the fetchNo retry in the generated client, so the wrapper and its faults are both yoursYou want retry generated rather than written
ToxiproxyAny client in any language, and the faults are latency, resets, and timeoutsA proxy process in the test setup, and no way to answer with an HTTP statusThe assertion is about 503 versus 400
WireMock fault and delay stubsDeclarative HTTP faults: statuses, delays, and connection resets per stubA JVM or container beside the tests, and faults only on requests it serves itselfTests should run in the client’s own process

Voxgig maintains sdkgen. It is one of five options here, not the recommendation.

Toxiproxy reaches every client because it sits on the TCP connection, but it cannot send a 503. WireMock sends any status and delay from a stub file, and needs a JVM to do it. A scripted fetch costs no process at all and only covers clients that take a fetch. netsim is deterministic and in process, but it tests sdkgen’s retry, not the code the agent wrote.

Put the faults beneath the retry code

The agent’s client passes retryingFetch(inner) as openapi-fetch’s fetch. A fault must sit in inner, under the loop, so every failure passes through the retry decision. faults.mjs provides two injectors with one script format: a status per call, reset for a dropped connection, and the last entry repeating.

// One entry per call: a status, 'reset' for a dropped connection, or 'ok'.
// After the script runs out, the last entry repeats, so ['503'] is an outage.
export function scriptedFetch(script, body = { id: 'b1', title: 'Dune' }) {
  const calls = []
  const fetch = async (input, init) => {
    const request = new Request(input, init)
    const step = script[Math.min(calls.length, script.length - 1)]
    calls.push(`${request.method} ${new URL(request.url).pathname} -> ${step}`)
    if (step === 'reset') throw new TypeError('fetch failed', { cause: { code: 'ECONNRESET' } })

faultServer serves the same script over TCP on 127.0.0.1 and destroys the socket on reset, so Node’s real fetch sees a dropped connection. That is the fault Toxiproxy’s reset_peer toxic produces.1 The wait comes from the client’s sleep option, which the demo replaces with a function that records each delay.

node --experimental-strip-types demo.mjs
503 then ok (in-process)
  calls 2, waits [100], result 200 Dune
400 (in-process)
  calls 1, waits [], result 400
503 for good (in-process)
  calls 3, waits [100,200], result 503
reset then ok (tcp)
  calls 2, waits [100], result 200 Dune
reset for good (tcp)
  calls 3, waits [100,200], result threw fetch failed

A 503 costs one retry and a 100 ms wait. The 400 comes back at once, because RFC 9110 frames a 400 as a client error.2 A permanent 503 stops after three calls. The demo finishes in milliseconds, since no wait was slept.

Run the same faults through sdkgen’s netsim feature

In an sdkgen SDK the injector is a feature. netsim sits beneath retry in the transport chain, and test answers from a seed, so the whole run stays in process. Add all three features when you scaffold. generate.sh ran these commands once and kept the compiled .js files in generated/:

npx --yes @voxgig/create-sdkgen@0.31.9 bookshelf --def api.json --target ts \
  --feature retry --feature netsim --feature test
(cd bookshelf-sdk/.sdk && npm run generate)
(cd bookshelf-sdk/ts && npm install --no-audit --no-fund && npm run build)

Each feature is off until its options say active: true. netsim.mjs turns on all three per client, passes the fault as netsim options, and gives retry a sleep that records:

export function client(netsim, sleeps) {
  return new BookshelfSDK({
    feature: {
      test: { active: true, entity: { book: { b1: { id: 'b1', title: 'Dune' } } } },
      netsim: { active: true, ...netsim },
      retry: { active: true, retries: 2, minDelay: 100, jitter: false, sleep: (ms) => { sleeps.push(ms) } },
    },
  })
}

The four cases pass { failTimes: 1, failStatus: 503 }, the same with 400, { errorTimes: 1 } for a connection error, and { offline: true }:

node netsim.mjs
503 then ok        calls 2, waits [100], ok Dune
400                calls 1, waits [], threw BookshelfSDK: load: request: 400: Simulated Failure
reset then ok      calls 2, waits [100], ok Dune
offline for good   calls 3, waits [100,200], threw BookshelfSDK: load: Simulated network offline

The numbers match the agent’s client because the options were chosen to match. client._retry counts retries rather than calls, so attempts: 1 means two calls. The run tests sdkgen’s retry feature, which matters when sdkgen generated the SDK. It says nothing about retry code the agent wrote.

Inject the same faults with openapi-fetch and Swagger Codegen

openapi-fetch 0.17.0 has no retry, and openapi-typescript 7.13.0 writes types only, so the agent’s wrapper is the retry and the scripted fetch is the fault. Inject through the fetch option, not through a middleware. A middleware onRequest that returns a Response skips the request, which the middleware page documents. A 503 returned there never passes through the retry wrapper, as the pitfall below shows. This pair beats sdkgen when the client must stay a thin typed layer over fetch with no generated runtime.

Swagger Codegen 3.0.85’s typescript-fetch client has no retry: its constructor is constructor(configuration?, basePath = BASE_PATH, fetch = isomorphicFetch). Wrap the third argument with the same retryingFetch, and put the faults under it.

node --experimental-strip-types swagger.mjs
generated client, 503 then ok  calls 1, waits [], rejected with a Response 503
wrapped, 503 then ok           calls 2, waits [100], resolved Dune
wrapped, 400                   calls 1, waits [], rejected with a Response 400

The generated client rejects with the Response itself, not an Error. It beats sdkgen for a team that ships ten languages from one Java tool and already has a retry policy per language.

Inject the faults beneath a Go transport

The leaf covers Go too, and a Go client takes its faults the same way, one layer down. The agent’s retry code is an http.RoundTripper whose Next is the injector and whose Sleep records:

func (t *Retry) RoundTrip(req *http.Request) (*http.Response, error) {
	if !slices.Contains(idempotent, req.Method) {
		return t.Next.RoundTrip(req)
	}
	for attempt := 0; ; attempt++ {
		resp, err := t.Next.RoundTrip(req)
		transient := err != nil || slices.Contains(t.Statuses, resp.StatusCode)
		if !transient || attempt >= t.Retries {
			return resp, err
		}
		if resp != nil {
			io.Copy(io.Discard, resp.Body)
			resp.Body.Close()
		}
		if err := t.Sleep(req.Context(), min(t.MaxDelay, t.BaseDelay<<attempt)); err != nil {
			return nil, err
		}
	}
}

go/faults.go holds the Go twin of scriptedFetch, a Script transport that returns a syscall.ECONNRESET error for reset. Run the cases:

go run -C go .
503 then ok      calls 2, waits [100ms], 200 {"id":"b1","title":"Dune"}
400              calls 1, waits [], 400 {"error":"status 400"}
503 for good     calls 3, waits [100ms 200ms], 503 {"error":"status 503"}
reset then ok    calls 2, waits [100ms], 200 {"id":"b1","title":"Dune"}
reset for good   calls 3, waits [100ms 200ms], error Get "http://bookshelf.test/v1/books/b1": read tcp: connection reset by peer
POST, 503        calls 1, waits [], 503 {"error":"status 503"}

The POST goes out once, because RFC 9110 does not count POST among the idempotent methods. client.ts applies the same list. The Go part has no sdkgen run: the sdkgen sample on this page is the TypeScript target only.

Check it worked

Ten tests cover both injectors, the give-up path, the POST that is not retried, the middleware trap, netsim, Swagger Codegen, and the Go transport, whose own go test suite the last test runs. The first asserts that the recorded 100 ms wait was not slept.

node --experimental-strip-types --test retry.test.mjs
1..10
# tests 10
# suites 0
# pass 10
# fail 0
# cancelled 0
# skipped 0
# todo 0

When it goes wrong

Only the happy retry was tested, and the client retries forever on an outage. pitfall.mjs holds a loop that retries every 5xx with no cap. Feed it a permanent 503 with a wait that stops the demo:

node --experimental-strip-types pitfall.mjs
naive retry, permanent 503
  calls 50, result: stopped after 50 waits
  the first 20 waits alone add up to 29 hours
capped retry, permanent 503
  calls 3, waits [100,200], result: 503
503 from a middleware, above the retry
  transport calls 0, waits [], result: 503

The uncapped loop has no exit and doubles its wait unbounded. Add a retry count and a maxDelay, and keep a test with a permanent outage in it.

The fault never reaches the retry code. It was injected outside the wrapper, in a middleware, so the transport made zero calls and nothing retried. Move the injector into the fetch the wrapper calls.

The suite takes minutes. A test exercises real waits. Pass sleep in every test, and assert on the recorded delays.

When not to do this

Do not use sdkgen’s netsim to verify retry code the agent wrote by hand. It exercises the retry feature sdkgen generated, so the agent’s loop is not under test. Use a scripted fetch or a fault server, and keep netsim for an SDK that sdkgen generated.

Do not reach for Toxiproxy to test status-code decisions. It works on TCP, so a 503 against a 400 is outside its reach. Pair it with an HTTP-level injector, or use WireMock, which does both.

Do not retry a POST because a test showed the retry works. A retried write that the server committed is a second record. Scope the retry to idempotent methods, as client.ts and the Go transport do, or add an idempotency key first.

Do not trust a retry test without a give-up case. One permanent outage per suite is the test that catches the loop with no exit.

Last verified

Verified 2026-11-20 against Node 22.23.5, @voxgig/create-sdkgen 0.31.9, @voxgig/sdkgen 4.44.3, openapi-fetch 0.17.0, openapi-typescript 7.13.0 and Swagger Codegen 3.0.85, compiled with TypeScript 5.9.3, and Go 1.24.7 for the Go part. Every output block is what the command preceding it printed. Swagger Codegen was run once and its compiled output is committed. Toxiproxy and WireMock were not run; their behavior comes from their documentation.

Footnotes

  1. The Toxiproxy README describes reset_peer as a simulated TCP RESET, and says the tool has been in all development and test environments at Shopify since October 2014. Its README lists nine entries under toxics, from latency to packet_loss, each a property of the connection rather than of HTTP. One of the nine, down, the README itself says is not technically a toxic. ↩︎ Back to text

  2. RFC 9110 describes a 503 as a temporary overload or scheduled maintenance that will likely pass after some delay, and lets the server say how long in Retry-After. A 400 is a request perceived as a client error. Retrying one is sending the same mistake again. ↩︎ Back to text

Read this page as markdown · All how-to guides

Generate the client instead of writing it#

Retries, timeouts, pagination and auth are the same problems in every client. Voxgig generates them from your OpenAPI description, in 23 languages, from one model.

Listen to Fireside, the Voxgig podcast

Since 2018, Richard Rodger has talked with developer advocates, event organizers, and the people who build developer-facing companies about the specifics of the work.