How-to › Make calls that survive failure

How to add a response cache to a third-party SDK#

Put a response cache at the fetch seam of an SDK you did not write, and count origin requests to prove a repeat call never leaves the process.

Audience
API consumer
Level
intermediate
Topic
Cache API responses
Languages
TypeScript and Go
Verified

Your service calls a vendor SDK for the same book record forty times a second, and every call goes to the network. The SDK has no cache option. Its methods build the request, send it, and decode the reply in one step, so the place to remember a response is not where your code calls the method.

What you get

You will end up with a cache at the transport seam of an SDK you cannot edit. A test counts origin requests to prove that a repeat call never leaves the process. This is for you if you consume a generated client and its vendor will not add caching for you.

Short answer

Find the seam where the SDK takes its transport: system.fetch in an sdkgen SDK, the fetch option of Hey API’s setConfig, orval’s override.mutator, or a Go http.RoundTripper. Pass a fetch-shaped cache keyed on method, URL, and body hash, never on headers. Count requests at the origin: a repeat call adds none, another id adds one, and the SDK still returns a decoded object.

You will need

Node 22.19 or later, Go 1.24 or later for the Go transport, and an SDK that accepts a fetch function or an HTTP client. Verified 2026-11-20 against Node 22.23.5 and Go 1.24.7. The sample runs offline against origin.mjs, a stand-in API on 127.0.0.1 that counts the requests it receives, with three SDKs generated once from the same five-operation Bookshelf document, api.json.

Approaches compared

ApproachWhen it fitsWhat it costs youWhen to pick something else
Go http.RoundTripperA Go SDK that takes an *http.ClientYou write the store, the key, and the locking, or adopt a library for itThe SDK builds its own client and offers no option
Hey API fetch optionA Hey API client, where setConfig takes any fetchThe fetch receives a Request object, so the key reads a cloned bodyYour SDK came from another generator, which has no setConfig
Method-level memoizingYou want decoded objects cached and the SDK has no seamKeys come from arguments, so two spellings of one call can missThe SDK exposes a transport you can wrap
orval mutatorAn orval client, where every function calls one mutatorThe mutator is yours to write and to keep in step with the configYou use orval’s default fetch output with no mutator
sdkgen cache featureAn sdkgen SDK and a plain freshness windowIt ignores Cache-Control and ETag, and each client instance has its own storeThe origin’s headers must decide what is stored
undici cache interceptorAny fetch seam on Node, and an origin that sends cache headersFollows the origin’s headers, so the run stored no reply that lacked Cache-ControlThe origin sends no Cache-Control

Voxgig maintains sdkgen. It is one of six options here, not the recommendation.

The transport rows see raw HTTP and cache exactly the bytes the wire carried, which is why the SDK still decodes a hit. undici reads the origin’s headers; the sdkgen feature and the hand-written caches read only a clock. A memoizer skips HTTP and caches decoded objects, so it works on any SDK at the price of deriving a key from arguments.

Write one cache for every fetch seam

Hey API, orval, and sdkgen all call a fetch-shaped function, so one cache serves all three. transport-cache.mjs builds the key from the method, the URL, and a SHA-256 of the body, and replays stored bytes as a fresh Response on every hit.

  async function fetchWithCache(input, init = {}) {
    const request = new Request(input, init)
    const method = request.method.toUpperCase()
    if (!methods.includes(method)) {
      stats.bypass++
      return inner(request)
    }

    const body = request.body ? Buffer.from(await request.clone().arrayBuffer()) : Buffer.alloc(0)
    const key = `${method} ${request.url} ${createHash('sha256').update(body).digest('hex')}`
    const entry = store.get(key)
    if (entry && entry.expires > now()) {
      stats.hit++
      return replay(entry)
    }

new Request(input, init) accepts both calling conventions. Hey API passes one Request object, and sdkgen passes a URL and an init. A body can be read once, so the key reads a clone.

Pass it to sdkgen, or turn on its cache feature

An sdkgen SDK reads its transport from the system.fetch constructor option, and falls back to global.fetch when the option is absent. The bundled cache feature is the other route. It is off until feature: { cache: { active: true } } is passed, and its options are ttl, max, methods, and an injectable now(). Its generated key is the method and the URL alone.

    async _through(ctx, url, fetchdef, inner) {
        const method = ((fetchdef && fetchdef.method) || 'GET').toUpperCase();
        const methods = this._options.methods || ['GET'];
        if (methods.indexOf(method) < 0) {
            return inner(ctx, url, fetchdef);
        }
        const key = method + ' ' + url;

The URL carries the query string, so list({ author: 'Austen' }) is a separate entry. The body is not in the key, which is safe for the default ['GET'] and wrong for a POST search added to methods.

demo.mjs builds five clients over one stand-in each, loads b1 twice and b2 once, and reads the origin’s counter.

node demo.mjs
same call twice, then a different id; requests counted at the origin
  injection point           b1 twice  b2 once  second b1 result
  sdkgen cache feature      1         1        BookEntity title=Emma published=number
  sdkgen system.fetch       1         1        BookEntity title=Emma published=number
  Hey API setConfig fetch   1         1        Object title=Emma published=number
  orval mutator             1         1        Object title=Emma published=number
  memoized getBook          1         1        Object title=Emma published=number

Every seam sends one request for two loads of b1, and b2 is a miss. The hit comes back decoded: sdkgen returns a BookEntity and published is a number, not a string of JSON.

Put the cache under Hey API and orval

Both generators ran on api.json. Hey API’s fetch client documents a custom fetch passed to client.setConfig(). The generated client calls it with a Request it has already built, so headers and the base URL are set before the cache sees them.

  'Hey API setConfig fetch': (base) => {
    heyClient.setConfig({ baseUrl: base, fetch: cachingFetch() })
    return async (id) => (await heyapi.getBook({ path: { bookId: id } })).data
  },
  'orval mutator': (base) => {
    configureTransport({ base, fetch: cachingFetch() })
    return async (id) => (await orval.getBook(id)).data
  },

orval’s fetch output calls the global fetch directly and takes no fetch argument. Its fetch guide adds a custom fetch through the mutator: override.mutator names a function, and every generated operation calls it with a path and a RequestInit.

export default {
  bookshelf: {
    input: './api.json',
    output: {
      target: './orval/bookshelf.ts',
      client: 'fetch',
      override: { mutator: { path: './orval/transport.ts', name: 'transport' } },
    },
  },
}

orval/transport.ts prefixes the base URL, calls the cache, and returns { data, status, headers }, the shape the generated types expect. Hey API wins where you want no hand-written file: its seam is one option. orval wins where you already own a mutator for auth, because the cache joins it.

Wrap the transport in Go

A Go SDK that accepts an *http.Client lets you set its Transport, an http.RoundTripper. The contract lets RoundTrip consume the request body but not modify the request, so the sample sends the next transport a clone.1

	sum := sha256.Sum256(body)
	key := req.Method + " " + req.URL.String() + " " + hex.EncodeToString(sum[:])
go run ./roundtrip
go RoundTripper: b1 twice -> origin 1, b2 once -> origin 1
second b1 decoded: main.Book {ID:b1 Title:Emma Published:1815}

Choose between a TTL and the origin’s headers

A transport seam can hold an HTTP-aware cache as well. http-aware.mjs passes undici’s fetch, composed with interceptors.cache(), as system.fetch, and sets it beside the sdkgen feature.

node http-aware.mjs
origin requests for the same load twice, by the Cache-Control the origin sends
  cache-control             undici cache in system.fetch  sdkgen cache feature
  public, max-age=60        1                             1
  (none)                    2                             1
  no-store                  2                             1

undici’s cache store is documented as expected to comply with RFC 9111, and it stored only what the origin allowed. The sdkgen feature caches every 2xx for ttl milliseconds, no-store included. That is correct for an origin that sends no headers, and wrong for one that forbids storage.

Check it worked

Fifteen tests run against the stand-in. One per seam asserts a single origin request for the repeat, a miss for another id, and a decoded book equal to the record.

node --test cache.test.mjs
1..15
# tests 15
# suites 0
# pass 15
# fail 0
# cancelled 0
# skipped 0
# todo 0
# duration_ms 348.029976

The others cover a body in the key, a 404 that is never stored, the request-id pitfall, and reordered memo keys. Four pin the feature: expiry at exactly ttl on its injected clock, a stored no-store reply, separate entries per query string, and a second POST answered from the first. Two more check that undici refuses no-store and that a second user gets the first user’s reply. The last shows DataCloneError from memoizing a whole Hey API result.

When it goes wrong

Every call misses, and the origin sees each one. The SDK stamps a fresh request id on every call, and the cache puts headers in its key. Key on method, URL, and body only. The sdkgen clienttrack feature adds a fresh X-Request-Id per call, which pitfall.mjs uses.

node pitfall.mjs
clienttrack on: every call carries a fresh X-Request-Id
  headers in the key          origin requests 2, distinct request ids seen 2
  method, URL and body only   origin requests 1, distinct request ids seen 1
  sdkgen cache feature        origin requests 1, distinct request ids seen 1

A memoized Hey API call throws DataCloneError. Its result holds the Request and Response, which structuredClone refuses to copy. Memoize the decoded data instead.

One caller changes a cached object and the next caller sees the change. The memoizer handed out its stored copy. Clone on the way in and on the way out, as memo.mjs does.

When not to do this

Do not cache under a header-blind key when credentials differ between calls on one client. The key omits Authorization, so a client shared across users would serve one user’s record to another. Give each caller its own client, or put the caller in the key.

Do not use the sdkgen cache feature against an origin that sends no-store or short max-age values. It reads no Cache-Control, so its TTL overrides the origin. Use undici’s interceptor at the system.fetch seam instead.

Do not switch generators to get a cache seam. If your SDK already comes from Hey API, keep it: its seam is one option. If it comes from orval, the mutator that may already hold your auth takes the cache in a few lines.

Do not memoize when a transport seam exists. A memoizer needs a key per method and per argument shape, and every new method is one more key to get right.

Last verified

Verified 2026-11-20 against Node 22.23.5, Go 1.24.7, @voxgig/create-sdkgen 0.31.9, @voxgig/sdkgen 4.44.3, @hey-api/openapi-ts 0.99.0, orval 8.48.0, TypeScript 5.9.3, and undici 8.12.4. Every output block is what the command preceding it printed. sdkgen, Hey API, and orval were each run once on api.json; their compiled output is committed with the generate.sh that made it.

Footnotes

  1. The RoundTripper documentation spends most of its paragraph on what an implementation should not do. It should not interpret the response or handle redirects, authentication, or cookies, and it should not modify the request beyond consuming and closing its body. A transport is defined largely by its abstentions. gregjones/httpcache, a Go cache built as a RoundTripper, describes itself as a mostly RFC 7234 compliant cache, and its author calls it done. Its README still cites RFC 7234, which RFC 9111 obsoleted in June 2022. ↩︎ Back to text

Read this page as markdown · All how-to guides

Generate the client instead of writing it#

Retries, timeouts, pagination and auth are the same problems in every client. Voxgig generates them from your OpenAPI description, in 23 languages, from one model.

Listen to Fireside, the Voxgig podcast

Since 2018, Richard Rodger has talked with developer advocates, event organizers, and the people who build developer-facing companies about the specifics of the work.