# How to turn typed SDK response objects into a pandas DataFrame

> Flatten every page of Pydantic, `attrs`, or sdkgen objects into one DataFrame, with enums as plain strings and the row count checked against the objects read.

Source: https://voxgig.com/howto/sdk-response-objects-into-a-pandas-dataframe

- Audience: api-consumer
- Level: beginner
- Languages: python
- Verified: 2026-11-20
- Published: 2026-11-20

## Short answer

To turn typed SDK responses into a pandas DataFrame, read every page, convert each object to a dict, and flatten the list with pd.json_normalize. On OpenAPI Generator's Pydantic models, call model_dump() and map the enum members with .value. On openapi-python-client models, call to_dict(), which leaves UNSET fields out. sdkgen's py-data package returns the frame from one call. Then check that df.shape[0] equals the objects read.

---
## You will need

Python 3.12 or later, pandas, and a Python SDK for the API installed from PyPI. Verified 2026-11-20
against Python 3.13.18, pandas 3.1.1, Pydantic 2.14.4, OpenAPI Generator 7.27.0,
`openapi-python-client` 0.30.0, and @voxgig/sdkgen 4.44.3. The document, `api.json`, describes an
order list behind a cursor envelope of `data`, `next_cursor`, and `has_more`, seven orders in
three pages. Every script runs against `mock_api.py`, a local stand-in that refuses a cursor it
did not issue.

## Approaches compared

| Approach | When it fits | What it costs you | When to pick something else |
| --- | --- | --- | --- |
| [OpenAPI Generator python models with model_dump()](https://openapi-generator.tech/docs/generators/python/) | The SDK is OpenAPI Generator's python client, with Pydantic models | A paging loop you write, Enum members to map, and float columns where integers go missing | Analysts want every page in one call with nullable integer columns |
| [`openapi-python-client` models with `to_dict()`](https://github.com/openapi-generators/openapi-python-client) | The SDK was generated by `openapi-python-client` and returns `attrs` models | A paging loop you write, dates as strings to parse, and UNSET wherever `to_dict()` is bypassed | You want Pydantic validation on every record |
| [sdkgen py-data method](https://github.com/voxgig/sdkgen) | The API owner generates with sdkgen and publishes the py-data package beside the SDK | pandas only, a second package to install, and string columns for dates until you parse them | The vendor publishes no py-data package, or the analysts work in Polars |
| [Speakeasy Python models with model_dump()](https://www.speakeasy.com/docs/sdks/languages/python/methodology-python) | The vendor's SDK is a Speakeasy Python SDK whose list operation has pagination configured | The same Enum and `datetime` handling as any Pydantic model, plus a `next()` loop per endpoint | The vendor's document has no pagination extension, so `next()` has nothing to follow |

Voxgig maintains sdkgen. It is one of four options here, not the recommendation.

The two Pydantic rows keep the SDK's validation until the last step, and a dict per object is the
price. The `to_dict()` that `openapi-python-client` generates writes strings for enums and dates,
so its frame needs one `to_datetime` and no enum mapping. The py-data package skips the object layer entirely, but it
exists only where the API owner generated and published it.

## See what the DataFrame constructor makes of objects

The constructor treats a list of objects as a list of rows, and what a row turns into depends on
the class.

```bash
PYTHONPATH=pydeps:generated/py:generated/py-data:generated/oag:generated/opc python3 one_column.py
```

```text output
sdkgen OrderEntity objects: shape (3, 1), columns [0], dtype object
OpenAPI Generator pydantic models: shape (3, 7), columns [0, 1, 2, 3, 4, 5, 6]
  first cell: ('id', 'ord_101')
```

The sdkgen entity is opaque to pandas, so each row is one object. A Pydantic model is iterable,
and [iterating it yields](https://pydantic.dev/docs/validation/latest/concepts/serialization/)
`(field_name, field_value)` pairs, so pandas builds a column per field and stores the pairs.[^1]
Group-bys and joins need a column per field, so each object has to become a dict first.

## Read every page through the sdkgen SDK

`generate.sh` built all three clients once, from the same `api.json`, and the output is committed
under `generated/`. The sdkgen part is two commands: the scaffold adds the `py` target, the `py-data`
target that wraps it, and the `paging` feature.

```bash title="generate.sh"
(cd "$WORK" && npx --yes @voxgig/create-sdkgen@0.31.9 orderdesk --def api.json \
  --target py --target py-data --feature paging)
(cd "$WORK/orderdesk-sdk/.sdk" && npm run generate)
```

The `py` SDK's `list()` returns `OrderEntity` objects, and `data_get()` unwraps each one into a
dict. The [`paging` feature](https://github.com/voxgig/sdkgen/blob/main/docs/reference/features.md)
writes the server's signals into the control dict you pass, so passing the same dict again
fetches the next page.

```py title="frames.py"
def sdkgen_records(sdk) -> list[dict]:
    """Every order through the sdkgen py SDK, unwrapped with data_get()."""
    orders = sdk.Order()
    ctrl = {"paging": {}}
    records: list[dict] = []
    seen: set[str] = set()
    while True:
        records.extend(order.data_get() for order in orders.list(None, ctrl))
        if not ctrl["paging"].get("hasMore"):
            return records
        cursor = ctrl["paging"].get("cursor")
        if cursor is not None and cursor in seen:
            raise RuntimeError(f"pagination repeated cursor {cursor!r}")
        seen.add(cursor)
```

`sdkgen_frame.py` runs that loop, then the py-data method, which drains the same pages and
returns the frame itself.

```bash
PYTHONPATH=pydeps:generated/py:generated/py-data:generated/oag:generated/opc python3 sdkgen_frame.py
```

```text output
py SDK, data_get() per entity: 7 objects, 7 rows, 3 requests
py-data, ad.orders(): 7 rows, 3 requests
  GET /v1/orders?page=1
  GET /v1/orders?cursor=cur_0003
  GET /v1/orders?cursor=cur_0006

        id    status  total  items             placed_at customer.country
0  ord_101      paid   42.5      3  2026-05-04T09:15:00Z               NG
1  ord_102   shipped   18.0      1  2026-05-04T08:40:00Z               NL
2  ord_103   pending   7.25   <NA>  2026-05-03T17:05:00Z               TW
3  ord_104  refunded   99.0      4  2026-05-03T11:30:00Z               IL
4  ord_105      paid  12.75      2  2026-05-02T20:00:00Z               KE
5  ord_106   shipped   64.0      5  2026-05-02T10:10:00Z               NG
6  ord_107      paid   30.0      2  2026-05-01T07:55:00Z               IN

id                   string
items                 Int64
note                 string
placed_at            string
status               string
total               Float64
customer.country        str
customer.name           str

placed_at with parse_dates: datetime64[us, UTC]
```

The first request carries `page=1`, because the feature sends a page number until a response hands
it a cursor; the stand-in ignores the parameter. The `dtypes` come from the
entity model, which records `status` as a `$STRING` and keeps none of its four enum values, so
this frame has no Enum members to map.
`items` stays an integer column with `<NA>` for the order that left it out, and `placed_at` stays
text until `parse_dates` names it.

## Dump Pydantic models from OpenAPI Generator and Speakeasy

OpenAPI Generator's python client has no paging helper; a search of the generated package for
`paginat` finds nothing. The loop is yours, and `model_dump()` turns each model into a dict.

```py title="frames.py"
def frame_from_pydantic(models: list) -> pd.DataFrame:
    """model_dump() keeps datetimes and Enum members; turn the members into strings."""
    df = pd.json_normalize([m.model_dump() for m in models])
    df["status"] = df["status"].map(lambda member: member.value)
    return check_rows(df, len(models))
```

```bash
PYTHONPATH=pydeps:generated/py:generated/py-data:generated/oag:generated/opc python3 oag_frame.py
```

```text output
7 models from 3 requests, 7 rows

                         model_dump()        status mapped
id                                str                  str
status                            str                  str
total                         float64              float64
items                         float64              float64
placed_at         datetime64[us, UTC]  datetime64[us, UTC]
note                              str                  str
customer.name                     str                  str
customer.country                  str                  str

status[0] after model_dump(): <OrderStatus.PAID: 'paid'>
status[0] after .value:       'paid'
placed_at[0]:                 2026-05-04 09:15:00+00:00
model_dump(mode='json'): status str, placed_at str
```

The two `dtypes` columns are identical, which is the trap: the enum column reads `str` while it
holds `OrderStatus` members.[^2] `placed_at` arrived as `datetime` objects, and pandas 3.1 typed
them on its own, so no `to_datetime` call was needed here. `model_dump(mode="json")` gives plain
strings for both, which trades the enum fix for a date column you parse yourself.
[`json_normalize`](https://pandas.pydata.org/docs/reference/api/pandas.json_normalize.html) turned
the nested `customer` into `customer.name` and `customer.country`.

Speakeasy's Python SDKs use [Pydantic for every generated
model](https://www.speakeasy.com/docs/sdks/languages/python/methodology-python), so the same
`frame_from_pydantic` applies. Its paging is the `x-speakeasy-pagination` extension in the
vendor's document, with `type: cursor` and an `outputs.nextCursor` path, and the response gains a
[`next()` method](https://www.speakeasy.com/docs/sdks/customize/runtime/pagination) that returns
nothing after the last page.[^3] Speakeasy was not run for this page, since generation needs an
account.

## Check it worked

`test_frames.py` runs every route against the stand-in and compares each frame's row count with the
objects iterated and the stand-in's request count.

```py title="test_frames.py"
    def test_the_pydantic_route_reads_every_page_and_maps_the_enum(self):
        models = oag_models(self.oag())
        df = frame_from_pydantic(models)
        self.assertEqual((len(models), df.shape[0], len(self.api.seen)), (ORDER_COUNT, ORDER_COUNT, PAGES))
        self.assertTrue(all(type(s) is str for s in df["status"]))
        self.assertEqual(str(df["placed_at"].dtype), "datetime64[us, UTC]")
```

```bash
PYTHONPATH=pydeps:generated/py:generated/py-data:generated/oag:generated/opc python3 test_frames.py
```

```text output
test_a_cursor_that_repeats_stops_the_loop (__main__.Guards.test_a_cursor_that_repeats_stops_the_loop) ... ok
test_a_cursor_that_repeats_stops_the_sdkgen_loop (__main__.Guards.test_a_cursor_that_repeats_stops_the_sdkgen_loop) ... ok
test_check_rows_refuses_a_short_frame (__main__.Guards.test_check_rows_refuses_a_short_frame) ... ok
test_the_entity_model_keeps_status_as_a_plain_string (__main__.Guards.test_the_entity_model_keeps_status_as_a_plain_string) ... ok
test_attrs_asdict_leaks_unset_into_the_frame (__main__.Routes.test_attrs_asdict_leaks_unset_into_the_frame) ... ok
test_model_dump_leaves_enum_members_in_a_str_column (__main__.Routes.test_model_dump_leaves_enum_members_in_a_str_column) ... ok
test_py_data_returns_every_page_as_one_frame (__main__.Routes.test_py_data_returns_every_page_as_one_frame) ... ok
test_py_data_types_columns_from_the_model (__main__.Routes.test_py_data_types_columns_from_the_model) ... ok
test_the_attrs_route_reads_every_page_without_sentinels (__main__.Routes.test_the_attrs_route_reads_every_page_without_sentinels) ... ok
test_the_constructor_makes_one_object_column (__main__.Routes.test_the_constructor_makes_one_object_column) ... ok
test_the_pydantic_route_reads_every_page_and_maps_the_enum (__main__.Routes.test_the_pydantic_route_reads_every_page_and_maps_the_enum) ... ok
test_the_sdkgen_py_sdk_loop_reads_every_page (__main__.Routes.test_the_sdkgen_py_sdk_loop_reads_every_page) ... ok
test_the_stand_in_refuses_cursors_and_limits_it_did_not_document (__main__.Routes.test_the_stand_in_refuses_cursors_and_limits_it_did_not_document) ... ok
test_without_paging_py_data_stops_after_one_page (__main__.Routes.test_without_paging_py_data_stops_after_one_page) ... ok
```

Fourteen tests print `ok` and the runner ends with `OK`. `check_rows` raises `ValueError` when
`df.shape[0]` differs from the count, so a frame built from a short loop fails where it is made.

## When it goes wrong

The group-by labels read `OrderStatus.PAID` in place of `paid`. `model_dump()` keeps
the Enum member, and printing one shows its class. Map the column with `.value`, as
`frame_from_pydantic` does.

```bash
PYTHONPATH=pydeps:generated/py:generated/py-data:generated/oag:generated/opc python3 pitfall.py
```

```text output
1. an enum column holds Enum members, and its dtype says str
   dtype: str, type of a value: OrderStatus
   status
   OrderStatus.PAID        3
   OrderStatus.PENDING     1
   OrderStatus.REFUNDED    1
   OrderStatus.SHIPPED     2
   after .map(lambda member: member.value):
   status
   paid        3
   pending     1
   refunded    1
   shipped     2

2. without the paging feature, the first page is the whole frame
   rows: 3 of 7, requests: ['/v1/orders']

3. attrs.asdict() keeps the UNSET sentinel that to_dict() leaves out
   attrs.asdict(): items object, row 3 items = Unset
   to_dict():      items float64, row 3 items = nan
```

The frame holds three rows of seven. The client was constructed without `feature={"paging": {"active":
True}}`, so py-data made one request and treated it as the only page. Pass the feature, and keep
the row-count check.

An `items` column turns to `object` and arithmetic on it raises. `attrs.asdict()` copied the
`Unset` sentinel that `openapi-python-client` puts in absent fields. Call the model's own
`to_dict()`, which leaves them out, so pandas fills `NaN`.

## When not to do this

Do not convert objects to a frame inside a service that only filters and forwards the records.
The SDK's types are the point there, and a frame throws them away for a table you never group.

Do not expect sdkgen's py-data package from a vendor that ships only an OpenAPI Generator or
Speakeasy SDK. It exists when the API owner generates with sdkgen and publishes it; otherwise the
`model_dump()` loop here is the route.

Do not choose the py-data package for analysts who work in Polars. pandas is the only frame
library it knows. A Speakeasy SDK with pagination configured is the better source there, because
`next()` and `model_dump()` feed any frame library.

Do not trust `dtypes` to show an enum problem. The column reads `str` while it holds Enum
members, and only a value check, as in the test, finds them.

## Related how-tos

- [Loop over every page of a REST collection](/howto/loop-over-every-page-of-a-collection)

- [Ship a DataFrame-first Python client for your analysts](/howto/dataframe-first-python-client-from-openapi)

- [Describe pagination so generators can follow it](/howto/describe-pagination-in-openapi-for-generators)

- [Generate your first SDK with sdkgen](/howto/first-sdk-with-sdkgen)

- [Backfill a year of API data under a rate limit](/howto/backfill-a-year-under-a-rate-limit)

## Last verified

Verified 2026-11-20 against Node 22.23.5, Python 3.13.18, @voxgig/create-sdkgen 0.31.9,
@voxgig/sdkgen 4.44.3, @voxgig/apidef 8.32.4, OpenAPI Generator 7.27.0, `openapi-python-client`
0.30.0, pandas 3.1.1, Pydantic 2.14.4, and `attrs` 26.1.0. Every output block is what the command
preceding it printed. The three generators ran once, in `generate.sh`, and their Python packages
are committed under `generated/`. Speakeasy was not run. Its options come from its documentation.

[^1]: Pydantic's serialization page says so in one sentence, then adds that field values are left
as is, so sub-models will not be converted to dictionaries. The [same
page](https://pydantic.dev/docs/validation/latest/concepts/serialization/) documents
`model_dump()` two sections earlier. pandas, offered an iterable of pairs per row, takes each pair
for a cell, which is a reasonable reading of an unreasonable input.

[^2]: The generated `OrderStatus` subclasses `str` and `Enum` together. Python 3.11 added
[`enum.StrEnum`](https://docs.python.org/3/library/enum.html), whose `__str__()` is
`str.__str__()`, so its members print as their values. OpenAPI Generator's [python
page](https://openapi-generator.tech/docs/generators/python/) gives the generator's language
version as 3.11+, and the template writes the mixin form that predates it. pandas 3.0 infers
`dtype="str"` for a collection of strings, and [the text
guide](https://pandas.pydata.org/docs/user_guide/text.html) says that without PyArrow it is backed by
a NumPy object array. An `OrderStatus` member is a `str`, so it goes in as it came, class
and all.

[^3]: Speakeasy's [pagination page](https://www.speakeasy.com/docs/sdks/customize/runtime/pagination)
shows the loop as `while response is not None`, which is Python, and then says the `next()`
function returns `nil, nil` when there are no more pages, which is Go. The page names no language
and stands for every target that Speakeasy's [maturity
table](https://www.speakeasy.com/docs/sdks/languages/maturity) marks for pagination, so one sentence
ended up in two.