Results Wrappers

Most endpoint methods return a lazy results wrapper instead of an eager list.

Shared interface

The main wrapper type is BaseResults. It exposes:

• to_dicts()
• to_json(indent: bool = False)
• to_arrow()
• to_polars()
• to_ipc_stream(schema=None, *, batch_size=128_000, queue_maxsize=4, compression=None)
• refresh()

The advanced hunting endpoint uses a dedicated wrapper, but it follows the same materialization interface.

Fetch behavior

The wrapper does not fetch on construction. The first terminal method triggers the HTTP request and caches the result in memory.

Pagination behavior

• collection endpoints paginate automatically by default
• if a query sets $top or $skip, the wrapper performs only that request
• single-item methods use single=True and normalize the response to a one-record list when materialized as dictionaries

File-backed behavior

Some wrappers use files=True. In that case the initial response is expected to contain export URLs, and the wrapper downloads the export blobs through ViaFiles before returning the final materialized result.

Materialization choices

• to_dicts() is the simplest Python-native format
• to_json() is useful for logging or serialization
• to_arrow() returns a pyarrow.Table
• to_polars() returns a Polars DataFrame
• refresh() clears the cache so the next terminal method fetches again

Streaming for memory-limited runtimes

to_ipc_stream() is an async terminal that streams results as Arrow IPC stream byte chunks instead of building a cached table. Each page is fetched, serialized into a RecordBatch, emitted, and released, so peak memory stays close to a single batch — suitable for exporting millions of rows from a 2 GiB Azure Function.

Unlike the other terminal methods, it is not cached: it issues fresh requests on every call and never populates the internal container. An explicit pyarrow.Schema is required (from schema= or the wrapper's SCHEMA) because the IPC stream header is written before any rows are fetched.

async for chunk in results.to_ipc_stream(compression="zstd"):
    ...  # forward each chunk to a streaming HTTP response

See Stream results as Arrow IPC.

See also

• Work with result formats
• Lazy results and caching

API

BaseResults

BaseResults(
    endpoint: BaseEndpoint,
    params: dict[str, str],
    *,
    path: str | None = None,
    single: bool = False,
    files: bool = False,
    method: str = "GET",
    request_kwargs: dict[str, Any] | None = None,
    use_concurrent_skip_pagination: bool | None = None,
    skip_page_size: int | None = None,
    skip_max_concurrent: int | None = None,
)

Base results wrapper for API endpoints.

This class provides lazy evaluation of results, deferring HTTP requests until a terminal method is called (e.g. to_dicts, to_arrow, to_polars, refresh).

to_dicts

to_dicts() -> list[dict]

Materialize results into a list of dicts.

to_json

to_json(indent: bool = False) -> str | bytes

Materialize results into a JSON string.

Returns a str if indent=False, bytes if indent=True (since orjson returns bytes). You can decode the bytes to get a str if needed.

to_arrow

to_arrow() -> Table

Materialize results into a PyArrow Table.

to_polars

to_polars() -> DataFrame

Materialize results into a Polars DataFrame.

to_ipc_stream async

to_ipc_stream(
    schema: Schema | None = None,
    *,
    batch_size: int = 128000,
    queue_maxsize: int = 4,
    compression: str | None = None,
) -> AsyncIterator[bytes]

Stream results as Arrow IPC stream byte chunks for memory-limited runtimes.

Unlike the cached terminal methods (to_dicts, to_json, to_arrow, to_polars), this method never materializes the full result set in memory. Each page is fetched, serialized into an Arrow RecordBatch, emitted as Arrow IPC stream bytes, and released, so peak memory stays close to a single batch. This makes it suitable for exporting millions of rows from, for example, a 2 GiB Azure Function.

The stream is not cached: it issues fresh requests on every call and does not populate or reuse the internal container used by the other terminal methods, so refresh() is unnecessary here.

Parameters:

Name	Type	Description	Default
schema	Schema | None	Explicit pyarrow.Schema for the emitted stream. Defaults to the wrapper's SCHEMA. An explicit schema is required because the IPC stream header is written before any rows are fetched and therefore cannot be inferred.	None
batch_size	int	Target row count per emitted RecordBatch.	128000
queue_maxsize	int	Bounded backpressure between page fetching and batch serialization.	4
compression	str | None	Optional Arrow IPC codec (e.g. "zstd") supported by your PyArrow build.	None

Yields:

Type	Description
AsyncIterator[bytes]	Arrow IPC stream byte chunks. Concatenated, they form a valid Arrow
AsyncIterator[bytes]	IPC stream readable via pyarrow.ipc.open_stream.

Raises:

Type	Description
ValueError	If no schema is available from schema or SCHEMA.

Example

results = client.machines.get_all()
async for chunk in results.to_ipc_stream(compression="zstd"):
    ...  # forward each chunk to a streaming HTTP response

refresh

refresh() -> BaseResults

Clear any cached results, forcing the next materialization to re-query the API.