Streams and errors

For normal PyArrow reads, use DeltaTable.to_reader() and close the reader with a with block. This page covers failures and the lower-level interface for handing results to another Arrow library.

Handle a failed read

Opening or planning can raise DeltaReaderError. Invalid Python arguments may raise built-in exceptions instead. While consuming rows, catch PyArrow errors around iteration or read_all() as well.

DeltaReaderError

Bases: Exception

A failure while opening a table or preparing a query.

Inspect phase and code to identify the failure without parsing the message. Messages omit sensitive details such as table paths and storage credentials.

Attributes:

Name	Type	Description
`phase`	`str`	The stage that failed, such as `schema` or `datafusion`.
`code`	`str`	The error category, such as `schema_conversion` or `sql_planning`.

Invalid Python arguments can raise built-in exceptions such as TypeError or ValueError instead. Failures while consuming a PyArrow reader use PyArrow exceptions; catch those around iteration or read_all() as well.

phase `instance-attribute`

phase: str

code `instance-attribute`

code: str

Export a scan to an Arrow consumer

Use DeltaTable.scan() when a consumer needs the Arrow stream protocol. to_reader() already performs this conversion for PyArrow. Once a stream is exported, close its consuming reader to release the scan.

RecordBatchStream

Hand a scan to a consumer that supports the Arrow stream protocol.

Created by DeltaTable.scan(); it has no public constructor. For normal PyArrow usage, DeltaTable.to_reader() does this conversion for you.

Examples:

import pyarrow as pa

stream = table.scan(limit=10)
with pa.RecordBatchReader.from_stream(stream) as reader:
    print(reader.read_all())

Each stream can be exported once. After export, close the consuming reader to release resources; closing this object does not close the consumer. If you decide not to export it, call close() to release the scan.

__arrow_c_stream__

__arrow_c_stream__(requested_schema: object | None = None) -> object

Transfer the stream to an Arrow consumer. Usually called by the consumer.

Parameters:

Name	Type	Description	Default
`requested_schema`	`object \| None`	`None` or an Arrow schema capsule identical to the scan schema. This method does not cast or convert the data.	`None`

Returns:

Type	Description
`object`	An Arrow stream capsule that owns the scan resources.

Raises:

Type	Description
`TypeError`	The requested schema is not a capsule.
`ValueError`	The schema capsule is invalid or incompatible.
`NotImplementedError`	The consumer requests an alternate representation.
`RuntimeError`	The stream has already been exported or closed.

A rejected schema request leaves the stream available for another attempt.

close

close() -> None

Release an unexported stream. Repeated calls have no effect.

After export, call close() on the consuming reader instead.

enter

__enter__() -> RecordBatchStream

Return this exporter for use in a with block.

exit

__exit__(
    exc_type: type[BaseException] | None,
    exc_value: BaseException | None,
    traceback: TracebackType | None,
) -> None

Close any unexported stream on leaving the block. Exceptions are not suppressed.

Threads and interruption

GIL behavior, Ctrl+C, and work already in progress

Loading a table, planning a scan, and reading batches release the GIL so other Python threads can run. During loading or planning on Python's main thread, Ctrl+C raises KeyboardInterrupt and cancels the pending operation. Interruption during a batch read reaches the caller as a PyArrow exception.

Synchronous Kernel work already running may still finish after interruption; cancellation does not stop it immediately.