Skip to content

DeltaTable

Open a table once, then reuse it for queries. The constructor below opens it; to_reader() selects rows and columns. Use schema to see the available columns, and refresh() when you want newer data.

For a complete example, use the Python quickstart. For SQL, register the loaded table with a Session. Opening-time settings are explained in Configuration.

DeltaTable 

DeltaTable(
    location: str | PathLike[str],
    *,
    version: int | None = None,
    storage_options: Mapping[str, str] | None = None,
    warmup: Literal["automatic", "none", "query_planning"] = "automatic",
    execution_options: ScanExecutionOptions | None = None,
)

Open a Delta table for repeated queries against the same version.

Use to_reader() to read batches or register the table with a Session to query it with SQL. The loaded version stays fixed even if another process writes to the table. Call refresh() to get a new table with the latest version.

Examples:

Replace the path with your table's directory or S3 URL.

from delta_arrow_reader import DeltaTable

table = DeltaTable("path/to/table")
with table.to_reader(limit=10) as reader:
    print(reader.read_all())

Parameters:

Name Type Description Default
location str | PathLike[str]

Local directory or storage URL containing the Delta table. Accepts a string or a path object that resolves to a string.

 required
version int | None

Version to load. None selects the latest available version. An explicit version must be an integer from 0 through 2**64 - 1; booleans are not accepted.

 None
storage_options Mapping[str, str] | None

Storage configuration as string keys and string values, for example {"aws_region": "us-west-2"} for S3.

 None
warmup Literal['automatic', 'none', 'query_planning']

How much preparation to do when opening the table. "automatic" prepares reusable file information and checks connection speed on supported S3 stores; local tables skip this extra work. "query_planning" prepares file information without test reads of data. "none" leaves that preparation to each query. Both explicit modes skip the connection check and keep partial-page reads inactive.

 'automatic'
execution_options ScanExecutionOptions | None

Defaults for this table's scan() and to_reader() calls. None uses ScanExecutionOptions(). SQL registrations have their own defaults.

 None

The automatic connection check has a five-second budget and may finish sooner. Loading file information takes additional time. Keep using the same table object to reuse this work; query results are not cached.

Raises:

Type Description
TypeError

An argument has the wrong type, including a Boolean version.
ValueError

The version is negative or the warmup mode is unknown.
OverflowError

The version exceeds 2**64 - 1.
DeltaReaderError

The table cannot be loaded or prepared.

to_reader 

to_reader(
    *,
    columns: list[str] | tuple[str, ...] | None = None,
    filters: list[_Filter] | list[list[_Filter]] | None = None,
    limit: int | None = None,
    target_partitions: int | None = None,
    execution_options: ScanExecutionOptions | None = None,
) -> pyarrow.RecordBatchReader

Select rows and columns, then read them as PyArrow batches.

Use a with block so the reader closes even when you stop early. Iterate over batches to process the result incrementally, or use read_all() to collect it in memory. Rows have no guaranteed order; use SQL with ORDER BY when order matters.

Examples:

For a table with id and region columns:

with table.to_reader(
    columns=["id", "region"],
    filters=[("id", ">=", 100)],
    limit=10,
) as reader:
    for batch in reader:
        print(batch.to_pydict())

Parameters:

Name Type Description Default
columns list[str] | tuple[str, ...] | None

Column names in output order. None selects all columns. An empty list selects no columns but preserves row counts.

 None
filters list[_Filter] | list[list[_Filter]] | None

Conditions written as (column, operator, value) tuples. A list of tuples combines conditions with AND. A list of lists combines AND groups with OR. None or [] disables filtering; an empty inner group is true. Filter columns need not be in the output. See the filtering rules below for operators and values.

 None
limit int | None

Maximum number of rows after filtering. None reads all matching rows; 0 returns no rows. Must be a nonnegative integer.

 None
target_partitions int | None

Target number of groups of files to process in parallel. None chooses automatically; an override must be a positive integer. This does not change the table's stored layout.

 None
execution_options ScanExecutionOptions | None

Settings for this scan. None inherits the table's settings. A supplied ScanExecutionOptions replaces the whole settings object for this scan without changing later scans.

 None

Returns:

Type Description
RecordBatchReader

A reader that yields pyarrow.RecordBatch objects and retains its snapshot independently of the table object. Result data is read when you request the first batch.

Raises:

Type Description
TypeError

An argument or filter value has the wrong type. Booleans are not accepted for integer settings or integer filter values.
ValueError

A limit, partition target, operator or filter value is invalid.
OverflowError

An integer does not fit its column type, or a limit or partition target exceeds 2 * sys.maxsize + 1.
DeltaReaderError

A column is unknown or scan planning fails.

Planning may read table metadata before this method returns. If automatic connection preparation was incomplete, an eligible filtered scan retries it once, which can add up to five seconds of test reads. Use warmup="none" or "query_planning" when opening the table to skip that check entirely. Errors during result consumption use PyArrow exceptions.

Filtering rules

Compare with ==, !=, <, <=, > or >=. Use "is" or "is not" with None to test for nulls; other comparisons with None are invalid. For example, [("region", "is not", None)] excludes null regions. [[("region", "==", "east")], [("id", "<", 10)]] selects rows matching either condition. Filters use top-level logical names. Deleted rows remain excluded.

Values must match the column type:

Column type Python value
Boolean bool
Signed integer int within the column's range
Float32, Float64 Finite float; Float32 rounds and rejects overflow
Utf8, LargeUtf8 UTF-8 encodable str
Binary, LargeBinary, FixedSizeBinary bytes; fixed-size values must match the width
Decimal128 Finite decimal.Decimal that exactly fits the precision and scale
Date32 datetime.date, excluding datetime.datetime
Microsecond timestamp datetime.datetime; naive for columns without a timezone, aware otherwise

Float comparisons distinguish -0.0 from 0.0. Decimal conversion is exact and does not depend on the decimal context. Timestamp conversion preserves microseconds without using the process timezone. Timestamps must be exact datetime.datetime instances, not subclasses, and UTC offsets must be exact datetime.timedelta instances or None.

refresh 

refresh() -> DeltaTable

Load the latest snapshot and return it as a new table.

The original table and its readers keep their version, even if refresh fails. The new table keeps the storage, warmup and execution settings.

Examples:

latest = table.refresh()
print(table.version, latest.version)

Returns:

Type Description
DeltaTable

A table at the latest version, which may be the same version as before.

Raises:

Type Description
DeltaReaderError

The latest snapshot cannot be loaded or prepared.

version property 

version: int

The version loaded by this table. It stays fixed until you open a new table.

schema property 

schema: Schema

Column names and types as a pyarrow.Schema, without reading data files.

The schema remains usable after the table object is released.

scan 

scan(
    *,
    columns: list[str] | tuple[str, ...] | None = None,
    filters: list[_Filter] | list[list[_Filter]] | None = None,
    limit: int | None = None,
    target_partitions: int | None = None,
    execution_options: ScanExecutionOptions | None = None,
) -> RecordBatchStream

Plan a scan and return a single-use Arrow stream exporter.

Use to_reader() for a PyArrow reader directly. This lower-level method is for consumers of the Arrow stream protocol. It accepts the same parameters and has the same filtering, planning and error behavior as to_reader().

Examples:

import pyarrow as pa

stream = table.scan(limit=10)
with pa.RecordBatchReader.from_stream(stream) as reader:
    print(reader.read_all())

Returns:

Type Description
RecordBatchStream

An exporter that keeps the table snapshot alive. Once exported, the consuming reader owns cleanup. Close that reader when finished; closing the exporter does not close the consumer.

__arrow_c_schema__ 

__arrow_c_schema__() -> object

Export the schema for Arrow-compatible consumers such as pyarrow.schema().

Usually, use the schema property instead. Each call returns a fresh Arrow schema capsule that owns its resources independently of this table.