Benchmarks

We compared Delta Arrow Reader with delta-rs, DuckDB, Polars, and Daft on four workloads. Delta Arrow Reader had the lowest median time on every workload an alternative could run. Polars completed all four workloads. Daft completed the text projection but rejected the other three because it does not support deletion vectors.

These numbers describe the workloads and machine shown below. Different tables, queries, and hardware can produce different results.

Results

The table below shows how long each reader took to load the Delta snapshot and stream the full result. Process startup and Python import time are not included. Each value is the median after one warmup run.

delta-rs took 1.34 times as long on the mixed-column projection, 3.73 times as long on the one-row deletion-vector read, and 4.80 times as long on the full deletion-vector scan.

Compared with Delta Arrow Reader, Polars took 3.39 times as long on the mixed-column projection, 1.55 times as long on the text projection, 22.7 times as long on the one-row deletion-vector read, and 7.39 times as long on the full deletion-vector scan. Daft took 1.95 times as long on the one workload it could run.

On the text projection, Delta Arrow Reader took 2.252-2.739 seconds and delta-rs took 2.447-2.969 seconds. Because those ranges overlap, we do not treat the difference between their medians as conclusive. The Delta Arrow Reader range did not overlap the Polars or Daft range on that workload.

Speed is only part of the picture. The charts and table below show median CPU time and median peak memory use:

Each cell shows CPU time followed by peak memory. Delta Arrow Reader used slightly more CPU and memory than delta-rs on the two projection workloads. It used less memory than DuckDB, Polars, and Daft on every comparable workload. On the deletion-vector workloads, it also used less CPU and memory than every alternative.

Workloads

The mixed-column workload is based on a real application query. It reads 28 string, numeric, boolean, and time columns from a 14.5-million-row table. It then reads 14 columns from a small reference table with two deletion vectors. The larger table contains 1,151 active Parquet files and 448 MB of compressed data.

The original schema is private, so the published query shapes use neutral table and column names. They preserve the number and types of projected columns without revealing the source data.

The text workload reads six columns from the public version-4 RedPajama StackExchange Delta snapshot. It contains 64 Parquet files, 3.03 GB of compressed data, and 2,697,774 rows.

Deletion vectors mark rows as deleted without rewriting their Parquet files. To test them, we generated a table with 200 files and 200 million physical rows. Each file has one deleted row. One query stops after the first available row. The other reads all 199,999,800 rows that remain. Together, these queries show both the startup cost of handling deletion vectors and the cost of applying them across a full scan.

Method

All readers used the same Delta snapshots stored in local MinIO. Each reader used the machine's 16 logical CPUs. Results were consumed as Arrow batches without retaining the full result, and output batches were limited to 8,192 rows.

For each workload, we ran every supported reader once to warm up the system and discarded those results. We then used balanced, process-isolated run orders: eight measured rounds for each four-reader workload and ten rounds for the five-reader text workload. Every reader appeared in every position twice, and every ordered pair of adjacent readers appeared twice. The exact run-order generator is checked in.

Delta Arrow Reader and delta-rs used two builds of the same Rust harness. Each reader kept the DataFusion version from its released dependency graph: 54.1.0 for Delta Arrow Reader and 53.1.0 for delta-rs. Both builds used Arrow and Parquet 58.4.0. DuckDB, Polars, and Daft each ran in a separate Python process and returned batches through PyArrow 25.0.1.

Wall time covers table opening through consumption of the final Arrow batch. The timer starts after process startup and, for the Python readers, after imports. CPU time and peak RSS come from Linux process accounting around the whole child process, so those figures include startup and imports.

All 146 measured processes completed successfully. Their row counts and logical schemas matched the frozen workload expectations. Both deletion vectors in the reference table removed the same 43 rows, and the generated table removed the expected 200 rows.

The anonymized per-run results include the timing, resource use, and parity outcome for all 146 measured processes, along with the three unsupported Daft probes. The chart generator validates the file and computes the plotted medians.

The readers did not always represent values in memory in exactly the same way. We allowed differences in string encoding, timestamp units, dictionary encoding, and where each reader split its Arrow batches because none of them changed the logical result.

Daft was not timed on the mixed-column or deletion-vector workloads. A separate capability probe confirmed that Daft 0.7.24 rejects tables that advertise the Delta deletion-vector reader feature. We did not bypass that check or ask Daft to ignore deleted rows.

Environment

We ran the benchmark on September 1, 2026, using one machine with the following hardware and software:

• AMD Ryzen 7 8845HS, with 8 cores and 16 threads
• 27.95 GiB of memory
• Crucial P3 Plus NVMe SSD
• Fedora Linux 43 with Linux 6.19.14

We kept all benchmark data in local MinIO so that public-network delays would not affect the results. Hardware and system load still affect absolute times, so we will rerun these comparisons when the readers or their execution settings change.

Focused benchmarks

• Selective S3 case study compares four existing selective queries across Lakehouse//RT Small (Beta), Serverless SQL Small, Delta Arrow Reader, and delta-rs, with every anonymized repetition available.
• Lazy and eager scan metadata compares when Delta scan metadata is loaded and reused across repeated queries.
• Parquet row-filter predicate decoding compares decoding only predicate columns with decoding unrelated columns as well.
• Parquet page-index range reads measures page-level reads for localized and scattered row-filter matches.
• Adaptive Parquet range planning compares exact and merged byte-range reads under controlled transport conditions.