Python API Walkthrough

This walkthrough uses the deltafunnel package. Install it from PyPI, or build the local wheel first with cargo xtask python-package-check when developing the repository.

Create a session

from deltafunnel import Session

connection_string = (
    "server=tcp:localhost,1433;"
    "database=warehouse;"
    "user id=etl_user;"
    "password=REPLACE_ME;"
    "encrypt=true;"
    "TrustServerCertificate=yes"
)

session = Session(default_mssql_connection_string=connection_string)

Delta Funnel accepts a SQL Server ADO-style connection string. It does not require an ODBC DSN.

Register a Delta source

orders = session.delta_lake("file:///path/to/orders-delta", name="orders")

Passing name registers the source immediately so SQL can reference it. Calling session.delta_lake(...) without name returns a pending source; call .alias("orders") before using it in SQL.

Read a private S3 Delta table from a local shell

storage_options are forwarded to the underlying object-store builder that Delta Funnel uses for S3 access. On the current S3 path, Delta Funnel does not auto-load shell AWS_* variables, AWS_PROFILE, or shared AWS config and credentials files.

For a private S3 Delta table from a local shell, pass explicit credentials and region in storage_options:

• AWS_ACCESS_KEY_ID
• AWS_SECRET_ACCESS_KEY
• AWS_SESSION_TOKEN as optional
• AWS_REGION

Delta Funnel also accepts these common lowercase aliases:

• aws_access_key_id
• aws_secret_access_key
• aws_session_token
• aws_region
• region

This works reliably:

import os
from deltafunnel import Session

storage_options = {
    "AWS_REGION": "us-east-1",
    "AWS_ACCESS_KEY_ID": os.environ["AWS_ACCESS_KEY_ID"],
    "AWS_SECRET_ACCESS_KEY": os.environ["AWS_SECRET_ACCESS_KEY"],
}
if os.environ.get("AWS_SESSION_TOKEN"):
    storage_options["AWS_SESSION_TOKEN"] = os.environ["AWS_SESSION_TOKEN"]

source = Session().delta_lake(
    "s3://<private-bucket>/<delta-table>",
    storage_options=storage_options,
    name="source",
)

This is not enough by itself:

Session().delta_lake(
    "s3://<private-bucket>/<delta-table>",
    storage_options={"region": "us-east-1"},
    name="source",
)

region is a supported key, but region alone only sets region. It does not provide credentials.

If the same table works in deltalake but fails in deltafunnel, the likely cause is a credential-discovery path mismatch, not a Delta snapshot or protocol problem.

Transform rows with SQL

daily_orders = session.table_from_sql("""
    select customer_id, order_date, total_amount
    from orders
    where order_date >= date '2026-01-01'
""")

table_from_sql creates a lazy table. It does not execute rows until a write or dry run needs the plan.

Write to SQL Server

report = daily_orders.write_to_mssql(
    schema="dbo",
    table="daily_orders",
    load_mode="create_and_load",
)

The returned report is a plain Python dict converted from Rust report types. Report formatting is designed to avoid exposing connection strings, credentials, and raw row values.