Apache Arrow is a language-independent columnar memory format and software ecosystem for flat and hierarchical data. It enables fast interchange between languages and systems through zero-copy reads, avoiding serialization overhead. section 6.5
Arrow connects analytical tools without forcing repeated copying or transformation. DuckDB can read Arrow-shaped data through ADBC, while Python workflows can exchange tables with Polars and other libraries. section 1.7.1 section 6.5
Move a DuckDB relation into Arrow
Convert a DuckDB relation to an Arrow table, then filter and select it with PyArrow. The resulting table retains typed columns such as string and int64 while avoiding a serialization step. section 6.5
import pyarrow.compute as pc
(arrow_table
.filter(pc.field("Region") == "NEAR EAST") #1
.select(["Country", "Region", "Population"]) #2
.slice(length=5) #3
)
Filter the Arrow table for one region, select three columns, and inspect the first five rows. section 6.5
import polars
population_table = con.table("population")
(population_table
.limit(5) #1
.pl() #2
[["Country", "Region", "Population"]] #3
)
Convert a DuckDB relation to a Polars DataFrame, limit it to five rows, and select the Country, Region, and Population columns. section 6.5
