polars_parquet::arrow::read

Function column_iter_to_arrays

Source 
pub fn column_iter_to_arrays(
    columns: Vec<BasicDecompressor>,
    types: Vec<&PrimitiveType>,
    field: Field,
    filter: Option<Filter>,
) -> PolarsResult<Box<dyn Array>>
Expand description

An iterator adapter that maps multiple iterators of [PagesIter] into an iterator of Arrays.

For a non-nested datatypes such as ArrowDataType::Int32, this function requires a single element in columns and types. For nested types, columns must be composed by all parquet columns with associated types types.

The arrays are guaranteed to be at most of size chunk_size and data type field.dtype.