polars_json::ndjson

Function iter_unique_dtypes

Source
pub fn iter_unique_dtypes<R: BufRead>(
    reader: &mut R,
    number_of_rows: Option<NonZeroUsize>,
) -> PolarsResult<impl Iterator<Item = ArrowDataType>>
Expand description

Infers the ArrowDataType from an NDJSON file, optionally only using number_of_rows rows.

ยงImplementation

This implementation reads the file line by line and infers the type of each line. It performs both O(N) IO and CPU-bounded operations where N is the number of rows.