polars_parquet::parquet::encoding::hybrid_rle::gatherer

Trait HybridRleGatherer

Source
pub trait HybridRleGatherer<O: Clone> {
    type Target;

    // Required methods
    fn target_reserve(&self, target: &mut Self::Target, n: usize);
    fn target_num_elements(&self, target: &Self::Target) -> usize;
    fn hybridrle_to_target(&self, value: u32) -> ParquetResult<O>;
    fn gather_one(
        &self,
        target: &mut Self::Target,
        value: O,
    ) -> ParquetResult<()>;
    fn gather_repeated(
        &self,
        target: &mut Self::Target,
        value: O,
        n: usize,
    ) -> ParquetResult<()>;

    // Provided methods
    fn gather_slice(
        &self,
        target: &mut Self::Target,
        source: &[u32],
    ) -> ParquetResult<()> { ... }
    fn gather_chunk(
        &self,
        target: &mut Self::Target,
        source: &<u32 as Unpackable>::Unpacked,
    ) -> ParquetResult<()> { ... }
    fn gather_bitpacked_all(
        &self,
        target: &mut Self::Target,
        decoder: Decoder<'_, u32>,
    ) -> ParquetResult<()> { ... }
    fn gather_bitpacked_limited<'a>(
        &self,
        target: &mut Self::Target,
        decoder: Decoder<'a, u32>,
        limit: usize,
    ) -> ParquetResult<BufferedBitpacked<'a>> { ... }
    fn gather_bitpacked<'a>(
        &self,
        target: &mut Self::Target,
        decoder: Decoder<'a, u32>,
        limit: Option<usize>,
    ) -> ParquetResult<(usize, Option<HybridRleBuffered<'a>>)> { ... }
}
Expand description

Trait that describes what to do with consumed Hybrid-RLE Encoded values.

This is quite a general trait that provides a lot of open space as to how to handle the Hybrid-RLE encoded values. There is also the Translator trait that is usually good enough if you want just want to map values to another set of values and collect them into a vector.

Although, this trait might seem quite over-engineered (it might be), it is very useful for performance. This allows for usage of the properties that HybridRleDecoder provides and for definition of efficient procedures for collecting slices, chunks, repeated elements and bit-packed elements.

The Translator doc-comment has a good description of why this trait is needed.

Required Associated Types§

Required Methods§

Source

fn target_reserve(&self, target: &mut Self::Target, n: usize)

Source

fn target_num_elements(&self, target: &Self::Target) -> usize

Source

fn hybridrle_to_target(&self, value: u32) -> ParquetResult<O>

Source

fn gather_one(&self, target: &mut Self::Target, value: O) -> ParquetResult<()>

Source

fn gather_repeated( &self, target: &mut Self::Target, value: O, n: usize, ) -> ParquetResult<()>

Provided Methods§

Source

fn gather_slice( &self, target: &mut Self::Target, source: &[u32], ) -> ParquetResult<()>

Source

fn gather_chunk( &self, target: &mut Self::Target, source: &<u32 as Unpackable>::Unpacked, ) -> ParquetResult<()>

Source

fn gather_bitpacked_all( &self, target: &mut Self::Target, decoder: Decoder<'_, u32>, ) -> ParquetResult<()>

Source

fn gather_bitpacked_limited<'a>( &self, target: &mut Self::Target, decoder: Decoder<'a, u32>, limit: usize, ) -> ParquetResult<BufferedBitpacked<'a>>

Source

fn gather_bitpacked<'a>( &self, target: &mut Self::Target, decoder: Decoder<'a, u32>, limit: Option<usize>, ) -> ParquetResult<(usize, Option<HybridRleBuffered<'a>>)>

Implementors§