Struct RunEndBuffer

pub struct RunEndBuffer<E: ArrowNativeType> { /* private fields */ }

A slice-able buffer of monotonically increasing, positive integers used to store run-ends

Logical vs Physical

A RunEndBuffer is used to encode runs of the same value, the index of each run is called the physical index. The logical index is then the corresponding index in the logical run-encoded array, i.e. a single run of length 3, would have the logical indices 0..3.

Each value in RunEndBuffer::values is the cumulative length of all runs in the logical array, up to that physical index.

Consider a RunEndBuffer containing [3, 4, 6]. The maximum physical index is 2, as there are 3 values, and the maximum logical index is 5, as the maximum run end is 6. The physical indices are therefore [0, 0, 0, 1, 2, 2]

    ┌─────────┐        ┌─────────┐           ┌─────────┐
    │    3    │        │    0    │ ─┬──────▶ │    0    │
    ├─────────┤        ├─────────┤  │        ├─────────┤
    │    4    │        │    1    │ ─┤ ┌────▶ │    1    │
    ├─────────┤        ├─────────┤  │ │      ├─────────┤
    │    6    │        │    2    │ ─┘ │ ┌──▶ │    2    │
    └─────────┘        ├─────────┤    │ │    └─────────┘
     run ends          │    3    │ ───┘ │  physical indices
                       ├─────────┤      │
                       │    4    │ ─────┤
                       ├─────────┤      │
                       │    5    │ ─────┘
                       └─────────┘
                     logical indices

Slicing

In order to provide zero-copy slicing, this container stores a separate offset and length

For example, a RunEndBuffer containing values [3, 6, 8] with offset and length 4 would describe the physical indices 1, 1, 2, 2

For example, a RunEndBuffer containing values [6, 8, 9] with offset 2 and length 5 would describe the physical indices 0, 0, 0, 0, 1

Implementations

impl<E> RunEndBuffer<E>

where E: ArrowNativeType,

pub fn new(run_ends: ScalarBuffer<E>, offset: usize, len: usize) -> Self

Create a new RunEndBuffer from a ScalarBuffer, an offset and len

Panics

• buffer does not contain strictly increasing values greater than zero
• the last value of buffer is less than offset + len

pub unsafe fn new_unchecked( run_ends: ScalarBuffer<E>, offset: usize, len: usize, ) -> Self

Create a new RunEndBuffer from an ScalarBuffer, an offset and len

Safety

• buffer must contain strictly increasing values greater than zero
• The last value of buffer must be greater than or equal to offset + len

pub fn offset(&self) -> usize

Returns the logical offset into the run-ends stored by this buffer

pub fn len(&self) -> usize

Returns the logical length of the run-ends stored by this buffer

pub fn is_empty(&self) -> bool

Returns true if this buffer is empty

pub fn shrink_to_fit(&mut self)

Free up unused memory.

pub fn values(&self) -> &[E]

Returns the values of this RunEndBuffer not including any offset

pub fn max_value(&self) -> usize

Returns the maximum run-end encoded in the underlying buffer

pub fn get_physical_index(&self, logical_index: usize) -> usize

Performs a binary search to find the physical index for the given logical index

The result is arbitrary if logical_index >= self.len()

pub fn get_start_physical_index(&self) -> usize

Returns the physical index at which the logical array starts

pub fn get_end_physical_index(&self) -> usize

Returns the physical index at which the logical array ends

pub fn slice(&self, offset: usize, len: usize) -> Self

Slices this RunEndBuffer by the provided offset and length

pub fn inner(&self) -> &ScalarBuffer<E>

Returns the inner ScalarBuffer

pub fn into_inner(self) -> ScalarBuffer<E>

Returns the inner ScalarBuffer, consuming self

Trait Implementations

impl<E: Clone + ArrowNativeType> Clone for RunEndBuffer<E>

fn clone(&self) -> RunEndBuffer<E>

Returns a copy of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl<E: Debug + ArrowNativeType> Debug for RunEndBuffer<E>

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.