datafusion_functions_aggregate_common::merge_arrays

Function merge_ordered_arrays

Source
pub fn merge_ordered_arrays(
    values: &mut [VecDeque<ScalarValue>],
    ordering_values: &mut [VecDeque<Vec<ScalarValue>>],
    sort_options: &[SortOptions],
) -> Result<(Vec<ScalarValue>, Vec<Vec<ScalarValue>>)>
Expand description

This functions merges values array (&[Vec<ScalarValue>]) into single array Vec<ScalarValue> Merging done according to ordering values stored inside ordering_values (&[Vec<Vec<ScalarValue>>]) Inner Vec<ScalarValue> in the ordering_values can be thought as ordering information for the each ScalarValue in the values array. Desired ordering specified by sort_options argument (Should have same size with inner Vec<ScalarValue> of the ordering_values array).

As an example values can be [ [1, 2, 3, 4, 5], [1, 2, 3, 4], [1, 2, 3, 4, 5, 6], ] In this case we will be merging three arrays (doesn’t have to be same size) and produce a merged array with size 15 (sum of 5+4+6) Merging will be done according to ordering at ordering_values vector. As an example ordering_values can be [ [(1, a), (2, b), (3, b), (4, a), (5, b) ], [(1, a), (2, b), (3, b), (4, a) ], [(1, b), (2, c), (3, d), (4, e), (5, a), (6, b) ], ] For each ScalarValue in the values we have a corresponding Vec<ScalarValue> (like timestamp of it) for the example above sort_options will have size two, that defines ordering requirement of the merge. Inner Vec<ScalarValue>s of the ordering_values will be compared according sort_options (Their sizes should match)