Module utils

This module provides the bisect function, which implements binary search.

Modules

datafusion_strsim

Adopted from strsim-rs for string similarity metrics

expr

Expression utilities

memory

This module provides a function to estimate the memory size of a HashTable prior to allocation

proxy

VecAllocExt and RawTableAllocExt to help tracking of memory allocations

string_utils

Utilities for working with strings

Structs

SingleRowListArrayBuilder

Creates single element ListArray, LargeListArray and FixedSizeListArray from other arrays

Functions

array_into_fixed_size_list_arrayDeprecated

array_into_fixed_size_list_array_with_field_nameDeprecated

array_into_large_list_arrayDeprecated

Wrap an array into a single element LargeListArray. For example [1, 2, 3] would be converted into [[1, 2, 3]]

array_into_large_list_array_with_field_nameDeprecated

array_into_list_arrayDeprecated

Wrap an array into a single element ListArray. For example [1, 2, 3] would be converted into [[1, 2, 3]]

array_into_list_array_nullableDeprecated

Wrap an array into a single element ListArray. For example [1, 2, 3] would be converted into [[1, 2, 3]] The field in the list array is nullable.

array_into_list_array_with_field_nameDeprecated

arrays_into_list_array

Wrap arrays into a single element ListArray.

base_type

Get the base type of a data type.

bisect

This function searches for a tuple of given values (target) among the given rows (item_columns) using the bisection algorithm. It assumes that item_columns is sorted according to sort_options and returns the insertion index of target. Template argument SIDE being true/false means left/right insertion.

coerced_fixed_size_list_to_list

Recursively coerce and FixedSizeList elements to List

coerced_type_with_base_type_only

A helper function to coerce base type in List.

combine_limit

Computes the skip and fetch parameters of a single limit that would be equivalent to two consecutive limits with the given skip/fetch parameters.

compare_rows

This function compares two tuples depending on the given sort options.

evaluate_partition_ranges

Given a list of 0 or more already sorted columns, finds the partition ranges that would partition equally across columns.

find_bisect_point

This function searches for a tuple of given values (target) among a slice of the given rows (item_columns) using the bisection algorithm. The slice starts at the index low and ends at the index high. The boolean-valued function compare_fn specifies whether we bisect on the left (by returning false), or on the right (by returning true) when we compare the target value with the current value as we iteratively bisect the input.

find_indices

Find indices of each element in targets inside items. If one of the elements is absent in items, returns an error.

fixed_size_list_to_arrays

Helper function to convert a FixedSizeListArray into a vector of ArrayRefs.

get_at_indices

This function “takes” the elements at indices from the slice items.

get_available_parallelism

Returns the estimated number of threads available for parallel execution.

get_row_at_idx

Given column vectors, returns row at idx.

is_sortedDeprecated

Checks whether the given index sequence is monotonically non-decreasing.

linear_search

This function searches for a tuple of given values (target) among the given rows (item_columns) via a linear scan. It assumes that item_columns is sorted according to sort_options and returns the insertion index of target. Template argument SIDE being true/false means left/right insertion.

list_ndims

Compute the number of dimensions in a list data type.

list_to_arrays

Helper function to convert a ListArray into a vector of ArrayRefs.

longest_consecutive_prefix

This function finds the longest prefix of the form 0, 1, 2, … within the collection sequence. Examples:

merge_and_order_indices

Merges collections first and second, removes duplicates and sorts the result, returning it as a Vec.

project_schema

Applies an optional projection to a SchemaRef, returning the projected schema

quote_identifier

Wraps identifier string in double quotes, escaping any double quotes in the identifier by replacing it with two double quotes

search_in_slice

This function searches for a tuple of given values (target) among a slice of the given rows (item_columns) via a linear scan. The slice starts at the index low and ends at the index high. The boolean-valued function compare_fn specifies the stopping criterion.

set_difference

Calculates the set difference between sequences first and second, returning the result as a Vec. Preserves the ordering of first.

transpose

Transposes the given vector of vectors.