Enum datafusion_physical_plan::ColumnarValue

pub enum ColumnarValue {
    Array(Arc<dyn Array>),
    Scalar(ScalarValue),
}

The result of evaluating an expression.

ColumnarValue::Scalar represents a single value repeated any number of times. This is an important performance optimization for handling values that do not change across rows.

ColumnarValue::Array represents a column of data, stored as an Arrow ArrayRef

A slice of ColumnarValues logically represents a table, with each column having the same number of rows. This means that all Arrays are the same length.

Example

A ColumnarValue::Array with an array of 5 elements and a ColumnarValue::Scalar with the value 100

┌──────────────┐
│ ┌──────────┐ │
│ │   "A"    │ │
│ ├──────────┤ │
│ │   "B"    │ │
│ ├──────────┤ │
│ │   "C"    │ │
│ ├──────────┤ │
│ │   "D"    │ │        ┌──────────────┐
│ ├──────────┤ │        │ ┌──────────┐ │
│ │   "E"    │ │        │ │   100    │ │
│ └──────────┘ │        │ └──────────┘ │
└──────────────┘        └──────────────┘

 ColumnarValue::        ColumnarValue::
      Array                 Scalar

Logically represents the following table:

Column 1	Column 2
A	100
B	100
C	100
D	100
E	100

Performance Notes

When implementing functions or operators, it is important to consider the performance implications of handling scalar values.

Because all functions must handle ArrayRef, it is convenient to convert ColumnarValue::Scalars using Self::into_array. For example, ColumnarValue::values_to_arrays converts multiple columnar values into arrays of the same length.

However, it is often much more performant to provide a different, implementation that handles scalar values differently

Variants

Array(Arc<dyn Array>)

Array of values

Scalar(ScalarValue)

A single value

Implementations

impl ColumnarValue

pub fn data_type(&self) -> DataType

pub fn into_array( self, num_rows: usize, ) -> Result<Arc<dyn Array>, DataFusionError>

Convert a columnar value into an Arrow ArrayRef with the specified number of rows. Self::Scalar is converted by repeating the same scalar multiple times which is not as efficient as handling the scalar directly.

See Self::values_to_arrays to convert multiple columnar values into arrays of the same length.

Errors

Errors if self is a Scalar that fails to be converted into an array of size

pub fn create_null_array(num_rows: usize) -> ColumnarValue

null columnar values are implemented as a null array in order to pass batch num_rows

pub fn values_to_arrays( args: &[ColumnarValue], ) -> Result<Vec<Arc<dyn Array>>, DataFusionError>

Converts ColumnarValues to ArrayRefs with the same length.

Performance Note

This function expands any ScalarValue to an array. This expansion permits using a single function in terms of arrays, but it can be inefficient compared to handling the scalar value directly.

Thus, it is recommended to provide specialized implementations for scalar values if performance is a concern.

Errors

If there are multiple array arguments that have different lengths

pub fn cast_to( &self, cast_type: &DataType, cast_options: Option<&CastOptions<'static>>, ) -> Result<ColumnarValue, DataFusionError>

Cast’s this ColumnarValue to the specified DataType

Trait Implementations

impl Clone for ColumnarValue

fn clone(&self) -> ColumnarValue

Returns a copy of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for ColumnarValue

fn fmt(&self, f: &mut Formatter<'_>) -> Result<(), Error>

Formats the value using the given formatter.

impl From<Arc<dyn Array>> for ColumnarValue

fn from(value: Arc<dyn Array>) -> ColumnarValue

Converts to this type from the input type.

impl From<ScalarValue> for ColumnarValue

fn from(value: ScalarValue) -> ColumnarValue

Converts to this type from the input type.