Struct datafusion_common::UnnestOptions

pub struct UnnestOptions {
    pub preserve_nulls: bool,
}

Options for unnesting a column that contains a list type, replicating values in the other, non nested rows.

Conceptually this operation is like joining each row with all the values in the list column.

If preserve_nulls is false, nulls and empty lists from the input column are not carried through to the output. This is the default behavior for other systems such as ClickHouse and DuckDB

If preserve_nulls is true (the default), nulls from the input column are carried through to the output.

Examples

Unnest(c1), preserve_nulls: false

     ┌─────────┐ ┌─────┐                ┌─────────┐ ┌─────┐
     │ {1, 2}  │ │  A  │   Unnest       │    1    │ │  A  │
     ├─────────┤ ├─────┤                ├─────────┤ ├─────┤
     │  null   │ │  B  │                │    2    │ │  A  │
     ├─────────┤ ├─────┤ ────────────▶  ├─────────┤ ├─────┤
     │   {}    │ │  D  │                │    3    │ │  E  │
     ├─────────┤ ├─────┤                └─────────┘ └─────┘
     │   {3}   │ │  E  │                    c1        c2
     └─────────┘ └─────┘
       c1         c2

Unnest(c1), preserve_nulls: true

     ┌─────────┐ ┌─────┐                ┌─────────┐ ┌─────┐
     │ {1, 2}  │ │  A  │   Unnest       │    1    │ │  A  │
     ├─────────┤ ├─────┤                ├─────────┤ ├─────┤
     │  null   │ │  B  │                │    2    │ │  A  │
     ├─────────┤ ├─────┤ ────────────▶  ├─────────┤ ├─────┤
     │   {}    │ │  D  │                │  null   │ │  B  │
     ├─────────┤ ├─────┤                ├─────────┤ ├─────┤
     │   {3}   │ │  E  │                │    3    │ │  E  │
     └─────────┘ └─────┘                └─────────┘ └─────┘
       c1         c2                        c1        c2

Fields

preserve_nulls: bool

Should nulls in the input be preserved? Defaults to true

Implementations

impl UnnestOptions

pub fn new() -> Self

Create a new UnnestOptions with default values

pub fn with_preserve_nulls(self, preserve_nulls: bool) -> Self

Set the behavior with nulls in the input as described on Self

Trait Implementations

impl Clone for UnnestOptions

fn clone(&self) -> UnnestOptions

Returns a copy of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for UnnestOptions

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Default for UnnestOptions

fn default() -> Self

Returns the “default value” for a type.

impl Hash for UnnestOptions

fn hash<__H: Hasher>(&self, state: &mut __H)

Feeds this value into the given Hasher.

fn hash_slice<H>(data: &[Self], state: &mut H)

where H: Hasher, Self: Sized,

Feeds a slice of this type into the given Hasher.

impl PartialEq for UnnestOptions

fn eq(&self, other: &UnnestOptions) -> bool

Tests for self and other values to be equal, and is used by ==.

fn ne(&self, other: &Rhs) -> bool

Tests for !=. The default implementation is almost always sufficient, and should not be overridden without very good reason.

impl PartialOrd for UnnestOptions

fn partial_cmp(&self, other: &UnnestOptions) -> Option<Ordering>

This method returns an ordering between self and other values if one exists.

fn lt(&self, other: &Rhs) -> bool

Tests less than (for self and other) and is used by the < operator.

fn le(&self, other: &Rhs) -> bool

Tests less than or equal to (for self and other) and is used by the <= operator.

fn gt(&self, other: &Rhs) -> bool

Tests greater than (for self and other) and is used by the > operator.

fn ge(&self, other: &Rhs) -> bool

Tests greater than or equal to (for self and other) and is used by the >= operator.

impl Eq for UnnestOptions

impl StructuralPartialEq for UnnestOptions