Enum datafusion_expr::expr::Expr

pub enum Expr {
    Alias(Alias),
    Column(Column),
    ScalarVariable(DataType, Vec<String>),
    Literal(ScalarValue),
    BinaryExpr(BinaryExpr),
    Like(Like),
    SimilarTo(Like),
    Not(Box<Expr>),
    IsNotNull(Box<Expr>),
    IsNull(Box<Expr>),
    IsTrue(Box<Expr>),
    IsFalse(Box<Expr>),
    IsUnknown(Box<Expr>),
    IsNotTrue(Box<Expr>),
    IsNotFalse(Box<Expr>),
    IsNotUnknown(Box<Expr>),
    Negative(Box<Expr>),
    GetIndexedField(GetIndexedField),
    Between(Between),
    Case(Case),
    Cast(Cast),
    TryCast(TryCast),
    Sort(Sort),
    ScalarFunction(ScalarFunction),
    ScalarUDF(ScalarUDF),
    AggregateFunction(AggregateFunction),
    WindowFunction(WindowFunction),
    AggregateUDF(AggregateUDF),
    InList(InList),
    Exists(Exists),
    InSubquery(InSubquery),
    ScalarSubquery(Subquery),
    Wildcard,
    QualifiedWildcard {
        qualifier: String,
    },
    GroupingSet(GroupingSet),
    Placeholder(Placeholder),
    OuterReferenceColumn(DataType, Column),
}

Expr is a central struct of DataFusion’s query API, and represent logical expressions such as A + 1, or CAST(c1 AS int).

An Expr can compute its DataType and nullability, and has functions for building up complex expressions.

Examples

Create an expression c1 referring to column named “c1”

let expr = col("c1");
assert_eq!(expr, Expr::Column(Column::from_name("c1")));

Create the expression c1 + c2 to add columns “c1” and “c2” together

let expr = col("c1") + col("c2");

assert!(matches!(expr, Expr::BinaryExpr { ..} ));
if let Expr::BinaryExpr(binary_expr) = expr {
  assert_eq!(*binary_expr.left, col("c1"));
  assert_eq!(*binary_expr.right, col("c2"));
  assert_eq!(binary_expr.op, Operator::Plus);
}

Create expression c1 = 42 to compare the value in column “c1” to the literal value 42

let expr = col("c1").eq(lit(42_i32));

assert!(matches!(expr, Expr::BinaryExpr { .. } ));
if let Expr::BinaryExpr(binary_expr) = expr {
  assert_eq!(*binary_expr.left, col("c1"));
  let scalar = ScalarValue::Int32(Some(42));
  assert_eq!(*binary_expr.right, Expr::Literal(scalar));
  assert_eq!(binary_expr.op, Operator::Eq);
}

Variants

Alias(Alias)

An expression with a specific name.

Column(Column)

A named reference to a qualified filed in a schema.

ScalarVariable(DataType, Vec<String>)

A named reference to a variable in a registry.

Literal(ScalarValue)

A constant value.

BinaryExpr(BinaryExpr)

A binary expression such as “age > 21”

Like(Like)

LIKE expression

SimilarTo(Like)

LIKE expression that uses regular expressions

Not(Box<Expr>)

Negation of an expression. The expression’s type must be a boolean to make sense.

IsNotNull(Box<Expr>)

Whether an expression is not Null. This expression is never null.

IsNull(Box<Expr>)

Whether an expression is Null. This expression is never null.

IsTrue(Box<Expr>)

Whether an expression is True. Boolean operation

IsFalse(Box<Expr>)

Whether an expression is False. Boolean operation

IsUnknown(Box<Expr>)

Whether an expression is Unknown. Boolean operation

IsNotTrue(Box<Expr>)

Whether an expression is not True. Boolean operation

IsNotFalse(Box<Expr>)

Whether an expression is not False. Boolean operation

IsNotUnknown(Box<Expr>)

Whether an expression is not Unknown. Boolean operation

Negative(Box<Expr>)

arithmetic negation of an expression, the operand must be of a signed numeric data type

GetIndexedField(GetIndexedField)

Returns the field of a [arrow::array::ListArray] or [arrow::array::StructArray] by index or range

Between(Between)

Whether an expression is between a given range.

Case(Case)

The CASE expression is similar to a series of nested if/else and there are two forms that can be used. The first form consists of a series of boolean “when” expressions with corresponding “then” expressions, and an optional “else” expression.

CASE WHEN condition THEN result [WHEN …] [ELSE result] END

The second form uses a base expression and then a series of “when” clauses that match on a literal value.

CASE expression WHEN value THEN result [WHEN …] [ELSE result] END

Cast(Cast)

Casts the expression to a given type and will return a runtime error if the expression cannot be cast. This expression is guaranteed to have a fixed type.

TryCast(TryCast)

Casts the expression to a given type and will return a null value if the expression cannot be cast. This expression is guaranteed to have a fixed type.

Sort(Sort)

A sort expression, that can be used to sort values.

ScalarFunction(ScalarFunction)

Represents the call of a built-in scalar function with a set of arguments.

ScalarUDF(ScalarUDF)

Represents the call of a user-defined scalar function with arguments.

AggregateFunction(AggregateFunction)

Represents the call of an aggregate built-in function with arguments.

WindowFunction(WindowFunction)

Represents the call of a window function with arguments.

AggregateUDF(AggregateUDF)

aggregate function

InList(InList)

Returns whether the list contains the expr value.

Exists(Exists)

EXISTS subquery

InSubquery(InSubquery)

IN subquery

ScalarSubquery(Subquery)

Scalar subquery

Wildcard

Represents a reference to all fields in a schema.

QualifiedWildcard

Fields

qualifier: String

Represents a reference to all fields in a specific schema.

GroupingSet(GroupingSet)

List of grouping set expressions. Only valid in the context of an aggregate GROUP BY expression list

Placeholder(Placeholder)

A place holder for parameters in a prepared statement (e.g. $foo or $1)

OuterReferenceColumn(DataType, Column)

A place holder which hold a reference to a qualified field in the outer query, used for correlated sub queries.

Implementations

impl Expr

pub fn display_name(&self) -> Result<String>

Returns the name of this expression as it should appear in a schema. This name will not include any CAST expressions.

pub fn name(&self) -> Result<String>

👎Deprecated since 14.0.0: please use display_name instead

Returns the name of this expression as it should appear in a schema. This name will not include any CAST expressions.

pub fn canonical_name(&self) -> String

Returns a full and complete string representation of this expression.

pub fn variant_name(&self) -> &str

Return String representation of the variant represented by self Useful for non-rust based bindings

pub fn eq(self, other: Expr) -> Expr

Return self == other

pub fn not_eq(self, other: Expr) -> Expr

Return self != other

pub fn gt(self, other: Expr) -> Expr

Return self > other

pub fn gt_eq(self, other: Expr) -> Expr

Return self >= other

pub fn lt(self, other: Expr) -> Expr

Return self < other

pub fn lt_eq(self, other: Expr) -> Expr

Return self <= other

pub fn and(self, other: Expr) -> Expr

Return self && other

pub fn or(self, other: Expr) -> Expr

Return self || other

pub fn like(self, other: Expr) -> Expr

Return self LIKE other

pub fn not_like(self, other: Expr) -> Expr

Return self NOT LIKE other

pub fn ilike(self, other: Expr) -> Expr

Return self ILIKE other

pub fn not_ilike(self, other: Expr) -> Expr

Return self NOT ILIKE other

pub fn name_for_alias(&self) -> Result<String>

Return the name to use for the specific Expr, recursing into Expr::Sort as appropriate

pub fn alias_if_changed(self, original_name: String) -> Result<Expr>

Ensure expr has the name as original_name by adding an alias if necessary.

pub fn alias(self, name: impl Into<String>) -> Expr

Return self AS name alias expression

pub fn unalias(self) -> Expr

Remove an alias from an expression if one exists.

pub fn in_list(self, list: Vec<Expr>, negated: bool) -> Expr

Return self IN <list> if negated is false, otherwise return self NOT IN <list>.a

pub fn is_null(self) -> Expr

Return `IsNull(Box(self))

pub fn is_not_null(self) -> Expr

Return `IsNotNull(Box(self))

pub fn sort(self, asc: bool, nulls_first: bool) -> Expr

Create a sort expression from an existing expression.

let sort_expr = col("foo").sort(true, true); // SORT ASC NULLS_FIRST

pub fn is_true(self) -> Expr

Return IsTrue(Box(self))

pub fn is_not_true(self) -> Expr

Return IsNotTrue(Box(self))

pub fn is_false(self) -> Expr

Return IsFalse(Box(self))

pub fn is_not_false(self) -> Expr

Return IsNotFalse(Box(self))

pub fn is_unknown(self) -> Expr

Return IsUnknown(Box(self))

pub fn is_not_unknown(self) -> Expr

Return IsNotUnknown(Box(self))

pub fn between(self, low: Expr, high: Expr) -> Expr

return self BETWEEN low AND high

pub fn not_between(self, low: Expr, high: Expr) -> Expr

return self NOT BETWEEN low AND high

pub fn field(self, name: impl Into<String>) -> Self

Return access to the named field. Example expr["name"]

Access field “my_field” from column “c1”

For example if column “c1” holds documents like this

{
  "my_field": 123.34,
  "other_field": "Boston",
}

You can access column “my_field” with

let expr = col("c1")
   .field("my_field");
assert_eq!(expr.display_name().unwrap(), "c1[my_field]");

pub fn index(self, key: Expr) -> Self

Return access to the element field. Example expr["name"]

Example Access element 2 from column “c1”

For example if column “c1” holds documents like this

[10, 20, 30, 40]

You can access the value “30” with

let expr = col("c1")
   .index(lit(3));
assert_eq!(expr.display_name().unwrap(), "c1[Int32(3)]");

pub fn range(self, start: Expr, stop: Expr) -> Self

Return elements between 1 based start and stop, for example expr[1:3]

Example: Access element 2, 3, 4 from column “c1”

For example if column “c1” holds documents like this

[10, 20, 30, 40]

You can access the value [20, 30, 40] with

let expr = col("c1")
   .range(lit(2), lit(4));
assert_eq!(expr.display_name().unwrap(), "c1[Int32(2):Int32(4)]");

pub fn try_into_col(&self) -> Result<Column>

pub fn to_columns(&self) -> Result<HashSet<Column>>

Return all referenced columns of this expression.

pub fn contains_outer(&self) -> bool

Return true when the expression contains out reference(correlated) expressions.

Trait Implementations

impl Add<Expr> for Expr

Support <expr> + <expr> fluent style

type Output = Expr

The resulting type after applying the + operator.

fn add(self, rhs: Self) -> Self

Performs the + operation.

impl BitAnd<Expr> for Expr

Support <expr> & <expr> fluent style

type Output = Expr

The resulting type after applying the & operator.

fn bitand(self, rhs: Self) -> Self

Performs the & operation.

impl BitOr<Expr> for Expr

Support <expr> | <expr> fluent style

type Output = Expr

The resulting type after applying the | operator.

fn bitor(self, rhs: Self) -> Self

Performs the | operation.

impl BitXor<Expr> for Expr

Support <expr> ^ <expr> fluent style

type Output = Expr

The resulting type after applying the ^ operator.

fn bitxor(self, rhs: Self) -> Self

Performs the ^ operation.

impl Clone for Expr

fn clone(&self) -> Expr

Returns a copy of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for Expr

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Display for Expr

Format expressions for display as part of a logical plan. In many cases, this will produce similar output to Expr.name() except that column names will be prefixed with ‘#’.

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Div<Expr> for Expr

Support <expr> / <expr> fluent style

type Output = Expr

The resulting type after applying the / operator.

fn div(self, rhs: Self) -> Self

Performs the / operation.

impl ExprSchemable for Expr

fn get_type<S: ExprSchema>(&self, schema: &S) -> Result<DataType>

Returns the [arrow::datatypes::DataType] of the expression based on ExprSchema

Note: DFSchema implements ExprSchema.

Errors

This function errors when it is not possible to compute its [arrow::datatypes::DataType]. This happens when e.g. the expression refers to a column that does not exist in the schema, or when the expression is incorrectly typed (e.g. [utf8] + [bool]).

fn nullable<S: ExprSchema>(&self, input_schema: &S) -> Result<bool>

Returns the nullability of the expression based on ExprSchema.

Note: DFSchema implements ExprSchema.

Errors

This function errors when it is not possible to compute its nullability. This happens when the expression refers to a column that does not exist in the schema.

fn to_field(&self, input_schema: &DFSchema) -> Result<DFField>

Returns a [arrow::datatypes::Field] compatible with this expression.

So for example, a projected expression col(c1) + col(c2) is placed in an output field named col(“c1 + c2”)

fn cast_to<S: ExprSchema>( self, cast_to_type: &DataType, schema: &S ) -> Result<Expr>

Wraps this expression in a cast to a target [arrow::datatypes::DataType].

Errors

This function errors when it is impossible to cast the expression to the target [arrow::datatypes::DataType].

fn metadata<S: ExprSchema>(&self, schema: &S) -> Result<HashMap<String, String>>

given a schema, return the expr’s optional metadata

impl Hash for Expr

fn hash<__H: Hasher>(&self, state: &mut __H)

Feeds this value into the given Hasher.

fn hash_slice<H>(data: &[Self], state: &mut H)where H: Hasher, Self: Sized,

Feeds a slice of this type into the given Hasher.

impl Mul<Expr> for Expr

Support <expr> * <expr> fluent style

type Output = Expr

The resulting type after applying the * operator.

fn mul(self, rhs: Self) -> Self

Performs the * operation.

impl Neg for Expr

Support - <expr> fluent style

type Output = Expr

The resulting type after applying the - operator.

fn neg(self) -> Self::Output

Performs the unary - operation.

impl Not for Expr

type Output = Expr

The resulting type after applying the ! operator.

fn not(self) -> Self::Output

Performs the unary ! operation.

impl PartialEq<Expr> for Expr

fn eq(&self, other: &Expr) -> bool

This method tests for self and other values to be equal, and is used by ==.

fn ne(&self, other: &Rhs) -> bool

This method tests for !=. The default implementation is almost always sufficient, and should not be overridden without very good reason.

impl PartialOrd<Expr> for Expr

fn partial_cmp(&self, other: &Self) -> Option<Ordering>

This method returns an ordering between self and other values if one exists.

fn lt(&self, other: &Rhs) -> bool

This method tests less than (for self and other) and is used by the < operator.

fn le(&self, other: &Rhs) -> bool

This method tests less than or equal to (for self and other) and is used by the <= operator.

fn gt(&self, other: &Rhs) -> bool

This method tests greater than (for self and other) and is used by the > operator.

fn ge(&self, other: &Rhs) -> bool

This method tests greater than or equal to (for self and other) and is used by the >= operator.

impl Rem<Expr> for Expr

Support <expr> % <expr> fluent style

type Output = Expr

The resulting type after applying the % operator.

fn rem(self, rhs: Self) -> Self

Performs the % operation.

impl Shl<Expr> for Expr

Support <expr> << <expr> fluent style

type Output = Expr

The resulting type after applying the << operator.

fn shl(self, rhs: Self) -> Self::Output

Performs the << operation.

impl Shr<Expr> for Expr

Support <expr> >> <expr> fluent style

type Output = Expr

The resulting type after applying the >> operator.

fn shr(self, rhs: Self) -> Self::Output

Performs the >> operation.

impl Sub<Expr> for Expr

Support <expr> - <expr> fluent style

type Output = Expr

The resulting type after applying the - operator.

fn sub(self, rhs: Self) -> Self

Performs the - operation.

impl TreeNode for Expr

fn apply_children<F>(&self, op: &mut F) -> Result<VisitRecursion>where F: FnMut(&Self) -> Result<VisitRecursion>,

Apply the closure F to the node’s children

fn map_children<F>(self, transform: F) -> Result<Self>where F: FnMut(Self) -> Result<Self>,

Apply transform F to the node’s children, the transform F might have a direction(Preorder or Postorder)

fn apply<F>(&self, op: &mut F) -> Result<VisitRecursion, DataFusionError>where F: FnMut(&Self) -> Result<VisitRecursion, DataFusionError>,

Use preorder to iterate the node on the tree so that we can stop fast for some cases.

fn visit<V>(&self, visitor: &mut V) -> Result<VisitRecursion, DataFusionError>where V: TreeNodeVisitor<N = Self>,

Visit the tree node using the given TreeNodeVisitor It performs a depth first walk of an node and its children.

fn transform<F>(self, op: &F) -> Result<Self, DataFusionError>where F: Fn(Self) -> Result<Transformed<Self>, DataFusionError>,

Convenience utils for writing optimizers rule: recursively apply the given op to the node tree. When op does not apply to a given node, it is left unchanged. The default tree traversal direction is transform_up(Postorder Traversal).

fn transform_down<F>(self, op: &F) -> Result<Self, DataFusionError>where F: Fn(Self) -> Result<Transformed<Self>, DataFusionError>,

Convenience utils for writing optimizers rule: recursively apply the given ‘op’ to the node and all of its children(Preorder Traversal). When the op does not apply to a given node, it is left unchanged.

fn transform_up<F>(self, op: &F) -> Result<Self, DataFusionError>where F: Fn(Self) -> Result<Transformed<Self>, DataFusionError>,

Convenience utils for writing optimizers rule: recursively apply the given ‘op’ first to all of its children and then itself(Postorder Traversal). When the op does not apply to a given node, it is left unchanged.

fn rewrite<R>(self, rewriter: &mut R) -> Result<Self, DataFusionError>where R: TreeNodeRewriter<N = Self>,

Transform the tree node using the given TreeNodeRewriter It performs a depth first walk of an node and its children.

impl Eq for Expr

impl StructuralEq for Expr

impl StructuralPartialEq for Expr