Enum datafusion_expr::logical_plan::LogicalPlan

pub enum LogicalPlan {
    Projection(Projection),
    Filter(Filter),
    Window(Window),
    Aggregate(Aggregate),
    Sort(Sort),
    Join(Join),
    CrossJoin(CrossJoin),
    Repartition(Repartition),
    Union(Union),
    TableScan(TableScan),
    EmptyRelation(EmptyRelation),
    Subquery(Subquery),
    SubqueryAlias(SubqueryAlias),
    Limit(Limit),
    Statement(Statement),
    Values(Values),
    Explain(Explain),
    Analyze(Analyze),
    Extension(Extension),
    Distinct(Distinct),
    Prepare(Prepare),
    Dml(DmlStatement),
    Ddl(DdlStatement),
    Copy(CopyTo),
    DescribeTable(DescribeTable),
    Unnest(Unnest),
}

A LogicalPlan represents the different types of relational operators (such as Projection, Filter, etc) and can be created by the SQL query planner and the DataFrame API.

A LogicalPlan represents transforming an input relation (table) to an output relation (table) with a (potentially) different schema. A plan represents a dataflow tree where data flows from leaves up to the root to produce the query result.

Variants

Projection(Projection)

Evaluates an arbitrary list of expressions (essentially a SELECT with an expression list) on its input.

Filter(Filter)

Filters rows from its input that do not match an expression (essentially a WHERE clause with a predicate expression).

Semantically, <predicate> is evaluated for each row of the input; If the value of <predicate> is true, the input row is passed to the output. If the value of <predicate> is false (or null), the row is discarded.

Window(Window)

Windows input based on a set of window spec and window function (e.g. SUM or RANK). This is used to implement SQL window functions, and the OVER clause.

Aggregate(Aggregate)

Aggregates its input based on a set of grouping and aggregate expressions (e.g. SUM). This is used to implement SQL aggregates and GROUP BY.

Sort(Sort)

Sorts its input according to a list of sort expressions. This is used to implement SQL ORDER BY

Join(Join)

Join two logical plans on one or more join columns. This is used to implement SQL JOIN

CrossJoin(CrossJoin)

Apply Cross Join to two logical plans. This is used to implement SQL CROSS JOIN

Repartition(Repartition)

Repartitions the input based on a partitioning scheme. This is used to add parallelism and is sometimes referred to as an “exchange” operator in other systems

Union(Union)

Union multiple inputs with the same schema into a single output stream. This is used to implement SQL UNION [ALL] and INTERSECT [ALL].

TableScan(TableScan)

Produces rows from a TableSource, used to implement SQL FROM tables or views.

EmptyRelation(EmptyRelation)

Produces no rows: An empty relation with an empty schema that produces 0 or 1 row. This is used to implement SQL SELECT that has no values in the FROM clause.

Subquery(Subquery)

Produces the output of running another query. This is used to implement SQL subqueries

SubqueryAlias(SubqueryAlias)

Aliased relation provides, or changes, the name of a relation.

Limit(Limit)

Skip some number of rows, and then fetch some number of rows.

Statement(Statement)

A DataFusion Statement such as SET VARIABLE or START TRANSACTION

Values(Values)

Values expression. See Postgres VALUES documentation for more details. This is used to implement SQL such as VALUES (1, 2), (3, 4)

Explain(Explain)

Produces a relation with string representations of various parts of the plan. This is used to implement SQL EXPLAIN.

Analyze(Analyze)

Runs the input, and prints annotated physical plan as a string with execution metric. This is used to implement SQL EXPLAIN ANALYZE.

Extension(Extension)

Extension operator defined outside of DataFusion. This is used to extend DataFusion with custom relational operations that

Distinct(Distinct)

Remove duplicate rows from the input. This is used to implement SQL SELECT DISTINCT ....

Prepare(Prepare)

Prepare a statement and find any bind parameters (e.g. ?). This is used to implement SQL-prepared statements.

Dml(DmlStatement)

Data Manipulaton Language (DML): Insert / Update / Delete

Ddl(DdlStatement)

Data Definition Language (DDL): CREATE / DROP TABLES / VIEWS / SCHEMAs

Copy(CopyTo)

COPY TO for writing plan results to files

DescribeTable(DescribeTable)

Describe the schema of the table. This is used to implement the SQL DESCRIBE command from MySQL.

Unnest(Unnest)

Unnest a column that contains a nested list type such as an ARRAY. This is used to implement SQL UNNEST

Implementations

impl LogicalPlan

pub fn schema(&self) -> &DFSchemaRef

Get a reference to the logical plan’s schema

pub fn fallback_normalize_schemas(&self) -> Vec<&DFSchema>

Used for normalizing columns, as the fallback schemas to the main schema of the plan.

pub fn all_schemas(&self) -> Vec<&DFSchemaRef>

👎Deprecated since 20.0.0

Get all meaningful schemas of a plan and its children plan.

pub fn explain_schema() -> SchemaRef

Returns the (fixed) output schema for explain plans

pub fn expressions(self: &LogicalPlan) -> Vec<Expr>

returns all expressions (non-recursively) in the current logical plan node. This does not include expressions in any children

pub fn all_out_ref_exprs(self: &LogicalPlan) -> Vec<Expr>

Returns all the out reference(correlated) expressions (recursively) in the current logical plan nodes and all its descendant nodes.

pub fn inspect_expressions<F, E>(self: &LogicalPlan, f: F) -> Result<(), E>where F: FnMut(&Expr) -> Result<(), E>,

Calls f on all expressions (non-recursively) in the current logical plan node. This does not include expressions in any children.

pub fn inputs(&self) -> Vec<&LogicalPlan>

returns all inputs of this LogicalPlan node. Does not include inputs to inputs, or subqueries.

pub fn using_columns(&self) -> Result<Vec<HashSet<Column>>, DataFusionError>

returns all Using join columns in a logical plan

pub fn head_output_expr(&self) -> Result<Option<Expr>>

returns the first output expression of this LogicalPlan node.

pub fn with_new_inputs(&self, inputs: &[LogicalPlan]) -> Result<LogicalPlan>

Returns a copy of this LogicalPlan with the new inputs

pub fn with_new_exprs( &self, expr: Vec<Expr>, inputs: &[LogicalPlan] ) -> Result<LogicalPlan>

Returns a new LogicalPlan based on self with inputs and expressions replaced.

The exprs correspond to the same order of expressions returned by Self::expressions. This function is used by optimizers to rewrite plans using the following pattern:

let new_inputs = optimize_children(..., plan, props);

// get the plans expressions to optimize
let exprs = plan.expressions();

// potentially rewrite plan expressions
let rewritten_exprs = rewrite_exprs(exprs);

// create new plan using rewritten_exprs in same position
let new_plan = plan.new_with_exprs(rewritten_exprs, new_inputs);

Note: sometimes Self::with_new_exprs will use schema of original plan, it will not change the scheam. Such as Projection/Aggregate/Window

pub fn with_param_values( self, param_values: Vec<ScalarValue> ) -> Result<LogicalPlan>

Convert a prepared LogicalPlan into its inner logical plan with all params replaced with their corresponding values

pub fn max_rows(self: &LogicalPlan) -> Option<usize>

Returns the maximum number of rows that this plan can output, if known.

If None, the plan can return any number of rows. If Some(n) then the plan can return at most n rows but may return fewer.

impl LogicalPlan

pub fn replace_params_with_values( &self, param_values: &[ScalarValue] ) -> Result<LogicalPlan>

Return a logical plan with all placeholders/params (e.g $1 $2, …) replaced with corresponding values provided in the params_values

pub fn get_parameter_types( &self ) -> Result<HashMap<String, Option<DataType>>, DataFusionError>

Walk the logical plan, find any PlaceHolder tokens, and return a map of their IDs and DataTypes

impl LogicalPlan

pub fn display_indent(&self) -> impl Display + '_

Return a formatable structure that produces a single line per node. For example:

Projection: employee.id
   Filter: employee.state Eq Utf8(\"CO\")\
      CsvScan: employee projection=Some([0, 3])

use arrow::datatypes::{Field, Schema, DataType};
use datafusion_expr::{lit, col, LogicalPlanBuilder, logical_plan::table_scan};
let schema = Schema::new(vec![
    Field::new("id", DataType::Int32, false),
]);
let plan = table_scan(Some("t1"), &schema, None).unwrap()
    .filter(col("id").eq(lit(5))).unwrap()
    .build().unwrap();

// Format using display_indent
let display_string = format!("{}", plan.display_indent());

assert_eq!("Filter: t1.id = Int32(5)\n  TableScan: t1",
            display_string);

pub fn display_indent_schema(&self) -> impl Display + '_

Return a formatable structure that produces a single line per node that includes the output schema. For example:

Projection: employee.id [id:Int32]\
   Filter: employee.state = Utf8(\"CO\") [id:Int32, state:Utf8]\
     TableScan: employee projection=[0, 3] [id:Int32, state:Utf8]";

use arrow::datatypes::{Field, Schema, DataType};
use datafusion_expr::{lit, col, LogicalPlanBuilder, logical_plan::table_scan};
let schema = Schema::new(vec![
    Field::new("id", DataType::Int32, false),
]);
let plan = table_scan(Some("t1"), &schema, None).unwrap()
    .filter(col("id").eq(lit(5))).unwrap()
    .build().unwrap();

// Format using display_indent_schema
let display_string = format!("{}", plan.display_indent_schema());

assert_eq!("Filter: t1.id = Int32(5) [id:Int32]\
            \n  TableScan: t1 [id:Int32]",
            display_string);

pub fn display_graphviz(&self) -> impl Display + '_

Return a formatable structure that produces lines meant for graphical display using the DOT language. This format can be visualized using software from graphviz

This currently produces two graphs – one with the basic structure, and one with additional details such as schema.

use arrow::datatypes::{Field, Schema, DataType};
use datafusion_expr::{lit, col, LogicalPlanBuilder, logical_plan::table_scan};
let schema = Schema::new(vec![
    Field::new("id", DataType::Int32, false),
]);
let plan = table_scan(Some("t1"), &schema, None).unwrap()
    .filter(col("id").eq(lit(5))).unwrap()
    .build().unwrap();

// Format using display_graphviz
let graphviz_string = format!("{}", plan.display_graphviz());

If graphviz string is saved to a file such as /tmp/example.dot, the following commands can be used to render it as a pdf:

  dot -Tpdf < /tmp/example.dot  > /tmp/example.pdf

pub fn display(&self) -> impl Display + '_

Return a formatable structure with the a human readable description of this LogicalPlan node per node, not including children. For example:

Projection: id

use arrow::datatypes::{Field, Schema, DataType};
use datafusion_expr::{lit, col, LogicalPlanBuilder, logical_plan::table_scan};
let schema = Schema::new(vec![
    Field::new("id", DataType::Int32, false),
]);
let plan = table_scan(Some("t1"), &schema, None).unwrap()
    .build().unwrap();

// Format using display
let display_string = format!("{}", plan.display());

assert_eq!("TableScan: t1", display_string);

Trait Implementations

impl Clone for LogicalPlan

fn clone(&self) -> LogicalPlan

Returns a copy of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for LogicalPlan

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Hash for LogicalPlan

fn hash<__H: Hasher>(&self, state: &mut __H)

Feeds this value into the given Hasher.

fn hash_slice<H>(data: &[Self], state: &mut H)where H: Hasher, Self: Sized,

Feeds a slice of this type into the given Hasher.

impl PartialEq<LogicalPlan> for LogicalPlan

fn eq(&self, other: &LogicalPlan) -> bool

This method tests for self and other values to be equal, and is used by ==.

fn ne(&self, other: &Rhs) -> bool

This method tests for !=. The default implementation is almost always sufficient, and should not be overridden without very good reason.

impl ToStringifiedPlan for LogicalPlan

fn to_stringified(&self, plan_type: PlanType) -> StringifiedPlan

Create a stringified plan with the specified type

impl TreeNode for LogicalPlan

fn visit<V: TreeNodeVisitor<N = Self>>( &self, visitor: &mut V ) -> Result<VisitRecursion>

To use, define a struct that implements the trait TreeNodeVisitor and then invoke LogicalPlan::visit.

For example, for a logical plan like:

Projection: id
   Filter: state Eq Utf8(\"CO\")\
      CsvScan: employee.csv projection=Some([0, 3])";

The sequence of visit operations would be:

visitor.pre_visit(Projection)
visitor.pre_visit(Filter)
visitor.pre_visit(CsvScan)
visitor.post_visit(CsvScan)
visitor.post_visit(Filter)
visitor.post_visit(Projection)

fn apply<F>(&self, op: &mut F) -> Result<VisitRecursion>where F: FnMut(&Self) -> Result<VisitRecursion>,

Use preorder to iterate the node on the tree so that we can stop fast for some cases.

fn apply_children<F>(&self, op: &mut F) -> Result<VisitRecursion>where F: FnMut(&Self) -> Result<VisitRecursion>,

Apply the closure F to the node’s children

fn map_children<F>(self, transform: F) -> Result<Self>where F: FnMut(Self) -> Result<Self>,

Apply transform F to the node’s children, the transform F might have a direction(Preorder or Postorder)

fn transform<F>(self, op: &F) -> Result<Self, DataFusionError>where F: Fn(Self) -> Result<Transformed<Self>, DataFusionError>,

Convenience utils for writing optimizers rule: recursively apply the given op to the node tree. When op does not apply to a given node, it is left unchanged. The default tree traversal direction is transform_up(Postorder Traversal).

fn transform_down<F>(self, op: &F) -> Result<Self, DataFusionError>where F: Fn(Self) -> Result<Transformed<Self>, DataFusionError>,

Convenience utils for writing optimizers rule: recursively apply the given ‘op’ to the node and all of its children(Preorder Traversal). When the op does not apply to a given node, it is left unchanged.

fn transform_up<F>(self, op: &F) -> Result<Self, DataFusionError>where F: Fn(Self) -> Result<Transformed<Self>, DataFusionError>,

Convenience utils for writing optimizers rule: recursively apply the given ‘op’ first to all of its children and then itself(Postorder Traversal). When the op does not apply to a given node, it is left unchanged.

fn rewrite<R>(self, rewriter: &mut R) -> Result<Self, DataFusionError>where R: TreeNodeRewriter<N = Self>,

Transform the tree node using the given TreeNodeRewriter It performs a depth first walk of an node and its children.

impl Eq for LogicalPlan

impl StructuralEq for LogicalPlan

impl StructuralPartialEq for LogicalPlan