Module physical_plan

re-export of datafusion_physical_plan crate

Modules

aggregates

Aggregates functionalities

analyze

Defines the ANALYZE operator

coalesce

coalesce_batches

CoalesceBatchesExec combines small batches into larger batches.

coalesce_partitions

Defines the merge plan for executing partitions in parallel and then merging the results into a single partition

common

Defines common code used in execution plans

display

Implementation of physical plan display. See crate::displayable for examples of how to format

empty

EmptyRelation with produce_one_row=false execution plan

execution_plan

explain

Defines the EXPLAIN operator

filter

insert

Execution plan for writing data to DataSinks

joins

DataFusion Join implementations

limit

Defines the LIMIT plan

memory

Execution plan for reading in-memory batches of data

metrics

Metrics for recording information about execution

placeholder_row

EmptyRelation produce_one_row=true execution plan

projection

Defines the projection execution plan. A projection determines which columns or expressions are returned from a query. The SQL statement SELECT a, b, a+b FROM t1 is an example of a projection on table t1 where the expressions a, b, and a+b are the projection expressions. SELECT without FROM will only evaluate expressions.

recursive_query

Defines the recursive query plan

repartition

This file implements the RepartitionExec operator, which maps N input partitions to M output partitions based on a partitioning scheme, optionally maintaining the order of the input rows in the output.

sorts

Sort functionalities

spill

Defines the spilling functions

stream

Stream wrappers for physical operators

streaming

Generic plans for deferred execution: StreamingTableExec and PartitionStream

tree_node

This module provides common traits for visiting or rewriting tree nodes easily.

udaf

union

The Union operator combines multiple inputs with the same schema

unnest

Define a plan for unnesting values in columns that contain a list type.

values

Values execution plan

windows

Physical expressions for window functions

work_table

Defines the work table query plan

Macros

handle_state

The handle_state macro is designed to process the result of a state-changing operation. It operates on a StatefulStreamResult by matching its variants and executing corresponding actions. This macro is used to streamline code that deals with state transitions, reducing boilerplate and improving readability.

internal_err

Macro wraps Err($ERR) to add backtrace feature

Structs

ColumnStatistics

Statistics for a column within a relation

DefaultDisplay

A new type wrapper to display T implementingDisplayAs using the Default mode

EmptyRecordBatchStream

EmptyRecordBatchStream can be used to create a RecordBatchStream that will produce no results

Metric

Something that tracks a value of interest (metric) of a DataFusion ExecutionPlan execution.

PlanProperties

Stores certain, often expensive to compute, plan properties used in query optimization.

Statistics

Statistics for a relation Fields are optional and can be inexact because the sources sometimes provide approximate estimates for performance reasons and the transformations output are not always predictable.

TopK

Global TopK

VerboseDisplay

A new type wrapper to display T implementing DisplayAs using the Verbose mode

Enums

ColumnarValue

The result of evaluating an expression.

DisplayFormatType

Options for controlling how each ExecutionPlan should format itself

Distribution

How data is distributed amongst partitions. See Partitioning for more details.

InputOrderMode

Specifies how the input to an aggregation or window operator is ordered relative to their GROUP BY or PARTITION BY expressions.

Partitioning

Output partitioning supported by ExecutionPlans.

Traits

Accumulator

Tracks an aggregate function’s state.

DisplayAs

Trait for types which could have additional details when formatted in Verbose mode

ExecutionPlan

Represent nodes in the DataFusion Physical Plan.

ExecutionPlanProperties

Extension trait provides an easy API to fetch various properties of ExecutionPlan objects based on ExecutionPlan::properties.

ExecutionPlanVisitor

Trait that implements the Visitor pattern for a depth first walk of ExecutionPlan nodes. pre_visit is called before any children are visited, and then post_visit is called after all children have been visited.

PhysicalExpr

PhysicalExprs represent expressions such as A + 1 or CAST(c1 AS int).

RecordBatchStream

Trait for types that stream RecordBatch

WindowExpr

Common trait for window function implementations

Functions

accept

Visit all children of this plan, according to the order defined on ExecutionPlanVisitor.

collect

Execute the ExecutionPlan and collect the results in memory

collect_partitioned

Execute the ExecutionPlan and collect the results in memory

displayable

Return a DisplayableExecutionPlan wrapper around an ExecutionPlan which can be displayed in various easier to understand ways.

execute_input_stream

Executes an input stream and ensures that the resulting stream adheres to the not null constraints specified in the sink_schema.

execute_stream

Execute the ExecutionPlan and return a single stream of RecordBatches.

execute_stream_partitioned

Execute the ExecutionPlan and return a vec with one stream per output partition

get_plan_string

Utility function yielding a string representation of the given ExecutionPlan.

project_schema

Applies an optional projection to a SchemaRef, returning the projected schema

visit_execution_plan

Recursively calls pre_visit and post_visit for this node and all of its children, as described on ExecutionPlanVisitor

with_new_children_if_necessary

Returns a copy of this plan if we change any child according to the pointer comparison. The size of children must be equal to the size of ExecutionPlan::children().

Type Aliases

SendableRecordBatchStream

Trait for a Stream of RecordBatches that can be passed between threads