Module physical_plan

re-export of datafusion_physical_plan crate

Modules

• aggregates
  Aggregates functionalities
• analyze
  Defines the ANALYZE operator
• coalesce
• coalesce_batches
  CoalesceBatchesExec combines small batches into larger batches.
• coalesce_partitions
  Defines the merge plan for executing partitions in parallel and then merging the results into a single partition
• common
  Defines common code used in execution plans
• display
  Implementation of physical plan display. See crate::displayable for examples of how to format
• empty
  EmptyRelation with produce_one_row=false execution plan
• execution_plan
• explain
  Defines the EXPLAIN operator
• filter
• insert
  Execution plan for writing data to DataSinks
• joins
  DataFusion Join implementations
• limit
  Defines the LIMIT plan
• memory
  Execution plan for reading in-memory batches of data
• metrics
  Metrics for recording information about execution
• placeholder_row
  EmptyRelation produce_one_row=true execution plan
• projection
  Defines the projection execution plan. A projection determines which columns or expressions are returned from a query. The SQL statement SELECT a, b, a+b FROM t1 is an example of a projection on table t1 where the expressions a, b, and a+b are the projection expressions. SELECT without FROM will only evaluate expressions.
• recursive_query
  Defines the recursive query plan
• repartition
  This file implements the RepartitionExec operator, which maps N input partitions to M output partitions based on a partitioning scheme, optionally maintaining the order of the input rows in the output.
• sorts
  Sort functionalities
• spill
  Defines the spilling functions
• stream
  Stream wrappers for physical operators
• streaming
  Generic plans for deferred execution: StreamingTableExec and PartitionStream
• tree_node
  This module provides common traits for visiting or rewriting tree nodes easily.
• udaf
• union
  The Union operator combines multiple inputs with the same schema
• unnest
  Define a plan for unnesting values in columns that contain a list type.
• values
  Values execution plan
• windows
  Physical expressions for window functions
• work_table
  Defines the work table query plan

Macros

• handle_state
  The handle_state macro is designed to process the result of a state-changing operation. It operates on a StatefulStreamResult by matching its variants and executing corresponding actions. This macro is used to streamline code that deals with state transitions, reducing boilerplate and improving readability.
• internal_err
  Macro wraps Err($ERR) to add backtrace feature

Structs

• ColumnStatistics
  Statistics for a column within a relation
• DefaultDisplay
  A newtype wrapper to display T implementingDisplayAs using the Default mode
• EmptyRecordBatchStream
  EmptyRecordBatchStream can be used to create a RecordBatchStream that will produce no results
• Metric
  Something that tracks a value of interest (metric) of a DataFusion ExecutionPlan execution.
• PlanProperties
  Stores certain, often expensive to compute, plan properties used in query optimization.
• Statistics
  Statistics for a relation Fields are optional and can be inexact because the sources sometimes provide approximate estimates for performance reasons and the transformations output are not always predictable.
• TopK
  Global TopK
• VerboseDisplay
  A newtype wrapper to display T implementing DisplayAs using the Verbose mode

Enums

• ColumnarValue
  The result of evaluating an expression.
• DisplayFormatType
  Options for controlling how each ExecutionPlan should format itself
• Distribution
  How data is distributed amongst partitions. See Partitioning for more details.
• ExecutionMode
  Describes the execution mode of the result of calling ExecutionPlan::execute with respect to its size and behavior.
• InputOrderMode
  Specifies how the input to an aggregation or window operator is ordered relative to their GROUP BY or PARTITION BY expressions.
• Partitioning
  Output partitioning supported by ExecutionPlans.

Traits

• Accumulator
  Tracks an aggregate function’s state.
• DisplayAs
  Trait for types which could have additional details when formatted in Verbose mode
• ExecutionPlan
  Represent nodes in the DataFusion Physical Plan.
• ExecutionPlanProperties
  Extension trait provides an easy API to fetch various properties of ExecutionPlan objects based on ExecutionPlan::properties.
• ExecutionPlanVisitor
  Trait that implements the Visitor pattern for a depth first walk of ExecutionPlan nodes. pre_visit is called before any children are visited, and then post_visit is called after all children have been visited.
• PhysicalExpr
  PhysicalExprs represent expressions such as A + 1 or CAST(c1 AS int).
• RecordBatchStream
  Trait for types that stream RecordBatch
• WindowExpr
  Common trait for window function implementations

Functions

• accept
  Visit all children of this plan, according to the order defined on ExecutionPlanVisitor.
• collect
  Execute the ExecutionPlan and collect the results in memory
• collect_partitioned
  Execute the ExecutionPlan and collect the results in memory
• displayable
  Return a wrapper around an ExecutionPlan which can be displayed in various easier to understand ways.
• execute_input_stream
  Executes an input stream and ensures that the resulting stream adheres to the not null constraints specified in the sink_schema.
• execute_stream
  Execute the ExecutionPlan and return a single stream of RecordBatches.
• execute_stream_partitioned
  Execute the ExecutionPlan and return a vec with one stream per output partition
• get_plan_string
  Utility function yielding a string representation of the given ExecutionPlan.
• project_schema
  Applies an optional projection to a SchemaRef, returning the projected schema
• visit_execution_plan
  Recursively calls pre_visit and post_visit for this node and all of its children, as described on ExecutionPlanVisitor
• with_new_children_if_necessary
  Returns a copy of this plan if we change any child according to the pointer comparison. The size of children must be equal to the size of ExecutionPlan::children().

Type Aliases

• SendableRecordBatchStream
  Trait for a Stream of RecordBatches that can be passed between threads