Module physical_optimizer

Optimizer that rewrites ExecutionPlans.

These rules take advantage of physical plan properties , such as “Repartition” or “Sortedness”

Modules

• aggregate_statistics
  Utilizing exact statistics from sources to avoid scanning data
• coalesce_batches
  CoalesceBatches optimizer that groups batches together rows in bigger batches to avoid overhead with small batches
• combine_partial_final_agg
  CombinePartialFinalAggregate optimizer rule checks the adjacent Partial and Final AggregateExecs and try to combine them if necessary
• enforce_distribution
  EnforceDistribution optimizer rule inspects the physical plan with respect to distribution requirements and adds RepartitionExecs to satisfy them when necessary. If increasing parallelism is beneficial (and also desirable according to the configuration), this rule increases partition counts in the physical plan.
• enforce_sorting
  EnforceSorting optimizer rule inspects the physical plan with respect to local sorting requirements and does the following:
• join_selection
  The JoinSelection rule tries to modify a given plan so that it can accommodate infinite sources and utilize statistical information (if there is any) to obtain more performant plans. To achieve the first goal, it tries to transform a non-runnable query (with the given infinite sources) into a runnable query by replacing pipeline-breaking join operations with pipeline-friendly ones. To achieve the second goal, it selects the proper PartitionMode and the build side using the available statistics for hash joins.
• limit_pushdown
  LimitPushdown pushes LIMIT down through ExecutionPlans to reduce data transfer as much as possible.
• limited_distinct_aggregation
  A special-case optimizer rule that pushes limit into a grouped aggregation which has no aggregate expressions or sorting requirements
• optimizer
  Physical optimizer traits
• output_requirements
  The GlobalOrderRequire optimizer rule either:
• projection_pushdown
  This file implements the ProjectionPushdown physical optimization rule. The function remove_unnecessary_projections tries to push down all projections one by one if the operator below is amenable to this. If a projection reaches a source, it can even disappear from the plan entirely.
• pruning
  PruningPredicate to apply filter Expr to prune “containers” based on statistics (e.g. Parquet Row Groups)
• replace_with_order_preserving_variants
  Optimizer rule that replaces executors that lose ordering with their order-preserving variants when it is helpful; either in terms of performance or to accommodate unbounded streams by fixing the pipeline.
• sanity_checker
  The SanityCheckPlan rule ensures that a given plan can accommodate its infinite sources, if there are any. It will reject non-runnable query plans that use pipeline-breaking operators on infinite input(s). In addition, it will check if all order and distribution requirements of a plan are satisfied by its children.
• topk_aggregation
  An optimizer rule that detects aggregate operations that could use a limited bucket count
• update_aggr_exprs
  An optimizer rule that checks ordering requirements of aggregate expressions and modifies the expressions to work more efficiently if possible.

Traits

• PhysicalOptimizerRule
  PhysicalOptimizerRule transforms one [‘ExecutionPlan’] into another which computes the same results, but in a potentially more efficient way.