Module enforce_distribution

EnforceDistribution optimizer rule inspects the physical plan with respect to distribution requirements and adds RepartitionExecs to satisfy them when necessary. If increasing parallelism is beneficial (and also desirable according to the configuration), this rule increases partition counts in the physical plan.

Structs

EnforceDistribution

The EnforceDistribution rule ensures that distribution requirements are met. In doing so, this rule will increase the parallelism in the plan by introducing repartitioning operators to the physical plan.

Functions

adjust_input_keys_ordering

When the physical planner creates the Joins, the ordering of join keys is from the original query. That might not match with the output partitioning of the join node’s children A Top-Down process will use this method to adjust children’s output partitioning based on the parent key reordering requirements:

ensure_distribution

This function checks whether we need to add additional data exchange operators to satisfy distribution requirements. Since this function takes care of such requirements, we should avoid manually adding data exchange operators in other places.

reorder_aggregate_keys

reorder_join_keys_to_inputs

When the physical planner creates the Joins, the ordering of join keys is from the original query. That might not match with the output partitioning of the join node’s children This method will try to change the ordering of the join keys to match with the partitioning of the join nodes’ children. If it can not match with both sides, it will try to match with one, either the left side or the right side.

reorder_partitioned_join_keys

Type Aliases

DistributionContext

Keeps track of distribution changing operators (like RepartitionExec, SortPreservingMergeExec, CoalescePartitionsExec) and their ancestors. Using this information, we can optimize distribution of the plan if/when necessary.

PlanWithKeyRequirements

Keeps track of parent required key orderings.