Struct datafusion_optimizer::push_down_filter::PushDownFilter

pub struct PushDownFilter {}

Push Down Filter optimizer rule pushes filter clauses down the plan

Introduction

A filter-commutative operation is an operation whose result of filter(op(data)) = op(filter(data)). An example of a filter-commutative operation is a projection; a counter-example is limit.

The filter-commutative property is column-specific. An aggregate grouped by A on SUM(B) can commute with a filter that depends on A only, but does not commute with a filter that depends on SUM(B).

This optimizer commutes filters with filter-commutative operations to push the filters the closest possible to the scans, re-writing the filter expressions by every projection that changes the filter’s expression.

Filter: b Gt Int64(10) Projection: a AS b

is optimized to

Projection: a AS b Filter: a Gt Int64(10) <— changed from b to a

This performs a single pass through the plan. When it passes through a filter, it stores that filter, and when it reaches a node that does not commute with it, it adds the filter to that place. When it passes through a projection, it re-writes the filter’s expression taking into account that projection. When multiple filters would have been written, it AND their expressions into a single expression.

Implementations

impl PushDownFilter

pub fn new() -> Self

Trait Implementations

impl Default for PushDownFilter

fn default() -> PushDownFilter

Returns the “default value” for a type.

impl OptimizerRule for PushDownFilter

fn name(&self) -> &str

A human readable name for this optimizer rule

fn apply_order(&self) -> Option<ApplyOrder>

How should the rule be applied by the optimizer? See comments on ApplyOrder for details.

fn try_optimize( &self, plan: &LogicalPlan, _config: &dyn OptimizerConfig ) -> Result<Option<LogicalPlan>>

Try and rewrite plan to an optimized form, returning None if the plan cannot be optimized by this rule.