Trait datafusion_physical_plan::execution_plan::WindowExpr

pub trait WindowExpr:
    Send
    + Sync
    + Debug {
    // Required methods
    fn as_any(&self) -> &(dyn Any + 'static);
    fn field(&self) -> Result<Field, DataFusionError>;
    fn expressions(&self) -> Vec<Arc<dyn PhysicalExpr>>;
    fn evaluate(
        &self,
        batch: &RecordBatch,
    ) -> Result<Arc<dyn Array>, DataFusionError>;
    fn partition_by(&self) -> &[Arc<dyn PhysicalExpr>];
    fn order_by(&self) -> &[PhysicalSortExpr];
    fn get_window_frame(&self) -> &Arc<WindowFrame>;
    fn uses_bounded_memory(&self) -> bool;
    fn get_reverse_expr(&self) -> Option<Arc<dyn WindowExpr>>;

    // Provided methods
    fn name(&self) -> &str { ... }
    fn evaluate_args(
        &self,
        batch: &RecordBatch,
    ) -> Result<Vec<Arc<dyn Array>>, DataFusionError> { ... }
    fn evaluate_stateful(
        &self,
        _partition_batches: &IndexMap<Vec<ScalarValue>, PartitionBatchState>,
        _window_agg_state: &mut IndexMap<Vec<ScalarValue>, WindowState>,
    ) -> Result<(), DataFusionError> { ... }
    fn order_by_columns(
        &self,
        batch: &RecordBatch,
    ) -> Result<Vec<SortColumn>, DataFusionError> { ... }
    fn all_expressions(&self) -> WindowPhysicalExpressions { ... }
    fn with_new_expressions(
        &self,
        _args: Vec<Arc<dyn PhysicalExpr>>,
        _partition_bys: Vec<Arc<dyn PhysicalExpr>>,
        _order_by_exprs: Vec<Arc<dyn PhysicalExpr>>,
    ) -> Option<Arc<dyn WindowExpr>> { ... }
}

Common trait for window function implementations

Aggregate Window Expressions

These expressions take the form

OVER({ROWS | RANGE| GROUPS} BETWEEN UNBOUNDED PRECEDING AND ...)

For example, cumulative window frames uses PlainAggregateWindowExpr.

Non Aggregate Window Expressions

The expressions have the form

OVER({ROWS | RANGE| GROUPS} BETWEEN M {PRECEDING| FOLLOWING} AND ...)

For example, sliding window frames use SlidingAggregateWindowExpr.

Required Methods

fn as_any(&self) -> &(dyn Any + 'static)

Returns the window expression as Any so that it can be downcast to a specific implementation.

fn field(&self) -> Result<Field, DataFusionError>

The field of the final result of this window function.

fn expressions(&self) -> Vec<Arc<dyn PhysicalExpr>>

Expressions that are passed to the WindowAccumulator. Functions which take a single input argument, such as sum, return a single datafusion_expr::expr::Expr, others (e.g. cov) return many.

fn evaluate( &self, batch: &RecordBatch, ) -> Result<Arc<dyn Array>, DataFusionError>

Evaluate the window function values against the batch

fn partition_by(&self) -> &[Arc<dyn PhysicalExpr>]

Expressions that’s from the window function’s partition by clause, empty if absent

fn order_by(&self) -> &[PhysicalSortExpr]

Expressions that’s from the window function’s order by clause, empty if absent

fn get_window_frame(&self) -> &Arc<WindowFrame>

Get the window frame of this WindowExpr.

fn uses_bounded_memory(&self) -> bool

Return a flag indicating whether this WindowExpr can run with bounded memory.

fn get_reverse_expr(&self) -> Option<Arc<dyn WindowExpr>>

Get the reverse expression of this WindowExpr.

Provided Methods

fn name(&self) -> &str

Human readable name such as "MIN(c2)" or "RANK()". The default implementation returns placeholder text.

fn evaluate_args( &self, batch: &RecordBatch, ) -> Result<Vec<Arc<dyn Array>>, DataFusionError>

Evaluate the window function arguments against the batch and return array ref, normally the resulting Vec is a single element one.

fn evaluate_stateful( &self, _partition_batches: &IndexMap<Vec<ScalarValue>, PartitionBatchState>, _window_agg_state: &mut IndexMap<Vec<ScalarValue>, WindowState>, ) -> Result<(), DataFusionError>

Evaluate the window function against the batch. This function facilitates stateful, bounded-memory implementations.

fn order_by_columns( &self, batch: &RecordBatch, ) -> Result<Vec<SortColumn>, DataFusionError>

Get order by columns, empty if absent

fn all_expressions(&self) -> WindowPhysicalExpressions

Returns all expressions used in the WindowExpr. These expressions are (1) function arguments, (2) partition by expressions, (3) order by expressions.

fn with_new_expressions( &self, _args: Vec<Arc<dyn PhysicalExpr>>, _partition_bys: Vec<Arc<dyn PhysicalExpr>>, _order_by_exprs: Vec<Arc<dyn PhysicalExpr>>, ) -> Option<Arc<dyn WindowExpr>>

Rewrites WindowExpr, with new expressions given. The argument should be consistent with the return value of the WindowExpr::all_expressions method. Returns Some(Arc<dyn WindowExpr>) if re-write is supported, otherwise returns None.

Implementors

impl WindowExpr for BuiltInWindowExpr

impl WindowExpr for PlainAggregateWindowExpr

peer based evaluation based on the fact that batch is pre-sorted given the sort columns and then per partition point we’ll evaluate the peer group (e.g. SUM or MAX gives the same results for peers) and concatenate the results.

impl WindowExpr for SlidingAggregateWindowExpr

Incrementally update window function using the fact that batch is pre-sorted given the sort columns and then per partition point.

Evaluates the peer group (e.g. SUM or MAX gives the same results for peers) and concatenate the results.