Struct CorrelationGroupsAccumulator

pub struct CorrelationGroupsAccumulator { /* private fields */ }

Implementations

impl CorrelationGroupsAccumulator

pub fn new() -> Self

Trait Implementations

impl Default for CorrelationGroupsAccumulator

fn default() -> CorrelationGroupsAccumulator

Returns the “default value” for a type.

impl GroupsAccumulator for CorrelationGroupsAccumulator

GroupsAccumulator implementation for corr(x, y) that computes the Pearson correlation coefficient between two numeric columns.

Online algorithm for correlation:

r = (n * sum_xy - sum_x * sum_y) / sqrt((n * sum_xx - sum_x^2) * (n * sum_yy - sum_y^2)) where: n = number of observations sum_x = sum of x values sum_y = sum of y values
sum_xy = sum of (x * y) sum_xx = sum of x^2 values sum_yy = sum of y^2 values

Reference: https://en.wikipedia.org/wiki/Pearson_correlation_coefficient#For_a_sample

fn update_batch( &mut self, values: &[ArrayRef], group_indices: &[usize], opt_filter: Option<&BooleanArray>, total_num_groups: usize, ) -> Result<()>

Updates the accumulator’s state from its arguments, encoded as a vector of ArrayRefs.

fn merge_batch( &mut self, values: &[ArrayRef], group_indices: &[usize], opt_filter: Option<&BooleanArray>, total_num_groups: usize, ) -> Result<()>

Merges intermediate state (the output from Self::state) into this accumulator’s current state.

fn evaluate(&mut self, emit_to: EmitTo) -> Result<ArrayRef>

Returns the final aggregate value for each group as a single RecordBatch, resetting the internal state.

fn state(&mut self, emit_to: EmitTo) -> Result<Vec<ArrayRef>>

Returns the intermediate aggregate state for this accumulator, used for multi-phase grouping, resetting its internal state.

fn size(&self) -> usize

Amount of memory used to store the state of this accumulator, in bytes.

fn convert_to_state( &self, _values: &[Arc<dyn Array>], _opt_filter: Option<&BooleanArray>, ) -> Result<Vec<Arc<dyn Array>>, DataFusionError>

Converts an input batch directly to the intermediate aggregate state.

fn supports_convert_to_state(&self) -> bool

Returns true if Self::convert_to_state is implemented to support intermediate aggregate state conversion.