Struct datafusion::functions_aggregate::regr::RegrAccumulator

pub struct RegrAccumulator { /* private fields */ }

RegrAccumulator is used to compute linear regression aggregate functions by maintaining statistics needed to compute them in an online fashion.

This struct uses Welford’s online algorithm for calculating variance and covariance: https://en.wikipedia.org/wiki/Algorithms_for_calculating_variance#Welford's_online_algorithm

Given the statistics, the following aggregate functions can be calculated:

• regr_slope(y, x): Slope of the linear regression line, calculated as: cov_pop(x, y) / var_pop(x). It represents the expected change in Y for a one-unit change in X.

• regr_intercept(y, x): Intercept of the linear regression line, calculated as: mean_y - (regr_slope(y, x) * mean_x). It represents the expected value of Y when X is 0.

• regr_count(y, x): Count of the non-null(both x and y) input rows.

• regr_r2(y, x): R-squared value (coefficient of determination), calculated as: (cov_pop(x, y) ^ 2) / (var_pop(x) * var_pop(y)). It provides a measure of how well the model’s predictions match the observed data.

• regr_avgx(y, x): Average of the independent variable X, calculated as: mean_x.

• regr_avgy(y, x): Average of the dependent variable Y, calculated as: mean_y.

• regr_sxx(y, x): Sum of squares of the independent variable X, calculated as: m2_x.

• regr_syy(y, x): Sum of squares of the dependent variable Y, calculated as: m2_y.

• regr_sxy(y, x): Sum of products of paired values, calculated as: algo_const.

Here’s how the statistics maintained in this struct are calculated:

• cov_pop(x, y): algo_const / count.
• var_pop(x): m2_x / count.
• var_pop(y): m2_y / count.

Implementations

impl RegrAccumulator

pub fn try_new(regr_type: &RegrType) -> Result<RegrAccumulator, DataFusionError>

Creates a new RegrAccumulator

Trait Implementations

impl Accumulator for RegrAccumulator

fn state(&mut self) -> Result<Vec<ScalarValue>, DataFusionError>

Returns the intermediate state of the accumulator, consuming the intermediate state.

fn update_batch( &mut self, values: &[Arc<dyn Array>], ) -> Result<(), DataFusionError>

Updates the accumulator’s state from its input.

fn supports_retract_batch(&self) -> bool

Does the accumulator support incrementally updating its value by removing values.

fn retract_batch( &mut self, values: &[Arc<dyn Array>], ) -> Result<(), DataFusionError>

Retracts (removed) an update (caused by the given inputs) to accumulator’s state.

fn merge_batch( &mut self, states: &[Arc<dyn Array>], ) -> Result<(), DataFusionError>

Updates the accumulator’s state from an Array containing one or more intermediate values.

fn evaluate(&mut self) -> Result<ScalarValue, DataFusionError>

Returns the final aggregate value, consuming the internal state.

fn size(&self) -> usize

Returns the allocated size required for this accumulator, in bytes, including Self.

impl Debug for RegrAccumulator

fn fmt(&self, f: &mut Formatter<'_>) -> Result<(), Error>

Formats the value using the given formatter.