Trait Shuffler

pub trait Shuffler: Send + Sync {
    // Required method
    fn shuffle<'life0, 'async_trait>(
        &'life0 self,
        data: Box<dyn RecordBatchStream + Unpin + 'static>,
    ) -> Pin<Box<dyn Future<Output = Result<Box<dyn ShuffleReader>>> + Send + 'async_trait>>
       where Self: 'async_trait,
             'life0: 'async_trait;
}

A shuffler that can shuffle the incoming stream of record batches into IVF partitions. Returns a IvfShuffleReader that can be used to read the shuffled partitions.

Required Methods

fn shuffle<'life0, 'async_trait>( &'life0 self, data: Box<dyn RecordBatchStream + Unpin + 'static>, ) -> Pin<Box<dyn Future<Output = Result<Box<dyn ShuffleReader>>> + Send + 'async_trait>>

where Self: 'async_trait, 'life0: 'async_trait,

Shuffle the incoming stream of record batches into IVF partitions. Returns a IvfShuffleReader that can be used to read the shuffled partitions.

Implementors

impl Shuffler for IvfShuffler