Struct BlobFieldScheduler

pub struct BlobFieldScheduler { /* private fields */ }

A field scheduler for large binary data

Large binary data (1MiB+) can be inefficient if we store as a regular primitive. We essentially end up with 1 page per row (or a few rows) and the overhead of the metadata can be significant.

At the same time the benefits of using pages (contiguous arrays) are pretty small since we can generally perform random access at these sizes without much penalty.

This encoder gives up the random access and stores the large binary data out of line. This keeps the metadata small.

Implementations

impl BlobFieldScheduler

pub fn new(descriptions_scheduler: Arc<dyn FieldScheduler>) -> Self

Trait Implementations

impl Debug for BlobFieldScheduler

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl FieldScheduler for BlobFieldScheduler

fn schedule_ranges<'a>( &'a self, ranges: &[Range<u64>], filter: &FilterExpression, ) -> Result<Box<dyn SchedulingJob + 'a>>

Schedules I/O for the requested portions of the field.

fn num_rows(&self) -> u64

The number of rows in this field

fn initialize<'a>( &'a self, filter: &'a FilterExpression, context: &'a SchedulerContext, ) -> BoxFuture<'a, Result<()>>

Called at the beginning of scheduling to initialize the scheduler