lance_encoding

Trait EncodingsIo

Source 
pub trait EncodingsIo:
    Debug
    + Send
    + Sync {
    // Required method
    fn submit_request(
        &self,
        range: Vec<Range<u64>>,
        priority: u64,
    ) -> BoxFuture<'static, Result<Vec<Bytes>>>;

    // Provided method
    fn submit_single(
        &self,
        range: Range<u64>,
        priority: u64,
    ) -> BoxFuture<'static, Result<Bytes>> { ... }
}
Expand description

A trait for an I/O service

This represents the I/O API that the encoders and decoders need in order to operate. We specify this as a trait so that lance-encodings does not need to depend on lance-io

In general, it is assumed that this trait will be implemented by some kind of “file reader” or “file scheduler”. The encodings here are all limited to accessing a single file.

Required Methods§

Source

fn submit_request( &self, range: Vec<Range<u64>>, priority: u64, ) -> BoxFuture<'static, Result<Vec<Bytes>>>

Submit an I/O request

The response must contain a Bytes object for each range requested even if the underlying I/O was coalesced into fewer actual requests.

§Arguments
  • ranges - the byte ranges to request
  • priority - the priority of the request

Priority should be set to the lowest row number that this request is delivering data for. This is important in cases where indirect I/O causes high priority requests to be submitted after low priority requests. We want to fulfill the indirect I/O more quickly so that we can decode as quickly as possible.

The implementation should be able to handle empty ranges, and should return an empty byte buffer for each empty range.

Provided Methods§

Source

fn submit_single( &self, range: Range<u64>, priority: u64, ) -> BoxFuture<'static, Result<Bytes>>

Submit an I/O request with a single range

This is just a utitliy function that wraps EncodingsIo::submit_request for the common case of a single range request.

Implementors§