Struct tantivy_columnar::Dictionary

pub struct Dictionary<TSSTable = VoidSSTable>
where
    TSSTable: SSTable,
{
    pub sstable_slice: FileSlice,
    pub sstable_index: SSTableIndex,
    /* private fields */
}

An SSTable is a sorted map that associates sorted &[u8] keys to any kind of typed values.

The SSTable is organized in blocks. In each block, keys and values are encoded separately.

The keys are encoded using incremental encoding. The values on the other hand, are encoded according to a value-specific codec defined in the TSSTable generic argument.

Finally, an index is joined to the Dictionary to make it possible, given a key to identify which block contains this key.

The codec was designed in such a way that the sstable reader is not aware of block, and yet can read any sequence of blocks, as long as the slice of bytes it is given starts and stops at block boundary.

(See also README.md)

Fields

sstable_slice: FileSlice

sstable_index: SSTableIndex

Implementations

impl Dictionary

pub fn build_for_tests(terms: &[&str]) -> Dictionary

impl<TSSTable> Dictionary<TSSTable>

where TSSTable: SSTable,

pub fn builder<W>( wrt: W ) -> Result<Writer<W, <TSSTable as SSTable>::ValueWriter>, Error>

where W: Write,

pub fn file_slice_for_range( &self, key_range: impl RangeBounds<[u8]>, limit: Option<u64> ) -> FileSlice

This function returns a file slice covering a set of sstable blocks that include the key range passed in arguments. Optionally returns only block for up to limit matching terms.

It works by identifying

• first_block: the block containing the start boundary key
• last_block: the block containing the end boundary key.

And then returning the range that spans over all blocks between. and including first_block and last_block, aka: [first_block.start_offset .. last_block.end_offset)

Technically this function does not provide the tightest fit, as for simplification, it treats the start bound of the key_range as if it was inclusive, even if it is exclusive. On the rare edge case where a user asks for (start_key, end_key] and start_key happens to be the last key of a block, we return a slice that is the first block was not necessary.

pub fn open( term_dictionary_file: FileSlice ) -> Result<Dictionary<TSSTable>, Error>

Opens a TermDictionary.

pub fn from_bytes( owned_bytes: OwnedBytes ) -> Result<Dictionary<TSSTable>, Error>

Creates a term dictionary from the supplied bytes.

pub fn empty() -> Dictionary<TSSTable>

Creates an empty term dictionary which contains no terms.

pub fn num_terms(&self) -> usize

Returns the number of terms in the dictionary. Term ordinals range from 0 to num_terms() - 1.

pub fn term_ord<K>(&self, key: K) -> Result<Option<u64>, Error>

where K: AsRef<[u8]>,

Returns the ordinal associated with a given term.

pub fn ord_to_term(&self, ord: u64, bytes: &mut Vec<u8>) -> Result<bool, Error>

Returns the term associated with a given term ordinal.

Term ordinals are defined as the position of the term in the sorted list of terms.

Returns true if and only if the term has been found.

Regardless of whether the term is found or not, the buffer may be modified.

pub fn term_info_from_ord( &self, term_ord: u64 ) -> Result<Option<<TSSTable as SSTable>::Value>, Error>

Returns the number of terms in the dictionary.

pub fn get<K>( &self, key: K ) -> Result<Option<<TSSTable as SSTable>::Value>, Error>

where K: AsRef<[u8]>,

Lookups the value corresponding to the key.

pub async fn get_async<K>( &self, key: K ) -> Result<Option<<TSSTable as SSTable>::Value>, Error>

where K: AsRef<[u8]>,

Lookups the value corresponding to the key.

pub fn range(&self) -> StreamerBuilder<'_, TSSTable>

Returns a range builder, to stream all of the terms within an interval.

pub fn stream(&self) -> Result<Streamer<'_, TSSTable>, Error>

A stream of all the sorted terms.

pub fn search<'a, A>(&'a self, automaton: A) -> StreamerBuilder<'a, TSSTable, A>

where A: Automaton + 'a, <A as Automaton>::State: Clone,

Returns a search builder, to stream all of the terms within the Automaton

Trait Implementations

impl<TSSTable> Clone for Dictionary<TSSTable>

where TSSTable: Clone + SSTable,

fn clone(&self) -> Dictionary<TSSTable>

Returns a copy of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl<TSSTable> Debug for Dictionary<TSSTable>

where TSSTable: Debug + SSTable,

fn fmt(&self, f: &mut Formatter<'_>) -> Result<(), Error>

Formats the value using the given formatter.