Struct Manifest

pub struct Manifest {

    pub schema: Schema,
    pub local_schema: Schema,
    pub version: u64,
    pub writer_version: Option<WriterVersion>,
    pub fragments: Arc<Vec<Fragment>>,
    pub version_aux_data: usize,
    pub index_section: Option<usize>,
    pub timestamp_nanos: u128,
    pub tag: Option<String>,
    pub reader_feature_flags: u64,
    pub writer_feature_flags: u64,
    pub max_fragment_id: u32,
    pub transaction_file: Option<String>,
    pub next_row_id: u64,
    pub data_storage_format: DataStorageFormat,
    pub config: HashMap<String, String>,
    pub blob_dataset_version: Option<u64>,
    /* private fields */
}

Manifest of a dataset

• Schema
• Version
• Fragments.
• Indices.

Fields

schema: Schema

Dataset schema.

local_schema: Schema

Local schema, only containing fields with the default storage class (not blobs)

version: u64

Dataset version

writer_version: Option<WriterVersion>

Version of the writer library that wrote this manifest.

fragments: Arc<Vec<Fragment>>

Fragments, the pieces to build the dataset.

This list is stored in order, sorted by fragment id. However, the fragment id sequence may have gaps.

version_aux_data: usize

The file position of the version aux data.

index_section: Option<usize>

The file position of the index metadata.

timestamp_nanos: u128

The creation timestamp with nanosecond resolution as 128-bit integer

tag: Option<String>

An optional string tag for this version

reader_feature_flags: u64

The reader flags

writer_feature_flags: u64

The writer flags

max_fragment_id: u32

The max fragment id used so far

transaction_file: Option<String>

The path to the transaction file, relative to the root of the dataset

next_row_id: u64

The max row id used so far.

data_storage_format: DataStorageFormat

The storage format of the data files.

config: HashMap<String, String>

Table configuration.

blob_dataset_version: Option<u64>

Blob dataset version

Implementations

impl Manifest

pub fn new( schema: Schema, fragments: Arc<Vec<Fragment>>, data_storage_format: DataStorageFormat, blob_dataset_version: Option<u64>, ) -> Self

pub fn new_from_previous( previous: &Self, schema: Schema, fragments: Arc<Vec<Fragment>>, new_blob_version: Option<u64>, ) -> Self

pub fn timestamp(&self) -> DateTime<Utc>

Return the timestamp_nanos value as a Utc DateTime

pub fn set_timestamp(&mut self, nanos: u128)

Set the timestamp_nanos value from a Utc DateTime

pub fn update_config( &mut self, upsert_values: impl IntoIterator<Item = (String, String)>, )

Set the config from an iterator

pub fn delete_config_keys(&mut self, delete_keys: &[&str])

Delete config keys using a slice of keys

pub fn update_schema_metadata(&mut self, new_metadata: HashMap<String, String>)

Replaces the schema metadata with the given key-value pairs.

pub fn update_field_metadata( &mut self, field_id: i32, new_metadata: HashMap<String, String>, )

Replaces the metadata of the field with the given id with the given key-value pairs.

If the field does not exist in the schema, this is a no-op.

pub fn update_max_fragment_id(&mut self)

Check the current fragment list and update the high water mark

pub fn max_fragment_id(&self) -> Option<u64>

Return the max fragment id. Note this does not support recycling of fragment ids.

This will return None if there are no fragments.

pub fn max_field_id(&self) -> i32

Get the max used field id

This is different than Schema::max_field_id because it also considers the field ids in the data files that have been dropped from the schema.

pub fn fragments_since(&self, since: &Self) -> Result<Vec<Fragment>>

Return the fragments that are newer than the given manifest. Note this does not support recycling of fragment ids.

pub fn fragments_by_offset_range( &self, range: Range<usize>, ) -> Vec<(usize, &Fragment)>

Find the fragments that contain the rows, identified by the offset range.

Note that the offsets are the logical offsets of rows, not row IDs.

Parameters

range: Range Offset range

Returns

Vec<(usize, Fragment)> A vector of (starting_offset_of_fragment, fragment) pairs.

pub fn uses_move_stable_row_ids(&self) -> bool

Whether the dataset uses move-stable row ids.

pub fn serialized(&self) -> Vec<u8>

Creates a serialized copy of the manifest, suitable for IPC or temp storage and can be used to create a dataset

pub fn should_use_legacy_format(&self) -> bool

Trait Implementations

impl Clone for Manifest

fn clone(&self) -> Manifest

Returns a copy of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for Manifest

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl From<&Manifest> for Manifest

fn from(m: &Manifest) -> Self

Converts to this type from the input type.

impl PartialEq for Manifest

fn eq(&self, other: &Manifest) -> bool

Tests for self and other values to be equal, and is used by ==.

fn ne(&self, other: &Rhs) -> bool

Tests for !=. The default implementation is almost always sufficient, and should not be overridden without very good reason.

impl ProtoStruct for Manifest

type Proto = Manifest

impl TryFrom<Manifest> for Manifest

type Error = Error

The type returned in the event of a conversion error.

fn try_from(p: Manifest) -> Result<Self>

Performs the conversion.

impl StructuralPartialEq for Manifest