Struct DeltaTableConfig

pub struct DeltaTableConfig {
    pub require_tombstones: bool,
    pub require_files: bool,
    pub log_buffer_size: usize,
    pub log_batch_size: usize,
    pub io_runtime: Option<IORuntime>,
}

Configuration options for delta table

Fields

require_tombstones: bool

Indicates whether our use case requires tracking tombstones. This defaults to true

Read-only applications never require tombstones. Tombstones are only required when writing checkpoints, so even many writers may want to skip them.

require_files: bool

Indicates whether DeltaTable should track files. This defaults to true

Some append-only applications might have no need of tracking any files. Hence, DeltaTable will be loaded with significant memory reduction.

log_buffer_size: usize

Controls how many files to buffer from the commit log when updating the table. This defaults to 4 * number of cpus

Setting a value greater than 1 results in concurrent calls to the storage api. This can decrease latency if there are many files in the log since the last checkpoint, but will also increase memory usage. Possible rate limits of the storage backend should also be considered for optimal performance.

log_batch_size: usize

Control the number of records to read / process from the commit / checkpoint files when processing record batches.

io_runtime: Option<IORuntime>

When a runtime handler is provided, all IO tasks are spawn in that handle

Trait Implementations

impl Clone for DeltaTableConfig

fn clone(&self) -> DeltaTableConfig

Returns a copy of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for DeltaTableConfig

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Default for DeltaTableConfig

fn default() -> Self

Returns the “default value” for a type.

impl<'de> Deserialize<'de> for DeltaTableConfig

fn deserialize<__D>(__deserializer: __D) -> Result<Self, __D::Error>

where __D: Deserializer<'de>,

Deserialize this value from the given Serde deserializer.

impl PartialEq for DeltaTableConfig

fn eq(&self, other: &Self) -> bool

Tests for self and other values to be equal, and is used by ==.

fn ne(&self, other: &Rhs) -> bool

Tests for !=. The default implementation is almost always sufficient, and should not be overridden without very good reason.

impl Serialize for DeltaTableConfig

fn serialize<__S>(&self, __serializer: __S) -> Result<__S::Ok, __S::Error>

where __S: Serializer,

Serialize this value into the given Serde serializer.