Module optimize

Optimize a Delta Table

Perform bin-packing on a Delta Table which merges small files into a large file. Bin-packing reduces the number of API calls required for read operations.

Optimize will fail if a concurrent write operation removes files from the table (such as in an overwrite). It will always succeed if concurrent writers are only appending.

Optimize increments the table’s version and creates remove actions for optimized files. Optimize does not delete files from storage. To delete files that were removed, call vacuum on DeltaTable.

See OptimizeBuilder for configuration.

Example

let table = open_table("../path/to/table")?;
let (table, metrics) = OptimizeBuilder::new(table.object_store(), table.state).await?;

Structs

• MergePlan
  Encapsulates the operations required to optimize a Delta Table
• MergeTaskParameters
  Parameters passed to individual merge tasks
• MetricDetails
  Statistics on files for a particular operation Operation can be remove or add
• Metrics
  Metrics from Optimize
• OptimizeBuilder
  Optimize a Delta table with given options
• PartialMetrics
  Metrics for a single partition

Enums

• OptimizeType
  Type of optimization to perform.

Functions

• create_merge_plan
  Build a Plan on which files to merge together. See OptimizeBuilder