deltalake_core::operations

Module write

Source
Expand description

New Table Semantics

  • The schema of the RecordBatch is used to initialize the table.
  • The partition columns will be used to partition the table.

Existing Table Semantics

  • The save mode will control how existing data is handled (i.e. overwrite, append, etc)
  • (NOT YET IMPLEMENTED) The schema of the RecordBatch will be checked and if there are new columns present they will be added to the tables schema. Conflicting columns (i.e. a INT, and a STRING) will result in an exception.
  • The partition columns, if present, are validated against the existing metadata. If not present, then the partitioning of the table is respected.

In combination with Overwrite, a replaceWhere option can be used to transactionally replace data that matches a predicate.

§Example

ⓘ
let id_field = arrow::datatypes::Field::new("id", arrow::datatypes::DataType::Int32, false);
let schema = Arc::new(arrow::datatypes::Schema::new(vec![id_field]));
let ids = arrow::array::Int32Array::from(vec![1, 2, 3, 4, 5]);
let batch = RecordBatch::try_new(schema, vec![Arc::new(ids)])?;
let ops = DeltaOps::try_from_uri("../path/to/empty/dir").await?;
let table = ops.write(vec![batch]).await?;

Structs§

Enums§