Struct MergeBuilder

pub struct MergeBuilder { /* private fields */ }

Merge records into a Delta Table.

Implementations

impl MergeBuilder

pub fn new<E: Into<Expression>>( log_store: LogStoreRef, snapshot: DeltaTableState, predicate: E, source: DataFrame, ) -> Self

Create a new MergeBuilder

pub fn when_matched_update<F>(self, builder: F) -> DeltaResult<MergeBuilder>

where F: FnOnce(UpdateBuilder) -> UpdateBuilder,

Update a target record when it matches with a source record

The update expressions can specify both source and target columns.

Multiple match clauses can be specified and their predicates are evaluated to determine if the corresponding operation are performed. Only the first clause that results in a satisfy predicate is executed. The order of match clauses matter.

#Example

let table = open_table("../path/to/table")?;
let (table, metrics) = DeltaOps(table)
    .merge(source, col("target.id").eq(col("source.id")))
    .with_source_alias("source")
    .with_target_alias("target")
    .when_matched_update(|update| {
        update
            .predicate(col("source.value").lt(lit(0)))
            .update("value", lit(0))
            .update("modified", col("source.modified"))
    })?
    .when_matched_update(|update| {
        update
            .update("value", col("source.value") + lit(1))
            .update("modified", col("source.modified"))
    })?
    .await?

pub fn when_matched_delete<F>(self, builder: F) -> DeltaResult<MergeBuilder>

where F: FnOnce(DeleteBuilder) -> DeleteBuilder,

Delete a target record when it matches with a source record

Multiple match clauses can be specified and their predicates are evaluated to determine if the corresponding operation are performed. Only the first clause that results in a satisfy predicate is executed. The order of match clauses matter.

#Example

let table = open_table("../path/to/table")?;
let (table, metrics) = DeltaOps(table)
    .merge(source, col("target.id").eq(col("source.id")))
    .with_source_alias("source")
    .with_target_alias("target")
    .when_matched_delete(|delete| {
        delete.predicate(col("source.delete"))
    })?
    .await?

pub fn when_not_matched_insert<F>(self, builder: F) -> DeltaResult<MergeBuilder>

where F: FnOnce(InsertBuilder) -> InsertBuilder,

Insert a source record when it does not match with a target record

Multiple not match clauses can be specified and their predicates are evaluated to determine if the corresponding operation are performed. Only the first clause that results in a satisfy predicate is executed. The order of not match clauses matter.

#Example

let table = open_table("../path/to/table")?;
let (table, metrics) = DeltaOps(table)
    .merge(source, col("target.id").eq(col("source.id")))
    .with_source_alias("source")
    .with_target_alias("target")
    .when_not_matched_insert(|insert| {
        insert
            .set("id", col("source.id"))
            .set("value", col("source.value"))
            .set("modified", col("source.modified"))
    })?
    .await?

pub fn when_not_matched_by_source_update<F>( self, builder: F, ) -> DeltaResult<MergeBuilder>

where F: FnOnce(UpdateBuilder) -> UpdateBuilder,

Update a target record when it does not match with a source record

The update expressions can specify only target columns.

Multiple source not match clauses can be specified and their predicates are evaluated to determine if the corresponding operation are performed. Only the first clause that results in a satisfy predicate is executed. The order of source not match clauses matter.

#Example

let table = open_table("../path/to/table")?;
let (table, metrics) = DeltaOps(table)
    .merge(source, col("target.id").eq(col("source.id")))
    .with_source_alias("source")
    .with_target_alias("target")
    .when_not_matched_by_source_update(|update| {
        update
            .update("active", lit(false))
            .update("to_dt", lit("2023-07-11"))
    })?
    .await?

pub fn when_not_matched_by_source_delete<F>( self, builder: F, ) -> DeltaResult<MergeBuilder>

where F: FnOnce(DeleteBuilder) -> DeleteBuilder,

Delete a target record when it does not match with a source record

Multiple source “not match” clauses can be specified and their predicates are evaluated to determine if the corresponding operations are performed. Only the first clause that results in a satisfy predicate is executed. The order of source “not match” clauses matter.

#Example

let table = open_table("../path/to/table")?;
let (table, metrics) = DeltaOps(table)
    .merge(source, col("target.id").eq(col("source.id")))
    .with_source_alias("source")
    .with_target_alias("target")
    .when_not_matched_by_source_delete(|delete| {
        delete
    })?
    .await?

pub fn with_source_alias<S: ToString>(self, alias: S) -> Self

Rename columns in the source dataset to have a prefix of alias.original column name

pub fn with_target_alias<S: ToString>(self, alias: S) -> Self

Rename columns in the target dataset to have a prefix of alias.original column name

pub fn with_session_state(self, state: SessionState) -> Self

The Datafusion session state to use

pub fn with_commit_properties(self, commit_properties: CommitProperties) -> Self

Additional metadata to be added to commit info

pub fn with_writer_properties(self, writer_properties: WriterProperties) -> Self

Writer properties passed to parquet writer for when fiiles are rewritten

pub fn with_safe_cast(self, safe_cast: bool) -> Self

Specify the cast options to use when casting columns that do not match the table’s schema. When cast_options.safe is set true then any failures to cast a datatype will use null instead of returning an error to the user.

Example (column’s type is int): Input Output 123 -> 123 Test123 -> null

Trait Implementations

impl IntoFuture for MergeBuilder

type Output = Result<(DeltaTable, MergeMetrics), DeltaTableError>

The output that the future will produce on completion.

type IntoFuture = Pin<Box<dyn Future<Output = <MergeBuilder as IntoFuture>::Output> + Send>>

Which kind of future are we turning this into?

fn into_future(self) -> Self::IntoFuture

Creates a future from a value.