Module checkpoints

Implementation for writing delta checkpoints.

Constants

• CHECKPOINT_RECORD_BATCH_SIZE
  The record batch size for checkpoint parquet file

Functions

• cleanup_expired_logs_for
  Deletes all delta log commits that are older than the cutoff time and less than the specified version.
• cleanup_metadata
  Delete expires log files before given version from table. The table log retention is based on the logRetentionDuration property of the Delta Table, 30 days by default.
• create_checkpoint
  Creates checkpoint at current table version
• create_checkpoint_for
  Creates checkpoint for a given table version, table state and object store
• create_checkpoint_from_table_uri_and_cleanup
  Loads table from given table_uri at given version and creates checkpoint for it. The cleanup param decides whether to run metadata cleanup of obsolete logs. If it’s empty then the table’s enableExpiredLogCleanup is used.