Module commit

Trait for commit implementations.

In Lance, a transaction is committed by writing the next manifest file. However, care should be taken to ensure that the manifest file is written only once, even if there are concurrent writers. Different stores have different abilities to handle concurrent writes, so a trait is provided to allow for different implementations.

The trait CommitHandler can be implemented to provide different commit strategies. The default implementation for most object stores is RenameCommitHandler, which writes the manifest to a temporary path, then renames the temporary path to the final path if no object already exists at the final path. This is an atomic operation in most object stores, but not in AWS S3. So for AWS S3, the default commit handler is UnsafeCommitHandler, which writes the manifest to the final path without any checks.

When providing your own commit handler, most often you are implementing in terms of a lock. The trait CommitLock can be implemented as a simpler alternative to CommitHandler.

Modules

• external_manifest
  Trait for external manifest handler.

Structs

• CommitConfig
• ManifestLocation
• RenameCommitHandler
  A commit implementation that uses a temporary path and renames the object.
• UnsafeCommitHandler
  A naive commit implementation that does not prevent conflicting writes.

Enums

• CommitError
  Errors that can occur when committing a manifest.
• ManifestNamingScheme
  How manifest files should be named.

Traits

• CommitHandler
  Handle commits that prevent conflicting writes.
• CommitLease
• CommitLock
  A commit implementation that uses a lock to prevent conflicting writes.

Functions

• commit_handler_from_url
• migrate_scheme_to_v2
  Migrate all V1 manifests to V2 naming scheme.
• parse_version_from_path

Type Aliases

• ManifestWriter
  Function that writes the manifest to the object store.