Module commit

Trait for commit implementations.

In Lance, a transaction is committed by writing the next manifest file. However, care should be taken to ensure that the manifest file is written only once, even if there are concurrent writers. Different stores have different abilities to handle concurrent writes, so a trait is provided to allow for different implementations.

The trait CommitHandler can be implemented to provide different commit strategies. The default implementation for most object stores is RenameCommitHandler, which writes the manifest to a temporary path, then renames the temporary path to the final path if no object already exists at the final path. This is an atomic operation in most object stores, but not in AWS S3. So for AWS S3, the default commit handler is UnsafeCommitHandler, which writes the manifest to the final path without any checks.

When providing your own commit handler, most often you are implementing in terms of a lock. The trait CommitLock can be implemented as a simpler alternative to CommitHandler.

Modules

external_manifest

Trait for external manifest handler.

Structs

CommitConfig

ManifestLocation

RenameCommitHandler

A commit implementation that uses a temporary path and renames the object.

UnsafeCommitHandler

A naive commit implementation that does not prevent conflicting writes.

Enums

CommitError

Errors that can occur when committing a manifest.

ManifestNamingScheme

How manifest files should be named.

Traits

CommitHandler

Handle commits that prevent conflicting writes.

CommitLease

CommitLock

A commit implementation that uses a lock to prevent conflicting writes.

Functions

commit_handler_from_url

migrate_scheme_to_v2

Migrate all V1 manifests to V2 naming scheme.

parse_version_from_path

Type Aliases

ManifestWriter

Function that writes the manifest to the object store.