Trait ObjectStoreRetryExt

pub trait ObjectStoreRetryExt: ObjectStore {
    // Provided methods
    fn put_with_retries<'life0, 'life1, 'async_trait>(
        &'life0 self,
        location: &'life1 Path,
        bytes: PutPayload,
        max_retries: usize,
    ) -> Pin<Box<dyn Future<Output = Result<PutResult>> + Send + 'async_trait>>
       where Self: Sync + 'async_trait,
             'life0: 'async_trait,
             'life1: 'async_trait { ... }
    fn delete_with_retries<'life0, 'life1, 'async_trait>(
        &'life0 self,
        location: &'life1 Path,
        max_retries: usize,
    ) -> Pin<Box<dyn Future<Output = Result<()>> + Send + 'async_trait>>
       where Self: Sync + 'async_trait,
             'life0: 'async_trait,
             'life1: 'async_trait { ... }
}

Retry extension for ObjectStore

Read-only operations are retried by ObjectStore internally. However, PUT/DELETE operations are not retried even thought they are technically idempotent. ObjectStore does not retry those operations because having preconditions may produce different results for the same request. PUT/DELETE operations without preconditions are idempotent and can be retried. Unfortunately, ObjectStore’s retry mechanism only works on HTTP request level, thus there is no way to distinguish whether a request has preconditions or not.

This trait provides additional methods for working with ObjectStore that automatically retry unconditional operations when they fail.

See also:

• https://github.com/apache/arrow-rs/pull/5278

Provided Methods

fn put_with_retries<'life0, 'life1, 'async_trait>( &'life0 self, location: &'life1 Path, bytes: PutPayload, max_retries: usize, ) -> Pin<Box<dyn Future<Output = Result<PutResult>> + Send + 'async_trait>>

where Self: Sync + 'async_trait, 'life0: 'async_trait, 'life1: 'async_trait,

Save the provided bytes to the specified location

The operation is guaranteed to be atomic, it will either successfully write the entirety of bytes to location, or fail. No clients should be able to observe a partially written object

Note that put_with_opts may have precondition semantics, and thus may not be retriable.

fn delete_with_retries<'life0, 'life1, 'async_trait>( &'life0 self, location: &'life1 Path, max_retries: usize, ) -> Pin<Box<dyn Future<Output = Result<()>> + Send + 'async_trait>>

where Self: Sync + 'async_trait, 'life0: 'async_trait, 'life1: 'async_trait,

Delete the object at the specified location

Implementors

impl<T: ObjectStore + ?Sized> ObjectStoreRetryExt for T