Struct JsonWriter

pub struct JsonWriter { /* private fields */ }

Writes messages to a delta lake table.

Implementations

impl JsonWriter

pub async fn try_new( table_uri: String, schema_ref: ArrowSchemaRef, partition_columns: Option<Vec<String>>, storage_options: Option<HashMap<String, String>>, ) -> Result<Self, DeltaTableError>

Create a new JsonWriter instance

pub fn for_table(table: &DeltaTable) -> Result<JsonWriter, DeltaTableError>

Creates a JsonWriter to write to the given table

pub fn buffer_len(&self) -> usize

Returns the current byte length of the in memory buffer. This may be used by the caller to decide when to finalize the file write.

pub fn buffered_record_batch_count(&self) -> usize

Returns the number of records held in the current buffer.

pub fn reset(&mut self)

Resets internal state.

pub fn arrow_schema(&self) -> Arc<Schema>

Returns the user-defined arrow schema representation or the schema defined for the wrapped table.

Trait Implementations

impl Debug for JsonWriter

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl DeltaWriter<Vec<Value>> for JsonWriter

fn write<'life0, 'async_trait>( &'life0 mut self, values: Vec<Value>, ) -> Pin<Box<dyn Future<Output = Result<(), DeltaTableError>> + Send + 'async_trait>>

where Self: 'async_trait, 'life0: 'async_trait,

Write a chunk of values into the internal write buffers with the default write mode

fn write_with_mode<'life0, 'async_trait>( &'life0 mut self, values: Vec<Value>, mode: WriteMode, ) -> Pin<Box<dyn Future<Output = Result<(), DeltaTableError>> + Send + 'async_trait>>

where Self: 'async_trait, 'life0: 'async_trait,

Writes the given values to internal parquet buffers for each represented partition.

fn flush<'life0, 'async_trait>( &'life0 mut self, ) -> Pin<Box<dyn Future<Output = Result<Vec<Add>, DeltaTableError>> + Send + 'async_trait>>

where Self: 'async_trait, 'life0: 'async_trait,

Writes the existing parquet bytes to storage and resets internal state to handle another file.

This function returns the Add actions which should be committed to the DeltaTable for the written data files

fn flush_and_commit<'life0, 'life1, 'async_trait>( &'life0 mut self, table: &'life1 mut DeltaTable, ) -> Pin<Box<dyn Future<Output = Result<i64, DeltaTableError>> + Send + 'async_trait>>

where Self: Send + 'async_trait, 'life0: 'async_trait, 'life1: 'async_trait,

Flush the internal write buffers to files in the delta table folder structure. and commit the changes to the Delta log, creating a new table version.