Crate fuel_data_parser

Fuel Data Parser

A utility library for encoding and decoding data in the Fuel Data Systems project.

📚 Documentation   🐛 Report Bug   ✨ Request Feature

📝 About

The DataParser struct provides functionality for encoding and decoding data through compression and serialization. It offers flexibility in choosing compression strategies and serialization formats, allowing for optimization of memory usage and I/O bandwidth. This utility is particularly useful when dealing with large datasets or when efficient data transfer is crucial.

🛠️ Usage

This library is intended for internal use within the Fuel Data Systems project. This is an example of usage outside of this crate within the project:

use fuel_data_parser::{DataEncoder, DataParser, SerializationType, DataParserError};
use serde::{Serialize, Deserialize};

#[derive(Debug, Clone, Serialize, Deserialize)]
struct YourDataType {
    // Your data fields here
}

impl DataEncoder for YourDataType {
    type Err = DataParserError;
}

async fn example_usage() -> Result<(), Box<dyn std::error::Error>> {
    let parser = DataParser::default()
        .with_serialization_type(SerializationType::Bincode);

    // Encoding data
    let data = YourDataType { /* ... */ };
    let encoded = parser.encode(&data).await?;

    // Decoding data
    let decoded: YourDataType = parser.decode(&encoded).await?;

    Ok(())
}

🏎️ Benchmarks

To run the benchmarks and measure performance of different serialization and compression strategies:

cargo bench -p data-parser

[!INFO] The benchmarks are located in the ../../benches folder.

🤝 Contributing

Contributions are welcome! Please feel free to submit a Pull Request.

For more information on contributing, please see the CONTRIBUTING.md file in the root of the repository.

📜 License

This repo is licensed under the Apache-2.0 license. See LICENSE for more information.

Modules

compression_encoders_and_decoders

Types which operate over AsyncWrite streams, both encoders and decoders for various formats.

Structs

ALL_COMPRESSION_STRATEGIES

DEFAULT_COMPRESSION_STRATEGY

DataParser

DataParser is a utility struct for encoding (serializing and optionally compressing) and decoding (deserializing and optionally decompressing) data. It is useful for optimizing memory usage and I/O bandwidth by applying different serialization formats and optional compression strategies.

SerializationTypeIter

An iterator over the variants of SerializationType

ZstdCompressionStrategy

Enums

CompressionError

Compression error types

CompressionLevel

Level of compression data should be compressed with.

DataParserError

Data parser error types.

SerdeError

Serialization/Deserialization error types.

SerializationType

Serialization types supported for data parsing

Traits

CompressionStrategy

The CompressionStrategy trait defines the interface for compression and decompression strategies. It is sealed to restrict external implementations.

DataEncoder