Crate fuel_data_parser

Fuel Data Parser

A utility library for encoding and decoding data in the Fuel Data Systems project.

📚 Documentation   🐛 Report Bug   ✨ Request Feature

📝 About

The DataParser struct provides functionality for encoding and decoding data through compression and serialization. It offers flexibility in choosing compression strategies and serialization formats, allowing for optimization of memory usage and I/O bandwidth. This utility is particularly useful when dealing with large datasets or when efficient data transfer is crucial.

🛠️ Usage

This library is intended for internal use within the Fuel Data Systems project. This is an example of usage outside of this crate within the project:

use fuel_data_parser::{DataParser, SerializationType, DataParseable};
use serde::{Serialize, Deserialize};

#[derive(Debug, Clone, Serialize, Deserialize)]
struct YourDataType {
    // Your data fields here
}

async fn example_usage() -> Result<(), Box<dyn std::error::Error>> {
    let parser = DataParser::default()
        .with_serialization_type(SerializationType::Bincode);

    // Encoding data
    let data = YourDataType { /* ... */ };
    let encoded = parser.encode(&data).await?;

    // Decoding data
    let decoded: YourDataType = parser.decode(&encoded).await?;

    Ok(())
}

🏎️ Benchmarks

To run the benchmarks and measure performance of different serialization and compression strategies:

cargo bench -p data-parser -p nats-publisher -p bench-consumers

[!INFO] The benchmarks are located in the ../../benches folder.

🤝 Contributing

Contributions are welcome! Please feel free to submit a Pull Request.

For more information on contributing, please see the CONTRIBUTING.md file in the root of the repository.

📜 License

This repo is licensed under the Apache-2.0 license. See LICENSE for more information.

Modules

• compression_encoders_and_decoders
  Types which operate over AsyncWrite streams, both encoders and decoders for various formats.

Structs

• DEFAULT_COMPRESSION_STRATEGY
• DataParser
  DataParser is a utility struct for encoding (serializing and optionally compressing) and decoding (deserializing and optionally decompressing) data. It is useful for optimizing memory usage and I/O bandwidth by applying different serialization formats and optional compression strategies.
• SerializationTypeIter
  An iterator over the variants of SerializationType
• ZLibCompressionStrategy

Enums

• CompressionError
  Compression error types
• CompressionLevel
  Level of compression data should be compressed with.
• Error
  Data parser error types.
• SerdeError
  Serialization/Deserialization error types.
• SerializationType
  Serialization types supported for data parsing

Traits

• CompressionStrategy
  The CompressionStrategy trait defines the interface for compression and decompression strategies. It is sealed to restrict external implementations.
• DataParseable
  Traits required for a data type to be parseable