Crate pot

A concise storage format, written for BonsaiDb.

Pot is an encoding format used within BonsaiDb. Its purpose is to provide an encoding format for serde that:

• Is self-describing.

• Is safe to run in production.

• Is compact. While still being self-describing, Pot’s main space-saving feature is not repeating symbols/identifiers more than one time while serializing. When serializing arrays of structures, this can make a major difference. The logs.rs example demonstrates this:

  $ cargo test --example logs -- average_sizes --nocapture
  Generating 1000 LogArchives with 100 entries.
  +-----------------+-----------+-----------------+
  | Format          | Bytes     | Self-Describing |
  +-----------------+-----------+-----------------+
  | pot             | 2,627,586 | yes             |
  +-----------------+-----------+-----------------+
  | cbor            | 3,072,369 | yes             |
  +-----------------+-----------+-----------------+
  | msgpack(named)  | 3,059,915 | yes             |
  +-----------------+-----------+-----------------+
  | msgpack         | 2,559,907 | no              |
  +-----------------+-----------+-----------------+
  | bincode(varint) | 2,506,844 | no              |
  +-----------------+-----------+-----------------+
  | bincode         | 2,755,137 | no              |
  +-----------------+-----------+-----------------+
  

Example

use serde::{Deserialize, Serialize};

#[derive(Serialize, Deserialize, Debug, Eq, PartialEq)]
pub struct User {
    id: u64,
    name: String,
}

fn main() -> Result<(), pot::Error> {
    let user = User {
        id: 42,
        name: String::from("ecton"),
    };
    let serialized = pot::to_vec(&user)?;
    println!("User serialized: {serialized:02x?}");
    let deserialized: User = pot::from_slice(&serialized)?;
    assert_eq!(deserialized, user);

    // Pot also provides a "Value" type for serializing Pot encoded payloads
    // without needing the original structure.
    let user: pot::Value<'_> = pot::from_slice(&serialized)?;
    println!("User decoded as value: {user}");

    Ok(())
}

Outputs:

User serialized: [50, 6f, 74, 00, a2, c4, 69, 64, 40, 2a, c8, 6e, 61, 6d, 65, e5, 65, 63, 74, 6f, 6e]
User decoded as value: {id: 42, name: ecton}

Benchmarks

Because benchmarks can be subjective and often don’t mirror real-world usage, this library’s authors aren’t making any specific performance claims. The way Pot achieves space savings requires some computational overhead. As such, it is expected that a hypothetically perfect CBOR implementation could outperform a hypothetically perfect Pot implementation.

The results from the current benchmark suite executed on Github Actions are viewable here. The current suite is only aimed at comparing the default performance for each library.

Serialize into new Vec<u8>

Serialize into reused Vec<u8>

Deserialize

Modules

• de
  Types for deserializing pots.
• format
  Low-level interface for reading and writing the pot format.
• reader
  Types for reading data.
• ser
  Types for serializing pots.

Structs

• Config
  Serialization and deserialization configuration.
• OwnedValue
  A [Value<'static>] wrapper that supports DeserializeOwned.

Enums

• Error
  All errors that Pot may return.
• Value
  A Pot encoded value. This type can be used to deserialize to and from Pot without knowing the original data structure.
• ValueError
  An error from deserializing a type using Value::deserialize_as.

Functions

• from_reader
  Restore a previously Pot-serialized value from a Read implementor.
• from_slice
  Restore a previously Pot-serialized value from a slice.
• to_vec
  Serialize value using Pot into a Vec<u8>.
• to_writer
  Serialize value using Pot into writer.

Type Definitions

• Result
  A result alias that returns Error.