Crate tracing_bunyan_formatter

tracing-bunyan-formatter

Bunyan formatting for tokio-rs/tracing.

tracing-bunyan-formatter provides two Layers implementation to be used on top of a tracing Subscriber:

• JsonStorageLayer, to attach contextual information to spans for ease of consumption by downstream Layers, via JsonStorage and Span’s extensions;
• BunyanFormattingLayer, which emits a bunyan-compatible formatted record upon entering a span, exiting a span and event creation.

Important: each span will inherit all fields and properties attached to its parent - this is currently not the behaviour provided by tracing_subscriber::fmt::Layer.

Example

use tracing_bunyan_formatter::{BunyanFormattingLayer, JsonStorageLayer};
use tracing::instrument;
use tracing::info;
use tracing_subscriber::Registry;
use tracing_subscriber::layer::SubscriberExt;

#[instrument]
pub fn a_unit_of_work(first_parameter: u64) {
    for i in 0..2 {
        a_sub_unit_of_work(i);
    }
    info!(excited = "true", "Tracing is quite cool!");
}

#[instrument]
pub fn a_sub_unit_of_work(sub_parameter: u64) {
    info!("Events have the full context of their parent span!");
}

fn main() {
    let formatting_layer = BunyanFormattingLayer::new("tracing_demo".into(), std::io::stdout);
    let subscriber = Registry::default()
        .with(JsonStorageLayer)
        .with(formatting_layer);
    tracing::subscriber::set_global_default(subscriber).unwrap();

    info!("Orphan event without a parent span");
    a_unit_of_work(2);
}

Console output

---

If you pipe the output in the bunyan CLI:

---

As a pure-Rust alternative check out the bunyan crate. It includes a CLI binary with similar functionality to the original bunyan CLI written in JavaScript.

Implementation strategy

The layered approach we have pursued is not necessarily the most efficient, but it makes it easier to separate different concerns and re-use common logic across multiple Layers.

While the current crate has no ambition to provide any sort of general purpose framework on top of tracing-subscriber’s Layer trait, the information collected by JsonStorageLayer can be leveraged via its public API by other downstream layers outside of this crate whose main concern is formatting. It significantly lowers the amount of complexity you have to deal with if you are interested in implementing your own formatter, for whatever reason or purpose.

You can also add another enrichment layer following the JsonStorageLayer to collect additional information about each span and store it in JsonStorage. We could have pursued this compositional approach to add elapsed_milliseconds to each span instead of baking it in JsonStorage itself.

Optional features

You can enable the arbitrary_precision feature to handle numbers of arbitrary size losslessly. Be aware of a known issue with untagged deserialization.

valuable

The tracing crate has an unstable feature valuable to enable recording custom composite types like structs and enums. Custom types must implement the valuable crate’s Valuable trait, which can be derived with a macro.

To use tracing and tracing-bunyan-formatter with valuable, you must set the following configuration in your binary (as of the current crate versions on 2023-03-29):

1. Enable the feature flag valuable for the tracing dependency.

2. Add the --cfg tracing_unstable arguments to your rustc flags (see tracing’s documentation on this). This can be done in a few ways:

   1. Adding the arguments to your binary package’s .cargo/config.toml under build.rustflags. See the cargo config reference documentation).

      Example:

      [build]
      rustflags = "--cfg tracing_unstable"

   2. Adding the arguments to the RUSTFLAGS environment variable when you run cargo. See the cargo environment variable docs).

      Example:

      RUSTFLAGS="--cfg tracing_unstable" cargo build

3. Enable the feature flag valuable for the tracing-bunyan-formatter dependency.

4. Add dependency valuable.

5. Optional: if you want to derive the Valuable trait for your custom types, enable the feature flag derive for the valuable dependency.

See more details in the example in examples/valuable.rs.

Testing

Just run cargo test.

To run extra tests with the valuable feature enabled, run:

RUSTFLAGS='--cfg tracing_unstable' \
cargo test --target-dir target/debug_valuable --features "valuable valuable/derive"

RUSTFLAGS='--cfg tracing_unstable' \
cargo run --example valuable --target-dir target/debug_valuable --features "valuable valuable/derive"

Structs

• BunyanFormattingLayer
  This layer is exclusively concerned with formatting information using the Bunyan format. It relies on the upstream JsonStorageLayer to get access to the fields attached to each span.
• JsonStorage
  JsonStorage will collect information about a span when it’s created (new_span handler) or when new records are attached to it (on_record handler) and store it in its extensions for future retrieval from other layers interested in formatting or further enrichment.
• JsonStorageLayer
  This layer is only concerned with information storage, it does not do any formatting or provide any output.
• SkipFieldError
  This error will be returned in BunyanFormattingLayer::skip_fields if trying to skip a core field.

Enums

• Type
  The type of record we are dealing with: entering a span, exiting a span, an event.