Struct Bencher

pub struct Bencher<'a, M: Measurement = WallTime> { /* private fields */ }

Available on non-crate feature codspeed only.

Timer struct used to iterate a benchmarked function and measure the runtime.

This struct provides different timing loops as methods. Each timing loop provides a different way to time a routine and each has advantages and disadvantages.

• If you want to do the iteration and measurement yourself (eg. passing the iteration count to a separate process), use iter_custom.
• If your routine requires no per-iteration setup and returns a value with an expensive drop method, use iter_with_large_drop.
• If your routine requires some per-iteration setup that shouldn’t be timed, use iter_batched or iter_batched_ref. See BatchSize for a discussion of batch sizes. If the setup value implements Drop and you don’t want to include the drop time in the measurement, use iter_batched_ref, otherwise use iter_batched. These methods are also suitable for benchmarking routines which return a value with an expensive drop method, but are more complex than iter_with_large_drop.
• Otherwise, use iter.

Implementations

impl<'a, M: Measurement> Bencher<'a, M>

pub fn iter<O, R>(&mut self, routine: R)

where R: FnMut() -> O,

Times a routine by executing it many times and timing the total elapsed time.

Prefer this timing loop when routine returns a value that doesn’t have a destructor.

Timing model

Note that the Bencher also times the time required to destroy the output of routine(). Therefore prefer this timing loop when the runtime of mem::drop(O) is negligible compared to the runtime of the routine.

elapsed = Instant::now + iters * (routine + mem::drop(O) + Range::next)

Example

use criterion::*;

// The function to benchmark
fn foo() {
    // ...
}

fn bench(c: &mut Criterion) {
    c.bench_function("iter", move |b| {
        b.iter(|| foo())
    });
}

criterion_group!(benches, bench);
criterion_main!(benches);

pub fn iter_custom<R>(&mut self, routine: R)

where R: FnMut(u64) -> M::Value,

Times a routine by executing it many times and relying on routine to measure its own execution time.

Prefer this timing loop in cases where routine has to do its own measurements to get accurate timing information (for example in multi-threaded scenarios where you spawn and coordinate with multiple threads).

Timing model

Custom, the timing model is whatever is returned as the Duration from routine.

Example

use criterion::*;
use criterion::black_box;
use std::time::Instant;

fn foo() {
    // ...
}

fn bench(c: &mut Criterion) {
    c.bench_function("iter", move |b| {
        b.iter_custom(|iters| {
            let start = Instant::now();
            for _i in 0..iters {
                black_box(foo());
            }
            start.elapsed()
        })
    });
}

criterion_group!(benches, bench);
criterion_main!(benches);

pub fn iter_with_large_drop<O, R>(&mut self, routine: R)

where R: FnMut() -> O,

Times a routine by collecting its output on each iteration. This avoids timing the destructor of the value returned by routine.

WARNING: This requires O(iters * mem::size_of::<O>()) of memory, and iters is not under the control of the caller. If this causes out-of-memory errors, use iter_batched instead.

Timing model

elapsed = Instant::now + iters * (routine) + Iterator::collect::<Vec<_>>

Example

use criterion::*;

fn create_vector() -> Vec<u64> {
    // ...
}

fn bench(c: &mut Criterion) {
    c.bench_function("with_drop", move |b| {
        // This will avoid timing the Vec::drop.
        b.iter_with_large_drop(|| create_vector())
    });
}

criterion_group!(benches, bench);
criterion_main!(benches);

pub fn iter_batched<I, O, S, R>( &mut self, setup: S, routine: R, size: BatchSize, )

where S: FnMut() -> I, R: FnMut(I) -> O,

Times a routine that requires some input by generating a batch of input, then timing the iteration of the benchmark over the input. See BatchSize for details on choosing the batch size. Use this when the routine must consume its input.

For example, use this loop to benchmark sorting algorithms, because they require unsorted data on each iteration.

Timing model

elapsed = (Instant::now * num_batches) + (iters * (routine + O::drop)) + Vec::extend

Example

use criterion::*;

fn create_scrambled_data() -> Vec<u64> {
    // ...
}

// The sorting algorithm to test
fn sort(data: &mut [u64]) {
    // ...
}

fn bench(c: &mut Criterion) {
    let data = create_scrambled_data();

    c.bench_function("with_setup", move |b| {
        // This will avoid timing the clone call.
        b.iter_batched(|| data.clone(), |mut data| sort(&mut data), BatchSize::SmallInput)
    });
}

criterion_group!(benches, bench);
criterion_main!(benches);

pub fn iter_batched_ref<I, O, S, R>( &mut self, setup: S, routine: R, size: BatchSize, )

where S: FnMut() -> I, R: FnMut(&mut I) -> O,

Times a routine that requires some input by generating a batch of input, then timing the iteration of the benchmark over the input. See BatchSize for details on choosing the batch size. Use this when the routine should accept the input by mutable reference.

For example, use this loop to benchmark sorting algorithms, because they require unsorted data on each iteration.

Timing model

elapsed = (Instant::now * num_batches) + (iters * routine) + Vec::extend

Example

use criterion::*;

fn create_scrambled_data() -> Vec<u64> {
    // ...
}

// The sorting algorithm to test
fn sort(data: &mut [u64]) {
    // ...
}

fn bench(c: &mut Criterion) {
    let data = create_scrambled_data();

    c.bench_function("with_setup", move |b| {
        // This will avoid timing the clone call.
        b.iter_batched(|| data.clone(), |mut data| sort(&mut data), BatchSize::SmallInput)
    });
}

criterion_group!(benches, bench);
criterion_main!(benches);

pub fn iter_with_setup_wrapper<S>(&mut self, setup: S)

where S: FnMut(&mut WrapperRunner<'a, '_, M>),

Times a routine that requires some setup which mutably borrows data from outside the setup function.

The setup function is passed a [WrapperRunner]. It should perform whatever setup is required and then call run with the routine function. Only the execution time of the routine function is measured.

Each iteration of the benchmark is executed in series. So setup can mutably borrow data from outside its closure mutably and know that it has exclusive access to that data throughout each setup + routine iteration. i.e. equivalent to BatchSize::PerIteration.

Value returned by routine is returned from run. If you do not wish include drop time of a value in the measurement, return it from routine so it is dropped outside of the measured section.

Example

use criterion::*;

fn create_global_data() -> Vec<u64> {
    // ...
}

fn reset_global_data(data: &mut Vec<u64>) {
    // ...
}

// The algorithm to test
fn do_something_with(data: &mut [u64]) -> Vec<u64> {
    // ...
}

fn bench(c: &mut Criterion) {
    let mut data = create_global_data();

    c.bench_function("with_setup_wrapper", |b| {
        b.iter_with_setup_wrapper(|runner| {
            // Perform setup on each iteration. Not included in measurement.
            reset_global_data(&mut data);

            runner.run(|| {
                // Code in this closure is measured
                let result = do_something_with(&mut data);
                // Return result if do not want to include time dropping it in measure
                result
            });
        });
    });
}

criterion_group!(benches, bench);
criterion_main!(benches);

pub fn to_async<'b, A: AsyncExecutor>( &'b mut self, runner: A, ) -> AsyncBencher<'a, 'b, A, M>

Available on crate feature async only.

Convert this bencher into an AsyncBencher, which enables async/await support.