criterion 0.2.0

Statistics-driven micro-benchmarking library
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86

# Changelog
All notable changes to this project will be documented in this file.

The format is based on [Keep a Changelog](http://keepachangelog.com/en/1.0.0/)
and this project adheres to [Semantic Versioning](http://semver.org/spec/v2.0.0.html).

## [Unreleased]

## [0.2.0]
### Added
- Added `Criterion.bench` function, which accepts either a `Benchmark` or
  `ParameterizedBenchmark`. These new structures allow for custom per-benchmark
  configuration as well as more complex benchmark grouping (eg. comparing a Rust
  function against an external program over a range of inputs) which was not
  possible previously.
- Criterion.rs can now report the throughput of the benchmarked code in units of
  bytes or elements per second. See the `Benchmark.throughput` and
  `ParameterizedBenchmark.throughput` functions for further details.
- Criterion.rs now generates a basic HTML report for each benchmark.
- Added `--noplot` command line option to disable plot generation.

### Changed
- The builder methods on the Criterion struct now take and return self by value
  for easier chaining. Functions which configure a Criterion structure will need
  to be updated accordingly, or will need to be changed to work with the
  `Benchmark` or `ParameterizedBenchmark` types to do per-benchmark configuration
  instead.
- The closures taken by `Criterion.bench_*` must now have a `'static` lifetime.
  This means that you may need to change your closures from `|bencher| {...}`
  to `move |bencher| {...}`.
- `Criterion.bench_functions` now takes `I` as an input parameter, not `&I`.
- Input values must now implement `Debug` rather than `Display`.
- The generated plots are stored in `target/criterion` rather than `.criterion`.

### Removed
- The hidden `criterion::ConfidenceInterval` and`criterion::Estimate` types are
  no longer publicly accessible.
- The `Criterion.summarize` function has been removed.

### Fixed
- Fixed the relative mean and median reports.
- Fixed panic while summarizing benchmarks.

## [0.1.2]
### Changed
- Criterion.rs is now stable-compatible!
- Criterion.rs now includes its own stable-compatible `black_box` function.
  Some benchmarks may now be affected by dead-code-elimination where they
  previously weren't and may have to be updated.
- Criterion.rs now uses `serde` to save results. Existing results files will
  be automatically removed when benchmarks are run.
- Redesigned the command-line output to highlight the important information
  and reduce noise.

### Added
- Running benchmarks with the variable "CRITERION_DEBUG" in the environment will
  cause Criterion.rs to generate extra debug output and save the gnuplot scripts
  alongside the generated plots.

### Fixed
- Don't panic on IO errors or gnuplot failures
- Fix generation of invalid gnuplot scripts when benchmarking over inputs and inputs include values <= 0.
- Bug where benchmarks would run one sample fewer than was configured.

### Removed
- Generated plots will no longer use log-scale.

## [0.1.1]
### Added
- A changelog file.
- Added a chapter to the book on how Criterion.rs collects and analyzes data.
- Added macro rules to generate a test harness for use with `cargo bench`.
  Benchmarks defined without these macros should continue to work.
- New contribution guidelines
- Criterion.rs can selectively run benchmarks. See the Command-line page for
more details

## 0.1.0 - 2017-12-02
### Added
- Initial release on Crates.io.


[Unreleased]: https://github.com/japaric/criterion.rs/compare/0.2.0...HEAD
[0.1.1]: https://github.com/japaric/criterion.rs/compare/0.1.0...0.1.1
[0.1.2]: https://github.com/japaric/criterion.rs/compare/0.1.1...0.1.2
[0.2.0]: https://github.com/japaric/criterion.rs/compare/0.1.2...0.2.0