gix_features::parallel

Function in_parallel_with_finalize

Source
pub fn in_parallel_with_finalize<I, S, O, R>(
    input: impl Iterator<Item = I> + Send,
    thread_limit: Option<usize>,
    new_thread_state: impl FnOnce(usize) -> S + Send + Clone,
    consume: impl FnMut(I, &mut S) -> O + Send + Clone,
    finalize: impl FnOnce(S) -> O + Send + Clone,
    reducer: R,
) -> Result<<R as Reduce>::Output, <R as Reduce>::Error>
where R: Reduce<Input = O>, I: Send, O: Send,
Available on crate feature parallel only.
Expand description

Read items from input and consume them in multiple threads, whose output output is collected by a reducer. Its task is to aggregate these outputs into the final result returned by this function with the benefit of not having to be thread-safe. Caall finalize to finish the computation, once per thread, if there was no error sending results earlier.

  • if thread_limit is Some, the given amount of threads will be used. If None, all logical cores will be used.
  • new_thread_state(thread_number) -> State produces thread-local state once per thread to be based to consume
  • consume(Item, &mut State) -> Output produces an output given an input obtained by input along with mutable state initially created by new_thread_state(…).
  • finalize(State) -> Output is called to potentially process remaining work that was placed in State.
  • For reducer, see the Reduce trait