Function gix_features::parallel::in_parallel_with_slice

pub fn in_parallel_with_slice<I, S, R, E>(
    input: &mut [I],
    thread_limit: Option<usize>,
    new_thread_state: impl FnOnce(usize) -> S + Send + Clone,
    consume: impl FnMut(&mut I, &mut S, &AtomicIsize, &AtomicBool) -> Result<(), E> + Send + Clone,
    periodic: impl FnMut() -> Option<Duration> + Send,
    state_to_rval: impl FnOnce(S) -> R + Send + Clone
) -> Result<Vec<R>, E>
where
    I: Send,
    E: Send,
    R: Send,

Available on crate feature parallel only.

An experiment to have fine-grained per-item parallelization with built-in aggregation via thread state. This is only good for operations where near-random access isn’t detrimental, so it’s not usually great for file-io as it won’t make use of sorted inputs well. Note that periodic is not guaranteed to be called in case other threads come up first and finish too fast. consume(&mut item, &mut stat, &Scope, &threads_available, &should_interrupt) is called for performing the actual computation. Note that threads_available should be decremented to start a thread that can steal your own work (as stored in item), which allows callees to implement their own work-stealing in case the work is distributed unevenly. Work stealing should only start after having processed at least one item to give all threads naturally operating on the slice some time to start. Starting threads while slice-workers are still starting up would lead to over-allocation of threads, which is why the number of threads left may turn negative. Once threads are started and stopped, be sure to adjust the thread-count accordingly.