Struct TokenBucketRateLimiter

pub struct TokenBucketRateLimiter { /* private fields */ }

A classic token bucket rate limiter that treats non-conformant calls by waiting indefinitely for tokens to become available.

The token bucket scheme allows users to “burst” up to at most max-tokens during some period, but replaces tokens at a fixed rate so users have some flexibility, but the overall load of the server is still mediated.

The behavior of it’s implementation of VariableCostRateLimiter can be seen in this example.

TokenBucketRateLimiters implement Clone, so cloning a rate limiter will clone the underlying state, resulting in two separate rate limiters that share the same underlying state. This allows for structs containing a rate limiter, like an HTTP or REST client, to be Clone while both instances will respect rate limits.

Trying to acquire more than the possible available amount of tokens will deadlock.

Implementations

impl TokenBucketRateLimiter

pub fn new(token_bucket_state: Arc<Mutex<TokenBucketState>>) -> Self

Create a new TokenBucketRateLimiter using an established TokenBucketState.

token_bucket_state can be a reference for just this rate limiter, or it can be shared across many different rate limiters.

Trait Implementations

impl Clone for TokenBucketRateLimiter

fn clone(&self) -> TokenBucketRateLimiter

Returns a copy of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for TokenBucketRateLimiter

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl VariableCostRateLimiter for TokenBucketRateLimiter

async fn wait_with_cost(&mut self, cost: usize)

Example: A Variable Cost API Rate Limit

An API enforces a rate limit by allotting 10 tokens, and replenishes used tokens at a rate of 1 per second. An endpoint being called requires 4 tokens per call.

use tokio::time::{Instant, Duration};
use async_rate_limit::limiters::VariableCostRateLimiter;
use async_rate_limit::sliding_window::SlidingWindowRateLimiter;
use async_rate_limit::token_bucket::{TokenBucketState, TokenBucketRateLimiter};
use std::sync::Arc;
use tokio::sync::Mutex;

#[tokio::main]
async fn main() -> () {
    let state = TokenBucketState::new(10, 1, Duration::from_secs(1));
    let state_mutex = Arc::new(Mutex::new(state));
    let mut limiter = TokenBucketRateLimiter::new(state_mutex);
     
    // these calls proceed immediately, using 8 tokens
    get_something(&mut limiter).await;
    get_something(&mut limiter).await;

    // this call waits ~2 seconds to acquire additional tokens before proceeding
    get_something(&mut limiter).await;
}

// note the use of the `VariableCostRateLimiter` trait, rather than the direct type
async fn get_something<T>(limiter: &mut T) where T: VariableCostRateLimiter {
    limiter.wait_with_cost(4).await;
    println!("{:?}", Instant::now());
}