Struct LlamaTokenDataArray

pub struct LlamaTokenDataArray {
    pub data: Vec<LlamaTokenData>,
    pub selected: Option<usize>,
    pub sorted: bool,
}

a safe wrapper around llama_token_data_array.

Fields

data: Vec<LlamaTokenData>

the underlying data

selected: Option<usize>

the index of the selected token in data

sorted: bool

is the data sorted?

Implementations

impl LlamaTokenDataArray

pub fn new(data: Vec<LlamaTokenData>, sorted: bool) -> Self

Create a new LlamaTokenDataArray from a vector and whether or not the data is sorted.

let array = LlamaTokenDataArray::new(vec![
        LlamaTokenData::new(LlamaToken(0), 0.0, 0.0),
        LlamaTokenData::new(LlamaToken(1), 0.1, 0.1)
   ], false);
assert_eq!(array.data.len(), 2);
assert_eq!(array.sorted, false);

pub fn from_iter<T>(data: T, sorted: bool) -> LlamaTokenDataArray

where T: IntoIterator<Item = LlamaTokenData>,

Create a new LlamaTokenDataArray from an iterator and whether or not the data is sorted.

let array = LlamaTokenDataArray::from_iter([
    LlamaTokenData::new(LlamaToken(0), 0.0, 0.0),
    LlamaTokenData::new(LlamaToken(1), 0.1, 0.1)
], false);
assert_eq!(array.data.len(), 2);
assert_eq!(array.sorted, false);

pub fn selected_token(&self) -> Option<LlamaToken>

Returns the current selected token, if one exists.

impl LlamaTokenDataArray

pub fn apply_sampler(&mut self, sampler: &LlamaSampler)

Modifies the data array by applying a sampler to it

pub fn with_sampler(self, sampler: &mut LlamaSampler) -> Self

Modifies the data array by applying a sampler to it

pub fn sample_token(&mut self, seed: u32) -> LlamaToken

Randomly selects a token from the candidates based on their probabilities.

Panics

If the internal llama.cpp sampler fails to select a token.

pub fn sample_token_greedy(&mut self) -> LlamaToken

Selects the token with the highest probability.

Panics

If the internal llama.cpp sampler fails to select a token.

Trait Implementations

impl Clone for LlamaTokenDataArray

fn clone(&self) -> LlamaTokenDataArray

Returns a copy of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for LlamaTokenDataArray

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl PartialEq for LlamaTokenDataArray

fn eq(&self, other: &LlamaTokenDataArray) -> bool

Tests for self and other values to be equal, and is used by ==.

fn ne(&self, other: &Rhs) -> bool

Tests for !=. The default implementation is almost always sufficient, and should not be overridden without very good reason.

impl StructuralPartialEq for LlamaTokenDataArray