Struct Executor

pub struct Executor { /* private fields */ }

Executor is responsible for running the LLM and managing its context.

Trait Implementations

impl Executor for Executor

type StepTokenizer<'a> = MockTokenizer

fn new_with_options(options: Options) -> Result<Self, ExecutorCreationError>

Create a new executor with the given options. If you don’t need to set any options, you can use the new method instead.

fn execute<'life0, 'life1, 'life2, 'async_trait>( &'life0 self, options: &'life1 Options, prompt: &'life2 Prompt, ) -> Pin<Box<dyn Future<Output = Result<Output, ExecutorError>> + Send + 'async_trait>>

where Self: 'async_trait, 'life0: 'async_trait, 'life1: 'async_trait, 'life2: 'async_trait,

fn tokens_used( &self, options: &Options, prompt: &Prompt, ) -> Result<TokenCount, PromptTokensError>

Calculates the number of tokens used by the step given a set of parameters.

fn max_tokens_allowed(&self, _: &Options) -> i32

Returns the maximum number of input tokens allowed by the model used.

fn answer_prefix(&self, _prompt: &Prompt) -> Option<String>

Returns a possible answer prefix inserted by the model, during a certain prompt mode

fn get_tokenizer( &self, _: &Options, ) -> Result<Self::StepTokenizer<'_>, TokenizerError>

Creates a tokenizer, depending on the model used by step.

fn new() -> Result<Self, ExecutorCreationError>