llm_chain::traits

Trait Executor

Source 
pub trait Executor: Sized {
    type StepTokenizer<'a>: Tokenizer
       where Self: 'a;

    // Required methods
    fn new_with_options(options: Options) -> Result<Self, ExecutorCreationError>;
    fn execute<'life0, 'life1, 'life2, 'async_trait>(
        &'life0 self,
        options: &'life1 Options,
        prompt: &'life2 Prompt,
    ) -> Pin<Box<dyn Future<Output = Result<Output, ExecutorError>> + Send + 'async_trait>>
       where Self: 'async_trait,
             'life0: 'async_trait,
             'life1: 'async_trait,
             'life2: 'async_trait;
    fn tokens_used(
        &self,
        options: &Options,
        prompt: &Prompt,
    ) -> Result<TokenCount, PromptTokensError>;
    fn max_tokens_allowed(&self, options: &Options) -> i32;
    fn answer_prefix(&self, prompt: &Prompt) -> Option<String>;
    fn get_tokenizer(
        &self,
        options: &Options,
    ) -> Result<Self::StepTokenizer<'_>, TokenizerError>;

    // Provided method
    fn new() -> Result<Self, ExecutorCreationError> { ... }
}
Expand description

The Executor trait represents an executor that performs a single step in a chain. It takes a step, executes it, and returns the output.

Required Associated Types§

Source

type StepTokenizer<'a>: Tokenizer where Self: 'a

Required Methods§

Source

fn new_with_options(options: Options) -> Result<Self, ExecutorCreationError>

Create a new executor with the given options. If you don’t need to set any options, you can use the new method instead.

§Parameters
  • options: The options to set.
Source

fn execute<'life0, 'life1, 'life2, 'async_trait>( &'life0 self, options: &'life1 Options, prompt: &'life2 Prompt, ) -> Pin<Box<dyn Future<Output = Result<Output, ExecutorError>> + Send + 'async_trait>>
where Self: 'async_trait, 'life0: 'async_trait, 'life1: 'async_trait, 'life2: 'async_trait,

Source

fn tokens_used( &self, options: &Options, prompt: &Prompt, ) -> Result<TokenCount, PromptTokensError>

Calculates the number of tokens used by the step given a set of parameters.

The step and the parameters together are used to form full prompt, which is then tokenized and the token count is returned.

§Parameters
  • options: The per-invocation options that affect the token allowance.
  • prompt: The prompt passed into step
§Returns

A Result containing the token count, or an error if there was a problem.

Source

fn max_tokens_allowed(&self, options: &Options) -> i32

Returns the maximum number of input tokens allowed by the model used.

§Parameters
  • options: The per-invocation options that affect the token allowance.
§Returns

The max token count for the step

Source

fn answer_prefix(&self, prompt: &Prompt) -> Option<String>

Returns a possible answer prefix inserted by the model, during a certain prompt mode

§Parameters
  • prompt: The prompt passed into step
§Returns

A Option containing a String if prefix exists, or none if there is no prefix

Source

fn get_tokenizer( &self, options: &Options, ) -> Result<Self::StepTokenizer<'_>, TokenizerError>

Creates a tokenizer, depending on the model used by step.

§Parameters
  • step: The step to get an associated tokenizer for.
§Returns

A Result containing a tokenizer, or an error if there was a problem.

Provided Methods§

Source

fn new() -> Result<Self, ExecutorCreationError>

Dyn Compatibility§

This trait is not dyn compatible.

In older versions of Rust, dyn compatibility was called "object safety", so this trait is not object safe.

Implementors§