spin_sdk::llm

Function infer_with_options

Source
pub fn infer_with_options(
    model: InferencingModel<'_>,
    prompt: &str,
    options: InferencingParams,
) -> Result<InferencingResult, Error>
Expand description

Perform inferencing using the provided model, prompt, and options