Module llm

Structs

• EmbeddingsResult
  Result of generating embeddings
• EmbeddingsUsage
  Usage related to an embeddings generation request
• InferencingParams
  Inference request parameters
• InferencingResult
  An inferencing result
• InferencingUsage
  Usage information related to the inferencing result

Enums

• Error
  The set of errors which may be raised by functions in this interface

Functions

• generate_embeddings
  Generate embeddings for the supplied list of text
• infer
  Perform inferencing using the provided model and prompt with the given optional params

Type Aliases

• EmbeddingModel
  The model used for generating embeddings
• InferencingModel
  A Large Language Model.