Struct LlamaBackend

pub struct LlamaBackend {}

Representation of an initialized llama backend This is required as a parameter for most llama functions as the backend must be initialized before any llama functions are called. This type is proof of initialization.

Implementations

impl LlamaBackend

pub fn init() -> Result<LlamaBackend>

Initialize the llama backend (without numa).

Examples

let backend = LlamaBackend::init()?;
// the llama backend can only be initialized once
assert_eq!(Err(LLamaCppError::BackendAlreadyInitialized), LlamaBackend::init());

pub fn init_numa(strategy: NumaStrategy) -> Result<LlamaBackend>

Initialize the llama backend (with numa).

let llama_backend = LlamaBackend::init_numa(NumaStrategy::MIRROR)?;

pub fn void_logs(&mut self)

Change the output of llama.cpp’s logging to be voided instead of pushed to stderr.

Trait Implementations

impl Debug for LlamaBackend

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Drop for LlamaBackend

Drops the llama backend.

let backend = LlamaBackend::init()?;
drop(backend);
// can be initialized again after being dropped
let backend = LlamaBackend::init()?;

fn drop(&mut self)

Executes the destructor for this type.

impl PartialEq for LlamaBackend

fn eq(&self, other: &LlamaBackend) -> bool

Tests for self and other values to be equal, and is used by ==.

fn ne(&self, other: &Rhs) -> bool

Tests for !=. The default implementation is almost always sufficient, and should not be overridden without very good reason.

impl Eq for LlamaBackend

impl StructuralPartialEq for LlamaBackend