llama-cpp-sys
Raw bindings to llama.cpp with cuda support, including new Sampler API from llama-cpp.
See llama-cpp-4 for a safe API.
OpenMPI support
brew install open-mpi
cargo build -F mpi
Raw bindings to llama.cpp with cuda support, including new Sampler API from llama-cpp.
See llama-cpp-4 for a safe API.
brew install open-mpi
cargo build -F mpi