Zum Inhalt springen
Kitbay
Skill

llama-cpp

MITAktualisiert am 11. Nov. 2026262.606GitHub-Sterne

An optional skill in the Hermes Agent repository, credited to Orchestra Research, for running language models locally with llama.cpp and GGUF files.

Was der Skill vermittelt

An optional skill in the Hermes Agent repository, credited to Orchestra Research, for running language models locally with llama.cpp and GGUF files. The agent installs llama.cpp with Homebrew, winget or a CMake build, finds GGUF repositories on the Hugging Face Hub filtered by size, lists a repository's files to pick the right quantisation for the user's memory, and builds llama-cli or llama-server commands, including the server's OpenAI-compatible endpoint. It also covers the Python bindings for chat, streaming and embeddings, and runs on CPUs, Apple Silicon and NVIDIA, AMD or Intel GPUs. Six reference files cover the server, quantisation, tuning and troubleshooting.

Funktionen

  • Find GGUF models on the Hugging Face Hub
  • Pick a quantisation that fits available memory
  • Run llama-server with an OpenAI-compatible endpoint
  • Use the llama-cpp-python bindings