本文へスキップ
Kitbay
スキル

llama-cpp

MIT2026年11月11日 更新262,606GitHubスター

An optional skill in the Hermes Agent repository, credited to Orchestra Research, for running language models locally with llama.cpp and GGUF files.

スキルの内容

An optional skill in the Hermes Agent repository, credited to Orchestra Research, for running language models locally with llama.cpp and GGUF files. The agent installs llama.cpp with Homebrew, winget or a CMake build, finds GGUF repositories on the Hugging Face Hub filtered by size, lists a repository's files to pick the right quantisation for the user's memory, and builds llama-cli or llama-server commands, including the server's OpenAI-compatible endpoint. It also covers the Python bindings for chat, streaming and embeddings, and runs on CPUs, Apple Silicon and NVIDIA, AMD or Intel GPUs. Six reference files cover the server, quantisation, tuning and troubleshooting.

できること

  • Find GGUF models on the Hugging Face Hub
  • Pick a quantisation that fits available memory
  • Run llama-server with an OpenAI-compatible endpoint
  • Use the llama-cpp-python bindings