Pular para o conteúdo
Kitbay
Skill

llama-cpp

MITAtualizado em 11 de nov. de 2026262.606Estrelas no GitHub

An optional skill in the Hermes Agent repository, credited to Orchestra Research, for running language models locally with llama.cpp and GGUF files.

O que a skill ensina

An optional skill in the Hermes Agent repository, credited to Orchestra Research, for running language models locally with llama.cpp and GGUF files. The agent installs llama.cpp with Homebrew, winget or a CMake build, finds GGUF repositories on the Hugging Face Hub filtered by size, lists a repository's files to pick the right quantisation for the user's memory, and builds llama-cli or llama-server commands, including the server's OpenAI-compatible endpoint. It also covers the Python bindings for chat, streaming and embeddings, and runs on CPUs, Apple Silicon and NVIDIA, AMD or Intel GPUs. Six reference files cover the server, quantisation, tuning and troubleshooting.

O que faz

  • Find GGUF models on the Hugging Face Hub
  • Pick a quantisation that fits available memory
  • Run llama-server with an OpenAI-compatible endpoint
  • Use the llama-cpp-python bindings