Skip to content
Kitbay

huggingface-local-models

Apache-2.0Updated on Nov 11, 202611,334GitHub stars

A Hugging Face skill for running models on your own machine with llama.cpp.

What the skill teaches

A Hugging Face skill for running models on your own machine with llama.cpp. The agent searches the Hub for repositories that ship GGUF files, opens a model's llama.cpp local-app page for the suggested command and quantisation, confirms exact file names through the Hub API, and starts the model with llama-cli or llama-server, which can serve an OpenAI-compatible endpoint. It covers CPU, Apple Metal, CUDA and ROCm setups, choosing a quantisation for the available memory, and converting Transformers weights to GGUF when no GGUF exists. Gated models need a Hugging Face login.

Capabilities

  • Find GGUF model repositories on the Hub
  • Choose a quantisation for the available hardware
  • Launch models with llama-cli or llama-server
  • Convert Transformers weights to GGUF