Zum Inhalt springen
Kitbay

huggingface-local-models

Apache-2.0Aktualisiert am 11. Nov. 202611.334GitHub-Sterne

A Hugging Face skill for running models on your own machine with llama.cpp.

Was der Skill vermittelt

A Hugging Face skill for running models on your own machine with llama.cpp. The agent searches the Hub for repositories that ship GGUF files, opens a model's llama.cpp local-app page for the suggested command and quantisation, confirms exact file names through the Hub API, and starts the model with llama-cli or llama-server, which can serve an OpenAI-compatible endpoint. It covers CPU, Apple Metal, CUDA and ROCm setups, choosing a quantisation for the available memory, and converting Transformers weights to GGUF when no GGUF exists. Gated models need a Hugging Face login.

Funktionen

  • Find GGUF model repositories on the Hub
  • Choose a quantisation for the available hardware
  • Launch models with llama-cli or llama-server
  • Convert Transformers weights to GGUF