Zum Inhalt springen
Kitbay

hf-mem

Apache-2.0Aktualisiert am 11. Nov. 202611.334GitHub-Sterne

A short Hugging Face skill for estimating how much memory a model on the Hub needs for inference before downloading it.

Was der Skill vermittelt

A short Hugging Face skill for estimating how much memory a model on the Hub needs for inference before downloading it. The agent runs the hf-mem tool through uvx with a model id, and the tool reads the Safetensors or GGUF weight metadata over HTTP range requests, optionally adding an estimate for the KV cache, so no weights are fetched or loaded locally. It is meant for questions such as whether a model fits on a given GPU. It needs uv installed, and a Hugging Face token only for gated or private models.

Funktionen

  • Estimate inference memory for a Hub model
  • Include KV cache in the estimate
  • Check whether a model fits a GPU without downloading it