Skip to content
Kitbay

hf-mem

Apache-2.0Updated on Nov 11, 202611,334GitHub stars

A short Hugging Face skill for estimating how much memory a model on the Hub needs for inference before downloading it.

What the skill teaches

A short Hugging Face skill for estimating how much memory a model on the Hub needs for inference before downloading it. The agent runs the hf-mem tool through uvx with a model id, and the tool reads the Safetensors or GGUF weight metadata over HTTP range requests, optionally adding an estimate for the KV cache, so no weights are fetched or loaded locally. It is meant for questions such as whether a model fits on a given GPU. It needs uv installed, and a Hugging Face token only for gated or private models.

Capabilities

  • Estimate inference memory for a Hub model
  • Include KV cache in the estimate
  • Check whether a model fits a GPU without downloading it