本文へスキップ
Kitbay
スキル公式

huggingface-community-evals

Apache-2.02026年11月11日 更新11,334GitHubスタースクリプトあり

Hugging Face's skill for evaluating Hub models on the user's own hardware.

スキルの内容

Hugging Face's skill for evaluating Hub models on the user's own hardware. It helps the agent choose between inspect-ai and lighteval, and between vLLM, Transformers and accelerate as the inference backend, then run smoke tests and full task runs with bundled uv scripts. One script evaluates a model through inference providers without a local GPU; the others drive local GPU runs. It deliberately stops at producing results: running the same evaluation on Hugging Face Jobs, editing model cards or publishing scores is handed to other skills or workflows. A Hugging Face token is needed for gated models and provider calls.

できること

  • Run inspect-ai evaluations against Hub models
  • Run lighteval with vLLM or accelerate on a local GPU
  • Pick an inference backend and fall back when one fails