huggingface-community-evals
Hugging Face's skill for evaluating Hub models on the user's own hardware.
What the skill teaches
Hugging Face's skill for evaluating Hub models on the user's own hardware. It helps the agent choose between inspect-ai and lighteval, and between vLLM, Transformers and accelerate as the inference backend, then run smoke tests and full task runs with bundled uv scripts. One script evaluates a model through inference providers without a local GPU; the others drive local GPU runs. It deliberately stops at producing results: running the same evaluation on Hugging Face Jobs, editing model cards or publishing scores is handed to other skills or workflows. A Hugging Face token is needed for gated models and provider calls.
Capabilities
- Run inspect-ai evaluations against Hub models
- Run lighteval with vLLM or accelerate on a local GPU
- Pick an inference backend and fall back when one fails
About
- Publisher
- Hugging Face
- Tool
- Hugging Face
Install
- Run this to add it to Any client:
hf skills add huggingface-community-evals --global
What's inside
- examples/.env.example
- examples/USAGE_EXAMPLES.md
- scripts/inspect_eval_uv.py
- scripts/inspect_vllm_uv.py
- scripts/lighteval_vllm_uv.py
MCP servers and skills can run code on your computer and act on your data. Review the source before installing.
This skill includes scripts that your agent may run.
Details
| Fact | Value | Source |
|---|---|---|
| Type | Skill | |
| Tool | Hugging Face | |
| Publisher | Hugging Face | source |
| Official status | OfficialHugging Face Hub docs link to the huggingface/skills repository | source |
| Needs an API key or sign-in | Not stated in its documentation | |
| Includes scripts | Yes | source |
| Platforms | Not stated in its documentation | |
| AI clients | Not stated in its documentation | |
| Licence | Apache-2.0 | source |
| Pricing | Not stated in its documentation | |
| Activity | Last commit Nov 11, 202611,334 GitHub starsDownload data not available (not on npm or PyPI) | source |
| Links | Repository Website |
Other copies of this skill (3)
These differ from this skill's text. They may be adapted copies or older versions.
- huggingface-community-evalswaybarrios/opencode-power-pack
- huggingface-community-evalspracticalswan/agent-skills
- huggingface-community-evalsbg-szy/top-skills
More from Hugging Face
1 server · 26 skills
- hf-mcp (Skill)
- hf-cli (Skill)
- hf-mem (Skill)
- huggingface-best (Skill)
- huggingface-datasets (Skill)
- hf-cloud-python-env-setup (Skill)
Badge for your README
Maintain this project? Add a badge that links to this page.
[](https://kitbay.app/skills/huggingface-huggingface-community-evals/)
Sources
- Skill file (SKILL.md)github.com/huggingface/skillsNov 10, 2026
- Web pagehuggingface.co · Backs: Official statusNov 12, 2026
- Licence file (LICENSE)github.com/huggingface/skills · Backs: LicenceNov 12, 2026
- GitHub repositorygithub.com/huggingface/skills · Backs: Publisher, Activity, LicenceNov 11, 2026
- Folder in the repository (huggingface-community-evals)github.com/huggingface/skills · Backs: Includes scriptsNov 10, 2026