hillclimb-skill
A skill from Caliper, a tool for evaluating agent skills, that improves an existing SKILL.md against its evaluation one change per round. It needs an evaluation with at least eight tasks.
What the skill teaches
A skill from Caliper, a tool for evaluating agent skills, that improves an existing SKILL.md against its evaluation one change per round. It needs an evaluation with at least eight tasks. After asking for the goal (a higher score or fewer tokens), the part to change, the smallest gain worth keeping and a round budget, it runs a baseline twice to measure noise and holds out a third of the tasks. Each round reads failures from the remaining tasks, makes the smallest general fix and keeps it only if the held-out tasks improve too. It finishes by restoring the highest-scoring version on held-out tasks, comparing it with the baseline, and leaving the edits uncommitted.
Capabilities
- Measure an evaluation's noise floor
- Split tasks into training and held-out sets
- Make and test one skill change per round
- Report held-out results against the baseline
About
- Publisher
- Emrick Donadei
- Tool
- Testing
Works with
Install
- Run this to add it to Any client:
npx skills@latest add edonadei/caliper
What's inside
- REFERENCE.md
- climb.py
- hillclimb-skill.eval.yaml
MCP servers and skills can run code on your computer and act on your data. Review the source before installing.
This skill includes scripts that your agent may run.
Details
| Fact | Value | Source |
|---|---|---|
| Type | Skill | |
| Tool | Testing | |
| Publisher | Emrick Donadei | source |
| Official status | Not stated in its documentation | |
| Needs an API key or sign-in | Not stated in its documentation | |
| Includes scripts | Yes | source |
| Platforms | Not stated in its documentation | |
| AI clients | Claude Code, Codex | source |
| Licence | MIT | source |
| Pricing | Not stated in its documentation | |
| Activity | Last commit Nov 10, 2026230 GitHub starsDownload data not available (not on npm or PyPI) | source |
| Links | Repository Website |
Also listed under: Testing
Sources
- GitHub repositorygithub.com/edonadei/caliper · Backs: AI clients, Licence, Publisher, ActivityNov 11, 2026
- Skill file (SKILL.md)github.com/edonadei/caliperNov 15, 2026
- Folder in the repository (hillclimb-skill)github.com/edonadei/caliper · Backs: Includes scriptsNov 15, 2026