Skip to content
Kitbay
Submit
Skill

bmad-eval

MITUpdated on Nov 11, 202655,252GitHub starsIncludes scripts

A skill from the BMAD Method that runs a skill's evaluations through the agent tool the user works in and reports the results.

What the skill teaches

A skill from the BMAD Method that runs a skill's evaluations through the agent tool the user works in and reports the results. It has four modes: whether the skill beats the bare model on the same input, whether a stripped or earlier version does as well, whether output meets a named rubric with a grader subagent, and whether the skill's description triggers on the right queries. Each case runs in a clean temporary workspace outside the project, and bundled Python scripts run cases, test triggers, convert case files and summarise repeated runs. It refuses to invent cases, labels results from fewer than three runs a quick look, and suggests retiring a skill that no longer beats the baseline.

Capabilities

  • Compare a skill against the bare model
  • Test whether a stripped version does as well
  • Grade output against a rubric
  • Check whether a skill description triggers correctly