Zum Inhalt springen
Kitbay
Einreichen
Skill

bmad-eval

MITAktualisiert am 11. Nov. 202655.252GitHub-SterneEnthält Skripte

A skill from the BMAD Method that runs a skill's evaluations through the agent tool the user works in and reports the results.

Was der Skill vermittelt

A skill from the BMAD Method that runs a skill's evaluations through the agent tool the user works in and reports the results. It has four modes: whether the skill beats the bare model on the same input, whether a stripped or earlier version does as well, whether output meets a named rubric with a grader subagent, and whether the skill's description triggers on the right queries. Each case runs in a clean temporary workspace outside the project, and bundled Python scripts run cases, test triggers, convert case files and summarise repeated runs. It refuses to invent cases, labels results from fewer than three runs a quick look, and suggests retiring a skill that no longer beats the baseline.

Funktionen

  • Compare a skill against the bare model
  • Test whether a stripped version does as well
  • Grade output against a rubric
  • Check whether a skill description triggers correctly