Zum Inhalt springen
Kitbay

agent-observability-replay-trace

MITAktualisiert am 9. Nov. 2026185GitHub-SterneEnthält Skripte

A Datadog skill for reworking one LLM Observability trace whose output was wrong.

Was der Skill vermittelt

A Datadog skill for reworking one LLM Observability trace whose output was wrong. You give it a trace ID and, optionally, a change to try; the agent fetches the trace through the Datadog MCP server or the pup CLI, finds the entry point in your local code, edits the code if asked, re-runs the application so it emits a new trace, and shows a short diff of old and new output with links to both. You repeat until satisfied. Python apps traced with ddtrace are the main target, and a Python runner template is included for the agent to copy and fill in.

Funktionen

  • Fetch a single LLM Observability trace
  • Re-run the traced request against local code
  • Diff the old and new outputs with links to both traces
  • Loop through edit, replay and compare