agent-observability-trace-rca
A Datadog skill for finding out why an LLM application is failing in production.
What the skill teaches
A Datadog skill for finding out why an LLM application is failing in production. Depending on the data available, the agent starts from failing evaluation verdicts, runtime errors or structural oddities in LLM Observability traces, collects the problem spans, groups them and follows each span tree from the symptom back to its cause, then reports the failure patterns it found. Its output can be passed to the eval bootstrap skill. It works through the Datadog MCP server or the pup CLI. The folder also contains evaluation data and a judge harness the maintainers use to test the skill.
Capabilities
- Find failing spans from eval verdicts or runtime errors
- Group failures by error type or pattern
- Trace a failure from symptom to root cause in the span tree
- Report failure patterns for evaluator generation
About
- Publisher
- Datadog
- Tool
- Datadog
Install
What's inside
- evals/__init__.py
- evals/data/llm_obs_trace_rca_v1.json
- evals/evaluator.py
- evals/example_config.json
- evals/executor.py
- evals/prompts/judge_rubric.txt
MCP servers and skills can run code on your computer and act on your data. Review the source before installing.
Details
| Fact | Value | Source |
|---|---|---|
| Type | Skill | |
| Tool | Datadog | |
| Publisher | Datadog | source |
| Official status | Officialdatadog-labs org; MIT licence copyright Datadog | source |
| Needs an API key or sign-in | Not stated in its documentation | |
| Includes scripts | No | source |
| Platforms | Not stated in its documentation | |
| AI clients | Not stated in its documentation | |
| Licence | MIT | source |
| Pricing | Not stated in its documentation | |
| Activity | Last commit Nov 9, 2026185 GitHub starsDownload data not available (not on npm or PyPI) | source |
| Links | Repository |
More from Datadog
2 servers · 70 skills
- agent-observability-auto-experiment (Skill)
- agent-observability-build-eval-from-annotations (Skill)
- agent-observability-eval-bootstrap (Skill)
- agent-observability-eval-pipeline (Skill)
- agent-observability-experiment-analyzer (Skill)
- agent-observability-experiment-bootstrap (Skill)
Sources
- Skill file (SKILL.md)github.com/datadog-labs/agent-skillsNov 10, 2026
- README (README.md)github.com/datadog-labs/agent-skillsNov 13, 2026
- Licence file (LICENSE)github.com/datadog-labs/agent-skills · Backs: Licence, Official statusNov 13, 2026
- GitHub repositorygithub.com/datadog-labs/agent-skills · Backs: Publisher, Activity, LicenceNov 11, 2026
- Folder in the repository (agent-observability-trace-rca)github.com/datadog-labs/agent-skills · Backs: Includes scriptsNov 10, 2026
Other MCP servers and skills for Datadog
- agent-observability-auto-experiment(Skill)
- agent-observability-build-eval-from-annotations(Skill)
- agent-observability-eval-bootstrap(Skill)
- agent-observability-eval-pipeline(Skill)
- agent-observability-experiment-analyzer(Skill)
- agent-observability-experiment-bootstrap(Skill)
- agent-observability-online-experiment(Skill)
- datadog-app(Skill)
- View all →