Zum Inhalt springen
Kitbay

agent-observability-session-classify

MITAktualisiert am 9. Nov. 2026185GitHub-Sterne

A Datadog skill that judges whether a user got what they wanted from an AI assistant interaction recorded in Datadog.

Was der Skill vermittelt

A Datadog skill that judges whether a user got what they wanted from an AI assistant interaction recorded in Datadog. It works on a single assistant session (adding Real User Monitoring and audit-trail signals), a single Agent Observability trace, or a sample of sessions or traces from one LLM application. The agent reads the conversation and tool results and returns a short verdict with a one-line reason by default. Its output can feed the trace root-cause and eval pipeline skills. It uses the Datadog MCP server or the pup CLI.

Funktionen

  • Classify a single session or trace as satisfied or not
  • Combine trace content with RUM behaviour signals
  • Sample and classify many sessions from one LLM app
  • Produce labels for root-cause analysis and evaluation