launchdarkly-online-evals
A partner-maintained skill for AWS DevOps Agent, from LaunchDarkly, for automatic quality scoring of AI config output in LaunchDarkly using judges, where one model rates another.
What the skill teaches
A partner-maintained skill for AWS DevOps Agent, from LaunchDarkly, for automatic quality scoring of AI config output in LaunchDarkly using judges, where one model rates another. It covers the built-in judges, creating custom judges, attaching them to config variations, setting sampling rates and watching quality scores, and notes that judges work only in completion mode. It needs the hosted LaunchDarkly MCP server, and SDK examples use the Python and Node.js AI SDKs.
Capabilities
- Attach judges to AI config variations
- Create custom judges
- Set evaluation sampling rates
- Monitor quality scores
About
- Publisher
- Amazon Web Services
Install
What's inside
- CHANGELOG.md
- README.md
- evals/exemptions.json
MCP servers and skills can run code on your computer and act on your data. Review the source before installing.
Details
| Fact | Value | Source |
|---|---|---|
| Type | Skill | |
| Tool | Amazon Web Services | |
| Publisher | Amazon Web Services | source |
| Official status | Officialin the aws GitHub organisation; README presents it as tools for AWS DevOps Agent and links AWS docs; this skill is maintained by a partner (tier partner-managed) | source |
| Needs an API key or sign-in | Not stated in its documentation | |
| Includes scripts | No | source |
| Platforms | Not stated in its documentation | |
| AI clients | Not stated in its documentation | |
| Licence | Apache-2.0 | source |
| Pricing | Not stated in its documentation | |
| Activity | Last commit Nov 11, 2026138 GitHub starsDownload data not available (not on npm or PyPI) | source |
| Links | Repository Website |
More from Amazon Web Services
4 servers · 169 skills
- analyzing-release-readiness (Skill)
- Amazon ECS MCP Server (MCP server)
- Agent Toolkit for AWS (MCP server)
- chatting-with-aws-devops-agent (Skill)
- coordinating-multi-space-devops-agent (Skill)
- diff-scanning-with-aws-security-agent (Skill)
Sources
- Skill file (SKILL.md)github.com/aws/tools-for-devops-agentNov 14, 2026
- File in the repository (getting-started.md)github.com/aws/tools-for-devops-agentNov 13, 2026
- Licence file (LICENSE)github.com/aws/tools-for-devops-agent · Backs: LicenceNov 13, 2026
- README (README.md)github.com/aws/tools-for-devops-agent · Backs: Official statusNov 13, 2026
- File in the repository (NOTICE)github.com/aws/tools-for-devops-agent · Backs: PublisherNov 13, 2026
- Folder in the repository (launchdarkly-online-evals)github.com/aws/tools-for-devops-agent · Backs: Includes scriptsNov 14, 2026
- GitHub repositorygithub.com/aws/tools-for-devops-agent · Backs: Activity, LicenceNov 11, 2026
Other MCP servers and skills for Amazon Web Services
- AWS Documentation MCP Server(MCP server)
- awslabs.amazon-mq-mcp-server(MCP server)
- awslabs.amazon-sns-sqs-mcp-server(MCP server)
- awslabs.amazon-translate-mcp-server(MCP server)
- awslabs.aws-iac-mcp-server(MCP server)
- awslabs.aws-location-mcp-server(MCP server)
- awslabs.aws-pricing-mcp-server(MCP server)
- analyzing-release-readiness(Skill)
- View all →