creating-mcp-agent-evals
A maintainer skill in Apify's MCP server repository for building Langfuse evaluation suites that test how well agents use one family of the server's tools, such as storage or runs.
What the skill teaches
A maintainer skill in Apify's MCP server repository for building Langfuse evaluation suites that test how well agents use one family of the server's tools, such as storage or runs. It sets out a process: list every tool and argument group, check real platform behaviour with small scripts before writing cases, write easy, medium and hard cases in rounds, calibrate on a strong model and then test weaker ones, and fix tool outputs before descriptions. A diagnosis table helps decide whether a failing case, the judge, the product, the tool text or the model is at fault. A reference file holds commands and patterns.
Capabilities
- Plan eval cases for an Apify MCP tool family
- Calibrate evals across stronger and weaker models
- Diagnose why an eval case fails
- Improve tool outputs based on eval results
About
- Publisher
- Apify
- Tool
- Apify
Install
The project doesn't document install steps that we could confirm. See the project's own instructions
What's inside
- reference.md
MCP servers and skills can run code on your computer and act on your data. Review the source before installing.
Details
| Fact | Value | Source |
|---|---|---|
| Type | Skill | |
| Tool | Apify | |
| Publisher | Apify | source |
| Official status | Officialin Apify's own GitHub organisation, inside the official Apify MCP server repository | source |
| Needs an API key or sign-in | Not stated in its documentation | |
| Includes scripts | No | source |
| Platforms | Not stated in its documentation | |
| AI clients | Not stated in its documentation | |
| Licence | MIT | source |
| Pricing | Not stated in its documentation | |
| Activity | Last commit Nov 10, 202615,564 GitHub starsDownload data not available (not on npm or PyPI) | source |
| Links | Repository Website |
Sources
- Skill file (SKILL.md)github.com/apify/apify-mcp-serverNov 10, 2026
- Licence file (LICENSE)github.com/apify/apify-mcp-server · Backs: Licence, PublisherNov 13, 2026
- GitHub repositorygithub.com/apify/apify-mcp-server · Backs: Official status, Activity, LicenceNov 10, 2026
- Folder in the repository (creating-mcp-agent-evals)github.com/apify/apify-mcp-server · Backs: Includes scriptsNov 10, 2026