Zum Inhalt springen
Kitbay
Skill

ai-crawler-audit

MITAktualisiert am 9. Nov. 2026606GitHub-SterneEnthält Skripte

A skill from the geo-doctor toolkit that checks whether AI crawlers such as GPTBot and ClaudeBot can fetch a given page and read its main text.

Was der Skill vermittelt

A skill from the geo-doctor toolkit that checks whether AI crawlers such as GPTBot and ClaudeBot can fetch a given page and read its main text. A bundled Node.js script with no dependencies requests the page, reads robots.txt, probes with AI crawler user agents and looks for noindex tags, firewall blocks, empty bodies and pages that only render with JavaScript, then scores the result from 0 to 100, capping it at 60 when a critical check fails. It can write a Markdown report, a badge or a share card and can fail a CI run below a threshold. The skill stresses that the score measures readability only, not ranking, and that untested items are marked unverified.

Funktionen

  • Check robots.txt rules for AI crawlers
  • Detect noindex, firewall blocks and JavaScript-only pages
  • Score a page's AI readability from 0 to 100
  • Write a Markdown audit report