Skip to content
Kitbay
Skill

apify-blocked-scrape-triage

Apache-2.0Updated on Nov 6, 2026294GitHub stars

A community skill in Apify's awesome-skills collection for working out why a scrape came back blocked, throttled, empty or partial before paying for a heavier Actor.

What the skill teaches

A community skill in Apify's awesome-skills collection for working out why a scrape came back blocked, throttled, empty or partial before paying for a heavier Actor. It has the agent first check its own egress address and DNS, then climb a ladder in order of cost: a cheaper published data source, whether the host answers at all, how fast and from how many addresses requests arrive, how closely they resemble a real browser, session tokens, the site's internal API, and only then a browser. It caps diagnostic traffic per host, changes one variable at a time, and notes that the Cheerio scraper used for test runs needs a one-time full-permission approval in Apify Console.

Capabilities

  • Diagnose 403, 429 and empty-result scrapes
  • Check the agent's own egress IP and DNS first
  • Escalate fixes in order of cost before using a browser