If a crawler you care about shows Blocked, the rule beside it is the robots.txt line to change. Your web designer can fix it in minutes.
AI Crawler Check
Enter your website. We read its robots.txt and show, crawler by crawler, whether ChatGPT, Claude, Perplexity, Google and others are allowed to read your home page.
Enter a website to see each AI crawler, what it does, and whether it can read the home page. Nothing is stored.
What to do with the result.
Training crawlers (GPTBot, ClaudeBot, CCBot) and search crawlers (OAI-SearchBot, PerplexityBot, Claude-SearchBot) are separate. You can block training and still allow search.
Allowed is necessary, not enough. AI still needs clear pages and outside sources before it names you.
Questions about this tool.
A small text file at the root of your website that tells crawlers which pages they may read. Most sites have one; many were written before AI crawlers existed.
That’s your call. Blocking training crawlers like GPTBot does not stop ChatGPT search from reading you, because that uses OAI-SearchBot. Blocking the search crawlers does keep you out of those answers.
Want it done for you?
These tools check the basics. We do the full job: fixes, pages, listings and reviews, checked every month with the same questions.
The score is free, no account needed. The call is 30 minutes.