AI Crawler
Find out if the assistant ever read your page
Being cited proves an assistant read you. Silence does not say whether it never came, or came and chose someone else, and those need opposite fixes. AI Crawler reads your own server and CDN records to tell them apart.
Free 7-day trial. No credit card required.
Fetches
1,087
1 Oct to 14 Oct
Assistants
6
Most: ChatGPT
Pages read
7
at least once
Missing pages
1
still asked for
Fetches a day, by assistant
ChatGPTClaudePerplexityGemini- /pricing6 assistants272
- /docs/getting-started6 assistants218
- /blog/how-it-works6 assistants164
- /features6 assistants163
- /about6 assistants111
The calls it helps you make, and the move after
The questions no answer-tracking tool can settle, because the answer is in your own records.
- 01
Do the AI assistants fetch our site at all?
Every fetch, named the way you know the assistant, across the 13 assistant families SKYA recognises.
In SKYA: AI Crawler
Open the record - 02
Which pages do they read, and which do they skip?
Pages ranked by fetches over 7, 14 and 30 days, with how many assistants read each one.
In SKYA: AI Crawler
Strengthen the skipped pages - 03
What are they asking for that no longer exists?
Fetches that hit a missing page, so a deleted address the assistants still want becomes a redirect or a restored page.
In SKYA: Missing pages
Fix the dead links - 04
Does any of it turn into visits?
The visits AI assistants send, counted by a short snippet, beside the crawler readings.
In SKYA: AI Referral Traffic
Compare fetches and visits
What the assistants keep asking for
When an assistant keeps fetching a page you deleted or moved, it is still reaching for something it learned to trust. AI Crawler lists those addresses with who asked and how often, so each one becomes a redirect or a restored page.
- Every fetch that hit a missing page, by address
- The assistant behind it, named the way you know it
- A fix beside each one
Addresses the assistants keep fetching that now answer with nothing.
- /blog/2024-launchMissing
41 fetches in 30 days
Redirect to /blog/launch
- /docs/v1/setupMissing
23 fetches in 30 days
Restore or redirect to /docs/setup
- /pricing-oldMissing
9 fetches in 30 days
Redirect to /pricing
Three steps, start to finish
Step 1: Connect a route
Cloudflare brings your history on day one. The SKYA Helper on WordPress, a webhook, a server snippet or a log upload work everywhere else.
Step 2: SKYA names every fetch
Each crawler is mapped to the assistant behind it, so the record reads ChatGPT, Claude or Perplexity rather than a user agent.
Step 3: Fix what they miss
Pages that are never fetched, pages that answer with an error and paths the assistants keep asking for, each with its fix.
Everything in it, shipping today
Read from your own records, because a page script cannot see a crawler.
Cloudflare, with history
Connect Cloudflare and the first screen has real numbers, not a chart that fills up next week.
WordPress in one install
The SKYA Helper plugin records crawler visits on your own server.
Webhook and server snippets
Send visits from a log drain, an edge network or a few lines on your own server.
Log upload
Upload a log file of up to 250 MB, CDN logs included, and read the visits in it.
Missing pages
The addresses assistants keep asking for that now return nothing, ready for a redirect.
AI referral visits
People who arrive from an AI assistant, counted beside the crawlers that read the page first.
Figures you can check
A day with no reading is not a day with no visits. Where a source has not reported yet, the figure is withheld rather than printed as zero.
How every number is calculated- assistant families named
- 13Every known AI crawler is mapped to the assistant behind it.
- day window, by page and by assistant
- 30Or 7 and 14 days.
- on every plan
- FreeCrawler tracking is not an upgrade.
Which plans include it
AI Crawler is on every plan, with every free route.
Included on every plan, with every free route
- Starter$59/mo
- Brand Pro$199/mo
- Agency$239/mo
- Agency Pro$649/mo
SKYA Edge
SKYA Edge records every AI crawler visit at the door of your site, whoever hosts it. Your pages pass through unchanged. Going live is one DNS record, and deleting that record undoes it within five minutes.
$15 a project a month, any plan
Tracking itself stays free through Cloudflare or WordPress. SKYA Edge is the route for a site that can use neither.
Compare every planWhat people ask first
See it on your own domain
Connect a route and see which assistants read you this month. Free 7-day trial. No credit card required.