AI Crawler Log Analyzer
Paste lines from your server or CDN access log and see every AI crawler that visited, how often, and the URLs each one fetched most. Everything runs in your browser; nothing is uploaded.
- Instant·Runs in your browser
- No signup
- Free
Access log
Apache, Nginx, Cloudflare or Vercel lines. Nothing leaves your browser.
0 lines pasted
Paste log lines to begin
You'll see every AI crawler that visited, its share of requests, and the pages it fetched most.
Next step
Tools that pair with this one
- Live checkAI Search Readiness CheckScan any site for the 12 things AI engines need to cite it.AI crawlersOpen
- Instantllms.txt GeneratorWrite the llms.txt file AI assistants read first.AI crawlersOpen
- InstantAI Crawler robots.txt GeneratorAllow search bots, decide on training bots, one file.AI crawlersOpen
4 steps
How to use it
- 1Export or copy recent lines from your access log (Apache, Nginx, Cloudflare, Vercel all work).
- 2Paste them into the box — the analyzer only needs the user agent and the request path.
- 3Read the per-bot breakdown: hits, share, and the pages each bot fetched most.
- 4Watch the user-triggered bots: each hit is a live conversation that needed your page.
FAQ
Frequently asked
Any line that contains a request path and a user agent string: Apache combined, Nginx default, Cloudflare Logpush lines and most CDN exports. It matches bots by their user-agent tokens, so column order doesn’t matter.
No. Parsing happens in your browser tab and nothing leaves it. Reload the page and it’s gone.
Yes, easily. Treat this as a first pass. To verify a hit, check the source IP against the vendor’s published ranges (openai.com/gptbot.json, claude.com/crawling/bots.json, perplexity.com/perplexitybot.json).
Keep exploring
Serve every crawler a page it can read
- FeatureCitation TrackingSee where AI answers quote and recommend your brand.
- SolutionAI search visibilityGet named in AI answers by building what engines cite, every day.
- Use caseRankbox for SEO agenciesAdd GEO to every retainer without adding writers.
- AI SEO guideGoogle AI Overviews & AI Mode SEO guideWin the links inside Google's AI answers
- PlaybookDynamic Rendering & Prerendering for JavaScript-Heavy AI CrawlersWhat crawlers that don't run JavaScript see, and the fix for each framework.
- PlaybookDynamic Rendering SEO: What Google Says Now and Safer AlternativesGoogle's current position on dynamic rendering, and safer alternatives.
Want this on autopilot?
Enter your site to see how often ChatGPT, Perplexity, Gemini, and Google cite your brand, and exactly what to publish next.
7-day free trial · Card required · Cancel anytime