AI Source Checklist: Get Picked by Perplexity, ChatGPT and Google AI

If you only have 15 minutes

1. Open yourdomain.com/robots.txt (3 min)

   Search the page for OAI-SearchBot, PerplexityBot, Claude-SearchBot, Googlebot, Bingbot. Any 'Disallow: /' under them? Remove it, or paste the block from the robots.txt section.

2. Check your CDN (3 min)

   In Cloudflare or your host, look for an 'AI bots' or 'bot fight' setting. Make sure search crawlers aren't blocked.

3. Ask one buyer question in all three tools (6 min)

   Write down who gets cited. Note any URL cited by two or more.

4. Check Search Console (3 min)

   Open the generative AI performance report. Is your main page getting AI impressions? Write the number down.

Bonus: Claude and Microsoft Copilot

Access checks (do these first)

• robots.txt allows Googlebot and Bingbot.

  CONFIRMED for Google: AI features only use indexed pages. Bing is needed for Microsoft's AI answers, and ChatGPT search is reported to use Bing as one provider (LIKELY).

• robots.txt allows OAI-SearchBot.

  CONFIRMED (OpenAI crawler docs): sites that opt out of OAI-SearchBot won't be shown in ChatGPT search answers, though they can still appear as navigational links.

• robots.txt allows PerplexityBot, and your firewall/CDN doesn't block its published IP ranges.

  CONFIRMED (Perplexity crawler docs): allow PerplexityBot in robots.txt and its published IP ranges.

• Decide separately on GPTBot and Google-Extended.

  These control training (and for Google-Extended, Gemini grounding). Google says Google-Extended doesn't affect inclusion in Google Search.

• Your CDN or bot-protection doesn't challenge AI crawlers by default.

  Google lists 'crawling is allowed in robots.txt, and by any CDN or hosting infrastructure'. Check your Cloudflare or host bot settings.

• Key pages are indexed in Google Search Console and Bing Webmaster Tools.

  Use URL Inspection. If it isn't indexed, it can't be a Google AI source.

• Important content is in the HTML text, not only in images, PDFs or behind a login.

  Google: make sure important content is available in textual form.

• No nosnippet or max-snippet:0 on pages you want quoted.

  Google uses these controls to limit what is shown, including in AI features.

• robots.txt allows Claude-SearchBot.

  CONFIRMED (Anthropic support): blocking it may reduce your visibility in Claude's search results.

Page checks (what cited pages tend to share)

These are not official ranking factors for any of the three tools. Each is tagged. Use them as the patterns to check when you study which pages get cited.

• The page answers one buyer question, and the heading says that question.

  LIKELY. A clear match between question and page makes the page easy to pick for that query.

• The first sentence under the heading is the direct answer.

  LIKELY. Easy to lift. Add detail after it.

• Facts are specific: prices, numbers, dates, places, names.

  STUDY. The GEO paper (Aggarwal et al., KDD 2024) found adding statistics raised visibility in AI answers in its benchmark, by up to 40% for the best methods. Lab result, not a promise.

• Claims link to their source.

  STUDY. Citing sources was one of the strongest methods in the GEO paper. It also helps people trust you.

• Author or business name is visible, with an About page that says who you are and since when.

  CONFIRMED as Google guidance ('Is it self-evident who authored your content?'). Effect on AI citation: UNKNOWN.

• Updated date is shown and true.

  LIKELY. AI answers about prices and 'best X' are time-sensitive.

• Structured data matches what's visible on the page.

  Google lists this in its AI features best practices. Schema doesn't replace visible text.

• Internal links point to the page from your menu or related pages.

  Google: make content findable through internal links.

• Comparison and pricing questions have their own pages.

  Buyers ask AI these constantly. A page that answers them directly is a candidate source.

Copy-paste robots.txt block

# Search and AI answer crawlers: allow
User-agent: Googlebot
Allow: /

User-agent: Bingbot
Allow: /

User-agent: OAI-SearchBot
Allow: /

User-agent: PerplexityBot
Allow: /

User-agent: Claude-SearchBot
Allow: /

# Training crawlers: your choice. Delete these lines to allow training.
User-agent: GPTBot
Disallow: /

User-agent: ClaudeBot
Disallow: /

User-agent: Google-Extended
Disallow: /

Sitemap: https://yourdomain.com/sitemap.xml

Off-site checks

• Google Business Profile is complete, verified and has recent reviews with replies.

  Google says complete info and reviews help local results, and lists 'Business Profile information is up-to-date' in its AI features best practices.

• You appear on the pages that rank for 'best [category]' in your market.

  When AI tools search, those list pages are often what they read. Ask for a fair listing or a correction.

• Your brand facts match everywhere: site, profiles, directories.

  Conflicting facts give AI tools conflicting answers.

• Buyers discuss you on forums like Reddit, honestly.

  Google and OpenAI both have Reddit data partnerships. See the Reddit playbook for the safe way.

The side-by-side test (monthly, 20 minutes)

1. Pick 5 buyer questions

   Questions a buyer asks before paying, e.g. 'best [category] in [city]', '[your product] vs [competitor]', 'how much does [service] cost'.

2. Run each in all three tools

   Perplexity, ChatGPT with search on, and Google (look at the AI Overview or open AI Mode). Use fresh, logged-out sessions.

3. Screenshot and log the cited URLs

   Sheet columns: date, tool, question, brands named, cited URLs, were you cited (Y/N).

4. Find overlap

   URLs cited by two or three tools are the most valuable. Study them: format, facts, author, date.

5. Fix one page

   Apply the page checks to the one page that should answer that question. Retest next month.

Side-by-side log (template + worked example)

Date | Tool (Perplexity / ChatGPT / Google AI) | Question | Brands named | Cited URLs | We were cited? (Y/N) | Cited by 2+ tools? | Page we should win with | Fix planned

Question: 'best accounting software for freelancers in India'
Perplexity: cites 6 URLs, including 2 comparison blogs and 1 Reddit thread
ChatGPT (search on): cites 4 URLs; 1 of the same comparison blogs
Google AI Overview: links 3 pages; same comparison blog again
Overlap: one comparison blog is cited by all three. That page is your first outreach target: ask for a fair listing or a correction.
Our page: not cited. Fix: add a pricing table and a 'who it's for' line at the top.

What the test tells you

Common mistakes

• Blocking GPTBot or ClaudeBot and thinking you've left AI search.

  CONFIRMED: those are training crawlers. Search uses OAI-SearchBot and Claude-SearchBot.

• Allowing crawlers in robots.txt but leaving Cloudflare 'block AI bots' on.

  The firewall wins. Check your CDN's bot settings.

• Optimising for one tool only.

  Our table shows each tool reads different pages. Overlap pages are worth most.

• Testing logged in, with history.

  Answers personalise. Use fresh, logged-out sessions where you can.

• Paying for an 'AI ranking' guarantee.

  No company publishes how it picks citations (UNKNOWN), so nobody can guarantee one.

• Reading Search Console's generative AI report as clicks.

  CONFIRMED: impressions only. Clicks sit inside the normal Performance report.