How AI Assistants Decide Which Brands to Recommend (and How to Get on the List)

When someone asks ChatGPT "what's the best platform for X", the answer is not pulled from a ranking you can buy your way into. The assistant runs a web search, fetches a handful of pages it can actually read, and assembles a recommendation from whatever those pages let it verify. Brands that are easy to retrieve and quote get recommended. Brands that hide their information behind JavaScript, PDFs, or vague marketing copy get skipped, even when their product is better.

TL;DR:

• AI assistants recommend what they can retrieve, verify, and quote. Retrievability is the entry ticket.
• The crawler that controls your ChatGPT search visibility is OAI-SearchBot, not GPTBot. Blocking the wrong one is a common self-inflicted wound.
• llms.txt is cheap to add, but no major AI system has confirmed using it. Treat it as a low-cost bet, not a strategy.
• The biggest wins are boring: open robots.txt, plain-text pricing, extractable answers near the top of the page, and third-party mentions.

How does an AI assistant actually pick a brand?

Three layers feed a recommendation:

1. Training data. What the model absorbed about your brand months ago. You influence this slowly, through consistent mentions across the web: reviews, comparisons, press, forums.

2. Live retrieval. For most commercial questions, assistants search the web at answer time. This is where you win or lose today. If the assistant's search bot can crawl your pages and extract a clear answer, you are a candidate. If not, a competitor is.

3. Verification. Assistants prefer claims they can pin to a source. Concrete numbers, named clients, transparent pricing, and third-party confirmation all make you safer to recommend than superlatives do.

Which crawlers do you need to allow?

Each AI company runs several bots with different jobs, and the names matter more than most robots.txt advice admits:

• OpenAI (documentation): GPTBot collects training data. OAI-SearchBot decides whether you appear in ChatGPT's search results. ChatGPT-User fetches pages when a user asks about you, and robots.txt rules may not apply to it. The practical takeaway: a site that blocks GPTBot for training-data reasons but forgets to allow OAI-SearchBot has opted out of ChatGPT recommendations without meaning to.
• Anthropic (documentation): ClaudeBot for training, Claude-SearchBot for search quality, Claude-User for user-initiated fetches.
• Perplexity (documentation): PerplexityBot indexes for its search results; Perplexity-User fetches pages on user request. Perplexity states that neither is used to train foundation models.

Check your robots.txt today. We keep ours fully open (User-agent: * Allow: /), and for most brands that is the right call: the training-data debate is real, but blocking search and user-fetch bots removes you from answers your buyers are already reading.

Does llms.txt help?

llms.txt is a community proposal by Jeremy Howard (September 2024, updated 2026): a markdown file at your domain root that summarizes your site for language models. Documentation platforms adopted it quickly, Chrome's Lighthouse now checks for it, and thousands of sites publish one.

Here is the uncomfortable part: Google's John Mueller has stated plainly that no AI system currently uses llms.txt, and Google Search does not use or endorse it. None of the major assistants have confirmed reading it either.

Our position: we publish one at increas.io/llms.txt anyway. It costs twenty minutes, it cannot hurt, and it forces you to write the one-paragraph description of your business that AI systems should find. Just do not mistake it for the work. If your pricing page is unreadable to crawlers, no llms.txt will save you.

What actually moves the needle

From the AI-visibility audits we run for ecommerce and SaaS brands, the gaps repeat:

1. Pricing that only exists in a JavaScript widget or a sales call. Pricing is among the hardest information for an assistant to retrieve, which means brands that state it in plain HTML get quoted while competitors get "pricing available on request", which reads as a warning. If you can publish prices or even ranges, do it.

2. Answers buried under storytelling. Assistants extract a short passage, not your whole page. Put the direct answer to the question a buyer would ask in the first paragraph, in plain language, with the number in it.

3. Zero third-party footprint. If the only source about your brand is your own site, you are a claim, not a fact. Reviews on G2 or Trustpilot, comparison mentions, a founder interview on a real publication: these are what let an assistant recommend you with confidence.

4. Server-rendered content that is not actually server-rendered. We still find "SSR" storefronts where the product data arrives client-side only. Crawlers that do not execute JavaScript see an empty shell. This is a solvable engineering problem, and it is the kind we fix on Next.js builds weekly.

How to test where you stand

Do this now, it takes ten minutes:

1. Ask ChatGPT, Claude, and Perplexity the exact question your buyer would ask ("best X for Y"). Note whether you appear and how you are described.
2. Ask each one "how much does [your product] cost?" If they cannot answer, your pricing is not retrievable.
3. Fetch your robots.txt and check it against the bot names above.

If the results are uncomfortable, that is fixable. We run AI-visibility audits that cover crawlability, extractability, and how assistants currently describe your brand, with a prioritized fix list your team can execute.