The words people use about AI search, explained in plain language, with diagrams and real examples from our research.
Your website1 of 94
AI agents and agentic commerce
AI assistants that act for a person: comparing prices, booking a call or buying something. If an agent can't find your price or send a message, it moves on.
AI agents can now browse sites, fill in forms and, on some platforms, complete purchases for the user. They rely on clear prices, plain buttons and forms that work without complicated steps.
An app you talk to in plain language, such as ChatGPT, Gemini, Claude, Perplexity or Microsoft Copilot. Most can search the web before they answer.
Assistants can answer questions, write, analyse files and, increasingly, act on the web. Most now decide for themselves when to search, especially for products, prices and recent events.
A bot that reads web pages for an AI company, either to train models or to answer questions live. Examples include GPTBot and OAI-SearchBot (OpenAI), ClaudeBot (Anthropic) and PerplexityBot.
Crawlers identify themselves by name in each request, and each serves a purpose: training (like GPTBot and ClaudeBot), search indexing (like OAI-SearchBot and PerplexityBot) or fetching a page a user asked about. Common Crawl's CCBot gathers data that many models have been trained on.
Google Search's conversational mode, for longer questions and follow-ups, answered by AI with links to sources.
AI Mode is a separate tab in Google Search for longer, more complex questions. Google says it uses query fan-out: it breaks a question into many related searches, runs them at once, and combines the results into one answer with links. You can ask follow-ups without starting again.
The AI-written summary Google shows at the top of some search results, with links to the sites it drew on.
AI Overviews grew out of Google's Search Generative Experience test and rolled out widely in May 2024. They appear on some searches, not all, and link to the pages they drew on. Since mid-2026 Google has been rolling out a Search Console setting that lets some site owners exclude their site from AI Overviews and AI Mode.
From our research
30% of the sites AI Overviews cited were Reddit, YouTube and forums, against 6% across all six AIs we asked.
Visits that arrive from an AI assistant, for example from chatgpt.com or perplexity.ai, usually seen in Google Analytics.
Analytics tools record these visits by their referrer, such as chatgpt.com, perplexity.ai or gemini.google.com. ChatGPT adds utm_source=chatgpt.com to many links, which helps. Some AI visits still arrive without a referrer and show up as direct traffic.
Finding things by asking an AI assistant instead of scanning a list of links. The assistant reads sources, writes one answer and often names a few brands.
A traditional search returns ten links and leaves the reading to you. AI search does the reading: it runs searches, opens pages, and writes a summary that usually recommends a few options. Many people now use both, starting with AI and then checking a website or two.
Product recommendations inside AI answers, sometimes with prices and where to buy. Whether your products appear depends on the same sources as any other answer.
Google, ChatGPT and Perplexity all show product recommendations, often with images, prices and links to buy. They draw on product data, merchant feeds, reviews and comparison articles.
How often, and how well, AI assistants mention your brand when people ask them questions about what you sell.
It covers more than whether you're named. It includes where you appear in the answer, how AI describes you, which competitors are named alongside you and which pages AI uses as sources. It also differs by platform: you can be visible in ChatGPT and missing from Google's AI Overviews.
A search tool that replies with a written answer rather than a list of results. Perplexity is the best-known example; ChatGPT, Gemini and Google's AI features work the same way when they search.
Answer engines combine a search index with a language model. They find relevant pages, pull out the useful parts and write them into one response, usually with numbered or linked sources. Perplexity was built this way from the start; ChatGPT, Gemini and Claude added web search later.
Work to make your brand the answer an answer engine or voice assistant gives. It's an older name for much the same work as GEO, and many people use the two interchangeably.
AEO started with featured snippets and voice assistants like Siri and Alexa, where one spoken answer replaced a page of results. The aim was to be the source that answer was taken from. Today it overlaps almost completely with GEO.
AI can answer the same question differently each time, and each AI answers it differently. That's why visibility is measured over many answers and days, not one screenshot.
Language models build answers with some randomness, and live searches can return different pages from day to day. Each platform also has its own sources and style.
ChatGPTBrand AGeminiBrand APerplexityBrand BAI OverviewsBrand CAI ModeBrand B
One question, five AIs, three different first picks.
Illustration
From our research
On only 21% of questions did every AI that named a brand first name the same one.
A link from another site to yours. Links help search rankings, and rankings shape which pages AI finds when it searches.
Search engines have treated links as votes of confidence since the early days of Google. Links from relevant, trusted sites count most; paid or spammy links can be ignored or penalised. They remain one of the signals search engines use to judge authority.
An article listing the best options in a category, such as “best CRM for small business 2026”. AI often searches for these and repeats who they name.
These are articles and listicles from publishers, bloggers and sometimes competitors. AI searches for them because they've already done the comparing, and it often repeats the brands they name.
Security that challenges automated visitors. Set too strictly, it can stop AI crawlers and assistants from reading your site at all.
Services like Cloudflare can challenge or block traffic that looks automated. In July 2025 Cloudflare began blocking known AI crawlers by default for new sites, letting owners choose which ones to allow.
A question that names your brand, such as “is Asana good for small teams?”. Useful for checking what AI says about you, but you'll almost always be named in the answer.
Branded questions include comparisons (“Asana vs Monday”), reviews (“is HubSpot worth it?”) and facts (“does Zoho CRM have a free plan?”). They show how AI describes you to people who already know your name.
The story each AI tells about your brand: what it says you're best for, who you suit, and how you compare. It can differ from AI to AI.
The narrative includes your category, who you're “best for”, your price level and what you're compared with. ChatGPT might call you the budget option while Gemini calls you the enterprise pick, depending on what each one has read.
Profiles such as Google Business Profile, Bing Places, Apple Business Connect and Yelp. AI answers to local and “near me” questions draw on them, so details should match everywhere.
Listings hold your name, address, phone number, website, opening hours, categories, photos and reviews. Local marketers call the core details NAP: name, address, phone. Directories and data aggregators copy these details across the web, so mistakes spread too.
Where a buyer is when they ask: learning about a problem, comparing options, or choosing one. AI names different brands at each stage.
Learning questions (“what is a CRM?”) often get explanations with few brands. Comparing questions (“best CRM for startups”) get lists. Choosing questions (“how much does it cost?”) get specifics about one or two options.
LearningWhat is a CRM?
ComparingBest CRM for startups
ChoosingHow much does it cost?
Illustration
From our research
Answers to learning questions named 2.0 brands on average, against 3.3 for comparing questions.
The main address of a page when the same content is reachable at several. Marking it tells crawlers which one to credit, so signals aren't split across copies.
A canonical is set with a link tag in the page's head. It's common for product pages with filter parameters, printer versions or pages reachable through several paths.
A short section of a page that a retrieval system pulls out on its own. Pages made of clear, self-contained sections give AI passages it can use as they are.
Retrieval systems split pages into chunks, often a heading and the paragraphs under it, and score each one against the question. The best chunks are passed to the model.
Our storyHow much does it cost? Plans start at $49 a month, billed monthly.Customer storiesContact us
AI retrieves the one section that answers the question.
When an AI answer links to a page as a source. A citation of your site isn't the same as a mention: AI can cite your page and still recommend someone else.
Citations appear as numbered references, links under the answer or source cards, depending on the platform. One answer can cite several pages, from your site, competitors' sites, review sites, forums or news.
For small teams, YourBrand and Competitor are the usual picks.
competitor.comyourbrand.com
Name Mention: what buyers readsite Citation: a source it used
Illustration
From our research
salesforce.com was cited in more CRM answers than hubspot.com, yet HubSpot was named far more often.
The share of AI answers that cite at least one page from your site. Read it next to visibility: a high citation rate with low visibility means AI uses your pages but recommends someone else.
It's counted like visibility, but for citations: answers that link to your domain, divided by all answers to your tracked questions.
A site where people discuss things in public, such as Reddit, Quora, YouTube or forums. AI cites them for real-world opinions, some platforms far more than others.
Google and OpenAI both signed data deals with Reddit in 2024. Discussions show real experiences, complaints and recommendations, which AI uses to judge what people think.
From our research
reddit.com was tied with runrepeat.com as the most cited site across all the answers.
A page that compares you with named alternatives, such as “X vs Y”. Buyers weighing options ask AI to compare, and a fair comparison on your own site puts your side in front of it.
Good comparison pages are fair: they say where the alternative is stronger, show prices and features side by side, and help the reader choose.
A question AI answers with a competitor and not you, or a page AI cites for a competitor and never for you. Each gap is something to fix.
Gaps show up in two ways: questions where competitors are named and you aren't, and source pages that mention competitors but not you. Each points to a specific fix, like a page to write or a site to get listed on.
The brands you track alongside your own. Share of voice and position only mean something when they're measured against the competitors your buyers actually consider.
A good set includes the brands AI actually names for your questions, which aren't always the ones you think of as competitors.
How recently a page was updated. AI searches often include the current year, so dated pages can lose out to newer ones.
Freshness signals include a visible updated date, current year in titles, recent statistics and up-to-date prices. AI searches often add the current year, so older pages may not even be retrieved.
How much text a model can consider at once, counted in tokens. Pages it retrieves have to fit, so it usually works from selected passages rather than whole websites.
Context windows have grown from a few thousand tokens to hundreds of thousands or more. Even so, an AI answering a search reads a handful of pages or passages, not your whole site.
Searching through a back-and-forth chat, where each question can build on the last, instead of one search at a time.
In a conversation the AI remembers what was said earlier, so “which of those is cheapest?” makes sense. Google's AI Mode, ChatGPT and Perplexity all work this way.
Requests to your site from AI crawlers, counted from server logs or your hosting provider's analytics. They show whether AI companies are reading your pages at all.
Server logs record every request with the bot's name. Some hosting and network providers, such as Cloudflare, show AI crawler traffic in their dashboards.
Earning coverage and mentions on news sites, blogs, podcasts and industry publications: the kind of pages AI reads and cites.
Digital PR includes press coverage, expert commentary for journalists, data stories, podcast appearances and inclusion in industry roundups and awards. Unlike classic PR, it's measured by online coverage, links and mentions on sites people and search engines trust.
Experience, expertise, authoritativeness and trustworthiness: Google's framework for judging content quality. The same signals help a page earn trust as a source.
E-E-A-T comes from Google's Search Quality Rater Guidelines; the extra E, for experience, was added in December 2022. It isn't a single ranking score; it describes the qualities Google's systems try to reward, like first-hand experience and clear authorship.
A list of numbers that stands for the meaning of a piece of text. Similar meanings get similar numbers, which lets a system match a question to a passage that uses different words.
Embeddings place text in a kind of map where similar meanings sit close together. Search systems compare the question's embedding with each passage's to find the closest matches.
A distinct thing, such as a brand, person or product, that search engines and AI recognise. Consistent facts about you across the web make you easier to recognise.
Search engines store entities with their attributes, like a company's name, website, founders and products, and their links to other entities. Sources like Wikipedia, Wikidata, business listings and your own structured data feed this picture.
Checking the claims AI makes about your brand against your own facts, so wrong prices, features or details get caught.
A fact check compares what AI says, such as prices, plans, features, locations or founding year, with the facts you supply. Mismatches are flagged so you can fix the source of the error.
YourBrand starts at $99 a month $49 a month and has no free plan.
A fact check catches the wrong price before buyers do.
A page of real customer questions, each with a short, direct answer. It matches how people ask AI and gives it clear sentences to use.
Build it from real questions: sales calls, support tickets and the questions AI shows it searches for. Google stopped showing FAQ rich results for most sites in 2023, but the page itself still helps people and AI.
A question that builds on the previous answer, such as “which of those is cheapest?”. It narrows the choice among the brands already named.
Follow-ups can narrow (“which is cheapest?”), compare (“how does the first one compare to the second?”) or check (“is it good for beginners?”). Each one draws on the earlier answer and sometimes new searches.
AI that creates new text, images or other content, rather than only sorting or predicting. Assistants like ChatGPT, Gemini and Claude are generative AI.
Generative AI learns patterns from large amounts of data and uses them to produce something new: a paragraph, an image, code or audio. Language models are the kind behind chat assistants and AI search.
Work to get a brand named and cited in answers from generative AI such as ChatGPT, Gemini and Perplexity. The name comes from a 2023 research paper on the subject.
The term comes from “GEO: Generative Engine Optimization”, a 2023 paper by researchers at Princeton, Georgia Tech, the Allen Institute for AI and IIT Delhi. In their tests, adding citations, quotations and statistics to a page made it more visible in AI answers. In practice GEO covers your own pages, what other sites say about you, and whether AI can reach your site at all.
Basing an answer on sources retrieved at the time of the question, so it reflects current information and can point to where it came from.
Grounding ties each part of an answer to retrieved sources, which reduces made-up facts and lets the AI show citations. Google, for example, offers “Grounding with Google Search” to developers building on Gemini.
When an AI states something false with confidence: a wrong price, a feature you don't have, a founder who doesn't exist.
Hallucinations happen because a model writes what sounds likely, not what it has checked. They're more common when information about a subject is thin, outdated or contradictory. Searching the web reduces them but doesn't remove them.
YourBrand starts at $99 a month $49 a month and has no free plan.
A fact check catches the wrong price before buyers do.
A question on your own contact or booking form. It catches buyers who found you through AI but arrived by typing your name, which no analytics tool can see.
Add “an AI assistant, like ChatGPT” as an option, or leave a free-text field. Buyers who heard about you in an AI answer often search your name or type your address, so analytics can't connect the visit to AI.
An open protocol that tells participating search engines, including Microsoft Bing, the moment a page is added or changed, instead of waiting for them to crawl it.
IndexNow was launched by Microsoft Bing and Yandex in 2021. You ping one endpoint and participating engines share the notification. Google hasn't adopted it.
Running a page's scripts to build what it shows. Many AI crawlers skip this step: a 2024 Vercel study found that the crawlers from OpenAI, Anthropic and Perplexity read pages without running JavaScript.
Many modern sites send a nearly empty page and use JavaScript to fill it in. Googlebot runs JavaScript, and Vercel's study noted that Gemini crawls through Googlebot. Most other AI crawlers read only the initial HTML.
The date a model's training data ends. Anything newer only reaches its answers if it searches the web.
A model's cutoff is usually months before its release, because training and testing take time. Questions about anything newer, such as this year's prices or a product you launched last month, can only be answered well if the AI searches the web.
Google's database of facts about entities and how they connect. It feeds the information panels in Search and helps Google recognise your brand.
Google introduced the Knowledge Graph in 2012. It holds billions of facts about people, places, companies and things, drawn from sources like Wikipedia, Wikidata, licensed data and websites' structured data. The knowledge panel beside some search results is the most visible part of it.
The kind of AI model behind ChatGPT, Gemini, Claude and others: trained on large amounts of text to understand questions and write answers.
An LLM learns patterns in language from huge amounts of text and predicts what words should come next, which lets it answer questions, summarise and write. Most modern LLMs are built on the transformer design, introduced by Google researchers in 2017. On its own an LLM only knows its training data; connected to search, it can also read the web.
When an AI looks things up on the web while answering, instead of relying only on what it learned in training. Answers that search usually show their sources.
OpenAI launched ChatGPT search in October 2024, and Claude added web search in 2025. Perplexity, Gemini and Google's AI features have searched from the start. The AI decides when to search, often for anything about products, prices or recent events, and reads the pages it finds.
Another name for GEO, focused on large language models: making sure the models, and the searches they run, find accurate and favourable information about you.
Some people use LLMO to stress the model itself: what it learned in training, as well as what it finds when it searches. Others use it as a plain synonym for GEO. There's no standard body defining these terms, so the names overlap.
A proposed standard: a plain-text file at your site's root that summarises the site and links its key pages for AI models. Not every AI platform reads it yet.
llms.txt was proposed by Jeremy Howard of Answer.AI in September 2024. It's written in Markdown: a short description of the site and links to its most useful pages. Adoption is still early, and major AI platforms haven't said they rely on it.
# YourBrand
> What we sell, and who for, in one line.
## Key pages
- [Pricing](https://yourbrand.com/pricing)
- [Reviews](https://yourbrand.com/reviews)
The country, and sometimes the city, a question is asked from. AI can recommend different brands in different markets, so tracking should match where your buyers are.
AI considers where the person is. A question about the best accounting software can name different brands in Canada, the UK and India, based on local sources and availability.
When an AI answer names your brand. A mention is what buyers read.
A mention is any time the brand name appears in the answer text, whether as a recommendation, a comparison or a warning. Good tracking separates positive mentions from negative ones.
For small teams, YourBrand and Competitor are the usual picks.
competitor.comyourbrand.com
Name Mention: what buyers readsite Citation: a source it used
Searching with a photo, your voice or a video as well as text, such as pointing your camera at a product with Google Lens.
Multimodal models can read images and audio as well as text. Someone can photograph a pair of shoes and ask where to buy them, or ask out loud while driving.
Data, surveys or tests you publish first. Other sites cite it, and AI can repeat a specific finding with your name attached.
Examples include customer surveys, benchmark tests, pricing studies, industry reports or anonymised data from your own product. Publish the method and the headline numbers clearly, with a date, so others can cite it with confidence.
Owned media is your own site and channels. Earned media is what others publish about you: reviews, articles, forum threads and videos. AI answers draw on both, in proportions that vary by platform.
Marketers often add paid media, which is advertising. AI answers are written from owned and earned pages; ads, where an AI platform shows them, sit separately from the answer.
From our research
32% of ChatGPT's citations were the brands' own websites, against 5% for Claude.
When an AI tailors its answer to the person asking, using their location, earlier chats or saved memory. Two people can get different answers to the same question.
ChatGPT can remember details from earlier chats, Google can use location and search history, and most assistants consider the country you're in. Logged-in and logged-out answers can differ.
Where in an answer your brand is named: first, second, and so on. Being named first usually matters more than being named fifth.
AI answers often list options in order or lead with one recommendation. Position records where you appear: first, second, third, or later. It's usually reported as an average or as the share of answers where you're named first.
1Competitor
2YourBrand
3Another brand
Illustration
From our research
Asana was named in more project management answers than any competitor, but ClickUp was named first more often.
A page that states plainly what you charge. Buyers ask AI about cost, and a missing or vague price leaves AI to guess or to name someone whose price it can find.
Show prices, what each plan includes and any limits. If you quote custom prices, say what drives the cost and give a typical range or starting point.
From our research
39% of ChatGPT's web searches included “price” or “pricing”.
A file listing your products with prices, stock and details, sent to platforms such as Google Merchant Center. Shopping features use feeds to show current prices and availability.
Feeds include titles, descriptions, prices, availability, images and identifiers like GTINs. Google Merchant Center uses them for Shopping, and OpenAI also accepts merchant product feeds for shopping in ChatGPT.
What someone types into an AI. Tools that track AI visibility often call each tracked question a prompt.
Real prompts are often longer and more specific than search keywords, like “what's the best CRM for a 10-person real estate team that uses Gmail?”. Visibility tools track a fixed set of prompts so results can be compared day to day.
An estimate of how often people ask AI about a topic. It helps pick the questions worth tracking.
AI platforms don't publish how often each question is asked, the way Google's keyword data does for searches. Prompt volume is estimated from other signals, such as search demand, panels of users or traffic data, so treat it as a guide, not a count.
When an AI turns one question into several web searches before it answers. The searches show which pages it goes looking for.
A question like “best CRM for a small sales team” might become searches for “best CRM small business 2026”, “CRM pricing per user” and “HubSpot vs Pipedrive”. Each search brings back different pages, and the answer is built from all of them. Only some platforms, like ChatGPT and Claude, show these searches.
Best CRM for startups?
best CRM for startups 2026
CRM startups pricing free plan
HubSpot vs Pipedrive startups
Illustration
From our research
85% of ChatGPT's web searches included “2026”, and 58% included “best”.
The technique behind grounding: first retrieve relevant documents, then write the answer from them.
RAG was described in a 2020 research paper from Facebook AI. A retrieval step searches a collection of documents or the web, picks the most relevant passages, and passes them to the model with the question. The model then writes its answer from those passages.
Sites like G2, Capterra, Trustpilot or RunRepeat where customers or testers rate products. AI reads them often, especially Perplexity and Gemini.
Review sites aggregate ratings and written reviews from customers or testers. AI often treats them as neutral evidence and quotes them when recommending or warning about a product.
From our research
runrepeat.com was tied for the most cited site across all the answers.
A file at the root of your site that tells crawlers what they may read. Blocking an AI crawler here can keep your pages out of its answers. Google-Extended is the setting that limits some uses by Google's Gemini without leaving Google Search.
robots.txt has been used since the 1990s and became an official internet standard (RFC 9309) in 2022. Well-behaved crawlers follow it, but it's a request, not a lock. Google says blocking Google-Extended doesn't affect Search, including whether you appear in AI Overviews or AI Mode.
# Let OpenAI's crawler read the site
User-agent: GPTBot
Allow: /
# Opt out of some Gemini uses, stay in Google Search
User-agent: Google-Extended
Disallow: /
Work to rank higher in a search engine's list of results. It still matters for AI: assistants that search the web read search results, so pages that rank well are easier for them to find.
SEO covers technical work (a site search engines can crawl), content (pages that answer what people search for) and authority (links and mentions from other sites). Google's ranking systems use many signals that change over time.
Google's 2023 test of AI-written answers in Search. It rolled out widely as AI Overviews in May 2024.
SGE was announced at Google I/O in May 2023 and offered as an opt-in experiment through Search Labs. Google used it to test AI-written answers before launching them as AI Overviews.
The page a search engine shows after a search. On Google it can now open with an AI Overview above the usual links.
A modern results page can include ads, an AI Overview, local map results, videos, “People also ask” boxes and shopping results, as well as the classic blue links.
Using the right tags for headings, lists, tables and buttons, so a page's structure is clear to a machine that can't see the design.
Semantic HTML uses tags like h1 to h6 for headings, ul and ol for lists, table for tables and button for buttons, instead of styling generic boxes to look like them.
Search by meaning instead of exact words, so “cheap CRM for a small team” can find a page about “affordable CRM for startups”.
Older search matched the exact words you typed. Semantic search uses embeddings to match meaning, so synonyms, rephrasings and longer questions still find the right page. AI assistants rely on it to pull the passages most relevant to a question.
Whether AI talks about your brand positively, neutrally or negatively, and which strengths and doubts it mentions.
Sentiment goes beyond positive or negative. It picks up the specific strengths AI repeats (“easy to use”, “great support”) and the doubts (“expensive”, “steep learning curve”). These often come straight from reviews and articles AI has read.
Building the full page on the server, so the HTML a crawler receives already contains the text. It's the usual fix when a site's content only appears after JavaScript runs.
With server-side rendering or static generation, the server sends finished HTML. Frameworks like Next.js and Nuxt support it, and many website builders do it by default.
Your share of all the mentions of you and your competitors. If the brands you track are named 200 times and you're named 50 of them, your share of voice is 25%.
The idea comes from advertising, where it meant your share of all ad spend or exposure in a category. In AI visibility it compares mentions: yours against everyone you track, so it always adds up to 100% across the set.
You 25%Competitor A 40%B 20%C
All mentions of the brands you track, split by brand.
Whether AI can reach and read your site: crawlers allowed, no blocking, pages that load without JavaScript, sitemaps and clear facts.
A readiness check looks at robots.txt rules, firewall and bot settings, whether pages show their text without running JavaScript, sitemaps, structured data and basic facts like pricing and contact details.
A page an AI read to write its answer. Review sites, “best of” lists, forums and brands' own sites are the usual ones.
Some AIs show every page they read; others show only a few. Sources differ a lot by platform: some lean on brands' own sites, others on review sites or community discussions.
The kind of site a citation comes from: the brand's own site, a review or comparison site, a community like Reddit or YouTube, or everything else.
Sorting citations by type shows what each AI trusts. One platform may lean on brands' own sites, another on review sites, and another on Reddit and YouTube.
From our research
30% of what Google AI Overviews cited was Reddit, YouTube and forums. For Perplexity it was 0%.
Labels in a page's code, usually from schema.org, that state facts plainly: your product, price, reviews, opening hours or FAQ. They make facts easier for machines to read.
Schema.org was launched in 2011 by Google, Microsoft and Yahoo, with Yandex joining soon after. It's usually added as JSON-LD, a short block of code that states facts like “this is an Organization, with this name, logo and these profiles”.
A small chunk of text, often part of a word, that language models read and write. Model limits and prices are counted in tokens.
In English, a token averages about four characters, roughly three quarters of a word. Common words are one token; long or unusual words can be several.
A group of related questions, such as everything about pricing or one product line, so you can see your visibility for each part of the business.
Topics group questions by product, use case, audience or buying stage. You might be strong on pricing questions but weak on questions about agencies, or visible for one product line and missing for another.
Being a thorough, trusted source on one subject, built by covering it in depth over time rather than touching many subjects lightly.
Authority builds through coverage: many useful pages on one subject, linked together, kept up to date and referenced by others. A site with fifty clear pages on CRM for real estate teams looks more expert on that subject than one with a single article.
Some AI companies run separate bots for training models and for live answers. OpenAI uses GPTBot for training and OAI-SearchBot for ChatGPT search, so blocking one doesn't block the other.
Anthropic separates ClaudeBot (training), Claude-SearchBot (search) and Claude-User (pages a user asks about), and Perplexity separates PerplexityBot from Perplexity-User. Each can be allowed or blocked on its own line in robots.txt.
# Keep content out of OpenAI model training
User-agent: GPTBot
Disallow: /
# Stay visible in ChatGPT search
User-agent: OAI-SearchBot
Allow: /
The text a model learned from before release. What it 'knows' without searching comes from here, which is why older or thin information about a brand can stick around.
Models learn from large collections of text: public web pages, books, licensed data and other sources. Crawls like Common Crawl have been widely used to build these collections. Once a model is trained, that knowledge is fixed until the next version.
A question that doesn't name a brand, such as “best CRM for startups”. These are the ones that decide whether AI recommends you or a competitor.
Unbranded questions describe a need: “best project tool for agencies”, “CRM with built-in email”, “running shoes for flat feet”. The person hasn't chosen yet, so the brands in the answer become their shortlist.
From our research
Every question was unbranded, and the answers named 3 of the 5 brands we tracked, on average.
The name a browser or bot gives when it asks for a page, such as GPTBot or ClaudeBot. robots.txt rules and server logs use it to tell crawlers apart.
Each request includes a user agent string. Bots can lie about it, so major AI companies like OpenAI publish the IP addresses their crawlers use, letting you confirm a visit is genuine.
Content made by customers and users rather than the brand: reviews, forum posts, videos and social posts.
UGC is often trusted more than a brand's own words because it comes from people with no reason to sell. It includes star ratings, written reviews, Reddit threads, YouTube reviews, unboxing videos and social posts. AI reads a lot of it when judging what people really think of a product.
When an assistant opens a page because someone asked it to, such as OpenAI's ChatGPT-User. OpenAI says robots.txt rules may not apply to these visits, because a person started them.
These fetches happen when someone pastes a link or asks the assistant to look at a page. They aren't building an index; they read the page once for that conversation.
The share of AI answers that name your brand. If AI names you in 30 of 100 answers to your questions, your visibility is 30%.
It's the simplest AI visibility number: answers that name you, divided by all the answers to your tracked questions. It's usually reported per AI platform and as an overall average, and tracked over time.
A free, open database of facts about people, places, companies and products, run alongside Wikipedia. It's widely used as a reference for recognising entities.
Wikidata launched in 2012, run by the Wikimedia Foundation. Each item has an ID and facts like website, founding date, headquarters and social profiles. Anyone can edit it, within its notability and sourcing rules.
When someone gets what they need from the answer itself and doesn't click any link. Common with AI answers, which is why visits alone undercount AI's influence.
Zero-click searches existed before AI, with answer boxes and knowledge panels. AI answers make them more common, because a full answer with recommendations leaves less reason to click.
best CRM for startups
AI answerHubSpot, Pipedrive and Zoho are the usual picks...