What is llms.txt? The GEO File for AI Crawlers

Most businesses have spent years optimising for Google. They have sitemaps, robots.txt files, and structured data. These are all the signals Google needs to understand and rank their content. In 2024, a new standard appeared that does the same job for AI: llms.txt. At Geovise, we now implement it for every client from day one. Here is exactly what it is and why it matters.

llms.txt Definition

llms.txt is a Markdown file placed at the root of your website (yourdomain.com/llms.txt). It tells large language models and their AI crawlers what content to index and how to understand your business. As a GEO (Generative Engine Optimization) signal, it directly influences how AI search platforms such as Perplexity represent your brand in their answers. ClaudeBot and GPTBot are training crawlers that may also process it, though their official support for the standard differs.

The format uses lightweight Markdown: a top-level heading with your business name, a short description paragraph, and optional sections with links to your most important pages. It is far simpler than schema.org markup or XML sitemaps, but it is not entirely syntax-free. You do not need a developer to write the content itself. You need someone who understands your business well enough to describe it clearly in a few focused paragraphs.

The standard was proposed by Jeremy Howard on September 3, 2024 via his company Answer.AI. Howard is also co-founder of fast.ai and a lecturer at the University of Queensland and Stanford. The standard has since been adopted as a recommended Generative Engine Optimization (GEO) practice. Geovise includes llms.txt setup in every GEO engagement. You can see our own file at https://geovise.ai/llms.txt.

Why llms.txt Matters for GEO

When Perplexity crawls your website, it does not read your site the way a human does. It processes text, follows links, and builds a representation of what your business offers. Without any guidance, it will index what it finds. That may be your cookie policy or a three-year-old press release rather than your core services.

llms.txt gives the crawler a direct briefing: here is who we are, here is what we do, here are the pages that matter. Think of it as the equivalent of robots.txt. The difference is that instead of controlling access, it provides context.

Which AI crawlers actually use it? Perplexity is the most confirmed adopter. Their crawler reads llms.txt and uses it when deciding what to surface in answers. ClaudeBot is Anthropic's training data crawler and may process the file, but Anthropic's real-time search indexing is handled by a separate crawler called Claude-SearchBot. GPTBot is OpenAI's training crawler and has not officially confirmed llms.txt support as of March 2026; OpenAI's search indexing is handled by OAI-SearchBot. That said, adding a Markdown file to your site costs nothing and takes 20 minutes. Even if only one major AI platform reads it today, the visibility benefit is real and the risk is zero.

What to Put in llms.txt

The file should contain the following sections, in plain language:

• A brief description of your business covering what you do, who you serve, and where you operate
• A list of your key services with prices or ranges, if publicly available
• Information about your founder or team including their name, expertise, and years of experience
• Links to your most important pages: services, case studies, contact
• Contact information including email and phone number if applicable

Geovise publishes its own llms.txt at https://geovise.ai/llms.txt. It covers our services (GEO and SEO consulting for SMEs), our markets (Poland, DACH, UK), pricing tiers from €800/month, and direct links to our key pages. Use it as a reference for the level of detail that works well.

llms.txt vs robots.txt — What's the Difference?

These two files are often confused because they both sit at the root of your domain and both address crawler behaviour. They serve completely different purposes:

robots.txt vs llms.txt at a glance:

• robots.txt controls bot access by specifying which pages crawlers may or may not visit. llms.txt guides AI understanding by providing structured business context about who you are and what you do.
• robots.txt uses machine-readable directives that crawlers parse programmatically. llms.txt is a human-readable Markdown description that language models can interpret directly.
• robots.txt is mandatory for SEO and has been a standard since 1994. llms.txt is recommended for GEO and is an emerging standard, proposed in September 2024.
• robots.txt applies to all crawlers including Googlebot, Bingbot and others. llms.txt targets AI language model crawlers specifically, such as PerplexityBot, ClaudeBot and GPTBot.

You need both. robots.txt without llms.txt means Google can find you but AI cannot understand you. llms.txt without robots.txt means AI has context but Google's crawl may be misconfigured.

How to Create llms.txt for Your Business

Three steps, no developer required for the content itself:

1. Create a plain text file called llms.txt. Open any text editor (Notepad, TextEdit, or VS Code) and write a clear description of your business. Use your About page and Services page as source material. Aim for 300–600 words.
2. Place it in your website's public/ directory so it is accessible at yourdomain.com/llms.txt. For WordPress sites, upload it to your root folder via FTP or cPanel. For Next.js, drop it in the /public directory. For static sites, it goes in the site root.
3. Test it by visiting yourdomain.com/llms.txt directly in a browser. If you see plain text, it is working. If you get a 404, check the file location with your developer.

Konrad Kluz and the Geovise team include llms.txt creation and optimisation in our GEO audit service — if you want a professionally structured file that maximises AI crawler comprehension, that is part of what the audit covers. Details at geovise.ai/en/services.

Does llms.txt Guarantee AI Will Cite You?

No. llms.txt is one signal among many in GEO and not a shortcut to the top of every AI answer. Perplexity uses it to inform indexing decisions. ChatGPT and Google AI Overviews have not publicly confirmed they process the file. That may change, but right now it is not a guaranteed citation lever.

The analogy that fits: filing for a business licence does not guarantee customers, but it is still the right first step. llms.txt signals to AI that you are a legitimate, structured business that has taken the time to communicate clearly. That matters alongside authoritative content, proper schema.org markup, and consistent entity mentions across the web.

For businesses serious about GEO, llms.txt is the starting point, not the finish line. Our LLMO (Large Language Model Optimisation) service at geovise.ai/en/services/llmo covers the full stack: llms.txt, schema.org, entity consistency, content authority signals, and AI citation monitoring.

Photo by Wes Cockx on Unsplash, commissioned by Google DeepMind for the Visualising AI project.