Optimize for
The llms.txt file is the new standard for making your website AI-discoverable. Learn how to create, format, and deploy an llms.txt file that helps ChatGPT, Claude, Gemini, and Perplexity understand your brand.
An llms.txt file is a standardized markdown file placed at the root of your website (e.g., yourdomain.com/llms.txt) that provides AI models with a structured, human-readable overview of your site. Think of it as a "welcome packet" specifically designed for Large Language Models like ChatGPT, Claude, Gemini, and Perplexity.
While robots.txt tells crawlers what they can and cannot access, llms.txt tells AI models what your site is about, what your core products and services are, and where to find the most important information. It is the difference between giving someone a key to your building versus giving them a guided tour.
The standard was proposed in late 2024 and has rapidly gained adoption among forward-thinking brands that want to control how AI systems understand and represent them. By early 2026, major AI platforms are actively checking for and utilizing llms.txt files during content retrieval.
AI search is no longer a niche experiment—it is a mainstream discovery channel. ChatGPT processes over 400 million queries per week. Perplexity has grown 858% year-over-year. When someone asks an AI assistant about your industry, your brand needs to be part of the answer.
Without an llms.txt file, AI models are forced to figure out what your site is about by crawling individual pages, parsing navigation menus, and guessing which information is most important. This process is error-prone and often leads to incomplete or inaccurate brand representations in AI responses.
Here is what an llms.txt file solves:
Clarity: It gives AI models a single, authoritative source for understanding your brand, products, and value proposition
Control: You define the narrative instead of letting AI models piece it together from scattered web pages
Discoverability: It highlights your most important pages, documentation, and resources so AI retrieval systems can find them quickly
Accuracy: It reduces the chance of AI models hallucinating or misrepresenting your brand information
Competitive advantage: Most websites still do not have an llms.txt file—implementing one puts you ahead of competitors in AI search
The llms.txt file follows a simple markdown format designed to be both human-readable and machine-parseable. There is no complex configuration or schema to learn—if you can write a markdown document, you can create an llms.txt file.
The standard structure includes these sections:
Title (H1): Your project or brand name as a top-level heading
Blockquote summary: A one-line description of what your brand or product does
Body text: Additional context about your organization, mission, or key differentiators
Sections (H2): Organized groups of links and descriptions for different content areas
Links: Key URLs formatted as markdown links with optional descriptions
Example llms.txt structure:
A well-formatted llms.txt file starts with a heading, includes a brief description, then organizes your key resources under section headings. Each link is a standard markdown link, optionally followed by a colon and a short description of what the page contains.
Start with a clear, one-line summary of what your brand or product does. This should be the same elevator pitch you use across all your marketing channels. Consistency matters—AI models build confidence when they see the same description across multiple sources.
Keep it under 150 characters. Be specific about your category and value proposition. Avoid buzzwords that do not convey concrete meaning.
Map out the most important sections of your website that you want AI models to know about. Common sections include:
Documentation: Product guides, API references, getting-started tutorials
Blog: Your most authoritative and comprehensive articles
Features: Core product capabilities and use cases
Pricing: Plans, pricing tiers, and comparison information
About: Company information, team, and mission
Tools: Free tools, calculators, or utilities you offer
Support: Help center, FAQ, and contact information
For each important page, create a markdown link with a brief description. The description should tell the AI what information it will find on that page. This is critical—AI models use these descriptions to decide which pages to retrieve when answering specific queries.
Good example: A link to your pricing page with a description like "Detailed pricing for all plans including features, limits, and enterprise options."
Bad example: A link labeled just "Click here" or "Pricing" with no additional context.
Create a file named llms.txt in markdown format. Place it at the root of your domain so it is accessible at yourdomain.com/llms.txt. The file must be served with a text/plain or text/markdown content type.
For most web frameworks and CMS platforms:
Next.js: Add the file to your public/ directory
WordPress: Upload to your root directory via FTP or use a plugin
Shopify: Add as an asset or use a custom route
Static sites: Place in the build output root directory
The llms.txt standard also supports an extended version called llms-full.txt. While llms.txt provides a concise overview with links, llms-full.txt can contain the full text content of your key documentation pages in a single file. This is especially useful for AI models that prefer to consume content in one retrieval step.
This is optional but recommended for documentation-heavy sites, developer tools, and SaaS products where technical accuracy matters.
These three files serve different but complementary purposes in your website's AI and search optimization stack:
robots.txt is the gatekeeper. It tells crawlers (both traditional search engines and AI bots) which pages they are allowed or denied to access. It controls permission—can the bot visit this page or not?
sitemap.xml is the map. It lists all your indexable URLs with metadata like last modified dates and priority scores. It helps crawlers discover all your pages efficiently.
llms.txt is the guide. It explains what your site is about, what your key offerings are, and where AI models should look for the most important information. It provides context and understanding—something neither robots.txt nor sitemap.xml can do.
You need all three:
robots.txt ensures AI crawlers can access your content
sitemap.xml ensures AI crawlers can find all your pages
llms.txt ensures AI models understand what your site is about and which pages matter most
Your llms.txt file should be a curated overview, not a dump of every URL on your site. Include only your most important 20-50 links. AI models work better with focused, high-signal information than with exhaustive lists.
Whenever you launch a new product, publish a major piece of content, or change your pricing, update your llms.txt file to reflect the change. Stale files with broken links or outdated descriptions erode trust with AI retrieval systems.
The descriptions in your llms.txt should use the same language and positioning as your website, social media profiles, and directory listings. AI models build confidence through consistency. If your llms.txt says you are a "marketing analytics platform" but your homepage says "social media tool," the AI will be confused.
If you offer free tools, calculators, or utilities, make sure they are prominently featured in your llms.txt. Free tools are frequently cited in AI responses because they directly answer user questions like "Is there a free tool for X?"
Organize your links under clear H2 section headers. Use labels like "Documentation," "Blog Posts," "Free Tools," and "Product Features" rather than vague labels like "Resources" or "Links."
Including every page: Your llms.txt is not a sitemap. Curate the most important 20-50 links only
Missing descriptions: Links without descriptions force AI models to guess what the page contains. Always add a brief explanation
Broken links: Test all URLs before deploying. Broken links signal low quality to AI retrieval systems
Inconsistent brand messaging: Ensure your llms.txt descriptions match your website copy and external profiles
Forgetting to update: Treat your llms.txt as a living document, not a one-time setup. Review it monthly
Wrong file location: The file must be at the domain root (yourdomain.com/llms.txt), not in a subdirectory
Using HTML instead of Markdown: The standard calls for markdown formatting. Do not use HTML tags in your llms.txt file
After creating and deploying your llms.txt file, verify it works correctly:
Direct access test: Visit yourdomain.com/llms.txt in your browser. The file should render as plain text or markdown
Content type check: Use browser developer tools to verify the response header returns text/plain or text/markdown
Link validation: Click through every link in the file to ensure none are broken
Use Sourceable's LLMs.txt Generator: Our free tool can audit your existing llms.txt file and suggest improvements based on best practices
Monitor AI citations: After deploying, use a tool like Sourceable's Visibility Report to track whether your AI citation rate improves over the following weeks
The llms.txt file is a community-driven proposal that has gained significant adoption since late 2024. While it is not an official W3C standard, major AI platforms are actively supporting it, and its adoption is growing rapidly among SaaS companies, publishers, and e-commerce brands.
The llms.txt file is designed for AI search engines, not traditional Google search. It will not directly improve your Google rankings. However, improved AI visibility can indirectly boost your brand authority, which is a factor in Google's E-E-A-T evaluation.
Review and update your llms.txt file at least once a month, or whenever you make significant changes to your website, product, or content. Think of it as part of your regular content maintenance workflow.
Yes, and it is recommended. The llms.txt file serves as a concise overview with links, while llms-full.txt contains the complete text of your key documentation. AI models may use either or both depending on the query context.
No. These files serve completely different purposes. Robots.txt controls crawler access permissions. The llms.txt file provides context and understanding. You need both for a complete AI optimization strategy.
The llms.txt file is one of the simplest, highest-impact steps you can take to improve your brand's AI visibility. It takes less than an hour to create, costs nothing to deploy, and immediately gives AI models a better understanding of your brand.
If you want to get started quickly, use Sourceable's free LLMs.txt Generator tool. Enter your website URL and we will analyze your site structure and generate a complete, optimized llms.txt file ready for deployment.
The brands that make themselves easy for AI to understand will be the brands that AI recommends. Do not let your competitors get there first.
Continue reading our latest insights
Search Console now shows AI Overview impressions—but the biggest AI visibility story is still happening outside Google.
AI crawlers read millions of pages every day, but only a tiny fraction become citations. Here's how to improve your odds.