Yoast has introduced support for llms.txt in both the free and premium versions of its popular WordPress SEO plugin. This update aims to help websites prepare for the growing influence of artificial intelligence (AI) and large language models (LLMs) like ChatGPT and Google Gemini in how content is discovered and indexed.
What Is llms.txt?
The llms.txt file is a proposed standard designed to help LLMs more easily access and understand a website’s core content. The idea is similar to robots.txt, but instead of guiding search engine crawlers, it helps AI systems locate clean, ad-free versions of web pages—typically in markdown format (.md). These simplified duplicates are meant to present only the main content, without distractions aimed at human readers.
However, there are concerns about the reliability of this approach. Google’s John Mueller has pointed out that since these files can be manually edited, they could be manipulated by unethical SEOs to mislead AI models. Additionally, most LLMs today don’t actively look for or use llms.txt files, and it remains uncertain whether they will adopt this method in the future.
How Yoast SEO Implements llms.txt
Despite the debate around its effectiveness, Yoast has integrated llms.txt support directly into its plugin with the following benefits:
- Improved AI Understanding : Helps tools like ChatGPT and Gemini identify and interpret your most relevant content.
- Automatic Content Highlighting : Yoast SEO automatically detects and includes your most important and recently updated pages.
- No Technical Setup Needed : The llms.txt file is generated and refreshed weekly—no coding or manual configuration required.
- Future-Ready Websites : Positions your site to be compatible with emerging AI-driven search behaviors.
- Free and Accessible : Available with one click in both the free and premium versions of Yoast SEO.
Yoast emphasizes that this feature is an optional tool to help publishers stay ahead of evolving trends in AI and search, without requiring any additional effort from users.