LLMs.txt for website (LLM, AI sitemap)

What this solution does

The 'LLMs.txt for Site' module prepares 1C-Bitrix resources for accurate representation in artificial intelligence systems and large language models. The solution generates llms.txt and llms-full.txt files and adds a dedicated section to the admin panel for configuration, preview, generation, and manual editing.

Administrators can create a compact llms.txt file containing key links and instructions, as well as an extended llms-full.txt file that includes selected static pages and infoblock data. Separate configuration profiles are supported for each file type. Files can be published to the site root or to a specific site directory.

Features include content preview before publication, manual entry of links with descriptions in the llms.txt specification format, import of initial links, and presets for stores, service sites, and corporate websites. Static pages, files, and infoblocks with sections and elements serve as content sources.

The module generates an llms.txt file to optimize site interaction with language models and AI agents. It allows flexible configuration of description sources for infoblock elements, offering choices between automatic detection, short previews, detailed text, or no description. Link filtering is performed via include/exclude rules and blacklists, as well as by excluding URLs containing unwanted query parameters. The system automatically removes tracking parameters such as utm_*, gclid, yclid, fbclid, and similar tags. Deduplication is based on canonical URLs, while quality checks analyze HTTP status codes, canonical tags, and noindex attributes. Mapping rules distribute links across semantic sections. Configuration includes limits on the number of links, sections, elements, and the total file size. The process supports automatic regeneration on a schedule via the 1C-Bitrix agent and instant file updates when connected infoblock elements or sections change. An integrated editor allows direct modification of published llms.txt and llms-full.txt files within the system.

The llms.txt file is a Markdown document designed to help language models and AI agents quickly understand site structure, the purpose of key pages, and content usage rules. The file content may include the site name and description, instructions for AI services, contact information, sections with key links, page descriptions, and additional Markdown sections.

The llms-full.txt file is an extended version of the standard llms.txt file. It is designed to include not only key links but also detailed site content: selected static pages, sections, infoblock elements, descriptions, and additional materials. This format is useful when providing language models with more comprehensive information about the resource.

The module allows quickly creating a basic file version, preparing an extended version with page and infoblock content, and flexibly managing the list of links to include or exclude. The tool structures links into logical sections: services, catalog, knowledge base, contacts, and legal pages. The file is automatically updated when site content changes, supports pre-publication link validation, and allows manual editing of the generated file for targeted adjustments.

The solution does not require manual creation or maintenance of llms.txt files. It is compatible with the multisite configuration of 1C-Bitrix and utilizes native platform mechanisms, including module settings, agents, and infoblock events. This enables quick startup using preset configurations with subsequent rule customization for specific site requirements.

The module provides control over data sources, filtering, and the final structure of the LLMs.txt file. Access is available via the admin panel under Services → LLMs.txt for site, or through Settings → Product Settings → Module Settings → LLMs.txt for site.

General iT can install and configure this module and check how it works on your website.