llms.txt generator
Enter a public site URL. The generator reads robots.txt and the sitemap, then writes an llms.txt file in the format from llmstxt.org.
- Free
- No signup
- Nothing stored
The file shows up here
You can edit either file before you copy or download it. llms.txt is the short index. llms-full.txt adds a short excerpt of each page.
How it works
- 01
Enter a site or paste URLs
Submit one public site URL, or switch to manual mode and paste up to 30 page URLs.
- 02
Read the sitemap
The server fetches robots.txt and sitemap.xml, including Sitemap entries in robots.txt. If those are missing, it collects links from the homepage.
- 03
Review and download
Titles and meta descriptions are grouped into sections. Edit the preview, then copy or download llms.txt and the optional llms-full.txt.
What the file is for
An agent should not have to scrape a whole site to learn which pages matter. llmstxt.org describes a markdown index at /llms.txt. The H1 names the site, the blockquote summarizes it, and each H2 is a list of links. The Optional section is for pages an agent can skip, such as privacy and terms.
Paste URLs instead when the site has no sitemap. The same protections apply: http and https only, a DNS check that blocks private and link-local addresses, a timeout, a size cap, and a limit of 30 pages. Edit the preview, then copy or download it. llms-full.txt is included when you want excerpts too. For the publishing side, see Next.js SEO for AI search.
FAQ
What is llms.txt?
llms.txt is a markdown file, usually at /llms.txt, that gives agents a short index of a site. The format at llmstxt.org asks for an H1, a blockquote summary, then H2 sections of markdown links. An Optional section holds links an agent can skip.
What is llms-full.txt?
llms-full.txt is an expanded companion. This generator fills it with the same pages plus a short plain-text excerpt of each one. The short index stays in llms.txt.
How are the sections chosen?
Pages are grouped from the URL path: docs, blog, API, product, and about each get a section. Privacy, terms, and account pages go under Optional, which the spec treats as secondary.
Can I edit the file before downloading it?
Yes. The preview is a text area. Change the copy, then copy it or download llms.txt and llms-full.txt.
Which pages get left out?
Only public http and https URLs are fetched. Private, loopback, and link-local addresses are refused. The generator keeps at most 30 pages, and it skips files that are not HTML pages.
