LLMs.txt Lookup
Fetch and view a site’s /llms.txt file.
Enter a value above to run LLMs.txt Lookup
e.g. example.com
Quick answer
LLMs.txt Lookup fetches a site's /llms.txt file and shows its contents, revealing the curated links and guidance the site offers to AI models. If the file is missing, the site has not published an llms.txt.
LLMs.txt Lookup fetches the /llms.txt file from any domain and shows you its raw contents. llms.txt is a Markdown file some sites publish at their web root to give large language models a clean, curated map of their most important pages — documentation, API references, key articles — without the navigation, ads, and JavaScript that clutter normal HTML. Because the file is meant to be a public resource, this tool retrieves it server-side, so cross-origin browser restrictions never block the request. Use it to confirm your own file is live, or to inspect how another site exposes content to AI crawlers.
How it works
Enter a domain and the tool requests https://yourdomain/llms.txt over HTTPS from our server, then returns the response body verbatim. Running the fetch server-side sidesteps the CORS rules that would stop a browser from reading another origin's file directly. A successful lookup shows the plain Markdown text; a missing file returns a 404, which simply means the site has not published one. The content is displayed as-is, so you see exactly what an LLM crawler would receive.
How to read the results
A well-formed llms.txt opens with an H1 project name, an optional blockquote summarising the site, then H2 sections holding bullet lists of links — often pointing to Markdown versions of each page. Scan the headings to see what the author considered essential. Some sites also publish llms-full.txt, which inlines the full text instead of just linking to it. If the response looks like an HTML page rather than Markdown, the server likely returned a soft-404 or redirected to a normal page.
Common problems & fixes
A 404 means no file exists at the root — it must sit at /llms.txt, not in a subfolder. If you see your homepage HTML instead of Markdown, a catch-all route or SPA is intercepting the path; serve the file as a static asset with a text/plain or text/markdown content type. Remember that llms.txt does not control crawler access — that is robots.txt. It only offers content, and crawlers are free to ignore it, since it is a community proposal rather than an enforced standard.
Frequently asked questions
- Is llms.txt the same as robots.txt?
- No. robots.txt tells crawlers which paths they may or may not access, and is a long-standing web convention. llms.txt does the opposite job: it hands language models a curated list of your best content in clean Markdown so they can understand your site more accurately. One restricts access; the other invites and guides it. Publishing both is common and they do not conflict.
- Do AI systems actually read llms.txt?
- Adoption is early. The format was proposed in 2024 and is a voluntary community convention, not a ratified standard, so no crawler is required to fetch it. Some documentation platforms and AI tools now look for it, but many do not. Publishing one is low-cost and may help, but treat it as a useful hint rather than a guarantee that any model will use it.
- Where should the file live and what should it contain?
- Place it at your domain root, exactly /llms.txt, served as plain text or Markdown. Start with an H1 title, a short blockquote summary, then H2 sections listing links to your key pages — ideally to Markdown versions. Keep it concise and curated rather than a full sitemap. For sites that want to inline everything, add a separate llms-full.txt with the complete content.