Content Delivery

LLMify provides two ways to automatically serve Markdown instead of HTML:

1. Content Negotiation (autoServeMarkdown): Serves Markdown when a request contains an Accept: text/markdown header.
2. AI Bot Detection (enableBotDetection): Automatically detects known AI crawlers by their user agent and serves Markdown to them.

Both features are enabled by default and can be used independently or together.

Content Negotiation

When autoServeMarkdown is enabled, LLMify uses content negotiation to serve Markdown instead of HTML. If a request contains an Accept: text/markdown header, the same URL that normally returns HTML will return clean Markdown instead.

This approach is inspired by Cloudflare's Markdown for Agents. It allows AI agents to request your existing page URLs and receive structured Markdown without needing separate /raw/<slug>.md endpoints.

This works well in combination with the isRealUrlLlm setting. When both are enabled, llms.txt will point to your real page URLs, and AI agents can request those URLs with the Accept: text/markdown header to get the Markdown version.

AI Bot Detection

When enableBotDetection is enabled, LLMify detects known AI crawlers by their user agent and automatically serves Markdown to them. The following bots are detected by default:

GPTBot, ChatGPT-User, OAI-SearchBot, ClaudeBot, Claude-Web, Amazonbot, Bytespider, CCBot, Google-Extended, FacebookBot, PerplexityBot, Applebot-Extended, cohere-ai

You can add custom bot user agents via the additionalBotUserAgents config option.

Discovery Tag

When autoInjectDiscoveryTag is enabled (default), LLMify injects a <link> tag into the HTML head of every page that has Markdown available:

<link rel="alternate" type="text/markdown" href="https://your-site.com/raw/about.md">

This allows AI agents to discover the Markdown version of a page without needing to know about content negotiation.

How It Works

When a request hits a page:

1. LLMify checks if the request has an Accept: text/markdown header or if the user agent matches a known AI bot.
2. If either condition is met, it looks up the pre-generated Markdown for the requested entry.
3. If no pre-generated version exists, it generates the Markdown on-the-fly from the template output.
4. The response is returned with the appropriate headers.

Response Headers

All Markdown responses include the following headers:

Testing

You can test both auto-serve methods with curl:

curl -H "Accept: text/markdown" https://your-site.com/about

curl -A "GPTBot/1.0" https://your-site.com/about

curl -v -H "Accept: text/markdown" https://your-site.com/about 2>&1 | grep "< "

Blitz Integration

LLMify works with Blitz out of the box when using the default Blitz caching strategy. LLMify automatically tells Blitz to skip its cache for Accept: text/markdown requests, so the request is passed through to PHP where LLMify can handle it.

Blitz with Server Rewrites

If you use Blitz with Nginx server rewrites, Nginx serves cached pages directly from the file system without ever hitting PHP. In this case, LLMify cannot intercept the request.

To fix this, add a condition to your Nginx config that skips the Blitz cache when the Accept header contains text/markdown:

set $cache_path false;
if ($request_method = GET) {
    set $cache_path /cache/blitz/$host/$uri/index.html;
}
if ($args ~ "token=") {
    set $cache_path false;
}
# Skip Blitz cache for text/markdown requests (LLMify auto-serve)
if ($http_accept ~ "text/markdown") {
    set $cache_path false;
}

location / {
    try_files $cache_path $uri $uri/ /index.php?$query_string;
}