> ## Documentation Index
> Fetch the complete documentation index at: https://docs.llmtag.org/llms.txt
> Use this file to discover all available pages before exploring further.

# Directives Reference

> Complete reference for all LLMTAG protocol directives and their usage

## Core Directives

### `spec_version`

**Required.** Declares the specification version being used.

```txt theme={null}
# REQUIRED: Protocol version declaration
spec_version: 3.0
```

<ParamField name="spec_version" type="string" required>
  The version number of the LLMTAG specification this file follows. Must be exactly "3.0" for current implementations.

  **Purpose:** This directive tells AI agents which version of the LLMTAG protocol to use when interpreting your policy file. It ensures compatibility and proper parsing of directives.

  **Values:**

  * `3.0` - Current LLMTAG Protocol version (required)

  **Example:**

  ```txt theme={null}
  # REQUIRED: Protocol version declaration
  spec_version: 3.0
  ```
</ParamField>

<Warning>
  All `llmtag.txt` files must include this directive. Files without it are considered invalid and will be ignored by compliant AI agents.
</Warning>

## Content Usage Directives

### `ai_training_data`

Controls whether content can be used as training data for machine learning models.

```txt theme={null}
# AI Training Policy: Controls whether content can be used for AI model training
# Values: allow (permit training) | disallow (block training)
ai_training_data: disallow
```

<ParamField name="ai_training_data" type="string">
  Controls the use of content for AI training purposes.

  **Purpose:** This is the most critical directive for most publishers, as it directly controls whether their content can be used to train AI models. This affects how AI systems learn and generate responses.

  **Values:**

  * `allow` - Content may be used for training machine learning models
  * `disallow` - Content may not be used for training machine learning models

  **Default:** `allow` (if not specified)

  **Examples:**

  ```txt theme={null}
  # Block AI training to protect content
  ai_training_data: disallow

  # Allow AI training for educational content
  ai_training_data: allow
  ```
</ParamField>

<Info>
  This is the most critical directive for most publishers, as it directly controls whether their content can be used to train AI models.
</Info>

**Examples:**

```txt theme={null}
# Block all AI training
ai_training_data: disallow

# Allow AI training
ai_training_data: allow
```

### `ai_use`

Controls specific AI applications and use cases beyond training.

```txt theme={null}
# AI Use Policy: Defines how AI agents can use your content
# Values: search_indexing (for search engines) | generative_synthesis (for AI responses) | research (for academic research)
ai_use: search_indexing, generative_synthesis
```

<ParamField name="ai_use" type="array">
  Comma-separated list of allowed AI use cases. Each value represents a specific way AI can use your content.

  **Purpose:** This directive controls how AI agents can use your content beyond just training. It allows fine-grained control over different AI applications.

  **Available Values:**

  * `search_indexing` - Traditional search engine indexing and ranking
  * `generative_synthesis` - Generating direct answers, summaries, or new content
  * `commercial_products` - Use within paid AI features or commercial products
  * `research` - Academic or non-commercial research purposes
  * `personal_assistance` - Personal AI assistants and chatbots

  **Default:** `search_indexing` (if not specified)

  **Examples:**

  ```txt theme={null}
  # Allow only search indexing
  ai_use: search_indexing

  # Allow multiple use cases
  ai_use: search_indexing, generative_synthesis, research

  # Allow all use cases
  ai_use: search_indexing, generative_synthesis, commercial_products, research, personal_assistance
  ```
</ParamField>

**Examples:**

```txt theme={null}
# Allow only search indexing
ai_use: search_indexing

# Allow multiple use cases
ai_use: search_indexing, generative_synthesis, research

# Allow all use cases
ai_use: search_indexing, generative_synthesis, commercial_products, research, personal_assistance
```

## Advanced Directives

### `verification_challenge`

Establishes a cryptographic handshake to verify that an AI agent has actually read and understood the rules.

```txt theme={null}
verification_challenge: sha256:abc123def456...
```

<ParamField name="verification_challenge" type="string">
  Optional cryptographic challenge for advanced verification mechanisms.

  **Format:** `[algorithm]:[hash_value]`

  **Supported Algorithms:**

  * `sha256` - SHA-256 hash
  * `sha1` - SHA-1 hash (deprecated)

  **Default:** Not specified (no verification required)
</ParamField>

<Note>
  This is an advanced feature for publishers who want to implement verification mechanisms. Most implementations can ignore this directive.
</Note>

**Example:**

```txt theme={null}
# Require SHA-256 verification
verification_challenge: sha256:a1b2c3d4e5f6789012345678901234567890abcdef1234567890abcdef123456
```

## Scope Directives

### `User-agent`

Defines a scope block for specific AI agents or crawlers.

```txt theme={null}
User-agent: GPTBot
ai_training_data: allow
ai_use: search_indexing, generative_synthesis
```

<ParamField name="User-agent" type="string">
  The user-agent string of the AI agent or crawler. This creates a scope block where subsequent directives apply only to that specific agent.

  **Rules:**

  * Case-insensitive matching
  * Must match the exact user-agent string reported by the agent
  * Use the full user-agent string, not partial matches
</ParamField>

**Common AI Agent User-Agents:**

```txt theme={null}
# OpenAI's GPTBot
User-agent: GPTBot

# ChatGPT-User (when users browse with ChatGPT)
User-agent: ChatGPT-User

# Google's AI crawlers
User-agent: Google-Extended

# Anthropic's Claude
User-agent: Claude-Web

# Perplexity AI
User-agent: PerplexityBot
```

### `Path`

Defines a scope block for specific URL paths or patterns.

```txt theme={null}
Path: /premium/
ai_training_data: disallow
ai_use: search_indexing
```

<ParamField name="Path" type="string">
  The URL path pattern to which subsequent directives apply. This creates a scope block for content under the specified path.

  **Rules:**

  * Use forward slashes (`/`) as path separators
  * Trailing slashes are optional
  * Path matching is prefix-based (e.g., `/blog/` matches `/blog/post-1/`)
  * Wildcards are not supported in v3.0
  * Case-sensitive matching
</ParamField>

**Examples:**

```txt theme={null}
# Protect premium content
Path: /premium/
ai_training_data: disallow

# Allow AI training for blog content
Path: /blog/
ai_training_data: allow
ai_use: search_indexing, generative_synthesis

# Protect user-generated content
Path: /user-content/
ai_training_data: disallow
ai_use: search_indexing
```

## Directive Processing

### Precedence Order

Directives are processed in the following order of precedence (highest to lowest):

1. **Path-specific directives** - Most specific
2. **User-agent specific directives** - Agent-specific
3. **Global directives** - Least specific

### Inheritance

When a directive is not specified at a more specific level, it inherits from the global level:

```txt theme={null}
# Global policy
spec_version: 3.0
ai_training_data: disallow
ai_use: search_indexing

# This agent inherits global policy but overrides ai_use
User-agent: ResearchBot
ai_use: research
# ai_training_data remains "disallow" from global

# This path inherits from global but overrides ai_training_data
Path: /public-research/
ai_training_data: allow
# ai_use remains "search_indexing" from global
```

### Default Values

If no directive is specified at any level, these defaults apply:

* `ai_training_data: allow`
* `ai_use: search_indexing`

## Comments

Use `#` to add comments to your `llmtag.txt` file:

```txt theme={null}
# Global policy: No AI training by default
spec_version: 3.0
ai_training_data: disallow
ai_use: search_indexing

# Allow research use for specific agents
User-agent: AcademicBot
ai_training_data: allow
ai_use: research

# Protect premium content from all AI training
Path: /premium/
ai_training_data: disallow
ai_use: search_indexing
```

<Info>
  Comments are ignored by AI agents and are purely for human readability and documentation purposes.
</Info>

## Best Practices

### 1. Start Simple

Begin with a basic global policy and add complexity as needed:

```txt theme={null}
spec_version: 3.0
ai_training_data: disallow
ai_use: search_indexing
```

### 2. Use Clear Comments

Document your policies for future reference:

```txt theme={null}
# Block AI training but allow search indexing
spec_version: 3.0
ai_training_data: disallow
ai_use: search_indexing

# Exception: Allow research bots to use content for training
User-agent: ResearchBot
ai_training_data: allow
ai_use: research
```

### 3. Test Your Implementation

Always verify your `llmtag.txt` file is accessible and properly formatted:

```bash theme={null}
curl https://yourdomain.com/llmtag.txt
```

### 4. Keep It Maintainable

Use consistent formatting and logical grouping:

```txt theme={null}
spec_version: 3.0

# Global policy
ai_training_data: disallow
ai_use: search_indexing

# Agent-specific policies
User-agent: GPTBot
ai_training_data: allow
ai_use: search_indexing, generative_synthesis

User-agent: ResearchBot
ai_training_data: allow
ai_use: research

# Path-specific policies
Path: /premium/
ai_training_data: disallow
ai_use: search_indexing

Path: /blog/
ai_training_data: allow
ai_use: search_indexing, generative_synthesis
```