Set Up Automatic Robots.txt for Your WordPress Website

Step 1: Install the Plugin

• Navigate to the Known Agents Projects page and open your project
• Copy your access token from the Settings page
• Log in to your website's WordPress dashboard
• Click Plugins in the sidebar
• Search for Known Agents (Plugin Directory)
• Click Install Now
• Click Activate

Step 2: Block Agents by Category

• Click Known Agents in the sidebar
• Paste in your access token
• Select the agent types you want to block. Allowed categories include:
  • AI Agent
  • AI Assistant
  • AI Coding Agent
  • AI Data Provider
  • AI Data Scraper
  • AI Search Crawler
  • Archiver
  • Developer Helper
  • Fetcher
  • Automated Agent
  • Intelligence Gatherer
  • Scraper
  • SEO Crawler
  • Search Engine Crawler
  • Security Scanner
  • Undocumented AI Agent
  • Uncategorized
• Click Save Changes

Step 3: Test Your Integration

If your website is correctly connected, you should see the new rules in your website's robots.txt.