← Back

Generate a User Agent Token List With the REST API

Request

Call this endpoint to get all known user agent tokens for agents in the requested categories.

Endpoint

HTTP Method POST
URL https://api.knownagents.com/user-agent-token-lists

Headers

Header Value
Authorization A bearer token with your project's access token (e.g. Bearer YOUR_ACCESS_TOKEN). You can get your project's access token by navigating to the Known Agents Projects page, opening your project, and opening its settings page.
Content-Type Set to application/json.

Body

Field Type Required Description
agent_types string[] Yes An array of agent types whose user agent tokens should be included. Allowed agent types include:
  • "AI Agent"
  • "AI Assistant"
  • "AI Coding Agent"
  • "AI Data Provider"
  • "AI Data Scraper"
  • "AI Search Crawler"
  • "Archiver"
  • "Developer Helper"
  • "Fetcher"
  • "Automated Agent"
  • "Intelligence Gatherer"
  • "Scraper"
  • "SEO Crawler"
  • "Search Engine Crawler"
  • "Security Scanner"
  • "Undocumented AI Agent"
  • "Uncategorized"
excluded_agent_tokens string[] No An array of user agent tokens that should be excluded.
excluded_operator_names string[] No An array of operator names whose user agent tokens should be excluded.

Example

curl -X POST https://api.knownagents.com/user-agent-token-lists \
-H "Authorization: Bearer YOUR_ACCESS_TOKEN" \
-H "Content-Type: application/json" \
-d '{
    "agent_types": [
        "AI Data Scraper",
        "AI Data Provider",
        "Scraper"
    ]
}'

Response

The endpoint returns an array of user agent tokens in application/json format.

Example

[
    "GPTBot",
    "ClaudeBot",
    "meta-externalagent",
    "CCBot",
    "Scrapy",
    "Bytespider",
    "cohere-ai",
    "Diffbot",
    "omgilibot",
    "ImagesiftBot",
    ...
]