About Known Agents
The Company
Known Agents is a complete toolkit for harnessing the agentic web. It helps website owners track, control, and optimize for AI agents, crawlers, scrapers, and other automated traffic. It was founded in New York City by Gavin King, previously of Meta/Instagram, Vimeo, PayPal, and Intuit.
Known Agents began in 2023 as Dark Visitors, a side project documenting the eight known AI bots then on the web. Today, more than 5,000 websites use the platform. The company adopted the Known Agents name to reflect its broader mission of building a transparent, mutually beneficial agentic internet.
Our Mission
We are not an anti-bot company. Our mission is to accelerate the transition to an agentic internet where trusted AI traffic unlocks new levels of human value.
- Accelerate the agentic internet: The internet is inevitably transitioning to a place where more AI agents visit websites than humans.
- Build trust between humans and agents: Create a transparent ecosystem where humans and agents are easily able to self-identify.
- Turn agent traffic into human value: Maximize the productivity of humans operating AI agents, and the value for humans creating content that AI agents interact with.
What Known Agents Does
Agent and Bot Analytics
Agent Analytics identifies the AI agents, crawlers, and scrapers visiting a website. It shows traffic trends, page activity, session behavior, errors, and robots.txt violations for each agent.
AI Chat Referral Tracking
AI Chat Referral Tracking measures human visits from platforms such as ChatGPT, Claude, Gemini, and DeepSeek. It separates those referrals from automated traffic and shows which AI platforms and pages drive visits and conversions.
Agent Access Control
Automatic Robots.txt keeps crawler policies current as new agents emerge. Website owners can allow useful agents, restrict unwanted scrapers by category, and enforce those policies with supported integrations.
Agent Identification
The Agent Identification API identifies AI agents and other bots from incoming network requests and verifies supported identities. A public directory of thousands of agents and bots documents who operates each one, what it does, and how website owners can manage its access.
What Makes Known Agents Different
Purpose-Built for Automated Traffic
Traditional analytics platforms like Google Analytics filter out bot traffic. Known Agents makes that otherwise hidden activity visible by agent, category, page, and session.
Server-Side Data Collection
Known Agents collects data from websites, CDNs, and backends through WordPress, Cloudflare, Fastly, Akamai, AWS, Google Cloud, Vercel, Node.js, and REST API integrations. Server-side collection works when browser analytics are blocked and adds nothing to the human visitor's page load.
Analytics and Control in One Platform
Known Agents combines identification, analytics, AI referral attribution, and access controls. Website owners can support useful agents, restrict unwanted access, and make a different decision for each kind of automated visitor.
Who Uses Known Agents
Known Agents serves organizations across many industries, including:
- Publishers and creators measuring AI scraping, controlling content access, and tracking AI referrals
- Marketplaces and ecommerce teams tracking automated product discovery and AI-driven referrals
- SaaS and technology teams monitoring how agents read documentation and use their products
- Service businesses measuring visibility and referrals from AI platforms
- Universities and educational institutions managing access to research, courses, and other content
- Manufacturers tracking how agents access product information, documentation, and support resources
- Website owners and developers who need current agent data from the Agent Directory, an identification API, or automated robots.txt
- AI agent operators who want website owners to understand and trust their agents
How Known Agents Works
Connect a Website
Create a project, then follow the setup guide for your platform or connect through the REST API. Most websites can connect in less than five minutes without adding client-side code.
Review Agent Activity
Bot visits and human referrals from AI platforms appear separately in the dashboard. Teams can review activity by agent, category, page, and session to understand how automated visitors use their websites.
Set Policies and Optimize
Use the data to allow useful agents, restrict unwanted bots, improve pages for trusted agents, and grow AI-driven referrals. Automatic Robots.txt and supported enforcement integrations keep access policies current.
Get Support
Documentation and email support are available to all customers. Paid plans include priority support, and Enterprise customers receive dedicated live support. Contact the team at contact@knownagents.com.
Key Facts
| Company Name | Known Agents |
|---|---|
| Type | Agentic web software platform |
| Founded | 2023, originally as Dark Visitors |
| Founder | Gavin King |
| Headquarters | New York City, United States |
| Website | knownagents.com |
| Core Offering | A toolkit for tracking, controlling, and optimizing for AI agents and bots |
| Pricing | Free plan, paid Startup and Growth plans, and custom Enterprise pricing. See current pricing. |
| Contract Terms | Self-service monthly or yearly subscriptions, with custom terms for Enterprise customers |
| Services | Agent Analytics, AI Chat Referral Tracking, Automatic Robots.txt, and Agent Identification API |
| Communication | Documentation, email support, priority support on paid plans, and dedicated live support for Enterprise customers |
| Customers Served | 5,000+ websites |
| Social | LinkedIn, X, and GitHub |
Frequently Asked Questions
Is Known Agents free?
Yes. The Free plan provides visibility into bot traffic with no credit card required. Paid plans add higher monthly event limits, more projects, CSV exports, API access, and priority support. See the pricing page for current details.
Which AI agents and bots does Known Agents track?
Known Agents tracks thousands of known AI agents, web crawlers, and scrapers using user-agent strings and other request signals. This includes GPTBot, ClaudeBot, PerplexityBot, Googlebot, Bytespider, and CCBot. The complete list is available in the Agent Directory.
Why should I care about bot and AI agent traffic?
Automated traffic affects content access, AI visibility, infrastructure costs, and the experience of customers using agents. Known Agents research has found that about half of website traffic is automated, but traditional analytics platforms do not show most of it.
Can Known Agents block scrapers and protect content?
Yes. Automatic Robots.txt can restrict categories such as AI training crawlers, SEO bots, and unauthorized scrapers, then update those rules as new agents emerge. Supported integrations can also enforce those policies against bots that ignore robots.txt.
What is an AI chat referral?
An AI chat referral is a human visit that follows a link or recommendation from a service such as ChatGPT, Claude, Gemini, or DeepSeek. Known Agents measures these visitors separately from automated bot traffic so teams can understand which AI platforms drive traffic and conversions.
Does Known Agents affect human visitors or site performance?
Known Agents uses server-side integrations and adds nothing to the front-end page load. Blocking integrations can also reduce the work a website performs for unwanted bots without affecting human visitors.
Is website data private and secure?
Known Agents collects the request metadata needed to identify and analyze agent and bot activity, including user-agent strings, IP addresses, timestamps, and URLs. The Privacy Policy explains how that data is handled. Known Agents does not sell customer data to third parties.
How do I add an agent to the Agent Directory?
Email the team with details about the agent and its operator. Agent Directory listings are free and help website owners understand an agent's purpose before deciding whether to allow it.