What is Codex?
Codex is an AI coding agent by OpenAI that helps developers write, review, debug, and ship code. You can set up Agent Analytics to see when Codex visits your website.
Category
| Example User Agent | Codex news-rss-source-router/0.1 (+public RSS validation) |
| Operated By | OpenAI |
| Source | Official Website |
| Triggered By | User action |
| Used for AI Model Training | Not expected |
| Affects AI Chat & Search Visibility | Possibly |
| Expected To Follow Robots.txt | Yes |
Why Is Codex Visiting My Website?
A developer is working with a product or technology you document. Codex retrieved your documentation, API reference, or examples because they were relevant to the code being written.
Should I Block Codex?
Allow Codex if developers rely on your documentation or technical content. Blocking it can make your product harder to build with but does not affect traditional search rankings. For context, 7% of the top websites we track currently have robots.txt rules for Codex.
How Do I Block Codex Using Robots.txt?
Add this rule to your robots.txt file to block Codex from accessing your website, or use Automatic Robots.txt to block all AI coding agents at once. You can customize which pages are blocked by swapping out / for a different path.
User-agent: Codex # https://knownagents.com/agents/codex
Disallow: /Insights for Codex
As of November 9, 2026, this data reflects agent visits measured across thousands of websites using Agent Analytics, combined with daily scans of the top 1000 websites and their robots.txt files.
Robots.txt Compliance
Country of Origin
Robots.txt Blocked Percentage
Frequently Asked Questions
What Traffic Patterns Should I Expect From Codex?
Codex typically makes several requests within seconds. A single developer leaves quiet gaps, while overlapping sessions can produce steady traffic. Activity comes in short, targeted sessions rather than broad site-wide crawls.
Does Codex Follow Robots.txt Rules?
Codex is expected to follow robots.txt rules, so a disallow rule is the appropriate first step if you want to block it. Automatic Robots.txt can add and maintain that rule, while Agent Analytics lets you verify whether Codex respects it.
Is Codex Used for AI Model Training?
Not ordinarily. Codex is not categorized as an AI training scraper or data provider, so AI model training is not its expected purpose.
Does Blocking Codex Affect AI Search or Citations?
Blocking Codex can keep your documentation out of answers it gives developers. It does not directly affect general AI search indexes or traditional search rankings.
Does Codex Access Private Content?
Not unless a developer explicitly provides credentials. Codex otherwise reads only public documentation, references, and code examples.
How Can I Verify a Request From Codex?
Agent Analytics tracks Codex visits in real time. You can also check your server logs for requests whose user-agent string contains "Codex". Look for clustered requests across documentation, API reference, and code example pages. Because Codex does not publish a verification method, any client can claim its identity and a log match is only a clue.
How Is Codex Different From Other OpenAI Agents?
OpenAI operates separate agents for different purposes, so each agent should be evaluated and controlled independently.
- ChatGPT Agent: ChatGPT Agent is an AI agent created by OpenAI that can use a web browser. It can intelligently navigate and interact with websites to complete multi-step tasks on behalf…
- Dot: Dot is an OpenAI AI agent that uses a cloud browser to research websites and carry out ongoing tasks for users.
- ChatGPT-User: ChatGPT-User is OpenAI's web crawler that visits websites when ChatGPT users request information. This enables ChatGPT to include links in its responses.
- GPTBot: GPTBot is OpenAI's web crawler that collects public web content used to develop and improve its generative AI models.
- OAI-SearchBot: OAI-SearchBot is OpenAI's web crawler that indexes websites for SearchGPT, collecting and analyzing web content to power AI-driven search results and real-time informatio…