What is SlickBot?

SlickBot is Slickstream's web crawler that indexes customer websites to power engagement features like site search, content recommendations, and personalization for improved visitor experience on publisher sites. Agent Analytics can track when it visits your website.

Overview

Operated By Slickstream
Source Official Website
Expected To Follow Robots.txt Yes
Insights Last Updated August 11, 2026

Do you operate this agent? Contact us to suggest an update.

Category

Intelligence Gatherer
Analyzes web content for brand safety, competitive insights, and ad targeting

Expected Behavior

SlickBot usually returns to a small set of pages rather than crawling an entire site. Visits may follow an hourly, daily, or weekly schedule, while websites its clients do not monitor may never encounter it.

SlickBot's User Agent

User Agent SlickBot-News/1.0 (+https://slicksearchhq.com/hsw)

How To Block SlickBot With Robots.txt

Add this rule to your robots.txt file to block SlickBot from accessing your entire website, or use Automatic Robots.txt to block all intelligence gatherers at once. You can customize which pages are blocked by swapping out / for a different path.

User-agent: SlickBot # https://knownagents.com/agents/slickbot
Disallow: /

Global Statistics for SlickBot

As of August 11, 2026, this data reflects agent visits measured across thousands of websites using Agent Analytics, combined with daily scans of the top 1000 websites and their robots.txt files.

Robots.txt Blocked Percentage

0%
0% of top websites are blocking SlickBot
Learn How →

Country of Origin

Canada
SlickBot normally visits From Canada

Robots.txt Blocking Trend

0% of top websites block SlickBot in their robots.txt files.

Overall Intelligence Gatherer Traffic

3.5% of all web traffic came from intelligence gatherers.

Top Visited Website Categories

Food and Drink
Finance
News
Business and Industrial
Science

The types of websites most frequently visited by SlickBot.

Frequently Asked Questions

Should I Block SlickBot?

If you run ads, confirm SlickBot's purpose before blocking it. Some intelligence gatherers measure ad placements or evaluate pages for brand safety, and blocking those agents can interfere with advertising systems. If this one serves a different purpose, weigh any research visibility against sharing business or page-level data with third parties. Almost none of the top websites we track currently have robots.txt rules for SlickBot.


Does SlickBot Follow Robots.txt Rules?

Yes. SlickBot is expected to follow robots.txt directives, so a disallow rule is the appropriate first step. Automatic Robots.txt can add and maintain that rule, while Agent Analytics lets you verify whether SlickBot respects it.


Does SlickBot Access Private Content?

No special access. SlickBot can read publicly available pages but cannot bypass authentication. Assume anything you publish openly may be collected.


Why Is SlickBot Visiting My Website?

A client of Slickstream may be gathering competitive intelligence, measuring advertisements, or evaluating content for brand safety. The pages requested reflect that client's objective.


How Can I Tell if SlickBot Is Visiting My Website?

Agent Analytics tracks SlickBot visits in real time alongside every other known AI agent, crawler, and scraper. You can also check your server logs for requests whose user-agent string contains "SlickBot". Look for repeated requests to the same small group of commercial or ad-supported pages. Because SlickBot does not publish a verification method, any client can claim its identity and a log match is only a clue.