Become an affiliateStart your program

Technology and tracking

Bot filter

Also called bot filtering, bot detection, crawler filter, invalid traffic filter.

Short answer

A bot filter is the check that rejects automated traffic before it is counted as a click. It recognises known crawlers, preview bots and scanners by user agent, origin and behaviour, so statistics and commission rest on real people.

Definition of Bot filter.

Automated traffic falls into three groups. Search engine crawlers follow links to index pages. Preview bots fetch a page every time a link is shared in a chat, an email or on social media, so an image and a title can be shown. Security scanners and price robots call pages systematically. None of them are customers, and none of them buy anything.

A bot filter combines several signals. The user agent string reveals the crawlers that state their identity. The IP address and its ownership reveal traffic from data centres rather than from broadband and mobile networks. Behaviour reveals the rest: no JavaScript, no images fetched, calls in too rapid succession, no mouse movement. The more signals point the same way, the safer the rejection.

The effect is a measurement effect. Every bot click that slips through is added to the denominator of the conversion rate and of EPC without ever being able to contribute to the numerator. An affiliate link shared in a large group chat can therefore appear to have had many clicks and no effect, even though none of the clicks were a person.

What a bot click costs when you do not pay per click

In a CPS model a bot click costs no commission, but it still has a cost. It dilutes the conversion rate, so a good traffic source looks worse than it is. It distorts A/B tests, because the noise is spread unevenly between the variants. And it leads the store to misjudge an affiliate who is actually delivering.

For the affiliate the damage runs the other way: clicks without traffic. If the dashboard shows 400 clicks while the affiliate's own analytics show 60 sessions, the difference may be down to automation – and without a filter there is no way of knowing which of the two figures to act on.

What a bot filter is not

A bot filter is not fraud protection. It recognises automated agents that behave like automated agents. A deliberate fraudster uses an ordinary browser, an ordinary IP and a person or a well-equipped headless browser to click – and that kind of thing is not caught at the door.

What catches the rest sits after the click: deduplication on order ID, a window in which the store can reject a sale with a reason, and an event log you can go back through. The bot filter cleans up the noise, so what is left is worth looking at.

How it goes wrong

  • 01

    The filter casts too wide a net

    Block entire IP ranges and real visitors go with them – mobile users behind carrier-grade NAT and everyone on a VPN. A customer rejected as a bot loses their attribution, and the affiliate loses a commission they had earned.

  • 02

    Preview bots counted as clicks

    Every time an affiliate link is shared in a chat or an email, the sending platform's server fetches the page to build a preview. Without a filter this is registered as a click, and the affiliate sees clicks that never became visits.

  • 03

    The signature list is never updated

    New crawlers keep appearing, and old ones change their user agent. A filter that was written once and never touched since still catches yesterday's bots and lets the rest in.

At Affilyflow

Bot filter in practice

Known bots are not counted on Affilyflow. If a recognised crawler hits an affiliate link, no click is registered and no attribution is created – the visit does not become a click that later has to be explained away in the statistics. Bot traffic is logged separately and can be seen in the store's Analytics.

Every tracking event is logged, including those that did not become a sale. That makes it possible to see what actually hit a link, instead of reasoning backwards from a total that does not add up.

Frequently asked questions

How can you tell whether a click came from a bot?

By comparing several signals. The user agent string identifies the crawlers that say who they are. The IP address reveals whether the call came from a data centre rather than an ordinary network. Behaviour reveals the rest: no JavaScript run, no images fetched, calls in too rapid succession. A single signal is rarely enough.

Does Google's crawler count as a click on my affiliate link?

Not if it is filtered out. Search engine crawlers state their identity in the user agent and come from known IP ranges, so they are among the easiest to recognise. Without a filter, every crawl of your page would add clicks to the statistics that nobody made.

Can a bot filter block real visitors?

Yes, if it is configured too aggressively. Broad IP blocks hit mobile users behind shared NAT and everyone on a VPN. The consequence is not just a missing registration but a lost attribution for a purchase that actually happened. That is why filters should rely on several signals rather than on IP alone.

Does a bot filter protect against click fraud?

Only against the most primitive part of it. The filter recognises automated agents that behave like automated agents, not a person or a script deliberately imitating an ordinary browser. That kind of thing is caught after the click with deduplication, a rejection window for the store and an event log that can be reviewed.

Read on