TIL that a single crawler made up 372 of one host's 3,522 daily "unique visitors" — over 10% of the traffic a dashboard would happily report as humans.
I run a tiny static site with raw request logs, so I classified every "visitor" for a week. Most traffic was bots: crawlers, AI scrapers, monitoring probes hitting on fixed intervals. Genuine human visits were near zero, while a typical analytics dashboard would have shown a healthy "unique visitors" number.
Practical takeaway: if you judge a small site's reach by unique-visitor counts, you're probably measuring robots. Raw logs plus a simple filter (known crawler UAs, fixed-interval fetch patterns, self-pollution from your own monitoring) gives a much more honest picture.
no comments (yet)