Huge Spike in ‘Bot’ Traffic

BotTrafficWhen I checked my site statistics this morning I noticed that the bot (Robot) traffic over the last two days is up about 15x to 20x the normal—not that there really is a ‘normal’ count for bot traffic.

A very small number of these bots are good bots. Like the Microsoft Bing and Google indexing bots that crawl through your site in order to index it for searches.

There are also legitimate bots that are collecting statistics about Web content and Web sites, or are maintenance bots used by your hosting service to check for inconsistences and errors in the page code, or checking for unresolvable hyperlinks, so they can be reported to you for fixing.

But the vast majority of bots these days are not out to do any good. They have been developed and deployed by people either trying to cause some mischief (by tampering with your site, if they are able to depending on the security of your site) or trying to make money somehow.

Two favourite pastimes of bots are: (1) to make postings into your site, and (2) to harvest e-mail addresses.

Postings into your site:  Bots make postings into your site, either into the main pages of your site or into the comments or forum areas, usually in order to subtly advertise something. Hoping then that legitimate visitors to your site will see these (extremely interesting) inserts or comments and, out of curiosity, will go and visit the service or other Web site referenced.

Various site hosting services use various tricks to stop bots from doing this. One of the most common is to ask the ‘human’ making the post to decode a blurred or twisted code-sequence and to then enter it correctly. Something that a bot, at this stage anyway, is unable to do.

Harvesting e-mail address:  Bots trawl sites trying to discover e-mail addresses. The people controlling the bots then use these e-mail addresses to send out SPAM, generally with the aim of somehow making money—but sometimes just for the pure inconvenience it causes.

For this reason it is generally a bad idea to include an in-page textual e-mail address anywhere in a site or in any comments you might post to a site. This does not include e-mails addresses you enter that do not end up in-page, such as those used for validation, logon, or when completing a shopping cart.

But back to the original reason I started keying this post. The bot hits on my site have skyrocketed in the last two days. I am wondering why? Is there a generally massive increase in bot activity going on or is my site being targeted in some way? I have done a couple of Google searches to see if anyone else is reporting huge increases in bot activity but there is nothing obvious being reported that I can see.

I guess all I can do is monitor it and see what happens over the next few days.