01.16.10

Lies and Denial of Service Attacks from Microsoft Bing

Posted in Deception, Google, Microsoft, Search at 5:56 am by Dr. Roy Schestowitz

Summary: Microsoft-rewarded companies hail Bing while it’s said to be down; crucial Perl services go down after MSNBOT goes out of control

EVERY firm that provides US-only statistics on search engines seems to be showing Microsoft's Bing on the decline (for December 2009). There is one exception. comScore, which is a Microsoft partner [1, 2, 3, 4, 5, 6, 7], contradicts the rest of the bunch, including Nielsen and Hitwise. This shows either that they are inaccurate or that there is a bias somewhere, favouring Microsoft at comScore (comScore is being paid by Microsoft and it cannot be trusted once its employees accept money from the very same company whose performance they assess in comparison to others’).

The only thing they all agree on is that Google continues to gain.

Nielsen, Hitwise, and comScore are all in agreement, so onlookers can perhaps declare it official now: Google had a good December. comScore, the member of the trio that most recently released its search market data, didn’t put Google up by a whole lot, though, and unlike the other firms, saw Bing gain ground.

The Bing-sponsored blog continues to show its bias and it says nothing about the fact that Microsoft is just buying some market share. Verizon is paid by Microsoft for example [1, 2]. So there are two issues here:

Microsoft paid comScore, which is now contradicting Nielsen and Hitwise when it claims Microsoft gains
Microsoft pays companies to remove Google as an option while blogs that it pays are claiming gains, describing that as success without mentioning those incentives (allegedly as much as $500,000,000 was paid to Verizon alone, in order to remove Google, despite customers’ preference)

It is worth adding that the same Microsoft-rewarded blog shares some Xbox 360 sales figures (US-only, even though this is not mentioned in the headline); these show Nintendo completely smashing Microsoft. Why not show global figures? Why not explicitly mention and emphasise that the figures refer only to one country? It is Microsoft’s home country where it battles against Japanese companies.

“MSNBOT must die!” says the Perl blog, which complains about Microsoft’s bad bots overwhelming the server due to shoddy code. Even CPAN was affected.

If you’ve suffered any problems accessing any of the sites, the databases, the CPAN mirror, etc from the CPAN Testers server last night, please direct your wrath at Microsoft. Last night the msnbot took out the CPAN Testers server with a dedicated denial of service attack. As a consequence measures are now being put in place to completely ban the msnbot from accessing at least the Reports site, and probably all the sites on the server.

Microsoft in their incompetent wisdom decided to unleash 20-30 bots every few seconds. I know this because I can see the IP addresses in the logs. The ones spotted within a few minutes of rebooting the server this morning to clear the processes were

Microsoft can’t even implement search right. An engine that’s estimated to handle just 3% of the world’s web search is causing far too much damage for far too little in return. █

Share this post: These icons link to social bookmarking sites where readers can share and discover new web pages.

Permalink Mail Send this to a friend

If you liked this post, consider subscribing to the RSS feed or join us now at the IRC channels.

Pages that cross-reference this one

4 Comments

williami said,

January 16, 2010 at 2:24 pm

Speaking of MSNBOT, Open Watcom (a free and enhanced version of the Watcom C, C++, and FORTRAN compilers), was affected too by MSNBOT:

http://cmeerw.org/blog/594.html

Evil M$…
Dennis Murczak said,

January 16, 2010 at 3:12 pm

I was wondering about those contradictions (yesterday I read about Bing declining, today about gains). Thanks for shining some light on this.

Roy Schestowitz Reply:
January 16th, 2010 at 3:39 pm
Money on the table changes everything, as Larry Lessig explained with many examples:

http://lessig.org/blog/2008/07/netroots_nation_keynote.html

Someone mailed me about this a few hours ago, saying:

Regarding the part of your article on the denial of service attack by the msnbot against some Perl-related sites, I find it entirely credible. I have a lot of images on my own Website and, several years ago, I noticed that the msnbot would often enough download all of the images with little or no pause. I felt that was exceptionally rude behaviour which no other search engine does. I put it down to Microsoft’s insensitivity and arrogance.

At first, I tried excluding them via robots.txt but the frequent hits from msnbot still seemed to come. Apparently, others have had this problem:

http://www.webmasterworld.com/forum93/411.htm
http://www.webmasterworld.com/msn_microsoft_search/3065473.htm

I then felt that my only recourse was to make a best effort to firewall Microsoft from accessing my site. I first tried to do the blocking narrowly but soon found out that their search engine bots run from many diverse IP ranges. I haven’t had this problem since I adopted a more broad-based block on Microsoft’s IP ranges. In a way, it’s regretable since perhaps some Microsoft employee might want to access my site for the same reason that anyone else would do but, in the end, it’s just karma.

Roy Schestowitz Reply:
January 16th, 2010 at 6:14 pm
Later he added:

I might also add that I did consider that my decision to block Microsoft search bots from my site could negatively impact my site’s traffic. I decided to go ahead with it since Live/Bing/<whatever Microsoft wants to call it today> has such a small portion of the search market, that any impact would be minimal.