05.19.09

DDOS and Migration (Updated)

Posted in Boycott Novell, Site News at 7:33 pm by Dr. Roy Schestowitz

Summary: Boycott Novell had been under DDOS attacks for almost 4 days. We were struggling to just stay online while hosts investigated where the attacks came from. We moved between hosts (to semi-dedicated) and the same pattern of attack persisted until yesterday.

WE have kept silent about it in order not to encourage the attacker/s, but it’s true. We have been under heavy DDOS attacks since Thursday night. What has happened since then? Well, a lot. Our previous host is no more as far as we are concerned. After struggling with the botnet for like 10 hours (filtering to no avail) our Web site got isolated. It did not serve any pages for almost 2 days. A reader of Boycott Novell was kind enough to lend us room on his server (more or less dedicated), on which he fought the botnets for over a day. The attackers kept changing tactics. Some other readers offered filtering advice and we are grateful to all of them. Ultimately, the attacks halted yesterday afternoon.

“Ultimately, the attacks halted yesterday afternoon.”The migration from the old server was not simple because the site was disabled abruptly following the early attacks. But now we have ensured that all data has been migrated. The only ‘good’ thing which came out of this attack is that, as oiaohm put it, the ordeal sort of made us more robust to future attacks.

Now that we have a new host in place, we also have more features. Data on the site (comments, posts, etc.) was not lost in the migration, just heaps of time and effort affecting several people. We have moved to a bigger, more robust environment that will hopefully facilitate the needs of the Web site as it continues to grow (we served about 200GB of data last month). We apologise for the downtime, which is unprecedented.

The plan is to carry on exposing Comes exhibits next month and also organise the Wiki. There is enough for years of work.

Again: we would like to thank all those who helped during the downtime and especially our generous reader ( Copilotco) who offered to host the Web site, taking us away from shared hosting in the process. Dedicated servers on normal Web hosts are just far too expensive for us to afford and I swear that I never made a single dime from this Web site. The ads merely covered the hosting fees which Shane has been paying since 2006.

One last clarification for lunatics who are now suggesting that we DDOSed ourselves, where to even begin refuting such nonsense (coupled with personal abuse)?

The attacks came from many addresses, for example 88.198.60.8 which is “tor-proxy.va6.de”. Multiple such IPs hit us constantly and relentlessly (all tor exit nodes at first). At one stage it seemed like the front page alone received 3 page requests per second. But the IPs were also doing a HEAD on the Web site as many times as possible, bringing the server down to its knees (both the old server and new server, the former running Red Hat and the latter CentOS).

Update: Here is a report from the administrator.

I took over hosting of boycottnovell.com for Roy in the middle of the DDOS attack. I am looking at the squid log for boycottnovell.com during the DDOS. I have squid caching/proxying/url-rewriting for apache for various reasons.

The attack initially (or at least, at the time the DNS was re-pointed to my server) consisted of lots of HEAD requests. Then I wrote up a script to tail the log finding anyone doing lots of HEAD requests and putting the offending IP into the iptables packet filter while I cooked up a more permanent solution. Eventually they figured this out and switched to a full on GET of the root of the site and then I think they started getting random pages from the site as fast as they could although I’m not sure about that.

The interesting part starts around timestamp 1242543590.804 which is apparently when most of the world’s DNS cut over to me including that of the machines in whatever bot net was employed in the attack.

If we run this command on the logfile with the logfile being /tmp/bn.log:


grep " HEAD http://boycottnovell.com/ " /tmp/bn.log | awk
'{print $3}' | sort | uniq -c |sort -n | tail -10

we get:

   2716 81.175.61.4
   2960 212.24.147.228
   3056 204.209.56.56
   5637 87.236.199.73
   6645 145.100.100.190
   7261 212.42.236.140
   8487 88.198.14.120
   9640 62.141.58.13
  11008 87.118.104.203
  11269 88.198.60.8

and if we do:


grep " GET http://boycottnovell.com/ " /tmp/bn.log |
awk '{print $3}' | sort | uniq -c |sort -n|tail -10

we get:


   5801 94.136.16.242
   5854 85.25.152.185
   5865 212.24.147.228
   6367 66.35.1.170
   6682 205.209.142.210
   6977 87.118.104.203
   8102 83.140.125.188
   8300 85.25.145.98
   8441 212.42.236.140
  20065 66.230.230.230

So one IP did a get of the root of the site 20k times before I really effectively got everything blocked off and another did a HEAD around 11k times. You can get a feel for how the attack progressed using:


egrep ' GET http://boycottnovell.com/ | HEAD
http://boycottnovell.com/ ' /tmp/bn.log | less

Assuming that everyone who did a GET or a HEAD more than 100 times (a conservative estimate) is involved in the attack:


egrep ' GET http://boycottnovell.com/ | HEAD http://boycottnovell.com/ '
/tmp/bn.log | awk '{print $3}' | sort | uniq -c| sort -n > /tmp/attackers

and then counting only the lines with greater than 100 hits we can see that there were 281 unique IP addresses involved in the attack.

However, it looks like they switched to targeting various different parts of the site later on or maybe just random pages because if we look at all of the accesses to the site which made more than 100 requests we get 863 IPs involved the top 19 being the following:


   6193 62.141.53.224
   7153 85.25.151.22
   7764 145.100.100.190
   8524 66.35.1.170
   8757 94.136.16.242
   9256 85.25.152.185
  10369 83.140.125.188
  10464 212.24.147.228
  10874 205.209.142.210
  10935 87.236.199.73
  11441 88.198.14.120
  12094 62.141.58.13
  12208 88.198.60.8
  12994 66.249.70.134
  13940 85.25.145.98
  19119 212.42.236.140
  19867 87.118.104.203
  26480 216.105.40.113
  29854 66.230.230.230

So 66.230.230.230 made 29k requests to the site in total.

Putting some iptables rules in place (which I document here):

http://www.kernel-panic.org/pipermail/kplug-list/2009-May/108075.html

nicely cut the problem down to size and now the effect of the DOS is unnoticeable.

11M of gzipped log are used for this sample.

Share in other sites/networks: These icons link to social bookmarking sites where readers can share and discover new web pages.
  • Reddit
  • email

If you liked this post, consider subscribing to the RSS feed or join us now at the IRC channels.

Pages that cross-reference this one

4 Comments

  1. Test said,

    May 20, 2009 at 7:09 am

    Gravatar

    site is super zippy fast now !

    great work team !

  2. aeshna23 said,

    May 20, 2009 at 8:48 pm

    Gravatar

    It’s sad that some people are so afraid of the ideas on this website that they resort to such tactics.

    I do have one suggestion, if this happens in the future. Don’t put long, technical explanations in the main post. You could put a link to a page with the long, technical explanation. The long, technical explanation dilutes the moral indignation about our opponents tactics. I know that what I’m saying is a rhetorical change, but rhetoric really matters here.

  3. Shane Coyle said,

    May 21, 2009 at 7:53 pm

    Gravatar

    Actually, according to Google, I am still receiving hits and clicks from BN for Adsense. I guess when you copied over the site you left in the publisher id for my account.

    Roy Schestowitz Reply:

    Shane,

    Yes, I am still speaking to Tracy to figure out how we shall work this out. I moved all three domains to his DNS namehost and we will probably use the adverts to pay him for all he has done to save us, not to mention hosting. Last night we moved to account to a VM (to limit the damage of future attacks).

What Else is New


  1. IRC Proceedings: Tuesday, January 19, 2021

    IRC logs for Tuesday, January 19, 2021



  2. Links 20/1/2021: WireGuard for pfSense and New US President

    Links for the day



  3. Links 19/1/2021: Krita 4.4.2 Released and JingOS Hype

    Links for the day



  4. Team UPC Keeps Pretending That UPCA Can Still be Resurrected (Even Without the UK, Which is Strictly a Requirement)

    The latest distortion of facts regarding the Unified Patent Court (UPC) Agreement (UPCA) as seen from the lens of people who seek to profit from such distortion



  5. 'Ethical Source' is Not Ethical and Not a Movement But a Misguided Self-Serving PR Stunt

    Something which is neither enforceable nor ethical is being promoted by profoundly unethical media in the pockets of large corporations



  6. InteLeaks – Part XXI: Intel Seeking Advice From a Bunch of Clowns (Harbor 'Research')

    A firm called Harbor 'Research' is making dubious recommendations to Intel; as shown in the above video, there's also an obsession with buzzwords (typically suggestive of a lack of technical grasp/understanding)



  7. IRC Proceedings: Monday, January 18, 2021

    IRC logs for Monday, January 18, 2021



  8. The US Election Was Not Rigged, But the Nomination Process Was (Undermined to Maintain Control by Oligarchy)

    Cheating/driving the left out of the Democratic Party seems like a longstanding tradition and we know who stands to gain from it; moreover, problems remain in the voting process because it's controlled by secret code of companies like Microsoft (in spite of the openwashing)



  9. InteLeaks – Part XX: Redacted (for Names Only) Release of Intel File About Developer eXperience (DX) Meddling in GNU/Linux

    Today (or tonight) we release the first 'phase' of InteLeaks in a sensibly redacted form; coming up next is a surprise from Team Microsoft



  10. Sites in Bed With the EPO and UPC 'Covering' the 'News' Without Mentioning Any of the Overt Abuses

    It is rather sad that blogs like IP Kat have turned into proponents of abusive EPO management and Team UPC increasingly resorts to lying using pseudonyms (to avert criticism and accountability); much of the rebuttal or response that’s hinged on reality/facts can only be found in comments, which are still subjected to a face-saving moderation process (conducted by Team UPC)



  11. Suppressed Facts of the Free Software Movement and Its Community of Volunteers – Part IV: Stories From the Depths of the Free Software Foundation (FSF)

    To reduce or alleviate suspicions and a potential of mistrust the FSF needs to become more transparent and liberate information (such as the real reason Bradley Kuhn left, as noted in the previous part)



  12. Links 18/1/2021: GNU Radio 3.9, Wikipedia at 20

    Links for the day



  13. InteLeaks – Part XIX: Intel's Web 'Experts' Seen as Microsoft Champions Dealing With the Platform Microsoft is Looking to Destroy

    Things aren't rosy at Intel because the hires aren't suitable for the job of documenting and/or presenting GNU/Linux-centric products (whose target audience is Free software developers)



  14. Adding Images as Characters to the Daily Bulletins of Techrights

    Our daily bulletins now have inside them coarse graphics, depicted using characters alone, and the tool used to generate them announced a new release earlier today; we showcase some of its features (in a new video)



  15. Links 18/1/2021: Weekly Summaries and Linux 5.11 RC4

    Links for the day



  16. IRC Proceedings: Sunday, January 17, 2021

    IRC logs for Sunday, January 17, 2021



  17. The Oligarchs' Parties Will Never Choose the Side of Software Freedom Because Free Software Cannot Bribe Officials

    The tough reality is that next week's (or this coming week's, depending on what Sunday counts as) inauguration ceremony is partly symbolic as all the same and important issues remain largely untouched, for corporations control almost everything of significance



  18. COVID-19 Has Actually Helped Software Freedom Due to Financial and 'Spare Time' Factors

    Developers and users are increasingly exploring what the Free software world has to offer; this is actually measurable and it contradicts claims to the contrary



  19. Future Plans and Using Videos to Complement Text

    Remarks on recent and impending site changes; We are not replacing text with video, we're just trying to enhance the presentation a bit, especially where visuals help make a point or where browsing through Web sites (or leaks) is more suitable than static, linear presentation



  20. InteLeaks – Part XVIII: Intel Does Not Know How to Properly Do Research and It Seems Apparent Unscientific Methods Are Used to Justify Poor Documentation

    There appears to be a severe crisis at Intel; they cannot recruit scientists (or those whom they recruited are walking away) and as a result the company produces bad products with poor documentation (or highly defective chipsets that top-notch marketing cannot compensate for); in this video we walk through some examples of how studies are being conducted (as already noted in Part XVII)



  21. Suppressed Facts of the Free Software Movement and Its Community of Volunteers – Part III: The Free Software Foundation (FSF) Seems More Like a Victim of Destabilisation Campaigns

    The Free Software Foundation (FSF), which turns 36 later this year, is looking to raise money that helps support the GNU Project, soon 38 years old and likely the most important Free software project to exist (ever)



  22. Links 17/1/2021: EasyOS on Raspberry Pi and GNU libsigsegv 2.13

    Links for the day



  23. InteLeaks – Part XVII: The High Cost of Microsoft Windows Users in GNU/Linux Development Teams

    A look inside Intel explains what holds back the technical team, which bemoans the lesser technical people getting in the way and not even using the product that they are writing about



  24. IRC Proceedings: Saturday, January 16, 2021

    IRC logs for Saturday, January 16, 2021



  25. Suppressed Facts of the Free Software Movement and Its Community of Volunteers – Part II: Why Bradley Kuhn Left the Free Software Foundation (FSF)

    The founder of the FSF is still at the FSF (albeit not publicly) and the person who lobbied to oust him has basically been 'banished' by the founder



  26. Links 16/1/2021: LibreOffice 7.1 Release Candidate, Zeroshell 3.9.5, FreeBSD Report, and GhostBSD 21.01.15

    Links for the day



  27. Free Speech on the Web Not Respected by Companies That Used to Support Software Freedom

    Mozilla does not have to make its Web browser about politics; it can just make an excellent piece of software that is neutral about the Web pages that it renders, based on the user's personal preferences



  28. Suppressed Facts of the Free Software Movement and Its Community of Volunteers – Part I: We Are Under Attack by Corporations and Their Salaried Facilitators

    The corporate takeover (taking over the Commons, produced by volunteers who are motivated by altruism) is a subject we must speak about and somehow tackle; this series will highlight uncomfortable or difficult truths



  29. InteLeaks – Part XVI: Intel Cannot Do Command Line, Even When It's Vastly Simpler and More Suitable for Development

    The Developer eXperience (DX) team at Intel seems to be full of Microsoft drones instead of developers and/or mildly technical people; this has not only harmed the quality of documentation but also upset staff, alienating people who actually understand what developers need (more than buzzwords like "DX")



  30. IRC Proceedings: Friday, January 15, 2021

    IRC logs for Friday, January 15, 2021


RSS 64x64RSS Feed: subscribe to the RSS feed for regular updates

Home iconSite Wiki: You can improve this site by helping the extension of the site's content

Home iconSite Home: Background about the site and some key features in the front page

Chat iconIRC Channel: Come and chat with us in real time

Recent Posts