Bonum Certa Men Certa

DDOS and Migration (Updated)

Summary: Boycott Novell had been under DDOS attacks for almost 4 days. We were struggling to just stay online while hosts investigated where the attacks came from. We moved between hosts (to semi-dedicated) and the same pattern of attack persisted until yesterday.

WE have kept silent about it in order not to encourage the attacker/s, but it's true. We have been under heavy DDOS attacks since Thursday night. What has happened since then? Well, a lot. Our previous host is no more as far as we are concerned. After struggling with the botnet for like 10 hours (filtering to no avail) our Web site got isolated. It did not serve any pages for almost 2 days. A reader of Boycott Novell was kind enough to lend us room on his server (more or less dedicated), on which he fought the botnets for over a day. The attackers kept changing tactics. Some other readers offered filtering advice and we are grateful to all of them. Ultimately, the attacks halted yesterday afternoon.



“Ultimately, the attacks halted yesterday afternoon.”The migration from the old server was not simple because the site was disabled abruptly following the early attacks. But now we have ensured that all data has been migrated. The only 'good' thing which came out of this attack is that, as oiaohm put it, the ordeal sort of made us more robust to future attacks.

Now that we have a new host in place, we also have more features. Data on the site (comments, posts, etc.) was not lost in the migration, just heaps of time and effort affecting several people. We have moved to a bigger, more robust environment that will hopefully facilitate the needs of the Web site as it continues to grow (we served about 200GB of data last month). We apologise for the downtime, which is unprecedented.

The plan is to carry on exposing Comes exhibits next month and also organise the Wiki. There is enough for years of work.

Again: we would like to thank all those who helped during the downtime and especially our generous reader ( Copilotco) who offered to host the Web site, taking us away from shared hosting in the process. Dedicated servers on normal Web hosts are just far too expensive for us to afford and I swear that I never made a single dime from this Web site. The ads merely covered the hosting fees which Shane has been paying since 2006.

One last clarification for lunatics who are now suggesting that we DDOSed ourselves, where to even begin refuting such nonsense (coupled with personal abuse)?

The attacks came from many addresses, for example 88.198.60.8 which is "tor-proxy.va6.de". Multiple such IPs hit us constantly and relentlessly (all tor exit nodes at first). At one stage it seemed like the front page alone received 3 page requests per second. But the IPs were also doing a HEAD on the Web site as many times as possible, bringing the server down to its knees (both the old server and new server, the former running Red Hat and the latter CentOS).

Update: Here is a report from the administrator.

I took over hosting of boycottnovell.com for Roy in the middle of the DDOS attack. I am looking at the squid log for boycottnovell.com during the DDOS. I have squid caching/proxying/url-rewriting for apache for various reasons.

The attack initially (or at least, at the time the DNS was re-pointed to my server) consisted of lots of HEAD requests. Then I wrote up a script to tail the log finding anyone doing lots of HEAD requests and putting the offending IP into the iptables packet filter while I cooked up a more permanent solution. Eventually they figured this out and switched to a full on GET of the root of the site and then I think they started getting random pages from the site as fast as they could although I'm not sure about that.

The interesting part starts around timestamp 1242543590.804 which is apparently when most of the world's DNS cut over to me including that of the machines in whatever bot net was employed in the attack.

If we run this command on the logfile with the logfile being /tmp/bn.log:




grep " HEAD http://boycottnovell.com/ " /tmp/bn.log | awk '{print $3}' | sort | uniq -c |sort -n | tail -10



we get:

   2716 81.175.61.4
   2960 212.24.147.228
   3056 204.209.56.56
   5637 87.236.199.73
   6645 145.100.100.190
   7261 212.42.236.140
   8487 88.198.14.120
   9640 62.141.58.13
  11008 87.118.104.203
  11269 88.198.60.8


and if we do:




grep " GET http://boycottnovell.com/ " /tmp/bn.log | awk '{print $3}' | sort | uniq -c |sort -n|tail -10



we get:




5801 94.136.16.242 5854 85.25.152.185 5865 212.24.147.228 6367 66.35.1.170 6682 205.209.142.210 6977 87.118.104.203 8102 83.140.125.188 8300 85.25.145.98 8441 212.42.236.140 20065 66.230.230.230



So one IP did a get of the root of the site 20k times before I really effectively got everything blocked off and another did a HEAD around 11k times. You can get a feel for how the attack progressed using:




egrep ' GET http://boycottnovell.com/ | HEAD http://boycottnovell.com/ ' /tmp/bn.log | less



Assuming that everyone who did a GET or a HEAD more than 100 times (a conservative estimate) is involved in the attack:




egrep ' GET http://boycottnovell.com/ | HEAD http://boycottnovell.com/ ' /tmp/bn.log | awk '{print $3}' | sort | uniq -c| sort -n > /tmp/attackers



and then counting only the lines with greater than 100 hits we can see that there were 281 unique IP addresses involved in the attack.

However, it looks like they switched to targeting various different parts of the site later on or maybe just random pages because if we look at all of the accesses to the site which made more than 100 requests we get 863 IPs involved the top 19 being the following:




6193 62.141.53.224 7153 85.25.151.22 7764 145.100.100.190 8524 66.35.1.170 8757 94.136.16.242 9256 85.25.152.185 10369 83.140.125.188 10464 212.24.147.228 10874 205.209.142.210 10935 87.236.199.73 11441 88.198.14.120 12094 62.141.58.13 12208 88.198.60.8 12994 66.249.70.134 13940 85.25.145.98 19119 212.42.236.140 19867 87.118.104.203 26480 216.105.40.113 29854 66.230.230.230



So 66.230.230.230 made 29k requests to the site in total.

Putting some iptables rules in place (which I document here):

http://www.kernel-panic.org/pipermail/kplug-list/2009-May/108075.html

nicely cut the problem down to size and now the effect of the DOS is unnoticeable.

11M of gzipped log are used for this sample.

Comments

Recent Techrights' Posts

[Meme] The Doyen and the Colonel
EPO continues to prioritise lawbreaking over knowledge
Microsoft Poetterix is Work in Progress
Linux's New DRM Panic 'Blue Screen of Death' In Action
[Meme] Community of People to be Exploited, Then Thrown Away, Left Behind or Even Slandered
Debian.org front page
Alexandre Oliva's FSF disposition
During my recent trip for LibrePlanet, I was fortunate to have, or at least start, long conversations with nearly everyone in FSF staff
 
While Everyone is Furious at Vista 11 (Over TPM, Recall and Other Malicious 'Features') Canonical is Selling It to People
So the only thing Canonical says about Windows is that you should give it a try?
Links 18/06/2024: Adobe and Internet Archive in Trouble
Links for the day
Peter Duffy Explains SystemD
Ein Volk, Ein Reich, Ein Führer!
EPO Union Action: Next Week SUEPO The Hague and SUEPO Munich Talk About New Pension Scheme (NPS) and Salary Savings Plan (SSP)
So there are basically 32 days left for more people to intervene
[Meme] Wait Till Systemd-Recall
The only thing Linux still needs is a forensics backdoor
GNU/Linux Up This Month in India (or Why Famous Criminal Bill Gates Keeps Visiting Modi)
truth tends to catch up with people
24/7 Work Discipline
it's not so much about how much (or how long) one works, it's about how one works and whether one feels comfortable doing it
Adamant Conformism is an Enemy of Science
"The reasonable man adapts himself to the world; the unreasonable one persists in trying to adapt the world to himself. Therefore, all progress depends on the unreasonable man"
Over at Tux Machines...
GNU/Linux news for the past day
IRC Proceedings: Monday, June 17, 2024
IRC logs for Monday, June 17, 2024
Links 18/06/2024: Further Mass Layoffs and Gemini Leftovers
Links for the day
At IBM, "Brownnosing is the Norm."
Many of these comments are from IBM insiders
Myanmar/Burma: Google Gains One Percent, Microsoft Loses One Percent Since the LLM Hype ('Bing Chat')
it's not hard to understand LLMs didn't replace real search and didn't replace Google, either
[Meme] KISS, not SAAS
Gemini Protocol turns 5 in exactly 2 days
Hostageware: The Threat of Clown Computing (or 'SaaS', Another Misnomer or Buzzword) to Computer Users Everywhere
This problem isn't limited to Free software adopters
Jean-Pierre Giraud, Possible Forgeries & Debian: elections, judgments, trademark already canceled, archaeologist
Reprinted with permission from Daniel Pocock
Six on the Beach: After Losing Six Continents Microsoft is Losing Oceania Too
Based on the 6- or 7-continent view of the world
Links 17/06/2024: Mass Layoffs Accelerating in Tech, Concerns About Impact of the Net
Links for the day
Gemini Links 17/06/2024: Hyprland Analysed and No Use for Betrusted
Links for the day
Microsoft Can Never Make a Comeback Anymore, the Community is Shutting It Out
We're relying on the real community, not fake ones or coopted ones
The World is Becoming (or Has Already Become) Linux
An intercontinental success story
Georgia: Bing Share Fell by Half Since 'Bing Chat' (LLM Hype), Fell Behind Yandex As Well
Georgia's situation is interesting
[Meme] SPI and 'FSFE': Sponsored by Microsoft to...
women's instincts do not matter to these strongmen
One More (Failed) Attempt to Deplatform the Sites by Harassing and Threatening Webhosts
What we're seeing here is a person who abuses the system in Canada at Canadian taxpayers' expense trying to do the same in the UK, at British taxpayers' expense
[Meme] Shitburger of an LLM
IBM and the Hololens
Links 17/06/2024: Chatbot Nonsense Thrown Under the Bus (Severe Failure, Pure Hype), How to Finance Free Software 'Hackers'
Links for the day
Debian's Personal Attacks Are Upsetting Women, Too
Female Debian Developer: "I Believe Daniel [Pocock] is On the Right Track."
Microsoft's Bing is So Irrelevant in Moldova (1%) That Russia's Yandex is About 5 Times Bigger
How much longer before Microsoft throws in the towel?
12 Days Have Passed Since the Edward Brocklesby Revelations and Debian Project Has Said Absolutely Nothing About That
One must therefore assume they have nothing to say in their defence (covering up severe security failings)
Yes, You Can
Unless you live somewhere like Russia...
[Meme] Listen to the Experts
Bill Gates didn't even finish university]
Roy and Rianne's Righteously Royalty-free RSS Reader (R.R.R.R.R.R.) and the Front-End Interfaces
As the Web deteriorates the availability, quality and prevalence of RSS feeds is not improving, to put it mildly
Algeria Shows High GNU/Linux and Android Adoption, All-Time High and Almost Three-Quarters of Web Requests
GNU/Linux was below 3%, now it is above 3%
Mass Layoffs at Microsoft-owned GitHub (About 80 Percent of the Staff in India Laid Off)
It's not just in India
Over at Tux Machines...
GNU/Linux news for the past day
IRC Proceedings: Sunday, June 16, 2024
IRC logs for Sunday, June 16, 2024
Gemini Links 16/06/2024: Scarecrows, Moles, Ham Radio, and No IPs
Links for the day
Africa is Android and Green (Chrome, Not Just Android Logo)
In Africa Firefox is almost below 1% now
Coercion From the "Consent" and "CoC" Crowd is a Self-Defeating Tactic
Freedom of the press; Nothing less
Covering Abuses and Corruption
We'll never surrender to blackmail
According to statCounter, GNU/Linux Increased From 3.77% to 3.89% This Month (Worldwide), Windows Now Below 20% in 78 Nations, Below 10% in 27 Nations
Highest since March (for GNU/Linux)
Ubuntu Running Out of Energy
Its planet too is deteriorating
Links 16/06/2024: In Defence of Email and Why Recycling Symbol Lost All Meaning
Links for the day
Gemini Links 16/06/2024: Computer Science Course Union and Potentiometer
Links for the day
Cross border crime: sale of Swiss insurance in France and European Union without authorisation
Reprinted with permission from Daniel Pocock
Letting Microsoft systemd Manage /home Was a Terrible Idea All Along
systemd-tmpfiles, deleting /home
Patriotism is OK, But We Need Facts and Reason, Not Blind Obedience to Authority
Very seldom in the history of human civilisation has groupthink proven to be of real merit
When You Touch One of Us You Touch All of Us
We have a principled, uncompromising stance on this matter
Links 16/06/2024: New Sanctions Against Russia, Fentanylware (TikTok) Causing More Problems
Links for the day
Social Control Media in Japan: Twitter (X) Has Collapsed, YouTube Rising (Apparently)
What a genius Mr. Musk is!
Windows Cleansed in South Africa (Already Hovering Around 10% Market Share)
Plus Microsoft's mass layoffs in Africa
[Meme] Satya Nadella's Windows PC RECALLS Not What He Did
Satya got lucky
Usage of Let's Encrypt in Geminispace Has Collapsed (That's a Good Thing!)
Ideally, or eventually, all capsules will sign their own certificates or have their own CA
North Macedonia: Windows Down From 99.2% to 28.5%
Last year it was even measured at 26%
Over at Tux Machines...
GNU/Linux news for the past day
IRC Proceedings: Saturday, June 15, 2024
IRC logs for Saturday, June 15, 2024
Gemini Links 16/06/2024: Hand Held Maneuvering Unit and Hugo Static Files
Links for the day