Bonum Certa Men Certa

DDOS and Migration (Updated)

Summary: Boycott Novell had been under DDOS attacks for almost 4 days. We were struggling to just stay online while hosts investigated where the attacks came from. We moved between hosts (to semi-dedicated) and the same pattern of attack persisted until yesterday.

WE have kept silent about it in order not to encourage the attacker/s, but it's true. We have been under heavy DDOS attacks since Thursday night. What has happened since then? Well, a lot. Our previous host is no more as far as we are concerned. After struggling with the botnet for like 10 hours (filtering to no avail) our Web site got isolated. It did not serve any pages for almost 2 days. A reader of Boycott Novell was kind enough to lend us room on his server (more or less dedicated), on which he fought the botnets for over a day. The attackers kept changing tactics. Some other readers offered filtering advice and we are grateful to all of them. Ultimately, the attacks halted yesterday afternoon.



“Ultimately, the attacks halted yesterday afternoon.”The migration from the old server was not simple because the site was disabled abruptly following the early attacks. But now we have ensured that all data has been migrated. The only 'good' thing which came out of this attack is that, as oiaohm put it, the ordeal sort of made us more robust to future attacks.

Now that we have a new host in place, we also have more features. Data on the site (comments, posts, etc.) was not lost in the migration, just heaps of time and effort affecting several people. We have moved to a bigger, more robust environment that will hopefully facilitate the needs of the Web site as it continues to grow (we served about 200GB of data last month). We apologise for the downtime, which is unprecedented.

The plan is to carry on exposing Comes exhibits next month and also organise the Wiki. There is enough for years of work.

Again: we would like to thank all those who helped during the downtime and especially our generous reader ( Copilotco) who offered to host the Web site, taking us away from shared hosting in the process. Dedicated servers on normal Web hosts are just far too expensive for us to afford and I swear that I never made a single dime from this Web site. The ads merely covered the hosting fees which Shane has been paying since 2006.

One last clarification for lunatics who are now suggesting that we DDOSed ourselves, where to even begin refuting such nonsense (coupled with personal abuse)?

The attacks came from many addresses, for example 88.198.60.8 which is "tor-proxy.va6.de". Multiple such IPs hit us constantly and relentlessly (all tor exit nodes at first). At one stage it seemed like the front page alone received 3 page requests per second. But the IPs were also doing a HEAD on the Web site as many times as possible, bringing the server down to its knees (both the old server and new server, the former running Red Hat and the latter CentOS).

Update: Here is a report from the administrator.

I took over hosting of boycottnovell.com for Roy in the middle of the DDOS attack. I am looking at the squid log for boycottnovell.com during the DDOS. I have squid caching/proxying/url-rewriting for apache for various reasons.

The attack initially (or at least, at the time the DNS was re-pointed to my server) consisted of lots of HEAD requests. Then I wrote up a script to tail the log finding anyone doing lots of HEAD requests and putting the offending IP into the iptables packet filter while I cooked up a more permanent solution. Eventually they figured this out and switched to a full on GET of the root of the site and then I think they started getting random pages from the site as fast as they could although I'm not sure about that.

The interesting part starts around timestamp 1242543590.804 which is apparently when most of the world's DNS cut over to me including that of the machines in whatever bot net was employed in the attack.

If we run this command on the logfile with the logfile being /tmp/bn.log:




grep " HEAD http://boycottnovell.com/ " /tmp/bn.log | awk '{print $3}' | sort | uniq -c |sort -n | tail -10



we get:

   2716 81.175.61.4
   2960 212.24.147.228
   3056 204.209.56.56
   5637 87.236.199.73
   6645 145.100.100.190
   7261 212.42.236.140
   8487 88.198.14.120
   9640 62.141.58.13
  11008 87.118.104.203
  11269 88.198.60.8


and if we do:




grep " GET http://boycottnovell.com/ " /tmp/bn.log | awk '{print $3}' | sort | uniq -c |sort -n|tail -10



we get:




5801 94.136.16.242 5854 85.25.152.185 5865 212.24.147.228 6367 66.35.1.170 6682 205.209.142.210 6977 87.118.104.203 8102 83.140.125.188 8300 85.25.145.98 8441 212.42.236.140 20065 66.230.230.230



So one IP did a get of the root of the site 20k times before I really effectively got everything blocked off and another did a HEAD around 11k times. You can get a feel for how the attack progressed using:




egrep ' GET http://boycottnovell.com/ | HEAD http://boycottnovell.com/ ' /tmp/bn.log | less



Assuming that everyone who did a GET or a HEAD more than 100 times (a conservative estimate) is involved in the attack:




egrep ' GET http://boycottnovell.com/ | HEAD http://boycottnovell.com/ ' /tmp/bn.log | awk '{print $3}' | sort | uniq -c| sort -n > /tmp/attackers



and then counting only the lines with greater than 100 hits we can see that there were 281 unique IP addresses involved in the attack.

However, it looks like they switched to targeting various different parts of the site later on or maybe just random pages because if we look at all of the accesses to the site which made more than 100 requests we get 863 IPs involved the top 19 being the following:




6193 62.141.53.224 7153 85.25.151.22 7764 145.100.100.190 8524 66.35.1.170 8757 94.136.16.242 9256 85.25.152.185 10369 83.140.125.188 10464 212.24.147.228 10874 205.209.142.210 10935 87.236.199.73 11441 88.198.14.120 12094 62.141.58.13 12208 88.198.60.8 12994 66.249.70.134 13940 85.25.145.98 19119 212.42.236.140 19867 87.118.104.203 26480 216.105.40.113 29854 66.230.230.230



So 66.230.230.230 made 29k requests to the site in total.

Putting some iptables rules in place (which I document here):

http://www.kernel-panic.org/pipermail/kplug-list/2009-May/108075.html

nicely cut the problem down to size and now the effect of the DOS is unnoticeable.

11M of gzipped log are used for this sample.

Comments

Recent Techrights' Posts

Serial Sloppers Are Killing the Web (They Probably Don't Care, Either)
Slop is a disease on the Web
IBM's Debt Ballooned by 8.5 Billion Dollars in Just 3 Months!
Hallmark of a company in a state of disarray, trying to spend its way out of trouble
Big Trouble in GNOME
even GNOME people admit the CoC went wrong
New Article Explains How the GPL Came About and WordPress Having Copyleft Obligations
Having been involved in the WordPress development community since almost the beginning, I know why it chose the GPL and how it restricts abuse by Automattic
Dr Richard Stallman (RMS) Gives Talk in Oxford University in 4 Hours
If you live nearby, go there (it's free as in gratis)
 
Get Rid of Back Doors, Don't Obsess Over Bounties and Other Corporate PR Stunts (or Needless Reboot Rituals)
Security as a term has mostly lost its meaning due to repeated misuse for many years
Streaming Apps Are “Investor Fraud” That Kills the Planet
Reprinted with permission from Ryan Farmer
Things Get Increasingly Nasty at Microsoft Ahead of the Fake Results and May's Mass Layoffs Wave
They try to get people to 'resign' so that they won't count as layoffs and the company's 'wellbeing' will seem better
Slopping the Trough: Disney Plus Loses Billions and the Decline of Physical Media in America
Reprinted with permission from Ryan Farmer
Over at Tux Machines...
GNU/Linux news for the past day
IRC Proceedings: Thursday, April 24, 2025
IRC logs for Thursday, April 24, 2025
Links 24/04/2025: GAFAM Problems and No Peace (or Ceasefire) in Sight
Links for the day
Slopfarms on the Web Almost Always Generate Anti-Linux FUD When They Produce "Linux" Output
Welcome to the dying Web
Richard Stallman's Oxford Talk Has Just Ended, Here Are Some Photos
he might hop over to another European country
Gemini Links 24/04/2025: Birthday and Good Work of Academia in Esotericism
Links for the day
Links 24/04/2025: EU fines Apple and Facebook, Another Microsoft GitHub Security Blunder
Links for the day
IBM Gained Almost 6 Billion Dollars in "Goodwill" Value in Just 3 Months, According to IBM
Congrats to the management!
In Belarus, Yandex is Now Measured as 50 Times More 'Popular' (by Usage) Than Microsoft
Yandex continues to gain, whereas Bing cannot even register at 1%. Last month it was registered or measured at a measly 0.65%.
IBM Cannot Lie to Shareholders Anymore
"I would not be surprised if we see a layoff every quarter this year."
Using a Law Firm's Licence to Exercise Politics Through Frivolous SLAPPs and Nastygrams (to Silence People, Remove Pages, Demand Fake or Forced 'Apologies')
Things must be getting really bad when lawyers act for raving antisemites
We're Working to Make Full-Site Search Available
This site has over 1,000 'wiki' pages, many thousands of documents, several thousands of videos, and about 50,000 blog posts or articles. We need to make them easier to find/navigate.
Links 24/04/2025: IBM Loses Many Contracts, Intel to Lay Off Over 20% (Not Counting Those Who Leave 'Voluntarily')
Links for the day
Richard Stallman Can Explain to Oxford Artificial Intelligence Society Why LLM Slop is Not Artificial Intelligence and Why It Hurts Society
another 'crop' of LLM slop that damages GNU/Linux and facts
Over at Tux Machines...
GNU/Linux news for the past day
IRC Proceedings: Wednesday, April 23, 2025
IRC logs for Wednesday, April 23, 2025
Open Source Initiative (OSI) Promoting Microsoft and Proprietary Software Using Microsoft Operatives
Because nothing says "Open Source" like GPL violations facilitated by Microsoft
Another Site Bites the Dust: "Open Source For You" Becoming a Slopfarm (LLM Slop)
What a shame. Another dead site.
Links 23/04/2025: Crackdowns on Dissent, Palin Loses Libel Retrial Against New York Times
Links for the day
Links 23/04/2025: Hard Times and Digital Amnesia
Links for the day
The GNU/Linux Site Formerly Known as "linoxide.com" is Back... as an LLM Slopfarm!
Better for linoxide.com to go offline than to do this
Richard Stallman to Speak in Oxford University Exactly a Day From Now
outsourced to GAFAM
Links 23/04/2025: "Hiding Corruption" and "The Cost of Defunding Harvard"
Links for the day
Microsoft 'Studies' Again? Leon Musolff is Writing Papers With Microsoft.
Even if one can see/find a link to "the study" (in the Bezos-controlled publication), most people won't look any further and just take everything at face value.
Towards GNU World Domination
The FSF led by Geoffrey S. Knauth with his friend Richard Stallman in the FSF's Board [...] Let's encourage people to adopt GNU/Linux. There has never been a better time.
statCounter Helps Visualise Just How Deep in Trouble Microsoft is (Especially in Africa)
Microsoft sabotaged efforts to connect Africans and equip them with GNU/Linux laptops
The Register is Using Linux-Hostile Clickbait in Articles of Linux Proponents
Don't be a "whore" to advertisers, team El Reg
Microsoft Windows in Cyprus Lacking a Future
Most people access the Web there from mobile
Matrix Has a Severe Problem With Illegal Images
If Matrix cannot get the CP problem under control, many projects and people will dump Matrix
Never Try to Justify Strangulation of Women (Not in the US and Not in the UK)
Joint post by Mrs. Rianne Schestowitz and Dr. Roy Schestowitz
Links 23/04/2025: Tesla Profits Plunge 71%, Intel Ready to Lay Off 20% of Staff, Microsoft and IBM Layoffs
Links for the day
Microsoft's Most Profound Issue is That People Moved to 'Mobile' and "App Stores" (Microsoft's Presence There is Negligible)
Expect a wild ride for Microsoft this year
Google News is Amplifying FUD and Lies About Linux (and OpenSSH/SSH) by Promoting Slopfarms With Machine-Generated FUD and Slop Images
Google should know better
Gemini Links 23/04/2025: Librarians, Anubis, and Refactoring a Gemini Capsule
Links for the day
Over at Tux Machines...
GNU/Linux news for the past day
IRC Proceedings: Tuesday, April 22, 2025
IRC logs for Tuesday, April 22, 2025