Bonum Certa Men Certa

Good News, Bad News (and Back to Normal)

Video download link | md5sum 948cc5f102085cd52f4d356b486c2586 Major Incident and Recovery Creative Commons Attribution-No Derivative Works 4.0



Summary: When many services are reliant on the integrity of a single, very tiny MicroSD card you're only moments away from 2 days of intensive labour (recovery, investigation, migration, and further coding); we've learned our lessons and took advantage of this incident to upgrade the operating system, double the storage space, even improve the code slightly (for compatibility with newer systems)

THE good news is that Gemini is expanding faster than we predicted earlier this month. Lupa is now just 11 capsules short of 2,000 and yesterday we received some E-mails asking about Gemini downtime (we also got some inquiries over IRC, which means the Gemini capsule really matters to people).



"These things are inherently fragile; telling people to reduce the number of write operations is almost unreasonable because what good is a system you cannot use (or program) as you wish?"So why was it down? The short story is, it was a hardware failure. Not the fault of GNU/Linux or anything like that (in fact, credit to GNU/Linux for letting us fetch another complete backup of the entire system despite the whole file system being in read-only mode). There was no panic, just frustration, and based on what we heard about MicroSD-based (for boot) systems such an error was inevitable and almost predictable. The latest backup (before the "emergency" one was initiated) had been marked only a few days old (contents at most a couple of days behind).

All the services are now back online, the operating system was replaced by Debian 11, and the machine has twice as much storage space as before, which ought to permit us to do things we didn't even dare when space was tight. To reduce future downtime I also bought a spare disk (card actually) and will work on improving/reducing D-R time, as it's likely that a similar incident will happen later this year or next year. These things are inherently fragile; telling people to reduce the number of write operations is almost unreasonable because what good is a system you cannot use (or program) as you wish?

"We're hoping that tonight and tomorrow we can make up for the lost time..."Debian 11 is quite nice, but of course imperfect (perception is an impossibility). It's the first time I use Debian 11 (my wife, my sister and myself all use Debian 10 on our laptops) and maybe I'll get to write some positive things about it some time later this year (once I gain more experience/s with it).

We're hoping that tonight and tomorrow we can make up for the lost time; I hardly slept yesterday (stayed awake for about 20 hours straight, then just 4 hours of sleep) and we have a bunch of things lined up that I never managed to publish as restoring services (like IPFS and Gemini) was more pressing a task, more urgent a need.

The hardest part (to me personally) was having to go to Town for replacement components, knowing that few shops still exist (even fewer because of the pandemic) and the bigger shops are full of unmasked people who don't respect people's perimeter (it's not helping that our government likes to pretend COVID-19 is just some past event).

Recent Techrights' Posts

Microsoft and Windows Have Many Back Doors, But LLM Slop Keep Claiming That Linux Has "Backdoor"
It's another example of LLM slop as FUD amplifier, via slopfarms as well
 
About a Quarter of Today's "linux" News in Google News Came From One Domain and It's a Slopfarm
Not kidding!
Gemini Links 05/08/2025: Zombie Threat and Switching to NixOS
Links for the day
Over at Tux Machines...
GNU/Linux news for the past day
IRC Proceedings: Monday, August 04, 2025
IRC logs for Monday, August 04, 2025
ChatGPT in Trouble
Watch out for the newer buzzwords
The Register MS Links to the Wrong statCounter Page
They link to older data
Dr. Andy Farnell Explains How Google Turned From "Librarian" Into "Oracle", Telling Us What to Think Instead of Where to Look
Google was always a lousy librarian
Microsoft Layoffs Continue in August 2025
If Microsoft is doing so well, how come about 10 rounds of layoffs in about 7 months in 2025?
In Many Countries Vista 11 Adoption Stalled or Became Negative
Not just because people move to GNU/Linux
Microsofters' Lawyers Are Name-calling and Insulting Microsoft Critics, Even Their Spouses
How not to win arguments
Flagging or Tagging Slop That We Find Online
Right now we use ImageMagick
Links 04/08/2025: Very Bad Weather and Travel Restrictions in China
Links for the day
Gemini Links 04/08/2025: Misiamisia and Mobile Linux
Links for the day
Microsoft's Stock is Like a Religion, Microsoft Goes Into 'Hiding' (From Shareholders)
like a religious person or devout believer, the media just parrot anything Microsoft says
Links 04/08/2025: 80 Years Since Last Nuclear War, IPv6 in China
Links for the day
Groklaw Static Site Relaunches With New Theme, But Many Pages and All the Comments Are Missing
We suppose that's still a lot better than the site being offline, as it was for several months
"For Five decades; For freedoms; For all users" (Original EMACS Turns 50 Next Year)
Linus Benedict Torvalds was only 6 when EMACS started
In Spain, Microsoft's Search Engine Market Share Fell to 2%
16 years have passed since Bing was introduced
Protecting GNU/Linux-Centric Journalism From Serial Sloppers
Unoriginal slop is taking away traffic from the people who did all the real work
It Looks Like Managers at Oracle Now Use LLM Slop to Write Blog Posts
Did he cheat by prompting LLMs for mindless text "filler"?
Over at Tux Machines...
GNU/Linux news for the past day
IRC Proceedings: Sunday, August 03, 2025
IRC logs for Sunday, August 03, 2025
Gemini Links 04/08/2025: Qubes OS and Curious crypto case of certificates (CCCC)
Links for the day
They Tell Us That "Cloud Storage" is Safe and Robust to Incidents Like Fires
Do you have backups? Where are they and who controls them?
"Allowing SDL to default to Wayland caused a number of customer issues so keep the default at X11 for now"
2025 is another year of Wayland ambitions. It's also a year of self-fulfilling prophecies.
In The United Kingdom (UK), Microsoft Search (Bing) Falls to All-Time Low
Grow? What grow??? It's collapsing.
GNU/Linux Reaches 5% in Oman
Some GNU/Linux distros are made in Oman
Google's "AI Mode" is a Pathetic Joke Prematurely Introduced in the UK (Like "Bard", Which Sank the Company's Shares)
what Google "thinks" about PCLinuxOS
What the Free Software Foundation Started Four Decades Ago is Becoming Mainstream
"Four decades; Four freedoms; For all users"
Doing a Better Job at Labelling Slop Images
we'll label screenshots that contain slop, typically with red-coloured text overlay
Social Control Media is Out of Style
What's your excuse for wasting time on (or in) it?
Maldives: GNU/Linux at All-Time High, Windows at New Lows
data from statCounter shows a reassuring trend
Efficiency is Good, So Why Won't Governments Cull LLM Companies Using Stronger, Stringent Policies?
Like every bubble that ever existed, including some recent ones, an end will come
The Defunct Site LinuxConfig Has Published a Fake Article About Richard Stallman Using LLM Slop, Which Stallman Calls "Bullshit Generator"
Worse yet, it is writing using a "Bullshit Generator" (the term used by Stallman) about Stallman's health
Microsoft Windows Falls to All-Time Lows in Morocco and Algeria
About 70% or even less
StopGenAI in the Cyber Show (C|S)
covering a theme that we too covered a lot lately
Gemini Links 03/08/2025: Once-a-Decade Couch Shopping and Blessings in Disguise
Links for the day
Links 03/08/2025: Political Catch-up, Global Warming, and Hunger
Links for the day
Brittany Day Entered LLM Slop Into LinuxSecurity.com and Something Hilarious Happened: The Site is "Exploited"
The brainless, effortless copypasta of "slop artists" shows its limits
Links 03/08/2025: Microsoft Exchange 0-day Exploited and Avoidable Nuclear Escalation
Links for the day
Next Month 'New Techrights' Turns Two
Next month, on the fourth week, it'll be 2 years since the migration
Definitely Not a Ponzi Scheme
Bitcoin v Microsoft
Online Safety Act Tries to Accomplish the Impossible
All I can say is, "good luck with that!"
The Electronic Frontier Foundation (EFF) is a Billionaires' Lobby
Billionaires that control tech companies
Microsoft Borrows 3 Billion Dollars Per Month, a Company Truly Worth Trillions Would Not Do This
if Windows (and Office) "market share" fell from about 90% to barely 30%, how come Microsoft is now "valued" at 20 times more?
It's Even Worse Than Microsoft Lunduke Puts It; GNOME is SLAPPing Journalists
In our experience, GNOME is so malicious - some elements of it in particular - that it would launch multiple simultaneous SLAPP campaigns not only against journalists but also their spouses
GNU/Linux Adoption Reaches All-Time Highs in Chile, statCounter Indicates
This month marks 4 years since Vista 11 came out (as a fake "leak") and some surveys still measure its adoption at less than 40%
Slop Will Not Change the World
Some of us grow up sooner and leave that nonsense behind (or altogether avoid/skip it)
Gemini Links 03/08/2025: Nostalgia and TOFU
Links for the day
Over at Tux Machines...
GNU/Linux news for the past day
IRC Proceedings: Saturday, August 02, 2025
IRC logs for Saturday, August 02, 2025