Bonum Certa Men Certa

It Looks Like Red Hat's (IBM) Fedora Project May be 'Outsourced' to Amazon's Datacentres

Amazon datacentres in Ashburn
Amazon datacentres in Ashburn (source: Wikileaks)



Summary: In "seeking a more modern and cost effective location" for Fedora Infrastructure it seems to have been decided, privately, that Amazon (AWS) would be the new home of this project; but there's sufficient obfuscation surrounding the matter and many people seem to be totally unaware

OVERNIGHT we've spent several hours researching this and we also asked some Fedora people, who don't seem to know exactly what's going on. Surely some people higher up the corporate chain have a better and clearer idea, but they're not likely to speak about it because IBM pays their salary and there's fear of further layoffs. A lot of the finer details will be available in IRC logs (frustratingly enough, almost all the coding/code hosting was outsourced to Microsoft), but the gist of it all is this: while examining about a dozen services of the Fedora Project we've occasionally stumbled upon Amazon addresses. And then there's this:



dig copr.fedorainfracloud.org



; <<>> DiG 9.11.5-P4-5.1+deb10u1-Debian <<>> copr.fedorainfracloud.org ;; global options: +cmd ;; Got answer: ;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 34423 ;; flags: qr rd ra; QUERY: 1, ANSWER: 2, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION: ; EDNS: version: 0, flags:; udp: 512 ;; QUESTION SECTION: ;copr.fedorainfracloud.org. IN A

;; ANSWER SECTION: copr.fedorainfracloud.org. 300 IN CNAME copr-fe.aws.fedoraproject.org. copr-fe.aws.fedoraproject.org. 939 IN A 3.81.0.123


This probably merits a little bit of context. A few hours ago Fedora Infrastructure had another very major OpenShift failure (not the first such incident over the past few days). Well, it wasn't an issue until lately...

This so-called 'cloud computing' software is Red Hat's brand (with GUIs and everything) of something that's otherwise widely available as Free software but harder to use.

"Fedora Infrastructure Status: Major service disruption," said the notice, reading out the extent of the impact: "New status major: issues with openshift cluster for services: Ipsilon, Badges, Blockerbugs, Package Updates Manager, Fedora Infrastructure Cloud, COPR Build System, Documentation website, Fedora elections, Account System, Fedora Messaging Bus, Fedora Calendar, Fedora pastebin service, The Koji Buildsystem, Koschei Continuous Integration, Kerberos, Mailing Lists, Module Build Service, Mirror List, Mirror Manager, Fedora Packages App, Pagure, Fedora People, Package maintainers git repositories, Fedora Container Registry, ABRT Server, Fedora websites, Fedora Wiki, Zodbot IRC bot"

"...while examining about a dozen services of the Fedora Project we've occasionally stumbled upon Amazon addresses."And later on it said "New status good: Everything seems to be working."

"We are now operating from the new datacentre and are bringing back capacity as time allows. The known issues are documented here in our wiki," it adds. If one goes to the corresponding status page for pertinent services ("Fedora Project Datacentre Move") it is linking to this wiki page, which in turn says: "This is a page to track known issues after the Fedora Infrastructure Datacenter move in 2020 from phx2 to iad2 and rdu2."

Those are not descriptive names, as there's not even a domain name/suffix. We tried tracing back the IP addresses of these services to identity where at least some these services were being moved to. "I think it's more Red Hat management pressure to rely on Github and Google," Ryan said, but it seems to be related to neither. "You know," Ryan continued, "the infrastructure self-hosting was VERY reliable for many years before that." He mentioned "GMail, Google Drive, calendar stuff" as examples, citing the anonymous post that said about Jim Whitehurst: "He didn't use our products (used a Mac, along with other members of the executive management team), allowed RH to dump open source solutions for our own business to move things to Google's services (which is a huge message to our very clients of "we don't value or trust open source, so why should you?")"

"A few hours ago Fedora Infrastructure had another very major OpenShift failure (not the first such incident over the past few days)."The migration pages are edited mostly by Kevin Fenzi (IBM/Red Hat), who wrote about it less than a year ago in the devel-announce mailing list. No replies to it as of this date.

Is Copr on AWS now? Seems so. If the new DC of Fedora is Amazon-owned and Amazon-controlled, there may be something to conclude. Depending on one's opinion regarding Amazon and AWS, which turns Free/libre things into proprietary.

This message from last year says the hosting of various services is moving to the city/area in VA where AWS has many datacentres that already route some Fedora traffic. While we don't know for sure what's going on, as we need to better understand what plans are made in the boardroom (behind closed doors), it seems likely there's a move to Amazon.

Four years ago The Atlantic published "Where Are Amazon's Data Centers?" and it took note of Ashburn and Sterling. About a year later Wikileaks revealed that both places have lots of datacentres in them, mostly Ashburn (shown above is our screenshot of these).

Going back to Kevin, less than a year ago he wrote: "Fedora Infrastructure currently has the majority of its hardware in a datacenter in Arizona, USA. Red Hat leases this space for use by a number of teams, including Fedora."

"This message from last year says the hosting of various services is moving to the city/area in VA where AWS has many datacentres that already route some Fedora traffic."We can confirm, based on lots of checks, that many services still link to that datacentre.

"However," Kevin continued, "they've been seeking a more modern and cost effective location for some time and have decided on one: So, we will be migrating to a new datacenter located in Ashburn, Virginia in 2020."

Oh, Ashburn. Where Amazon has a densely-distributed cluster of datacentres.

Is Red Hat moving to 'the cloud' and, if so, why not its own? Does IBM feel like it needs to outsource that to a competitor/partner?

The details on this are still not certain and I could not get anybody from Fedora (in IRC) to confirm or deny. Quite frankly, many of them may not be aware.

Recent Techrights' Posts

Confirmed in the Mainstream Media: A Lot of Microsoft "Workloads" Were Just LLM Slop (Helping to Fake Growth for Years, as Microsoft Had Paid "Open" "AI" to Become a "Client") and Demand is Rapidly Waning, Datacentres Canceled and/or Shut Down
Anything to facilitate further accounting fraud
Taiwan's Media Covers Closure of Microsoft's "AI" Lab, It's Time to Talk About the Gradual Death of Windows and Implosion of the "AI" Bubble
Earlier this week we showed that mostly Asian media had the 'nerve' to mention Microsoft silently shutting down its 'AI' lab
More Gains for GNU/Linux, Based on Web Surveys
the Steam site shows rapid growth for "Linux" this month
 
Microsoft's Trouble in Africa and Asia
A new all-time high for GNU/Linux
Brett Wilson LLP Reported to the Solicitors Regulation Authority (SRA)
The saddest thing in all this is that law firms can maintain high standards shall they wish to
Links 03/04/2025: Tariff Pains and C.D.C. Cuts
Links for the day
StatCounter: Microsoft is Masking a Disaster, It's Way Behind DeepSeek Already and Interest in LLMs Has Waned
it turns out the money "raised" for "Open" "AI" may not even exist at all
Links 03/04/2025: SoftBank Money for Microsoft "Open" "AI" Probably Doesn't Even Exist, Wikimedia Foundation Blasts LLM Nuisance While Microsoft Admits Demand Has Shrunk
Links for the day
Gemini Links 03/04/2025: Patch Panel and Pictures
Links for the day
Islamic Republic of Iran: GNU/Linux at All-time High This Month, Windows Falls to 12%
Vista 10 is up this month despite being "end of life" (EoL) soon
Indonesia: All-Time Highs for GNU/Linux
What's noteworthy right now is the growth of GNU/Linux
statCounter Says GNU/Linux Usage is Up Again (Internationally)
some preliminary April data
Only on April 1st Can the Free Software Foundation Associate With Microsoft's Open Source Initiative (OSI)
We saw some pranks that day linking the FSF to Microsoft (e.g. "endorsing" Windows)
IBM Gets Rid of Kelly Chambliss as Mass Layoffs Reported in IBM Consulting, IBM Loses Key Contracts/Graft
IBM Consulting has been in disarray lately
Slopwatch: Anti-Linux Articles, Not Even Written by Humans
Why aren't Web sites more vocal about this problem?
Over at Tux Machines...
GNU/Linux news for the past day
IRC Proceedings: Wednesday, April 02, 2025
IRC logs for Wednesday, April 02, 2025
Links 03/04/2025: Apple Fined Over Secret Surveillance, "Elegant Writer For A More Civilized Age"
Links for the day
Gemini Links 02/04/2025: Books and Cold Tea
Links for the day
Links 02/04/2025: More Layoffs, Nokia Again Takes Advantage of Illegal and Unconstitutional Patent Court With Nokia Staff as 'Judges'
Links for the day
Links 02/04/2025: Seizures and Returns to Windows of 24 Years Ago
Links for the day
LLM Slop Helps Obscure and Distort News About Layoffs (IBM, GAFAM)
It's hard to find accurate information
Links 02/04/2025: Microsoft Developers Are Threatening to Go on Strike, World Backup Day Noted
Links for the day
Gemini Protocol Has Growing Appeal (the Web Got Too Bloated and Full of LLM Slop)
For any "data plan" with bandwidth limits or "tiers" it would be cheaper to use/browse Geminispace
The Web Can Survive LLM Slop, But Only If We Collectively Shun and Discourage Serial Sloppers
Doing nothing ought not be a possibility
Amid Secret Shut-downs and Mass Layoffs at Microsoft (4 Waves of Layoffs in 3 Months of 2025) Some Microsoft Staff Expected to Go On Strike
workers going on strike
Gemini Links 02/04/2025: No more on Mastodon and Gemini Mention Script in Go
Links for the day
Over at Tux Machines...
GNU/Linux news for the past day
IRC Proceedings: Tuesday, April 01, 2025
IRC logs for Tuesday, April 01, 2025
My Motion Disbarring or “Striking Off” Brett Wilson LLP for Enabling Violent Americans Who Try to Crush Microsoft Critics in the United Kingdom by Multiple SLAPPs
"Guns for hire" (for Microsoft people who received Microsoft salaries)
The U.S. Patent and Trademark Office Hijacked Again by Patent Litigation Industry, as President Cheeto Prioritises Aggressors
The "mafia" has taken over the "industry" and the Federal system (justice and constitutions trampled upon)
Ubuntu Slop and FUD Manufactured With LLMs and Funded (by Oneself) 'Studies'
Slop and FUD are ruining the Web
Gemini Links 01/04/2025: Games and More
Links for the day
Links 01/04/2025: Apple Fined $162M for Privacy Abuses, Disinformation Online a Growing Concern
Links for the day
Why We're Reporting Brett Wilson LLP for Apparently Misusing Their Licence to Protect American Microsofters Who Attack Women
For those who have not been keeping abreast
Newer Press Reports Confirm That Microsoft Shuts Down 'Hey Hi' (AI) Labs Despite All the Hype
The "hey hi" (AI) bubble is not sustainable
Links 01/04/2025: Mass Layoffs at Eidos and "Microsoft Pulls Back on Data Centers" (Demand Lacking); "Racist and Sexist" Slop From Microsoft
Links for the day
Stefano Maffulli and His Microsoft-Funded OSI Staff Are Killing the OSI and Killing "Open Source" (All for Money!)
This is far from over
Gemini Links 01/04/2025: XKCDpunk and worldclock.py
Links for the day
50 Years of Sabotage and a Gut Punch to Computer Science (and Science in General)
Will we get back to science-based computing rather than cult-like following?
Techrights Headlines as Semaphore
"If you are hearing this, thank you"
3 Months in 2025, 4 Waves of Mass Layoffs at Microsoft, Now Offices Shut Down Permanently
"A recent visit by the South China Morning Post confirmed that the office was dark, unoccupied, and had its logo removed."
Over at Tux Machines...
GNU/Linux news for the past day
IRC Proceedings: Monday, March 31, 2025
IRC logs for Monday, March 31, 2025