Bonum Certa Men Certa

It Looks Like Red Hat's (IBM) Fedora Project May be 'Outsourced' to Amazon's Datacentres

Amazon datacentres in Ashburn
Amazon datacentres in Ashburn (source: Wikileaks)



Summary: In "seeking a more modern and cost effective location" for Fedora Infrastructure it seems to have been decided, privately, that Amazon (AWS) would be the new home of this project; but there's sufficient obfuscation surrounding the matter and many people seem to be totally unaware

OVERNIGHT we've spent several hours researching this and we also asked some Fedora people, who don't seem to know exactly what's going on. Surely some people higher up the corporate chain have a better and clearer idea, but they're not likely to speak about it because IBM pays their salary and there's fear of further layoffs. A lot of the finer details will be available in IRC logs (frustratingly enough, almost all the coding/code hosting was outsourced to Microsoft), but the gist of it all is this: while examining about a dozen services of the Fedora Project we've occasionally stumbled upon Amazon addresses. And then there's this:



dig copr.fedorainfracloud.org



; <<>> DiG 9.11.5-P4-5.1+deb10u1-Debian <<>> copr.fedorainfracloud.org ;; global options: +cmd ;; Got answer: ;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 34423 ;; flags: qr rd ra; QUERY: 1, ANSWER: 2, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION: ; EDNS: version: 0, flags:; udp: 512 ;; QUESTION SECTION: ;copr.fedorainfracloud.org. IN A

;; ANSWER SECTION: copr.fedorainfracloud.org. 300 IN CNAME copr-fe.aws.fedoraproject.org. copr-fe.aws.fedoraproject.org. 939 IN A 3.81.0.123


This probably merits a little bit of context. A few hours ago Fedora Infrastructure had another very major OpenShift failure (not the first such incident over the past few days). Well, it wasn't an issue until lately...

This so-called 'cloud computing' software is Red Hat's brand (with GUIs and everything) of something that's otherwise widely available as Free software but harder to use.

"Fedora Infrastructure Status: Major service disruption," said the notice, reading out the extent of the impact: "New status major: issues with openshift cluster for services: Ipsilon, Badges, Blockerbugs, Package Updates Manager, Fedora Infrastructure Cloud, COPR Build System, Documentation website, Fedora elections, Account System, Fedora Messaging Bus, Fedora Calendar, Fedora pastebin service, The Koji Buildsystem, Koschei Continuous Integration, Kerberos, Mailing Lists, Module Build Service, Mirror List, Mirror Manager, Fedora Packages App, Pagure, Fedora People, Package maintainers git repositories, Fedora Container Registry, ABRT Server, Fedora websites, Fedora Wiki, Zodbot IRC bot"

"...while examining about a dozen services of the Fedora Project we've occasionally stumbled upon Amazon addresses."And later on it said "New status good: Everything seems to be working."

"We are now operating from the new datacentre and are bringing back capacity as time allows. The known issues are documented here in our wiki," it adds. If one goes to the corresponding status page for pertinent services ("Fedora Project Datacentre Move") it is linking to this wiki page, which in turn says: "This is a page to track known issues after the Fedora Infrastructure Datacenter move in 2020 from phx2 to iad2 and rdu2."

Those are not descriptive names, as there's not even a domain name/suffix. We tried tracing back the IP addresses of these services to identity where at least some these services were being moved to. "I think it's more Red Hat management pressure to rely on Github and Google," Ryan said, but it seems to be related to neither. "You know," Ryan continued, "the infrastructure self-hosting was VERY reliable for many years before that." He mentioned "GMail, Google Drive, calendar stuff" as examples, citing the anonymous post that said about Jim Whitehurst: "He didn't use our products (used a Mac, along with other members of the executive management team), allowed RH to dump open source solutions for our own business to move things to Google's services (which is a huge message to our very clients of "we don't value or trust open source, so why should you?")"

"A few hours ago Fedora Infrastructure had another very major OpenShift failure (not the first such incident over the past few days)."The migration pages are edited mostly by Kevin Fenzi (IBM/Red Hat), who wrote about it less than a year ago in the devel-announce mailing list. No replies to it as of this date.

Is Copr on AWS now? Seems so. If the new DC of Fedora is Amazon-owned and Amazon-controlled, there may be something to conclude. Depending on one's opinion regarding Amazon and AWS, which turns Free/libre things into proprietary.

This message from last year says the hosting of various services is moving to the city/area in VA where AWS has many datacentres that already route some Fedora traffic. While we don't know for sure what's going on, as we need to better understand what plans are made in the boardroom (behind closed doors), it seems likely there's a move to Amazon.

Four years ago The Atlantic published "Where Are Amazon's Data Centers?" and it took note of Ashburn and Sterling. About a year later Wikileaks revealed that both places have lots of datacentres in them, mostly Ashburn (shown above is our screenshot of these).

Going back to Kevin, less than a year ago he wrote: "Fedora Infrastructure currently has the majority of its hardware in a datacenter in Arizona, USA. Red Hat leases this space for use by a number of teams, including Fedora."

"This message from last year says the hosting of various services is moving to the city/area in VA where AWS has many datacentres that already route some Fedora traffic."We can confirm, based on lots of checks, that many services still link to that datacentre.

"However," Kevin continued, "they've been seeking a more modern and cost effective location for some time and have decided on one: So, we will be migrating to a new datacenter located in Ashburn, Virginia in 2020."

Oh, Ashburn. Where Amazon has a densely-distributed cluster of datacentres.

Is Red Hat moving to 'the cloud' and, if so, why not its own? Does IBM feel like it needs to outsource that to a competitor/partner?

The details on this are still not certain and I could not get anybody from Fedora (in IRC) to confirm or deny. Quite frankly, many of them may not be aware.

Recent Techrights' Posts

Hate Mail From Anonymous Cowards
if this persists, we'll need to escalate
Informal Open Letter to the Lawyer of the Microsofters (on Who's Funding the SLAPPs Against Techrights)
Whenever I ask about the funding they try to change the subject and act all aggressive
Microsoft Lunduke is Just Provoking People for Provocation's Sake
Be forewarned and remember where this guy came from: Microsoft
 
Eventually UEFI 'Secure Boot' Will be Dropped (Users Will Demand Its Removal and Boycott Its Pushers)
we expect OEMs will just listen to users
The Register MS: We Know Slop is a Bubble and Mindless Hype, But We Get Paid to Participate
Call out the culprits
There Are Probably Over a Million Pages in Geminispace
there are two many limitations which merit a mention when it comes to assessing magnitude
Besieged by Plagiarists Who Play With LLMs and Image Fusions
We really need to exercise or use our collective voice to oppose Serial Sloppers
Over at Tux Machines...
GNU/Linux news for the past day
IRC Proceedings: Friday, August 08, 2025
IRC logs for Friday, August 08, 2025
Gemini Links 09/08/2025: Water Painting and Political Violence
Links for the day
Slopwatch: LLM Sloppers in Google News, LinuxSecurity, and More
they also perpetuate some falsehoods as the LLMs lack any comprehension
Links 08/08/2025: China King of Plastics and US Dictator Plans to Meet Russian Dictator
Links for the day
Gemini Links 08/08/2025: Cracking a Family Member's Password and Overdose of Slop
Links for the day
Red Hat's Latest Talent Hunt, Day Ahead of Mass Layoffs, is Yet Another Microsoft Executive
Red Hat will apparently commence mass layoffs early this coming Monday
Links 08/08/2025: "Quit Facebook" and High Cost of Microsoft/Windows Shown Again ("BlackSuit")
Links for the day
Good Morning, Readers of The Register MS
Things The Register MS could (but does not) cover this morning
Why Gemini Protocol Has a Bright Future
Maybe Gemini Protocol's promise becomes more appealing as the Web turns to slop and bloat
It's a Lot Easier to Participate in the Unethical System Than to Oppose Injustices in It
Going after powerful and high-budget interests is never easy
Microsofters Filed Two SLAPPs Against Us, Now They Cannot Keep Up With Judges' Orders
For over 4 months already their facilitator in London has been under investigation by British authorities because of what's being done to my wife and I
Censorship Regarding Red Hat Layoffs
Talk about this? They'd rather not.
Struggling to Cut Costs, Microsoft Continues Shutting Down and Cancelling Stuff This Month
There are August layoffs at Microsoft
Over at Tux Machines...
GNU/Linux news for the past day
IRC Proceedings: Thursday, August 07, 2025
IRC logs for Thursday, August 07, 2025
Fake 'Linux' Articles, Written by Bots to Take Traffic Away From Real Articles
LLM slop helps replace information with junk or misinformation
When Google's Googlebombing of "Gemini" Was Not Enough; They Now Also Googlebomb "Gemini Space"?
We know GAFAM not only worries about Gemini Protocol but also attempts to 'infiltrate' Geminispace
The Register MS Promotes Microsoft Slop, Assumes All Readers Use Microsoft Windows
Microsoft really dominates the site
Gemini Links 08/08/2025: KDE/Qt Development and What's Missing From "Retro"
Links for the day
Links 07/08/2025: US Punishes India Instead of Russia, Attacks Law Firms to Prevent Scrutiny
Links for the day
Read Us in Geminispace as Well
it's definitely a lot simpler than using a Web browser
Once a Site About BSD and GNU/Linux, and After Months of Silence, LinuxBSDos.com Comes Back Only as a Slopfarm
very frustrating
Links 07/08/2025: Hardware Wars, Mass Recall of Colgate Total Clean Mint, More Microsoft Holes Found
Links for the day
Gemini Links 07/08/2025: "Right To Manage" and LoRa Analysis
Links for the day
For the First Time in a Month OSI's "OpenSource.org" Blogs and It's Basically a Microsoft Blog Post (Microsoft Controls OSI)
For the first time in a month OSI writes something and it is Microsoft propaganda composed by a Microsoft-salaried operative
Microsoft, Already Borrowing 3 Billion Dollars a Month, is Trying to Cause Many People to Resign
MSN (i.e. Microsoft) and others openly admit it
GAFAM 'Says' is Front Page "News"
The point of journalism is to check and assess facts, not parrot what people and companies merely claim
Links 07/08/2025: Apple Makes False Promises, More Trouble for Microsoft
Links for the day
OSS Didn't Always Mean Open Source Software
"oligarchs all the way down"
The Register MS Does More Microsoft Sez or GitHub Sez (Says) Pieces
60 minutes ago
They Want Activists to Just Barely Walk and Eat, Not Do Activism Anymore
It's sort of like the ending of '1984'
Quit Perpetuating the Narrative of Gemini Protocol 'Dying' (It's False)
The "whisper campaign" against Gemini Protocol
Criticising Social Control Media in Social Control Media
Many people are quitting Social Control Media (fewer of them announce this in public)
Non-Free JavaScript Programs in Banks Aren't Even the Biggest Problem
Technology was supposed to make life easier; in practice, however, for most of us the opposite effect can be observed
Slopfarms Are Typically Fake News
Slopfarms typically relay falsehoods
Gemini Links 06/08/2025: Replacing a Pocket Watch and Buying in Bulk
Links for the day
IBM is Obliterating Fedora
"Fedora releases were shipping with an increasing number of bugs on launch day even while I was using it for a several year stretch."
Over at Tux Machines...
GNU/Linux news for the past day
IRC Proceedings: Wednesday, August 06, 2025
IRC logs for Wednesday, August 06, 2025