Bonum Certa Men Certa

It Looks Like Red Hat's (IBM) Fedora Project May be 'Outsourced' to Amazon's Datacentres

Amazon datacentres in Ashburn
Amazon datacentres in Ashburn (source: Wikileaks)



Summary: In "seeking a more modern and cost effective location" for Fedora Infrastructure it seems to have been decided, privately, that Amazon (AWS) would be the new home of this project; but there's sufficient obfuscation surrounding the matter and many people seem to be totally unaware

OVERNIGHT we've spent several hours researching this and we also asked some Fedora people, who don't seem to know exactly what's going on. Surely some people higher up the corporate chain have a better and clearer idea, but they're not likely to speak about it because IBM pays their salary and there's fear of further layoffs. A lot of the finer details will be available in IRC logs (frustratingly enough, almost all the coding/code hosting was outsourced to Microsoft), but the gist of it all is this: while examining about a dozen services of the Fedora Project we've occasionally stumbled upon Amazon addresses. And then there's this:



dig copr.fedorainfracloud.org



; <<>> DiG 9.11.5-P4-5.1+deb10u1-Debian <<>> copr.fedorainfracloud.org ;; global options: +cmd ;; Got answer: ;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 34423 ;; flags: qr rd ra; QUERY: 1, ANSWER: 2, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION: ; EDNS: version: 0, flags:; udp: 512 ;; QUESTION SECTION: ;copr.fedorainfracloud.org. IN A

;; ANSWER SECTION: copr.fedorainfracloud.org. 300 IN CNAME copr-fe.aws.fedoraproject.org. copr-fe.aws.fedoraproject.org. 939 IN A 3.81.0.123


This probably merits a little bit of context. A few hours ago Fedora Infrastructure had another very major OpenShift failure (not the first such incident over the past few days). Well, it wasn't an issue until lately...

This so-called 'cloud computing' software is Red Hat's brand (with GUIs and everything) of something that's otherwise widely available as Free software but harder to use.

"Fedora Infrastructure Status: Major service disruption," said the notice, reading out the extent of the impact: "New status major: issues with openshift cluster for services: Ipsilon, Badges, Blockerbugs, Package Updates Manager, Fedora Infrastructure Cloud, COPR Build System, Documentation website, Fedora elections, Account System, Fedora Messaging Bus, Fedora Calendar, Fedora pastebin service, The Koji Buildsystem, Koschei Continuous Integration, Kerberos, Mailing Lists, Module Build Service, Mirror List, Mirror Manager, Fedora Packages App, Pagure, Fedora People, Package maintainers git repositories, Fedora Container Registry, ABRT Server, Fedora websites, Fedora Wiki, Zodbot IRC bot"

"...while examining about a dozen services of the Fedora Project we've occasionally stumbled upon Amazon addresses."And later on it said "New status good: Everything seems to be working."

"We are now operating from the new datacentre and are bringing back capacity as time allows. The known issues are documented here in our wiki," it adds. If one goes to the corresponding status page for pertinent services ("Fedora Project Datacentre Move") it is linking to this wiki page, which in turn says: "This is a page to track known issues after the Fedora Infrastructure Datacenter move in 2020 from phx2 to iad2 and rdu2."

Those are not descriptive names, as there's not even a domain name/suffix. We tried tracing back the IP addresses of these services to identity where at least some these services were being moved to. "I think it's more Red Hat management pressure to rely on Github and Google," Ryan said, but it seems to be related to neither. "You know," Ryan continued, "the infrastructure self-hosting was VERY reliable for many years before that." He mentioned "GMail, Google Drive, calendar stuff" as examples, citing the anonymous post that said about Jim Whitehurst: "He didn't use our products (used a Mac, along with other members of the executive management team), allowed RH to dump open source solutions for our own business to move things to Google's services (which is a huge message to our very clients of "we don't value or trust open source, so why should you?")"

"A few hours ago Fedora Infrastructure had another very major OpenShift failure (not the first such incident over the past few days)."The migration pages are edited mostly by Kevin Fenzi (IBM/Red Hat), who wrote about it less than a year ago in the devel-announce mailing list. No replies to it as of this date.

Is Copr on AWS now? Seems so. If the new DC of Fedora is Amazon-owned and Amazon-controlled, there may be something to conclude. Depending on one's opinion regarding Amazon and AWS, which turns Free/libre things into proprietary.

This message from last year says the hosting of various services is moving to the city/area in VA where AWS has many datacentres that already route some Fedora traffic. While we don't know for sure what's going on, as we need to better understand what plans are made in the boardroom (behind closed doors), it seems likely there's a move to Amazon.

Four years ago The Atlantic published "Where Are Amazon's Data Centers?" and it took note of Ashburn and Sterling. About a year later Wikileaks revealed that both places have lots of datacentres in them, mostly Ashburn (shown above is our screenshot of these).

Going back to Kevin, less than a year ago he wrote: "Fedora Infrastructure currently has the majority of its hardware in a datacenter in Arizona, USA. Red Hat leases this space for use by a number of teams, including Fedora."

"This message from last year says the hosting of various services is moving to the city/area in VA where AWS has many datacentres that already route some Fedora traffic."We can confirm, based on lots of checks, that many services still link to that datacentre.

"However," Kevin continued, "they've been seeking a more modern and cost effective location for some time and have decided on one: So, we will be migrating to a new datacenter located in Ashburn, Virginia in 2020."

Oh, Ashburn. Where Amazon has a densely-distributed cluster of datacentres.

Is Red Hat moving to 'the cloud' and, if so, why not its own? Does IBM feel like it needs to outsource that to a competitor/partner?

The details on this are still not certain and I could not get anybody from Fedora (in IRC) to confirm or deny. Quite frankly, many of them may not be aware.

Recent Techrights' Posts

The Price of Exposing Corruption in Poland (and Elsewhere)
It's easier to participate in corruption than to merely do the right thing and oppose it
Abuse Inside the Polish Patent Office (UPRP) - Part IX: Minimum Wages For You (Experienced Scientist), Alicante/EU Paydays For Me (Unproductive, Corrupt Official)
Does UPRP maladministration extend to the false belief that qualified and experienced scientists can play the role of circus clowns?
"The Liberating Power of Simply Telling People the Truth."
'polite' bullying
Who Imitates Who? Plagiarist as Client (From Microsoft), 'Plagiarism' at the Law Firm?
let's revisit the subject
EPO's Central Staff Committee (CSC) Scrutinises the Man Who Illegally Grants (and Forces Others to Illegally Participate in Granting) Software Patents in Europe
EPO compels examiners to break the law in the name of obeying illegal "rules" or "orders"
The Latest Rumour Says The Next (as Correctly Predicted Before) Wave of Layoffs at Microsoft is 3 Weeks Away, "Larger Than the First Wave"
Step 2
TV Licensing Used to SPAM Your Postbox, Now It Does the Same to E-mail
First they ask for your E-mail address; then they start nagging you via E-mail
 
The More Time Passes, the Better Our Advice on Social Control Media Seems
At the end of the day, any platform you do not control yourself is working for someone else
Twitter (X) is Dying, Now It's Just Like a Mafia-Type Operation of the Man Who Does Nazi Salutes in Public
a form of extortion
UK High Court Blasts Brett Wilson LLP for Misusing "GDPR" After Failed Efforts to Censor Critics Using 'Libel' Claims
No wonder this firm is rapidly shrinking
Recent Blunders in Microsoft GitHub (e.g. Slop-Generated Bug Reports or GPL Violations 'as a Service') Taking Their Toll?
Put bluntly, if you still use Microsoft GitHub, then you're slave to Microsoft
American Imperialism and Microsoft Plagiarism
Techrights will therefore do what Microsoft does not want it to do: it'll write even more about Microsoft
When They Have Nothing Left to Help Advance Abusive Litigation for Microsoft People... Other Than Throwing ~500 Pages of Someone Else's Work Into a PDF
Microsoft is having a very tough year
Slopwatch and Yet More Holes in 'Secure Boot' (as Usual!), Promoted Inside Linux by the Man We Are Suing
Today's Slopwatch will be short
Gemini Links 13/06/2025: People You've Left Behind, Life Update and OS Changes
Links for the day
Over at Tux Machines...
GNU/Linux news for the past day
IRC Proceedings: Thursday, June 12, 2025
IRC logs for Thursday, June 12, 2025
Links 12/06/2025: Portland Homeless Deaths Quadruple, COVID Cases Surge in Asia
Links for the day
EPO's Gareth Lord Asked About "Quality and Productivity" or, Put Another Way, Why the EPO Keeps Granting So Many Invalid/Illegal Patents
letter to Lord
The Toxic Playbook
Either you support Prince Mohammed bin Salman or you're a nazi
It's Possible That BetaNews Got Cracked, But Nobody Talks About It, The Site Contains an Outdated Old Image, No Activity
It's possible that they will never explain what happened to the site and users' accounts
Links 12/06/2025: Beach Boys’ Brian Wilson Dies
Links for the day
Gemini Links 12/06/2025: Video Game Diegesis and Steam Next Fest
Links for the day
Why the Militants Have Lost Every Battle Since 2022 (When Attacking My Wife and I in Various Ways, Even Attacking Our Employers)
This takes patience, sure, but at the end most evildoers face the consequences for their actions
Our Priority is Still Tackling Software Patents and Corruption in Patent Offices
Meanwhile we got compliments on our recent articles, which means that they are effective
Politics Will Impact Software Choices
Will those systems respect users' freedom?
EPO: Neglecting Children to Promote American Monopolies by Shielding Them From European Competition
Yesterday the Central Staff Committee at the EPO spoke about another "reform" at the Office
Slopwatch: Another Day, Another Slopfest, LLM Slop Scrapers Slow Down Our Site
We too have some slop issues; this past day this site and the sister site had to answer about 2.5 million requests (not counting Gemini Protocol) and it's slowing things down for everybody
Over at Tux Machines...
GNU/Linux news for the past day
IRC Proceedings: Wednesday, June 11, 2025
IRC logs for Wednesday, June 11, 2025
Links 11/06/2025: More Vulnerabilities Found in 'Smart' Phones, China Extends Reach in the Pacific
Links for the day
Gemini Links 11/06/2025: Grain and Steam Next Fest
Links for the day
Links 11/06/2025: "Quantum" Hype From IBM, US Closer to Martial Law, and “The Nation” Celebrates Milestone
Links for the day
IBM's CEO Roasted, Sizzled and Grilled for Dumb and Inconsistent Vapourware Promises
It looks like being a chronic liar is what it takes to lead the company once synonymous with computing
IBM's Goal Is Not (and Never Was) Computer Users' Freedom
More than 1.5 decades ago I found IBM to be an "ally of convenience" because of OpenDocument Format (ODF)
Wayland Shows the IBM/Red Hat Way of Doing Things
IBM is trying to 'kill' X
GitHub is Proprietary, Controlled by Microsoft, and GPL Violation Warehouse
"IRS tax filing software [will be] released to the people as free software" ... In general this is good news
Slopfarm Catastrophe
Seems like BetaNews (or BetaNoise) has just suffered a major data loss and restored the site from a week-old backup
Abuse Inside the Polish Patent Office (UPRP) - Part VIII: Illegal Working Conditions
How many people need to die for these people to get their massive salaries?
Over at Tux Machines...
GNU/Linux news for the past day
IRC Proceedings: Tuesday, June 10, 2025
IRC logs for Tuesday, June 10, 2025