Bonum Certa Men Certa

It Looks Like Red Hat's (IBM) Fedora Project May be 'Outsourced' to Amazon's Datacentres

Amazon datacentres in Ashburn
Amazon datacentres in Ashburn (source: Wikileaks)



Summary: In "seeking a more modern and cost effective location" for Fedora Infrastructure it seems to have been decided, privately, that Amazon (AWS) would be the new home of this project; but there's sufficient obfuscation surrounding the matter and many people seem to be totally unaware

OVERNIGHT we've spent several hours researching this and we also asked some Fedora people, who don't seem to know exactly what's going on. Surely some people higher up the corporate chain have a better and clearer idea, but they're not likely to speak about it because IBM pays their salary and there's fear of further layoffs. A lot of the finer details will be available in IRC logs (frustratingly enough, almost all the coding/code hosting was outsourced to Microsoft), but the gist of it all is this: while examining about a dozen services of the Fedora Project we've occasionally stumbled upon Amazon addresses. And then there's this:



dig copr.fedorainfracloud.org



; <<>> DiG 9.11.5-P4-5.1+deb10u1-Debian <<>> copr.fedorainfracloud.org ;; global options: +cmd ;; Got answer: ;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 34423 ;; flags: qr rd ra; QUERY: 1, ANSWER: 2, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION: ; EDNS: version: 0, flags:; udp: 512 ;; QUESTION SECTION: ;copr.fedorainfracloud.org. IN A

;; ANSWER SECTION: copr.fedorainfracloud.org. 300 IN CNAME copr-fe.aws.fedoraproject.org. copr-fe.aws.fedoraproject.org. 939 IN A 3.81.0.123


This probably merits a little bit of context. A few hours ago Fedora Infrastructure had another very major OpenShift failure (not the first such incident over the past few days). Well, it wasn't an issue until lately...

This so-called 'cloud computing' software is Red Hat's brand (with GUIs and everything) of something that's otherwise widely available as Free software but harder to use.

"Fedora Infrastructure Status: Major service disruption," said the notice, reading out the extent of the impact: "New status major: issues with openshift cluster for services: Ipsilon, Badges, Blockerbugs, Package Updates Manager, Fedora Infrastructure Cloud, COPR Build System, Documentation website, Fedora elections, Account System, Fedora Messaging Bus, Fedora Calendar, Fedora pastebin service, The Koji Buildsystem, Koschei Continuous Integration, Kerberos, Mailing Lists, Module Build Service, Mirror List, Mirror Manager, Fedora Packages App, Pagure, Fedora People, Package maintainers git repositories, Fedora Container Registry, ABRT Server, Fedora websites, Fedora Wiki, Zodbot IRC bot"

"...while examining about a dozen services of the Fedora Project we've occasionally stumbled upon Amazon addresses."And later on it said "New status good: Everything seems to be working."

"We are now operating from the new datacentre and are bringing back capacity as time allows. The known issues are documented here in our wiki," it adds. If one goes to the corresponding status page for pertinent services ("Fedora Project Datacentre Move") it is linking to this wiki page, which in turn says: "This is a page to track known issues after the Fedora Infrastructure Datacenter move in 2020 from phx2 to iad2 and rdu2."

Those are not descriptive names, as there's not even a domain name/suffix. We tried tracing back the IP addresses of these services to identity where at least some these services were being moved to. "I think it's more Red Hat management pressure to rely on Github and Google," Ryan said, but it seems to be related to neither. "You know," Ryan continued, "the infrastructure self-hosting was VERY reliable for many years before that." He mentioned "GMail, Google Drive, calendar stuff" as examples, citing the anonymous post that said about Jim Whitehurst: "He didn't use our products (used a Mac, along with other members of the executive management team), allowed RH to dump open source solutions for our own business to move things to Google's services (which is a huge message to our very clients of "we don't value or trust open source, so why should you?")"

"A few hours ago Fedora Infrastructure had another very major OpenShift failure (not the first such incident over the past few days)."The migration pages are edited mostly by Kevin Fenzi (IBM/Red Hat), who wrote about it less than a year ago in the devel-announce mailing list. No replies to it as of this date.

Is Copr on AWS now? Seems so. If the new DC of Fedora is Amazon-owned and Amazon-controlled, there may be something to conclude. Depending on one's opinion regarding Amazon and AWS, which turns Free/libre things into proprietary.

This message from last year says the hosting of various services is moving to the city/area in VA where AWS has many datacentres that already route some Fedora traffic. While we don't know for sure what's going on, as we need to better understand what plans are made in the boardroom (behind closed doors), it seems likely there's a move to Amazon.

Four years ago The Atlantic published "Where Are Amazon's Data Centers?" and it took note of Ashburn and Sterling. About a year later Wikileaks revealed that both places have lots of datacentres in them, mostly Ashburn (shown above is our screenshot of these).

Going back to Kevin, less than a year ago he wrote: "Fedora Infrastructure currently has the majority of its hardware in a datacenter in Arizona, USA. Red Hat leases this space for use by a number of teams, including Fedora."

"This message from last year says the hosting of various services is moving to the city/area in VA where AWS has many datacentres that already route some Fedora traffic."We can confirm, based on lots of checks, that many services still link to that datacentre.

"However," Kevin continued, "they've been seeking a more modern and cost effective location for some time and have decided on one: So, we will be migrating to a new datacenter located in Ashburn, Virginia in 2020."

Oh, Ashburn. Where Amazon has a densely-distributed cluster of datacentres.

Is Red Hat moving to 'the cloud' and, if so, why not its own? Does IBM feel like it needs to outsource that to a competitor/partner?

The details on this are still not certain and I could not get anybody from Fedora (in IRC) to confirm or deny. Quite frankly, many of them may not be aware.

Recent Techrights' Posts

Our Case is a Very Easy Win, the SLAPPs From Microsofters Were a Grave Error, and Censoring Information Won't Work (It'll Only Ever Backfire)
Censoring is what people do when they lose the argument
 
Slopwatch: Mindless Slop Pieces, Fake Images and Text, Linux FUD on the Cheap
spewed out by Microsoft-controlled LLMs
Links 04/06/2025: Workers' Strikes, Sudan Exodus
Links for the day
Links 04/06/2025: Linux Foundation PR Spam and Lee Jae-myung Wins Election
Links for the day
Gemini Links 04/06/2025: Future Leaders of the World and Platforming Jordan Peterson
Links for the day
Links 04/06/2025: WSL Backfiring on Microsoft and "Disney, Microsoft Announce Massive Layoffs"
Links for the day
Say the Truth, the Rest Will Follow
There's no guarantee that writing the truth will result in an audience (or readership), but over time - in the long run - people generally gravitate towards what they know or feel to be crude truth, not just what's comforting (albeit false or self-deluding, usually groupthink dictated from above)
How to Expose High-Level Corruption Without Getting in (Too Much) Trouble
Democracy depends on free press and freedom of the press depends on being able to safely publish (and keep available) material that bad people don't want to be known to anybody
In-Depth EPO Coverage at Techrights Turns Eleven
11 years is a very long time
Windows Measured Below 10% in Afghanistan, GNU/Linux Gaining a Lot
about 80% are Android (Linux) users, compared to only about 10% for Windows
Poland's Political Predicament and Social Control Media
Democracy and fake "tech" don't mix well; the latter tends to interfere with the former and that's why we get more "Putins" out there
EPO: Taking Away From the Staff to Give More to the Rich
The Central Staff Committee (CSC) wrote to EPO staff earlier this week
Over at Tux Machines...
GNU/Linux news for the past day
IRC Proceedings: Tuesday, June 03, 2025
IRC logs for Tuesday, June 03, 2025
Abuse Inside the Polish Patent Office (UPRP) - Part I: It's a Lot Like the EPO
we can commence a series soon
Gemini Links 04/06/2025: Inescapable Questions and Quitting All "Oligarch Tech"
Links for the day
Slopwatch: Linux FUD From Slopfarms, Blaming Linux for Microsoft Issues; Even WebProNews Has Become a Slopfarm (Googlebombing "Linux" With Slop Images and Fake/Plagiarised Text)
The Web is really getting bad; it's also overwhelmed by fake material or plagiarised material, wherein the plagiarism gets disguised/hidden by LLM sausage factories
Links 03/06/2025: Tiananmen Square Massacre Censorship and Growing Military Activities Around Taiwan
Links for the day
Linux is Already Dominant (Android), Let's Make GNU/Linux Dominant in Desktops/Laptops as Well
"Dr. Stallman recently warned everybody about Microsoft."
The Loyalty to Microsoft and the Salaries From Microsoft (Funding SLAPPs Against Techrights and Tux Machines)
Garrett always knows better. He knows everything best.
Windows Falls in Italy as GNU/Linux Jumps to 5%
Italy knows a thing or two about digital autonomy
Nigeria is All Android and Google
Windows down to almost nothing in Africa's largest population
Mass Layoffs at Microsoft (Second Wave) Not Limited to Redmond
"More layoffs at Microsoft as axe falls in Washington and California"
Gemini Links 03/06/2025: Forth System and "Common Lisp is a Dumpster"
Links for the day
The Leaks Were Right: Mass Layoffs at Microsoft in May, Then Another Wave in June
Just as we've been saying for over a month
Over at Tux Machines...
GNU/Linux news for the past day
IRC Proceedings: Monday, June 02, 2025
IRC logs for Monday, June 02, 2025
Last Article From Australia's Sam Varghese Was a Year Ago and It Covered the Release of Julian Assange, Who Will Apparently Come Back as 'Politician'
It'll soon be exactly 12 months
Hungary Seems Hungry for Linux
Windows down by a lot
Like in Europe, Bad News for Microsoft in US and Canada
If it loses those "regions", then what's left?
About 8 Waves of Mass Layoffs at Microsoft in 2025 (in Less Than 5 Months), Now Vista 11 "Market Share" Decreases
Really bad news for shareholders of Microsoft
statCounter Sees Bing "Share" Falling Over 0.5% in One Month, Now Lower Than Before the ChatGPT/Bing Chat Hype
Bing has been part of the mass layoffs for quite some time
After Microsoft's Bankruptcy in Russia Android (Linux) Will Dominate Asia Completely
Windows probably peaked in "XP" or "2000"
Microsoft's Demise is a Global Phenomenon
mass layoffs justified using mindless buzzwords
All-Time Highs for GNU/Linux in EU and the UK, All-Time Lows for Microsoft
Combining ChromeOS and GNU/Linux, it adds up to and almost reaches 6%
India: Windows Falls to 50% in Desktops/Laptops and 8% Overall
laptops/desktops fell to 16% of the whole
statCounter: GNU/Linux Up to 4.7% "Market Share" This Month
30,000 Microsoft jobs may be eliminated by year's end
Microsoft is in Trouble and Microsofters Know It
"I've been happy on Win 3.11 for years."
[Video] New Introduction to Richard Stallman's Contributions Including GNU Emacs, GNU/Linux, and Software Freedom
from the channel previously bullied for supporting RMS
Links 02/06/2025: South Korea to Vote, Russia Blitzed From Within
Links for the day
Links 02/06/2025: Political Leftovers, DRM, and Patents
Links for the day
Links 02/06/2025: Microsoft Spins Layoffs as "Slop", Frontier Settles Lawsuit
Links for the day
When You Publicly Boast About Wanting to Violently Attack People (Even Colleagues) Finding a Job Will Prove Difficult
there's a lesson to be learned here
The Web We Lost, the Information Lost Due to Microsoft's Attacks on Companies Like Yahoo! (Before the LLM Slop Frenzy)
When it comes to news sites, what can we say?
Covering Corruption in Poland, Including a War on Science (Due to Bad Politicians)
What we're about to show is that skilled and experienced scientists in Poland are besieged by bureaucrats
Gemini Links 02/06/2025: "Star Wars Day" and "Security Day"
Links for the day
Over at Tux Machines...
GNU/Linux news for the past day
IRC Proceedings: Sunday, June 01, 2025
IRC logs for Sunday, June 01, 2025