Bonum Certa Men Certa

It Looks Like Red Hat's (IBM) Fedora Project May be 'Outsourced' to Amazon's Datacentres

Amazon datacentres in Ashburn
Amazon datacentres in Ashburn (source: Wikileaks)



Summary: In "seeking a more modern and cost effective location" for Fedora Infrastructure it seems to have been decided, privately, that Amazon (AWS) would be the new home of this project; but there's sufficient obfuscation surrounding the matter and many people seem to be totally unaware

OVERNIGHT we've spent several hours researching this and we also asked some Fedora people, who don't seem to know exactly what's going on. Surely some people higher up the corporate chain have a better and clearer idea, but they're not likely to speak about it because IBM pays their salary and there's fear of further layoffs. A lot of the finer details will be available in IRC logs (frustratingly enough, almost all the coding/code hosting was outsourced to Microsoft), but the gist of it all is this: while examining about a dozen services of the Fedora Project we've occasionally stumbled upon Amazon addresses. And then there's this:



dig copr.fedorainfracloud.org



; <<>> DiG 9.11.5-P4-5.1+deb10u1-Debian <<>> copr.fedorainfracloud.org ;; global options: +cmd ;; Got answer: ;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 34423 ;; flags: qr rd ra; QUERY: 1, ANSWER: 2, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION: ; EDNS: version: 0, flags:; udp: 512 ;; QUESTION SECTION: ;copr.fedorainfracloud.org. IN A

;; ANSWER SECTION: copr.fedorainfracloud.org. 300 IN CNAME copr-fe.aws.fedoraproject.org. copr-fe.aws.fedoraproject.org. 939 IN A 3.81.0.123


This probably merits a little bit of context. A few hours ago Fedora Infrastructure had another very major OpenShift failure (not the first such incident over the past few days). Well, it wasn't an issue until lately...

This so-called 'cloud computing' software is Red Hat's brand (with GUIs and everything) of something that's otherwise widely available as Free software but harder to use.

"Fedora Infrastructure Status: Major service disruption," said the notice, reading out the extent of the impact: "New status major: issues with openshift cluster for services: Ipsilon, Badges, Blockerbugs, Package Updates Manager, Fedora Infrastructure Cloud, COPR Build System, Documentation website, Fedora elections, Account System, Fedora Messaging Bus, Fedora Calendar, Fedora pastebin service, The Koji Buildsystem, Koschei Continuous Integration, Kerberos, Mailing Lists, Module Build Service, Mirror List, Mirror Manager, Fedora Packages App, Pagure, Fedora People, Package maintainers git repositories, Fedora Container Registry, ABRT Server, Fedora websites, Fedora Wiki, Zodbot IRC bot"

"...while examining about a dozen services of the Fedora Project we've occasionally stumbled upon Amazon addresses."And later on it said "New status good: Everything seems to be working."

"We are now operating from the new datacentre and are bringing back capacity as time allows. The known issues are documented here in our wiki," it adds. If one goes to the corresponding status page for pertinent services ("Fedora Project Datacentre Move") it is linking to this wiki page, which in turn says: "This is a page to track known issues after the Fedora Infrastructure Datacenter move in 2020 from phx2 to iad2 and rdu2."

Those are not descriptive names, as there's not even a domain name/suffix. We tried tracing back the IP addresses of these services to identity where at least some these services were being moved to. "I think it's more Red Hat management pressure to rely on Github and Google," Ryan said, but it seems to be related to neither. "You know," Ryan continued, "the infrastructure self-hosting was VERY reliable for many years before that." He mentioned "GMail, Google Drive, calendar stuff" as examples, citing the anonymous post that said about Jim Whitehurst: "He didn't use our products (used a Mac, along with other members of the executive management team), allowed RH to dump open source solutions for our own business to move things to Google's services (which is a huge message to our very clients of "we don't value or trust open source, so why should you?")"

"A few hours ago Fedora Infrastructure had another very major OpenShift failure (not the first such incident over the past few days)."The migration pages are edited mostly by Kevin Fenzi (IBM/Red Hat), who wrote about it less than a year ago in the devel-announce mailing list. No replies to it as of this date.

Is Copr on AWS now? Seems so. If the new DC of Fedora is Amazon-owned and Amazon-controlled, there may be something to conclude. Depending on one's opinion regarding Amazon and AWS, which turns Free/libre things into proprietary.

This message from last year says the hosting of various services is moving to the city/area in VA where AWS has many datacentres that already route some Fedora traffic. While we don't know for sure what's going on, as we need to better understand what plans are made in the boardroom (behind closed doors), it seems likely there's a move to Amazon.

Four years ago The Atlantic published "Where Are Amazon's Data Centers?" and it took note of Ashburn and Sterling. About a year later Wikileaks revealed that both places have lots of datacentres in them, mostly Ashburn (shown above is our screenshot of these).

Going back to Kevin, less than a year ago he wrote: "Fedora Infrastructure currently has the majority of its hardware in a datacenter in Arizona, USA. Red Hat leases this space for use by a number of teams, including Fedora."

"This message from last year says the hosting of various services is moving to the city/area in VA where AWS has many datacentres that already route some Fedora traffic."We can confirm, based on lots of checks, that many services still link to that datacentre.

"However," Kevin continued, "they've been seeking a more modern and cost effective location for some time and have decided on one: So, we will be migrating to a new datacenter located in Ashburn, Virginia in 2020."

Oh, Ashburn. Where Amazon has a densely-distributed cluster of datacentres.

Is Red Hat moving to 'the cloud' and, if so, why not its own? Does IBM feel like it needs to outsource that to a competitor/partner?

The details on this are still not certain and I could not get anybody from Fedora (in IRC) to confirm or deny. Quite frankly, many of them may not be aware.

Recent Techrights' Posts

The Mainstream Media Took 4 Days to Realise Microsoft Shut Down Its Operations in Pakistan and Fired Everybody
We estimate that Microsoft has had about 29,000 layoffs since January
“Twibel” Actions Against Comedians (and Why It's a Truly Low Blow)
they try to make up in quantities for a lack of merit or quality
Rewriting Things in Rust
How far would you go?
 
PCLinuxOS Sites Coming Back, Gradually
let's just be patient
Social Control Media, Even If Based on Free Software, Still Has Many Problems
a distraction from what actually mattered and still matters
IBM is Not Your Master
IBM makes friends with people who exclude the majority of the population: women
Help Fund the Free Software Foundation (FSF)
If you have some dollars to spare, go support the FSF
Over at Tux Machines...
GNU/Linux news for the past day
IRC Proceedings: Saturday, July 05, 2025
IRC logs for Saturday, July 05, 2025
A Short History of Attacks on Techrights (and Boycott Novell Before That)
good opportunity to tell again the story of several (not all) attempts to silence us
Leadership in Free Software
Don't let IBM lead. It's a terrible flag bearer.
Linux Foundation Apparently Flirting With Slop (Marketing by LLM-Generated SPAM)
The Web is in a really bad state!
COVID-19 Sped Up Site Improvements in Techrights
A few months later we created our very own IRC network
Gemini Links 05/07/2025: Negative Questions and 'Touching Grass' (Going Outside)
Links for the day
Links 05/07/2025: Dalai Lama Succession as 90th Birthday Approaches, 40 deg C in China
Links for the day
Links 05/07/2025: Hungary and US Defecting to Russia, "Google's Hotseat Hypocrisy"
Links for the day
Gemini Links 05/07/2025: 4th of July 2025 and "Zig Roadmap 2026"
Links for the day
How to Combat the Exploitation and Abuse by Microsoft GitHub
Not to mention corruption and crimes against women
Bryan Lunduke is Actually Sending His Audience to Attack People
"[Lunduke] is actually sending his audience to attack people."
Even The Right Wing is Rejecting Bryan Lunduke
no wonder he became so irrelevant and marginal
Microsoft's MSN Helps Microsoft Spread Lies About the Layoffs' Scale (Well Over 25,000 People Laid Off This Year)
There seem to be monopolies on lies and on truth
The Death of X Has Been Greatly Exaggerated (by Compromised Media)
X.Org Server is alive and well
In 2025 Everything is "AI". Remember Blockchains?
Talk about what companies and things (services, products, software) actually do, not the labels they use
Julian Assange Has Been Free for a Year
Julian Assange and I disagreed on some things
Monopolies and Scalping
Monopolies gravitate towards price hikes
Over at Tux Machines...
GNU/Linux news for the past day
IRC Proceedings: Friday, July 04, 2025
IRC logs for Friday, July 04, 2025
Microsoft's August Layoffs Wave: "August is Confirmed for Additional Performance Based Cuts"
"August is confirmed for additional performance based cuts from the recent connects along with additional organizational cuts."
What Microsoft Reputation Laundering (With a Weaponised Law Degree) Looks Like in a Foreign Continent
You would expect this in uncivilised and primitive countries
Slopwatch: LLMs 'Write' Fake or Distorted 'News' About "Linux"
LLM slop disguised as news
Links 04/07/2025: Google Replaces the Web With Slop, "AI Might Kill Us All"
Links for the day
Gemini Links 04/07/2025: Mindfulness and F1
Links for the day
Weeks After Microsoft Bankruptcy in Russia the Company Shuts Down in Pakistan, Too
Last month Windows' share in Pakistan fell to an all-time low
Rob Musial's June 2025 Additions of Malware in Proprietary Software
Via the GNU Web site this week
Links 04/07/2025: Microsoft's H-1B Visa Applications Show Another Crisis Unfolding, Many More Deep Cuts and Shutdowns Revealed, Complete Microsoft Exits
Links for the day
Gemini Links 04/07/2025: A Day To Remember and "Stop Killing Games"
Links for the day
Crime and Corruption at Microsoft GitHub Cannot be Covered Up by SLAPPs in Another Continent
We'll write about this for a long time to come
Slop Videos Are Disappointing Garbage, Nothing New, Just Brute Force up on Display or a Pedestal of Slop
Slop videos aren't a new thing
Over at Tux Machines...
GNU/Linux news for the past day
IRC Proceedings: Thursday, July 03, 2025
IRC logs for Thursday, July 03, 2025
The War on Local Storage (People Hosting Their Files Locally and Privately)
There's nothing wrong with controlling one's computing
What Digital Independence Means
Independence in the digital realms means abandoning platforms like GitHub, not just rejecting proprietary software
NVidia is a Bubble
they temporarily see fortunes and wrongly assume perpetuity thereof
Fedora Does Not Care About Diversity and Inclusion, It's About Optics (Corporate Image)
any notion of inclusion is superficial and misleading
Don't Buy the Excuses for Microsoft's Mass Layoffs
Back in the 90s, Microsoft bought a lot of companies to get and stay ahead