Bonum Certa Men Certa

It Looks Like Red Hat's (IBM) Fedora Project May be 'Outsourced' to Amazon's Datacentres

Amazon datacentres in Ashburn
Amazon datacentres in Ashburn (source: Wikileaks)



Summary: In "seeking a more modern and cost effective location" for Fedora Infrastructure it seems to have been decided, privately, that Amazon (AWS) would be the new home of this project; but there's sufficient obfuscation surrounding the matter and many people seem to be totally unaware

OVERNIGHT we've spent several hours researching this and we also asked some Fedora people, who don't seem to know exactly what's going on. Surely some people higher up the corporate chain have a better and clearer idea, but they're not likely to speak about it because IBM pays their salary and there's fear of further layoffs. A lot of the finer details will be available in IRC logs (frustratingly enough, almost all the coding/code hosting was outsourced to Microsoft), but the gist of it all is this: while examining about a dozen services of the Fedora Project we've occasionally stumbled upon Amazon addresses. And then there's this:



dig copr.fedorainfracloud.org



; <<>> DiG 9.11.5-P4-5.1+deb10u1-Debian <<>> copr.fedorainfracloud.org ;; global options: +cmd ;; Got answer: ;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 34423 ;; flags: qr rd ra; QUERY: 1, ANSWER: 2, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION: ; EDNS: version: 0, flags:; udp: 512 ;; QUESTION SECTION: ;copr.fedorainfracloud.org. IN A

;; ANSWER SECTION: copr.fedorainfracloud.org. 300 IN CNAME copr-fe.aws.fedoraproject.org. copr-fe.aws.fedoraproject.org. 939 IN A 3.81.0.123


This probably merits a little bit of context. A few hours ago Fedora Infrastructure had another very major OpenShift failure (not the first such incident over the past few days). Well, it wasn't an issue until lately...

This so-called 'cloud computing' software is Red Hat's brand (with GUIs and everything) of something that's otherwise widely available as Free software but harder to use.

"Fedora Infrastructure Status: Major service disruption," said the notice, reading out the extent of the impact: "New status major: issues with openshift cluster for services: Ipsilon, Badges, Blockerbugs, Package Updates Manager, Fedora Infrastructure Cloud, COPR Build System, Documentation website, Fedora elections, Account System, Fedora Messaging Bus, Fedora Calendar, Fedora pastebin service, The Koji Buildsystem, Koschei Continuous Integration, Kerberos, Mailing Lists, Module Build Service, Mirror List, Mirror Manager, Fedora Packages App, Pagure, Fedora People, Package maintainers git repositories, Fedora Container Registry, ABRT Server, Fedora websites, Fedora Wiki, Zodbot IRC bot"

"...while examining about a dozen services of the Fedora Project we've occasionally stumbled upon Amazon addresses."And later on it said "New status good: Everything seems to be working."

"We are now operating from the new datacentre and are bringing back capacity as time allows. The known issues are documented here in our wiki," it adds. If one goes to the corresponding status page for pertinent services ("Fedora Project Datacentre Move") it is linking to this wiki page, which in turn says: "This is a page to track known issues after the Fedora Infrastructure Datacenter move in 2020 from phx2 to iad2 and rdu2."

Those are not descriptive names, as there's not even a domain name/suffix. We tried tracing back the IP addresses of these services to identity where at least some these services were being moved to. "I think it's more Red Hat management pressure to rely on Github and Google," Ryan said, but it seems to be related to neither. "You know," Ryan continued, "the infrastructure self-hosting was VERY reliable for many years before that." He mentioned "GMail, Google Drive, calendar stuff" as examples, citing the anonymous post that said about Jim Whitehurst: "He didn't use our products (used a Mac, along with other members of the executive management team), allowed RH to dump open source solutions for our own business to move things to Google's services (which is a huge message to our very clients of "we don't value or trust open source, so why should you?")"

"A few hours ago Fedora Infrastructure had another very major OpenShift failure (not the first such incident over the past few days)."The migration pages are edited mostly by Kevin Fenzi (IBM/Red Hat), who wrote about it less than a year ago in the devel-announce mailing list. No replies to it as of this date.

Is Copr on AWS now? Seems so. If the new DC of Fedora is Amazon-owned and Amazon-controlled, there may be something to conclude. Depending on one's opinion regarding Amazon and AWS, which turns Free/libre things into proprietary.

This message from last year says the hosting of various services is moving to the city/area in VA where AWS has many datacentres that already route some Fedora traffic. While we don't know for sure what's going on, as we need to better understand what plans are made in the boardroom (behind closed doors), it seems likely there's a move to Amazon.

Four years ago The Atlantic published "Where Are Amazon's Data Centers?" and it took note of Ashburn and Sterling. About a year later Wikileaks revealed that both places have lots of datacentres in them, mostly Ashburn (shown above is our screenshot of these).

Going back to Kevin, less than a year ago he wrote: "Fedora Infrastructure currently has the majority of its hardware in a datacenter in Arizona, USA. Red Hat leases this space for use by a number of teams, including Fedora."

"This message from last year says the hosting of various services is moving to the city/area in VA where AWS has many datacentres that already route some Fedora traffic."We can confirm, based on lots of checks, that many services still link to that datacentre.

"However," Kevin continued, "they've been seeking a more modern and cost effective location for some time and have decided on one: So, we will be migrating to a new datacenter located in Ashburn, Virginia in 2020."

Oh, Ashburn. Where Amazon has a densely-distributed cluster of datacentres.

Is Red Hat moving to 'the cloud' and, if so, why not its own? Does IBM feel like it needs to outsource that to a competitor/partner?

The details on this are still not certain and I could not get anybody from Fedora (in IRC) to confirm or deny. Quite frankly, many of them may not be aware.

Recent Techrights' Posts

IBM's Debt Ballooned by 8.5 Billion Dollars in Just 3 Months!
Hallmark of a company in a state of disarray, trying to spend its way out of trouble
Big Trouble in GNOME
even GNOME people admit the CoC went wrong
New Article Explains How the GPL Came About and WordPress Having Copyleft Obligations
Having been involved in the WordPress development community since almost the beginning, I know why it chose the GPL and how it restricts abuse by Automattic
Dr Richard Stallman (RMS) Gives Talk in Oxford University in 4 Hours
If you live nearby, go there (it's free as in gratis)
Using a Law Firm's Licence to Exercise Politics Through Frivolous SLAPPs and Nastygrams (to Silence People, Remove Pages, Demand Fake or Forced 'Apologies')
Things must be getting really bad when lawyers act for raving antisemites
 
Streaming Apps Are “Investor Fraud” That Kills the Planet
Reprinted with permission from Ryan Farmer
Things Get Increasingly Nasty at Microsoft Ahead of the Fake Results and May's Mass Layoffs Wave
They try to get people to 'resign' so that they won't count as layoffs and the company's 'wellbeing' will seem better
Serial Sloppers Are Killing the Web (They Probably Don't Care, Either)
Slop is a disease on the Web
Slopping the Trough: Disney Plus Loses Billions and the Decline of Physical Media in America
Reprinted with permission from Ryan Farmer
Over at Tux Machines...
GNU/Linux news for the past day
IRC Proceedings: Thursday, April 24, 2025
IRC logs for Thursday, April 24, 2025
Links 24/04/2025: GAFAM Problems and No Peace (or Ceasefire) in Sight
Links for the day
Slopfarms on the Web Almost Always Generate Anti-Linux FUD When They Produce "Linux" Output
Welcome to the dying Web
Richard Stallman's Oxford Talk Has Just Ended, Here Are Some Photos
he might hop over to another European country
Gemini Links 24/04/2025: Birthday and Good Work of Academia in Esotericism
Links for the day
Links 24/04/2025: EU fines Apple and Facebook, Another Microsoft GitHub Security Blunder
Links for the day
IBM Gained Almost 6 Billion Dollars in "Goodwill" Value in Just 3 Months, According to IBM
Congrats to the management!
In Belarus, Yandex is Now Measured as 50 Times More 'Popular' (by Usage) Than Microsoft
Yandex continues to gain, whereas Bing cannot even register at 1%. Last month it was registered or measured at a measly 0.65%.
IBM Cannot Lie to Shareholders Anymore
"I would not be surprised if we see a layoff every quarter this year."
We're Working to Make Full-Site Search Available
This site has over 1,000 'wiki' pages, many thousands of documents, several thousands of videos, and about 50,000 blog posts or articles. We need to make them easier to find/navigate.
Links 24/04/2025: IBM Loses Many Contracts, Intel to Lay Off Over 20% (Not Counting Those Who Leave 'Voluntarily')
Links for the day
Richard Stallman Can Explain to Oxford Artificial Intelligence Society Why LLM Slop is Not Artificial Intelligence and Why It Hurts Society
another 'crop' of LLM slop that damages GNU/Linux and facts
Over at Tux Machines...
GNU/Linux news for the past day
IRC Proceedings: Wednesday, April 23, 2025
IRC logs for Wednesday, April 23, 2025
Open Source Initiative (OSI) Promoting Microsoft and Proprietary Software Using Microsoft Operatives
Because nothing says "Open Source" like GPL violations facilitated by Microsoft
Another Site Bites the Dust: "Open Source For You" Becoming a Slopfarm (LLM Slop)
What a shame. Another dead site.
Links 23/04/2025: Crackdowns on Dissent, Palin Loses Libel Retrial Against New York Times
Links for the day
Links 23/04/2025: Hard Times and Digital Amnesia
Links for the day
The GNU/Linux Site Formerly Known as "linoxide.com" is Back... as an LLM Slopfarm!
Better for linoxide.com to go offline than to do this
Get Rid of Back Doors, Don't Obsess Over Bounties and Other Corporate PR Stunts (or Needless Reboot Rituals)
Security as a term has mostly lost its meaning due to repeated misuse for many years
Richard Stallman to Speak in Oxford University Exactly a Day From Now
outsourced to GAFAM
Links 23/04/2025: "Hiding Corruption" and "The Cost of Defunding Harvard"
Links for the day
Microsoft 'Studies' Again? Leon Musolff is Writing Papers With Microsoft.
Even if one can see/find a link to "the study" (in the Bezos-controlled publication), most people won't look any further and just take everything at face value.
Towards GNU World Domination
The FSF led by Geoffrey S. Knauth with his friend Richard Stallman in the FSF's Board [...] Let's encourage people to adopt GNU/Linux. There has never been a better time.
statCounter Helps Visualise Just How Deep in Trouble Microsoft is (Especially in Africa)
Microsoft sabotaged efforts to connect Africans and equip them with GNU/Linux laptops
The Register is Using Linux-Hostile Clickbait in Articles of Linux Proponents
Don't be a "whore" to advertisers, team El Reg
Microsoft Windows in Cyprus Lacking a Future
Most people access the Web there from mobile
Matrix Has a Severe Problem With Illegal Images
If Matrix cannot get the CP problem under control, many projects and people will dump Matrix
Never Try to Justify Strangulation of Women (Not in the US and Not in the UK)
Joint post by Mrs. Rianne Schestowitz and Dr. Roy Schestowitz
Links 23/04/2025: Tesla Profits Plunge 71%, Intel Ready to Lay Off 20% of Staff, Microsoft and IBM Layoffs
Links for the day
Microsoft's Most Profound Issue is That People Moved to 'Mobile' and "App Stores" (Microsoft's Presence There is Negligible)
Expect a wild ride for Microsoft this year
Google News is Amplifying FUD and Lies About Linux (and OpenSSH/SSH) by Promoting Slopfarms With Machine-Generated FUD and Slop Images
Google should know better
Gemini Links 23/04/2025: Librarians, Anubis, and Refactoring a Gemini Capsule
Links for the day
Over at Tux Machines...
GNU/Linux news for the past day
IRC Proceedings: Tuesday, April 22, 2025
IRC logs for Tuesday, April 22, 2025