Bonum Certa Men Certa

It Looks Like Red Hat's (IBM) Fedora Project May be 'Outsourced' to Amazon's Datacentres

Amazon datacentres in Ashburn
Amazon datacentres in Ashburn (source: Wikileaks)



Summary: In "seeking a more modern and cost effective location" for Fedora Infrastructure it seems to have been decided, privately, that Amazon (AWS) would be the new home of this project; but there's sufficient obfuscation surrounding the matter and many people seem to be totally unaware

OVERNIGHT we've spent several hours researching this and we also asked some Fedora people, who don't seem to know exactly what's going on. Surely some people higher up the corporate chain have a better and clearer idea, but they're not likely to speak about it because IBM pays their salary and there's fear of further layoffs. A lot of the finer details will be available in IRC logs (frustratingly enough, almost all the coding/code hosting was outsourced to Microsoft), but the gist of it all is this: while examining about a dozen services of the Fedora Project we've occasionally stumbled upon Amazon addresses. And then there's this:



dig copr.fedorainfracloud.org



; <<>> DiG 9.11.5-P4-5.1+deb10u1-Debian <<>> copr.fedorainfracloud.org ;; global options: +cmd ;; Got answer: ;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 34423 ;; flags: qr rd ra; QUERY: 1, ANSWER: 2, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION: ; EDNS: version: 0, flags:; udp: 512 ;; QUESTION SECTION: ;copr.fedorainfracloud.org. IN A

;; ANSWER SECTION: copr.fedorainfracloud.org. 300 IN CNAME copr-fe.aws.fedoraproject.org. copr-fe.aws.fedoraproject.org. 939 IN A 3.81.0.123


This probably merits a little bit of context. A few hours ago Fedora Infrastructure had another very major OpenShift failure (not the first such incident over the past few days). Well, it wasn't an issue until lately...

This so-called 'cloud computing' software is Red Hat's brand (with GUIs and everything) of something that's otherwise widely available as Free software but harder to use.

"Fedora Infrastructure Status: Major service disruption," said the notice, reading out the extent of the impact: "New status major: issues with openshift cluster for services: Ipsilon, Badges, Blockerbugs, Package Updates Manager, Fedora Infrastructure Cloud, COPR Build System, Documentation website, Fedora elections, Account System, Fedora Messaging Bus, Fedora Calendar, Fedora pastebin service, The Koji Buildsystem, Koschei Continuous Integration, Kerberos, Mailing Lists, Module Build Service, Mirror List, Mirror Manager, Fedora Packages App, Pagure, Fedora People, Package maintainers git repositories, Fedora Container Registry, ABRT Server, Fedora websites, Fedora Wiki, Zodbot IRC bot"

"...while examining about a dozen services of the Fedora Project we've occasionally stumbled upon Amazon addresses."And later on it said "New status good: Everything seems to be working."

"We are now operating from the new datacentre and are bringing back capacity as time allows. The known issues are documented here in our wiki," it adds. If one goes to the corresponding status page for pertinent services ("Fedora Project Datacentre Move") it is linking to this wiki page, which in turn says: "This is a page to track known issues after the Fedora Infrastructure Datacenter move in 2020 from phx2 to iad2 and rdu2."

Those are not descriptive names, as there's not even a domain name/suffix. We tried tracing back the IP addresses of these services to identity where at least some these services were being moved to. "I think it's more Red Hat management pressure to rely on Github and Google," Ryan said, but it seems to be related to neither. "You know," Ryan continued, "the infrastructure self-hosting was VERY reliable for many years before that." He mentioned "GMail, Google Drive, calendar stuff" as examples, citing the anonymous post that said about Jim Whitehurst: "He didn't use our products (used a Mac, along with other members of the executive management team), allowed RH to dump open source solutions for our own business to move things to Google's services (which is a huge message to our very clients of "we don't value or trust open source, so why should you?")"

"A few hours ago Fedora Infrastructure had another very major OpenShift failure (not the first such incident over the past few days)."The migration pages are edited mostly by Kevin Fenzi (IBM/Red Hat), who wrote about it less than a year ago in the devel-announce mailing list. No replies to it as of this date.

Is Copr on AWS now? Seems so. If the new DC of Fedora is Amazon-owned and Amazon-controlled, there may be something to conclude. Depending on one's opinion regarding Amazon and AWS, which turns Free/libre things into proprietary.

This message from last year says the hosting of various services is moving to the city/area in VA where AWS has many datacentres that already route some Fedora traffic. While we don't know for sure what's going on, as we need to better understand what plans are made in the boardroom (behind closed doors), it seems likely there's a move to Amazon.

Four years ago The Atlantic published "Where Are Amazon's Data Centers?" and it took note of Ashburn and Sterling. About a year later Wikileaks revealed that both places have lots of datacentres in them, mostly Ashburn (shown above is our screenshot of these).

Going back to Kevin, less than a year ago he wrote: "Fedora Infrastructure currently has the majority of its hardware in a datacenter in Arizona, USA. Red Hat leases this space for use by a number of teams, including Fedora."

"This message from last year says the hosting of various services is moving to the city/area in VA where AWS has many datacentres that already route some Fedora traffic."We can confirm, based on lots of checks, that many services still link to that datacentre.

"However," Kevin continued, "they've been seeking a more modern and cost effective location for some time and have decided on one: So, we will be migrating to a new datacenter located in Ashburn, Virginia in 2020."

Oh, Ashburn. Where Amazon has a densely-distributed cluster of datacentres.

Is Red Hat moving to 'the cloud' and, if so, why not its own? Does IBM feel like it needs to outsource that to a competitor/partner?

The details on this are still not certain and I could not get anybody from Fedora (in IRC) to confirm or deny. Quite frankly, many of them may not be aware.

Recent Techrights' Posts

[Meme] Clergy of GNU/Linux (Corporations Like IBM)
Volunteers as powerless "followers" of companies that "harvest" their labour
 
Links 29/02/2024: Layoffs at Apple, Expedia, and Electronic Arts
Links for the day
Gemini Links 29/02/2024: Web Enshittification and Firefox user-agents
Links for the day
Spiked Piece/Censoreed Piece: 'Microsoft Copilot is a gimmick', says top CIO
Issues relate to connectivity and cost
Enrico Zini, Mattia Rizzolo, Plagiarism & Debian
Reprinted with permission from Daniel Pocock
There Will Be Lots More Apple Layoffs (Already Years in the Making)
The corporate media still tries to shape the narrative to prevent panic or delay market hysteria
Latest SUEPO (Staff Union of EPO) Report For The Hague Reveals EPO Does Not Obey Court Orders, Refuses to Allow Workers to Freely Talk to One Another
working in a place where communication itself is restricted
[Meme] The Oppression Will Continue Until EPO 'Quality' Improves
wonder why EPO morale is so low?
Over at Tux Machines...
GNU/Linux news for the past day
IRC Proceedings: Wednesday, February 28, 2024
IRC logs for Wednesday, February 28, 2024
Outreachy, GSoC-mentors & Debian-Private may soon become public records in federal court
Reprinted with permission from Daniel Pocock
Links 28/02/2024: Many War Updates and Censorship
Links for the day
Gemini Links 28/02/2024: Social Control Media Notifications and Gemini Protocol Extended
Links for the day
Links 28/02/2024: Microsoft the Plagiarist is Projecting, Food Sector Adopts Surge Pricing
Links for the day
Helping Microsoft 'Hijack' Developers (to Make Them Work for Microsoft, Not the Competition)
VS Code is proprietary spyware of Microsoft. Jack Wallen keeps promoting its use.
Gemini Links 28/02/2024: Groupthink and the 'Problem' With Linux
Links for the day
Android Rising (Windows Down to All-Time Lows, Internationally)
This month was a bloodbath for Microsoft
HexChat Looks for Successors to Keep IRC Growing
IRC is far from dead
[Meme] Just Make Him Happy
Y U no produce more monopolies?
End of a Long February
top 10 posts
[Meme] The EPO's Relationship With Patent Examiners
Nobody is "safe"
New Pension Scheme (NPS) at the European Patent Office Explained at the General Assembly
Investing in the future, or...
Donald Trump & FSFE Matthias Kirschner election denial
Reprinted with permission from Daniel Pocock
Over at Tux Machines...
GNU/Linux news for the past day
IRC Proceedings: Tuesday, February 27, 2024
IRC logs for Tuesday, February 27, 2024
Links 27/02/2024: PlayStation Layoffs and More Oppressive New Laws for Hong Kong
Links for the day
Gemini Links 27/02/2024: Facebook as Containment Field and Depression Driven Development (DDD)
Links for the day
They're Adding Warnings Now: The Site "It's FOSS" is Not FOSS
It's better that they at least explicitly state this
Links 27/02/2024: Nevada Versus End-To-End Encryption, Birmingham Bankrupt
Links for the day
End of an Era
The Web isn't just filled with marketing spam but actual disinformation
[Meme] Onboarding New EPO Staff
You read the patent application and grant within hours
The Legacy Prolific Writers Leave Behind Them
"Free Software Credibility Index" after more than 15 years
The Ongoing Evolutionary Process of News-Reading (or News-Finding) on the World Wide Web
it gets worse
Phoronix in Google News
congratulating or welcoming Embrace, Extend, Extinguish (E.E.E.)
Google Fired Many Employees Working on Google News (Which Had Deteriorated and Became Gulag Noise, Littered and Gamed by Blogspam, Plagiarism, and Chatbot/Translator-Generated Spew), Now Comes the Likely 'Phase-out'
No wonder many yearn for the days of DMOZ and Web directories in general
IRC Proceedings: Monday, February 26, 2024
IRC logs for Monday, February 26, 2024
Over at Tux Machines...
GNU/Linux news for the past day
"It's Obvious There's No Future For Any of Us from Blizzard at Microsoft"
The rumours suggest that more Microsoft layoffs are on the way
[Meme] Who's the Boss?
"I thought EPC governed the Office"
Salary Adjustment Procedure (SAP) at the EPO and Why Workers' Salary is Actually Decreasing Each Year (Currency Loses Its Purchasing Power)
outline and update on a years-old blunder
Exposed: FSFE, Legal & Licensing Workshop (LLW), Legal Network & Modern Slavery
Reprinted with permission from Daniel Pocock