Bonum Certa Men Certa

It Looks Like Red Hat's (IBM) Fedora Project May be 'Outsourced' to Amazon's Datacentres

Amazon datacentres in Ashburn
Amazon datacentres in Ashburn (source: Wikileaks)



Summary: In "seeking a more modern and cost effective location" for Fedora Infrastructure it seems to have been decided, privately, that Amazon (AWS) would be the new home of this project; but there's sufficient obfuscation surrounding the matter and many people seem to be totally unaware

OVERNIGHT we've spent several hours researching this and we also asked some Fedora people, who don't seem to know exactly what's going on. Surely some people higher up the corporate chain have a better and clearer idea, but they're not likely to speak about it because IBM pays their salary and there's fear of further layoffs. A lot of the finer details will be available in IRC logs (frustratingly enough, almost all the coding/code hosting was outsourced to Microsoft), but the gist of it all is this: while examining about a dozen services of the Fedora Project we've occasionally stumbled upon Amazon addresses. And then there's this:



dig copr.fedorainfracloud.org



; <<>> DiG 9.11.5-P4-5.1+deb10u1-Debian <<>> copr.fedorainfracloud.org ;; global options: +cmd ;; Got answer: ;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 34423 ;; flags: qr rd ra; QUERY: 1, ANSWER: 2, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION: ; EDNS: version: 0, flags:; udp: 512 ;; QUESTION SECTION: ;copr.fedorainfracloud.org. IN A

;; ANSWER SECTION: copr.fedorainfracloud.org. 300 IN CNAME copr-fe.aws.fedoraproject.org. copr-fe.aws.fedoraproject.org. 939 IN A 3.81.0.123


This probably merits a little bit of context. A few hours ago Fedora Infrastructure had another very major OpenShift failure (not the first such incident over the past few days). Well, it wasn't an issue until lately...

This so-called 'cloud computing' software is Red Hat's brand (with GUIs and everything) of something that's otherwise widely available as Free software but harder to use.

"Fedora Infrastructure Status: Major service disruption," said the notice, reading out the extent of the impact: "New status major: issues with openshift cluster for services: Ipsilon, Badges, Blockerbugs, Package Updates Manager, Fedora Infrastructure Cloud, COPR Build System, Documentation website, Fedora elections, Account System, Fedora Messaging Bus, Fedora Calendar, Fedora pastebin service, The Koji Buildsystem, Koschei Continuous Integration, Kerberos, Mailing Lists, Module Build Service, Mirror List, Mirror Manager, Fedora Packages App, Pagure, Fedora People, Package maintainers git repositories, Fedora Container Registry, ABRT Server, Fedora websites, Fedora Wiki, Zodbot IRC bot"

"...while examining about a dozen services of the Fedora Project we've occasionally stumbled upon Amazon addresses."And later on it said "New status good: Everything seems to be working."

"We are now operating from the new datacentre and are bringing back capacity as time allows. The known issues are documented here in our wiki," it adds. If one goes to the corresponding status page for pertinent services ("Fedora Project Datacentre Move") it is linking to this wiki page, which in turn says: "This is a page to track known issues after the Fedora Infrastructure Datacenter move in 2020 from phx2 to iad2 and rdu2."

Those are not descriptive names, as there's not even a domain name/suffix. We tried tracing back the IP addresses of these services to identity where at least some these services were being moved to. "I think it's more Red Hat management pressure to rely on Github and Google," Ryan said, but it seems to be related to neither. "You know," Ryan continued, "the infrastructure self-hosting was VERY reliable for many years before that." He mentioned "GMail, Google Drive, calendar stuff" as examples, citing the anonymous post that said about Jim Whitehurst: "He didn't use our products (used a Mac, along with other members of the executive management team), allowed RH to dump open source solutions for our own business to move things to Google's services (which is a huge message to our very clients of "we don't value or trust open source, so why should you?")"

"A few hours ago Fedora Infrastructure had another very major OpenShift failure (not the first such incident over the past few days)."The migration pages are edited mostly by Kevin Fenzi (IBM/Red Hat), who wrote about it less than a year ago in the devel-announce mailing list. No replies to it as of this date.

Is Copr on AWS now? Seems so. If the new DC of Fedora is Amazon-owned and Amazon-controlled, there may be something to conclude. Depending on one's opinion regarding Amazon and AWS, which turns Free/libre things into proprietary.

This message from last year says the hosting of various services is moving to the city/area in VA where AWS has many datacentres that already route some Fedora traffic. While we don't know for sure what's going on, as we need to better understand what plans are made in the boardroom (behind closed doors), it seems likely there's a move to Amazon.

Four years ago The Atlantic published "Where Are Amazon's Data Centers?" and it took note of Ashburn and Sterling. About a year later Wikileaks revealed that both places have lots of datacentres in them, mostly Ashburn (shown above is our screenshot of these).

Going back to Kevin, less than a year ago he wrote: "Fedora Infrastructure currently has the majority of its hardware in a datacenter in Arizona, USA. Red Hat leases this space for use by a number of teams, including Fedora."

"This message from last year says the hosting of various services is moving to the city/area in VA where AWS has many datacentres that already route some Fedora traffic."We can confirm, based on lots of checks, that many services still link to that datacentre.

"However," Kevin continued, "they've been seeking a more modern and cost effective location for some time and have decided on one: So, we will be migrating to a new datacenter located in Ashburn, Virginia in 2020."

Oh, Ashburn. Where Amazon has a densely-distributed cluster of datacentres.

Is Red Hat moving to 'the cloud' and, if so, why not its own? Does IBM feel like it needs to outsource that to a competitor/partner?

The details on this are still not certain and I could not get anybody from Fedora (in IRC) to confirm or deny. Quite frankly, many of them may not be aware.

Recent Techrights' Posts

When (Almost) One-Man Operations Are Disguised as Medium-Sized Companies
the CEO hides in the US (hiding from his ex-wives, 4 daughters from those wives, and Sirius staff that he defrauded)
Microsoft Actually in Trouble, Microsofters Unable to Obey Judges' Orders
For the second time in a week, Microsofters are unable to obey orders
Over at Tux Machines...
GNU/Linux news for the past day
Microsoft's Debt Exploded by 15.4 Billion Dollars in the Past 9 Months Alone (Despite All the Layoffs)
As of minutes ago, at 6PM on a Friday, the numbers are made public
 
Video: The Rise of GNU/Linux and Free Software as Seen by RMS in 2004
DTP's founder argued that when Windows goes below 85% "market share", it'll lose its grip in the monopoly sense
Russia: GNU/Linux Rises to Highest Adoption Level Since Invasion of Ukraine
Moving up in the north
Microsoft's Latest Financial Report: We "Gained" 300 Million Dollars in "Goodwill" and Liabilities Grew by 32 Billion Dollars
Microsoft's debt has reached an all-time high
The Register US = The Register MS
Formerly The Register UK
Weeks After Microsoft Shut Down Its Operations in Pakistan Windows Falls to All-Time Lows
Only less than a month ago it was quietly revealed, based on laid-off staff, that Microsoft shut down in Pakistan
Criminal Behaviour is the Standard Operating Procedure at Microsoft
In the future I'll be able to tell how, when dealing with SLAPPs from Microsofters, their Microsoft services failed me and sometimes even blocked my contacts
GNU/Linux Rises to All-Time Highs in Europe
many people will get fired for buying Microsoft
All-Time Highs for GNU/Linux on the Client Desktop/Laptop, Based on Steam Survey
GNU/Linux rose to 2.89% in Steam
Links 02/08/2025: Blaugust 2025 and "Russia Declares Navalny Memoir ‘Extremist’"
Links for the day
Free Software is Not a Business Model
Go ahead, ask your friend, "how do you plan to monetise your children?"
LLM Slop Harms Real Literature, Real Web Sites, Real Journalism
LLM slop is a parasite and it'll run out of legitimate outputs
Upcoming OSI Scandal Series
The OSI is a rogue actor because it serves Microsoft in exchange for money
Slopwatch: The Issue Persists, But the Consensus in the Media Changes as Google Enrages It With LLM Plagiarism
We've meanwhile assessed the latest output from Linuxiac
IRC Proceedings: Friday, August 01, 2025
IRC logs for Friday, August 01, 2025
Links 02/08/2025: İstanbul Retail Inflation Reaches 42.48%, US FBI Opens Office in New Zealand
Links for the day
Gemini Links 02/08/2025: ZFS, LLM Hype, and Fake Modules
Links for the day
Links 01/08/2025: Health, Conflict, and Attacks on Freedom of the Press
Links for the day
Meeting (Webchat) With Maria Arranz Gomez, Florian Grundies, Jürgen Janda and Konstantinos Kortsaris Confronts EPO Management About Breaking Promises and Crushing Workers
The lack of consistent messages suggests plans other than what's advertised and the lack of consultation (secrecy) likewise
Links 01/08/2025: "The Great British Firewall" and U.S. Army Sponsors Palantir
Links for the day
For Second Day in a Row, Top Story in The Register MS is "Microsoft Says"
The editor in chief exercises control over everybody else
LLMs as Attack Method Against Free Software and Programming
DDoS in "hey hi" (slop) clothing
Stability and Reliability, Backward Compatibility
I don't fancy relying on social control media as "sources"
What "the News" Looks Like in 2025
The "says" (or "sez") phenomenon
History Will Be Distorted, Sometimes Intentionally, Under the Guise of Intelligence (Manipulated/Curated Slop)
Militarised misinformation or military-grade chaff is a national security threat, even domestically
Financial Engineering Companies: A Company Worth 4 Trillion Dollars Would Not Borrow 100+ Billion Dollars at Interest Rates Like Today's
Many headlines perpetuate the lie Microsoft had just 2 waves of layoffs
Microsoft is Googlebombing "Linux" While Paying Former News Sites to Publish SPAM
How much lower will IDG sink?
Google as a 'Bullshit Generator' Disguised as Intelligence
It'll probably cause Google to get sued a lot, both by individuals and companies
As Expected, Google in the UK Now Experiments With Slop Instead of Web Search
At this point more people ought to stop and think: Does Google's search engine deserve trust?
The Data You Don't Give Away is Your Advantage
stop sharing data that does not need to be shared
Being Obedient or Doing the Right Thing
The world always changes for the better because of people who think "Outside the Box", not the cogs
Gemini Links 01/08/2025: Happy Hacking Keyboards and New Gemini Arrivals
Links for the day
Over at Tux Machines...
GNU/Linux news for the past day
IRC Proceedings: Thursday, July 31, 2025
IRC logs for Thursday, July 31, 2025
Sabotaging Linux on Behalf of Microsoft With UEFI 'Secure' Boot (De Facto Remote 'Kill Switch'), Then Defaming, Stalking and Harassing Critics of 'Secure' Boot for 12 Years, Then SLAPPing Their Spouses and Them
The sorts of stubborn lunatics we've been dealing with
Moving on in Techrights, Geeks Gonna Geek
In the coming weeks we plan to focus (as we explained last week) on patents, GNU/Linux issues, and the occasional philosophical essays
Slopwatch: Google News Has Lost the Plot
Almost the majority of articles returned for "Linux" are fakes
Links 31/07/2025: Australia Restricts YouTube Access, Personal Privacy at Risk
Links for the day
Links 31/07/2025: Spotify Collapses and Spotify Now Forcing Some Users to Undergo Face-Scanning
Links for the day
A Lot of Supposedly "Successful" Businesses Are Just Debt-Racking Vessels Without Any Prospects of Financial Sustainability
The probability of bankruptcy of any business is more than 0%
theregister.com: The Voice of Microsoft US?
It basically sold out
Yes, You Can Love and Adore Things Whilst Also Criticising Them
Is society being divided and groomed/primed to be resistant to constructive criticism?
Links 31/07/2025: War in Ukraine, Security News, and Cyberattacks Against Journalists on the Rise
Links for the day
Gemini Links 31/07/2025: Fake Money and Gemini Diaries
Links for the day
An Illusion and Cult Worship of Magnitude (Ubiquity as "Victory")
GNU has been around for over 40 years and it'll likely continue to exist for another 40 (in some form)
Google: From Pointing to Relevant Sites to Pointing to Social Control Media to Actually Parroting Social Control Media as "Facts"
Google has become a misinformation company
Over at Tux Machines...
GNU/Linux news for the past day
IRC Proceedings: Wednesday, July 30, 2025
IRC logs for Wednesday, July 30, 2025