Bonum Certa Men Certa

It Looks Like Red Hat's (IBM) Fedora Project May be 'Outsourced' to Amazon's Datacentres

Amazon datacentres in Ashburn
Amazon datacentres in Ashburn (source: Wikileaks)



Summary: In "seeking a more modern and cost effective location" for Fedora Infrastructure it seems to have been decided, privately, that Amazon (AWS) would be the new home of this project; but there's sufficient obfuscation surrounding the matter and many people seem to be totally unaware

OVERNIGHT we've spent several hours researching this and we also asked some Fedora people, who don't seem to know exactly what's going on. Surely some people higher up the corporate chain have a better and clearer idea, but they're not likely to speak about it because IBM pays their salary and there's fear of further layoffs. A lot of the finer details will be available in IRC logs (frustratingly enough, almost all the coding/code hosting was outsourced to Microsoft), but the gist of it all is this: while examining about a dozen services of the Fedora Project we've occasionally stumbled upon Amazon addresses. And then there's this:



dig copr.fedorainfracloud.org



; <<>> DiG 9.11.5-P4-5.1+deb10u1-Debian <<>> copr.fedorainfracloud.org ;; global options: +cmd ;; Got answer: ;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 34423 ;; flags: qr rd ra; QUERY: 1, ANSWER: 2, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION: ; EDNS: version: 0, flags:; udp: 512 ;; QUESTION SECTION: ;copr.fedorainfracloud.org. IN A

;; ANSWER SECTION: copr.fedorainfracloud.org. 300 IN CNAME copr-fe.aws.fedoraproject.org. copr-fe.aws.fedoraproject.org. 939 IN A 3.81.0.123


This probably merits a little bit of context. A few hours ago Fedora Infrastructure had another very major OpenShift failure (not the first such incident over the past few days). Well, it wasn't an issue until lately...

This so-called 'cloud computing' software is Red Hat's brand (with GUIs and everything) of something that's otherwise widely available as Free software but harder to use.

"Fedora Infrastructure Status: Major service disruption," said the notice, reading out the extent of the impact: "New status major: issues with openshift cluster for services: Ipsilon, Badges, Blockerbugs, Package Updates Manager, Fedora Infrastructure Cloud, COPR Build System, Documentation website, Fedora elections, Account System, Fedora Messaging Bus, Fedora Calendar, Fedora pastebin service, The Koji Buildsystem, Koschei Continuous Integration, Kerberos, Mailing Lists, Module Build Service, Mirror List, Mirror Manager, Fedora Packages App, Pagure, Fedora People, Package maintainers git repositories, Fedora Container Registry, ABRT Server, Fedora websites, Fedora Wiki, Zodbot IRC bot"

"...while examining about a dozen services of the Fedora Project we've occasionally stumbled upon Amazon addresses."And later on it said "New status good: Everything seems to be working."

"We are now operating from the new datacentre and are bringing back capacity as time allows. The known issues are documented here in our wiki," it adds. If one goes to the corresponding status page for pertinent services ("Fedora Project Datacentre Move") it is linking to this wiki page, which in turn says: "This is a page to track known issues after the Fedora Infrastructure Datacenter move in 2020 from phx2 to iad2 and rdu2."

Those are not descriptive names, as there's not even a domain name/suffix. We tried tracing back the IP addresses of these services to identity where at least some these services were being moved to. "I think it's more Red Hat management pressure to rely on Github and Google," Ryan said, but it seems to be related to neither. "You know," Ryan continued, "the infrastructure self-hosting was VERY reliable for many years before that." He mentioned "GMail, Google Drive, calendar stuff" as examples, citing the anonymous post that said about Jim Whitehurst: "He didn't use our products (used a Mac, along with other members of the executive management team), allowed RH to dump open source solutions for our own business to move things to Google's services (which is a huge message to our very clients of "we don't value or trust open source, so why should you?")"

"A few hours ago Fedora Infrastructure had another very major OpenShift failure (not the first such incident over the past few days)."The migration pages are edited mostly by Kevin Fenzi (IBM/Red Hat), who wrote about it less than a year ago in the devel-announce mailing list. No replies to it as of this date.

Is Copr on AWS now? Seems so. If the new DC of Fedora is Amazon-owned and Amazon-controlled, there may be something to conclude. Depending on one's opinion regarding Amazon and AWS, which turns Free/libre things into proprietary.

This message from last year says the hosting of various services is moving to the city/area in VA where AWS has many datacentres that already route some Fedora traffic. While we don't know for sure what's going on, as we need to better understand what plans are made in the boardroom (behind closed doors), it seems likely there's a move to Amazon.

Four years ago The Atlantic published "Where Are Amazon's Data Centers?" and it took note of Ashburn and Sterling. About a year later Wikileaks revealed that both places have lots of datacentres in them, mostly Ashburn (shown above is our screenshot of these).

Going back to Kevin, less than a year ago he wrote: "Fedora Infrastructure currently has the majority of its hardware in a datacenter in Arizona, USA. Red Hat leases this space for use by a number of teams, including Fedora."

"This message from last year says the hosting of various services is moving to the city/area in VA where AWS has many datacentres that already route some Fedora traffic."We can confirm, based on lots of checks, that many services still link to that datacentre.

"However," Kevin continued, "they've been seeking a more modern and cost effective location for some time and have decided on one: So, we will be migrating to a new datacenter located in Ashburn, Virginia in 2020."

Oh, Ashburn. Where Amazon has a densely-distributed cluster of datacentres.

Is Red Hat moving to 'the cloud' and, if so, why not its own? Does IBM feel like it needs to outsource that to a competitor/partner?

The details on this are still not certain and I could not get anybody from Fedora (in IRC) to confirm or deny. Quite frankly, many of them may not be aware.

Recent Techrights' Posts

Nobody is "Replaced by AI", It's Just a Smokescreen for Jobs Being Eliminated by Lack of Money (Too Much Debt) and Offshoring
It's also why many make the jokes about the "I" in "AI" being "India" or "Indians"
The US Government is Now in the Business (Literally!) of Saving Microsoft and Intel
This means that President TACO/Cheeto now has greater financial incentive to also prop up Microsoft and Windows
 
The UEFI Restricted Boot 'Time Bomb' is About to Go Off in a Few Weeks
Garrett was the first person to face sanctions (like muting) in our IRC channels because of his abuse; worse yet, he hijacked other people's names and then locked them out of their own accounts
Should Currys PCWorld Start Voiding Warranties of Users of Vista 11?
If a person's laptop has a mechanical issue, should this person replace GNU/Linux with Vista 11 for the repair shop? Only to damage the SSD?
Newer is Not Always Better, and It's Possible That 'Peak' is the Past
People creating their own platforms means progress, whereas centralisation (like moving from blogs to social control media) is the opposite of progress
LLM Hype is Sowing Destruction: It Contributes to DDoS Attacks and Makes the Web Less Accessible (JavaScript "R U Human?" Tests)
If it was googlebot, it would be possible to argue that you'd at least then get referral traffic from Google Search. With LLMs, all you get is plagiarised.
Links 24/08/2025: New York Times Talks About Hey Hi (AI) Bubble
Links for the day
Gemini Links 24/08/2025: Upgrading Debian and Mobile-indifferent Design
Links for the day
Over at Tux Machines...
GNU/Linux news for the past day
IRC Proceedings: Saturday, August 23, 2025
IRC logs for Saturday, August 23, 2025
Richard Stallman's Talk in Buenos Aires Scheduled for 16 November 2025 (a Month After FSF Turns 40)
they've just updated their site and Stallman is listed first
Men Who Abuse Women Should Never Spend Over 3 Years of the UK High Court's Time
This demonstrates that we need a reform in the UK
Slopwatch: Linux Journal, WebProNews, LinuxSecurity, and the Serial Slopper
The bubble needs to burst, but even then the Web will be left with residues of these slopfarms
Links 23/08/2025: Science, War, and Important Win for the British Media Against SLAPPers Who Abuse Women
Links for the day
Gemini Links 23/08/2025: BaseLibre Numerical System and Back to Oldschool
Links for the day
"Deserved Victory" for "Women That Suffered"
"GNM defended its reporting as being both true and in the public interest and in a judgment on Friday"
Links 23/08/2025: onmicrosoft.com as Spam Cannon, The Cheeto-Intel Deal Is Official
Links for the day
Wired Complained About LLM Slop Only Days Before It Got Caught Doing That Itself
Never throw stones in a glass house
IBM "Value" Down 14.16% in a Month, Red Hat Layoffs Allegedly Discussed 12 Days Ago
"IBM is a dinosaur. Dinosaurs get extinct when the don't keep up."
We're Seeing More Countries Where Windows Isn't Even in Second Place Anymore (Third or Worse)
In a way, Microsoft can barely even hold onto second place anymore
Microsoft Workers on Canonical's Payroll
If you want something that's sort of like Ubuntu but is not controlled by Canonical, then look into Linux Mint, Debian, or LMDE
GNU/Linux Climbs to 4% in Sierra Leone
Sierra Leone isn't a very rich country (to say the least), but it's better off than some of its neighbours
The SLAPPS Run Out of Oxygen Because They're Abuse of Process
At the end of the day we plan to publish over 1,000 articles explaining what happened
The Register MS Gets Paid by the Employer of the Previous Editor in Chief to Promote the "AI" Ponzi Scheme, Which Does Considerable Damage to the Web and to Online Journalists
The Register MS can 'badmouth' slop all it wants; it gets paid to inflate this bubble. It's actively participating in it.
Soon It'll be Autumn, Time to Repair Things
Where they don't charge an arm and a leg
Doing Our Best to Cover Software Patents When the Mainstream Media Does Not
Even the FSF has its limits
Gemini Links 23/08/2025: August Questions and Network Solutions
Links for the day
Over at Tux Machines...
GNU/Linux news for the past day
IRC Proceedings: Friday, August 22, 2025
IRC logs for Friday, August 22, 2025
Microsoft Has Issues in Guyana
It's not just Guyana
About 25% of the "Linux" News/Results in Google News Today Are LLM Slop, Almost 20% From the Same Rogue Operators of Slopfarms
Google, which tries to market itself as an LLM giant, apparently fails to understand what's wrong with it
Harassing People on Holiday
There are "no-go areas"; but that assumes all laws firms have ethical standards
The Great, Undeniable Value of Paper Trail, Not Purely Digital Systems
Suppose you have nothing but bits on someone else's computer and "word of mouth"...
The Company Behind Ars Technica, Reddit and Wired Caught Publishing LLM Slop (It Also Admits It Now)
Condé Nast busted
Links 22/08/2025: Lagrange 1.18.8, Wired Magazine and Business Insider Caught Resorting to LLM Slop
Links for the day
This Saturday It's Gonna be 3.5 Years* Since Russia Invaded Ukraine. No Microsoft Protests Against Microsoft Having Provided Russia With Services.
Companies do not have consistent policies and enforcement of "corporate values" is somewhat of an egg salad
Slopwatch: Sites Gone Rogue, Google Promoting Lies, and DDoS Attacks by Plagiarism Giants
Charlatans and frauds engage in a war against artistic industries, mislabeling plagiarism as "AI"
Links 22/08/2025: Cisco Layoffs, LA Times Says "AI Hype is Fading Fast"
Links for the day
Gemini Links 22/08/2025: K for Kentucky and Caddy Versus LLM Slopbots
Links for the day
The "End Software Patents" Initiative of the FSF Explains "WHY [to] ABOLISH SOFTWARE PATENTS"
We hope to cover patent-related issues more and more as the big anniversary of the FSF approaches
Freenode Sniffing
The grown-ups left the building
The Only Thing Worse Than Misinformation is Misinformation Sold to Everyone as "Intelligence"
Misplaced trust is worse than none at all
The Register MS Now Openly Admits LLM Hype Does Damage, But It's Also Being Paid to Participate in the LLM Hype (With Paid 'Articles' and 'Webcasts' for Paying Advertisers)
The Register MS gets paid to do this
End of the Smartphone Era? No.
Maybe the media should focus on producing accurate, factual news
Over at Tux Machines...
GNU/Linux news for the past day
IRC Proceedings: Thursday, August 21, 2025
IRC logs for Thursday, August 21, 2025
Enshittification of Airports, Airlines, and Airplanes
If people are willing to tolerate standard declines and enshittification (nowadays sold as "pivot to AI" or "replaced by AI" or "AI layoffs") they will pay for it some other way
Latest Is Not Greatest: The Case of "Foldable" Tech
don't be shamed into abandoning old things just because the "fashion industry" of Apple and Samsung tells you to
Airlines and Their Tricks That Only Work in the 'Digital Age'
People sceptical of the direction technology has taken are not "Luddites"
Open Source Initiative (OSI), Which Became a Propaganda Front of Microsoft and "Hey Hi" (Hype, Misnomer), Wants You to Forget These Scandals
A lot of these issues won't be set aside until there's a resolution