08.09.20

Gemini version available ♊︎

It Looks Like Red Hat’s (IBM) Fedora Project May be ‘Outsourced’ to Amazon’s Datacentres

Posted in Red Hat at 3:11 am by Dr. Roy Schestowitz

Amazon datacentres in Ashburn
Amazon datacentres in Ashburn (source: Wikileaks)

Summary: In “seeking a more modern and cost effective location” for Fedora Infrastructure it seems to have been decided, privately, that Amazon (AWS) would be the new home of this project; but there’s sufficient obfuscation surrounding the matter and many people seem to be totally unaware

OVERNIGHT we’ve spent several hours researching this and we also asked some Fedora people, who don’t seem to know exactly what’s going on. Surely some people higher up the corporate chain have a better and clearer idea, but they’re not likely to speak about it because IBM pays their salary and there’s fear of further layoffs. A lot of the finer details will be available in IRC logs (frustratingly enough, almost all the coding/code hosting was outsourced to Microsoft), but the gist of it all is this: while examining about a dozen services of the Fedora Project we’ve occasionally stumbled upon Amazon addresses. And then there’s this:

dig copr.fedorainfracloud.org

; <<>> DiG 9.11.5-P4-5.1+deb10u1-Debian <<>> copr.fedorainfracloud.org
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 34423
;; flags: qr rd ra; QUERY: 1, ANSWER: 2, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 512
;; QUESTION SECTION:
;copr.fedorainfracloud.org.     IN      A

;; ANSWER SECTION:
copr.fedorainfracloud.org. 300  IN      CNAME   copr-fe.aws.fedoraproject.org.
copr-fe.aws.fedoraproject.org. 939 IN   A       3.81.0.123

This probably merits a little bit of context. A few hours ago Fedora Infrastructure had another very major OpenShift failure (not the first such incident over the past few days). Well, it wasn’t an issue until lately…

This so-called ‘cloud computing’ software is Red Hat’s brand (with GUIs and everything) of something that’s otherwise widely available as Free software but harder to use.

“Fedora Infrastructure Status: Major service disruption,” said the notice, reading out the extent of the impact: “New status major: issues with openshift cluster for services: Ipsilon, Badges, Blockerbugs, Package Updates Manager, Fedora Infrastructure Cloud, COPR Build System, Documentation website, Fedora elections, Account System, Fedora Messaging Bus, Fedora Calendar, Fedora pastebin service, The Koji Buildsystem, Koschei Continuous Integration, Kerberos, Mailing Lists, Module Build Service, Mirror List, Mirror Manager, Fedora Packages App, Pagure, Fedora People, Package maintainers git repositories, Fedora Container Registry, ABRT Server, Fedora websites, Fedora Wiki, Zodbot IRC bot”

“…while examining about a dozen services of the Fedora Project we’ve occasionally stumbled upon Amazon addresses.”And later on it said “New status good: Everything seems to be working.”

“We are now operating from the new datacentre and are bringing back capacity as time allows. The known issues are documented here in our wiki,” it adds. If one goes to the corresponding status page for pertinent services (“Fedora Project Datacentre Move”) it is linking to this wiki page, which in turn says: “This is a page to track known issues after the Fedora Infrastructure Datacenter move in 2020 from phx2 to iad2 and rdu2.”

Those are not descriptive names, as there’s not even a domain name/suffix. We tried tracing back the IP addresses of these services to identity where at least some these services were being moved to. “I think it’s more Red Hat management pressure to rely on Github and Google,” Ryan said, but it seems to be related to neither. “You know,” Ryan continued, “the infrastructure self-hosting was VERY reliable for many years before that.” He mentioned “GMail, Google Drive, calendar stuff” as examples, citing the anonymous post that said about Jim Whitehurst: “He didn’t use our products (used a Mac, along with other members of the executive management team), allowed RH to dump open source solutions for our own business to move things to Google’s services (which is a huge message to our very clients of “we don’t value or trust open source, so why should you?”)”

“A few hours ago Fedora Infrastructure had another very major OpenShift failure (not the first such incident over the past few days).”The migration pages are edited mostly by Kevin Fenzi (IBM/Red Hat), who wrote about it less than a year ago in the devel-announce mailing list. No replies to it as of this date.

Is Copr on AWS now? Seems so. If the new DC of Fedora is Amazon-owned and Amazon-controlled, there may be something to conclude. Depending on one’s opinion regarding Amazon and AWS, which turns Free/libre things into proprietary.

This message from last year says the hosting of various services is moving to the city/area in VA where AWS has many datacentres that already route some Fedora traffic. While we don’t know for sure what’s going on, as we need to better understand what plans are made in the boardroom (behind closed doors), it seems likely there’s a move to Amazon.

Four years ago The Atlantic published “Where Are Amazon’s Data Centers?” and it took note of Ashburn and Sterling. About a year later Wikileaks revealed that both places have lots of datacentres in them, mostly Ashburn (shown above is our screenshot of these).

Going back to Kevin, less than a year ago he wrote: “Fedora Infrastructure currently has the majority of its hardware in a datacenter in Arizona, USA. Red Hat leases this space for use by a number of teams, including Fedora.”

“This message from last year says the hosting of various services is moving to the city/area in VA where AWS has many datacentres that already route some Fedora traffic.”We can confirm, based on lots of checks, that many services still link to that datacentre.

“However,” Kevin continued, “they’ve been seeking a more modern and cost effective location for some time and have decided on one: So, we will be migrating to a new datacenter located in Ashburn, Virginia in 2020.”

Oh, Ashburn. Where Amazon has a densely-distributed cluster of datacentres.

Is Red Hat moving to ‘the cloud’ and, if so, why not its own? Does IBM feel like it needs to outsource that to a competitor/partner?

The details on this are still not certain and I could not get anybody from Fedora (in IRC) to confirm or deny. Quite frankly, many of them may not be aware.

Share in other sites/networks: These icons link to social bookmarking sites where readers can share and discover new web pages.
  • Reddit
  • email

Decor ᶃ Gemini Space

Below is a Web proxy. We recommend getting a Gemini client/browser.

Black/white/grey bullet button This post is also available in Gemini over at this address (requires a Gemini client/browser to open).

Decor ✐ Cross-references

Black/white/grey bullet button Pages that cross-reference this one, if any exist, are listed below or will be listed below over time.

Decor ▢ Respond and Discuss

Black/white/grey bullet button If you liked this post, consider subscribing to the RSS feed or join us now at the IRC channels.

DecorWhat Else is New


  1. Links 24/9/2021: 30 Years of Europe’s First Root Name Server, Repairability of Laptops Discussed

    Links for the day



  2. ZDNet Has Failed

    ZDNet is on the decline and its demise appears to have greatly accelerated in recent months; we take a quick look at this month's coverage and explain the conflict of interest (it's PR, not news, and it's far too shallow/blatant to simply overlook)



  3. [Meme] Some People Are Just Above the Law

    A lot of people are still flabbergasted or at least baffled/miffed to discover that some people are in effect above the law; not even Europol and Interpol can apprehend and hold them accountable; that needs to change. Had Benoît Battistelli worked for France Télécom S.A. (not the EPO), would he be arrested? What about António Campinos and his drunk son?



  4. NPR and PBS, Both Funded by Bill Gates, Try to Save Him

    Bill Gates continues to corrupt the media and corrupt social control media (such as Twitter) using his money



  5. The EPO Must Forsake Its Diplomatic Immunity and Quit Pretending It's About Patent Law (or Any Law)

    There's no sign of the EPO actually trying to obey the law and correct the mistakes of the past; to make matters worse, the existing administration adds yet more corruption to an already-massive pile while dismissing any form of oversight



  6. IRC Proceedings: Thursday, September 23, 2021

    IRC logs for Thursday, September 23, 2021



  7. Links 24/9/2021: Ubuntu 21.10 Beta, Istio 1.11.3, and More Milestones for Steam Deck

    Links for the day



  8. [Meme] President Campinos Addresses the Legacy of Battistelli's “Strike Regulations”

    A sequence of four EPO memes about those infamous and unlawful “strike regulations” that Benoît Battistelli and António Campinos have exploited to abuse thousands of workers



  9. [Meme] Bill Gates Keeps Digging Himself Deeper in the Grave Each Time He Speaks

    These sorts of ‘interviews’ with Gates’ own propaganda mills (he also pays Twitter now) aren’t going to improve his image; people aren’t infinitely gullible (Source)



  10. Linux Foundation and Other 'Diploma Mills' Say There's Demand for Their Products in Their New 'Research' (Marketing)

    The so-called ‘Linux’ Foundation (LF), together with edX, are basically marketing their services and products, but this is disguised as 'research' (a false narrative widely parroted by shallow and paid-for media partners of theirs), piggybacking brands like “Linux” and buzzwords like “Open Source” (even when they promote proprietary things, e.g. memorisation of proprietary GUIs)



  11. [Meme] The EPO's Carte Blanche and 'Diplomatic Immunity' Card

    EPO staff is being taken for another ride by António Campinos and his cohorts, whose popularity among staff has likely gone down to sub-zero levels already (even faster than Benoît Battistelli)



  12. As Expected, Minimal Pseudo Compliance From EPO Management, Adding Insult to Injury

    SUEPO Central, the core of the staff union of EPO staff (almost 7,000 workers at the EPO, most of whom are SUEPO members), has strong words about the EPO's attitude and stance, which is perhaps unsurprising but still extremely disappointing



  13. Links 23/9/2021: PostgreSQL 14 RC 1 and MidnightBSD 2.1

    Links for the day



  14. Links 23/9/2021: More UPC PR Stunts and IBM (Poettering) TPM for Linux

    Links for the day



  15. The EPO is on the Run (Escaping Negative Press Coverage)

    Aside from tens of millions of euros granted to media and academia (to keep them complicit or silent about EPO corruption, which also implicates the EU) there’s also SLAPP and threats against staff representatives; but Members of the European Parliament are becoming interested in what’s really going on in Europe’s second-largest institution, so this utter waste of EPO money (manipulating the press and gaming universities’ research) might in itself become a scandal sooner or later



  16. [Meme] Lowering the Standards...

    It's time for another round of fluff at the EPO, this time without even travelling (PR-over-'ViCo')



  17. Gemini HTTP/HTML/Web Proxies and Self-Hosting Your Own Proxy

    Gemini protocol (gemini://) and the fast-growing Geminispace (expected to exceed 2,000 known capsules by year’s end, in effect quadrupling in a single year!) are possible to access using Web browsers, at least for those who do not have Gemini clients/browsers just yet; today we examine and give an outline of the options



  18. IRC Proceedings: Wednesday, September 22, 2021

    IRC logs for Wednesday, September 22, 2021



  19. Links 23/9/2021: GNU Parallel 20210922, Moroccan Propaganda From EPO

    Links for the day



  20. Links 22/9/2021: Google 'Upstream First' in Linux and New Maui Report

    Links for the day



  21. Links 22/9/2021: Mesa 21.2.2, GNOME 41 Released

    Links for the day



  22. Socially- or Corporate- or Centrally-Controlled Surveillance, Censorship and Throttling is Not Media

    The 'social control media' situation is getting out of hand; in YouTube, for example, there's a broad revolt against strict editorial control by Google and in Twitter it seems like ordinary users aren't shown so much to people who actually "follow" them



  23. Links 22/9/2021: Panfrost's OpenGL ES 3.1 Conformanc and NovProg 3.2.0

    Links for the day



  24. IRC Proceedings: Tuesday, September 21, 2021

    IRC logs for Tuesday, September 21, 2021



  25. Agents of Monopoly: WIPO is Lobbying for or Reinforcing Microsoft Monopoly by Pushing Its Proprietary Software and Formats

    The World Intellectual Property [sic] Organization — like the EPO (where António Campinos outsourced IT systems to Microsoft) — is choosing the most notorious/corrupt ‘tech’ ‘company’ (cult) instead of open standards and, as the links above show, this is nowadays done inside the United States and outside the United States as well, raising legal questions/ire



  26. Links 21/9/2021: Windowsfx 11, New Chrome, and LF PR Noise

    Links for the day



  27. [Meme] The EPO-EUIPO “Good Brother” Network

    Jobs as bribes at the EPO and EU(IPO) are a lingering problem



  28. Links 21/9/2021: Samba 4.15 and Ubuntu 14.04/16.04 Support for 10 Years

    Links for the day



  29. Richard Stallman's First Public Talk (Delivered in Person) in Years, Now With a Free Format

    Full talk now available. The organisers of the conference have uploaded to YouTube, so we’ve converted everything to a free/libre format (and last night only an excerpt was published here).



  30. [Meme] The Best Quality Propaganda

    The António Campinos-led EPO is still a never-ending propaganda machine; the media isn’t fact-checking or investigating anything, so of course that propaganda goes largely unchallenged and the propagandists (like Joff Wild) profit from it


RSS 64x64RSS Feed: subscribe to the RSS feed for regular updates

Home iconSite Wiki: You can improve this site by helping the extension of the site's content

Home iconSite Home: Background about the site and some key features in the front page

Chat iconIRC Channel: Come and chat with us in real time

Recent Posts