08.09.20

Gemini version available ♊︎

It Looks Like Red Hat’s (IBM) Fedora Project May be ‘Outsourced’ to Amazon’s Datacentres

Posted in Red Hat at 3:11 am by Dr. Roy Schestowitz

Amazon datacentres in Ashburn
Amazon datacentres in Ashburn (source: Wikileaks)

Summary: In “seeking a more modern and cost effective location” for Fedora Infrastructure it seems to have been decided, privately, that Amazon (AWS) would be the new home of this project; but there’s sufficient obfuscation surrounding the matter and many people seem to be totally unaware

OVERNIGHT we’ve spent several hours researching this and we also asked some Fedora people, who don’t seem to know exactly what’s going on. Surely some people higher up the corporate chain have a better and clearer idea, but they’re not likely to speak about it because IBM pays their salary and there’s fear of further layoffs. A lot of the finer details will be available in IRC logs (frustratingly enough, almost all the coding/code hosting was outsourced to Microsoft), but the gist of it all is this: while examining about a dozen services of the Fedora Project we’ve occasionally stumbled upon Amazon addresses. And then there’s this:

dig copr.fedorainfracloud.org

; <<>> DiG 9.11.5-P4-5.1+deb10u1-Debian <<>> copr.fedorainfracloud.org
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 34423
;; flags: qr rd ra; QUERY: 1, ANSWER: 2, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 512
;; QUESTION SECTION:
;copr.fedorainfracloud.org.     IN      A

;; ANSWER SECTION:
copr.fedorainfracloud.org. 300  IN      CNAME   copr-fe.aws.fedoraproject.org.
copr-fe.aws.fedoraproject.org. 939 IN   A       3.81.0.123

This probably merits a little bit of context. A few hours ago Fedora Infrastructure had another very major OpenShift failure (not the first such incident over the past few days). Well, it wasn’t an issue until lately…

This so-called ‘cloud computing’ software is Red Hat’s brand (with GUIs and everything) of something that’s otherwise widely available as Free software but harder to use.

“Fedora Infrastructure Status: Major service disruption,” said the notice, reading out the extent of the impact: “New status major: issues with openshift cluster for services: Ipsilon, Badges, Blockerbugs, Package Updates Manager, Fedora Infrastructure Cloud, COPR Build System, Documentation website, Fedora elections, Account System, Fedora Messaging Bus, Fedora Calendar, Fedora pastebin service, The Koji Buildsystem, Koschei Continuous Integration, Kerberos, Mailing Lists, Module Build Service, Mirror List, Mirror Manager, Fedora Packages App, Pagure, Fedora People, Package maintainers git repositories, Fedora Container Registry, ABRT Server, Fedora websites, Fedora Wiki, Zodbot IRC bot”

“…while examining about a dozen services of the Fedora Project we’ve occasionally stumbled upon Amazon addresses.”And later on it said “New status good: Everything seems to be working.”

“We are now operating from the new datacentre and are bringing back capacity as time allows. The known issues are documented here in our wiki,” it adds. If one goes to the corresponding status page for pertinent services (“Fedora Project Datacentre Move”) it is linking to this wiki page, which in turn says: “This is a page to track known issues after the Fedora Infrastructure Datacenter move in 2020 from phx2 to iad2 and rdu2.”

Those are not descriptive names, as there’s not even a domain name/suffix. We tried tracing back the IP addresses of these services to identity where at least some these services were being moved to. “I think it’s more Red Hat management pressure to rely on Github and Google,” Ryan said, but it seems to be related to neither. “You know,” Ryan continued, “the infrastructure self-hosting was VERY reliable for many years before that.” He mentioned “GMail, Google Drive, calendar stuff” as examples, citing the anonymous post that said about Jim Whitehurst: “He didn’t use our products (used a Mac, along with other members of the executive management team), allowed RH to dump open source solutions for our own business to move things to Google’s services (which is a huge message to our very clients of “we don’t value or trust open source, so why should you?”)”

“A few hours ago Fedora Infrastructure had another very major OpenShift failure (not the first such incident over the past few days).”The migration pages are edited mostly by Kevin Fenzi (IBM/Red Hat), who wrote about it less than a year ago in the devel-announce mailing list. No replies to it as of this date.

Is Copr on AWS now? Seems so. If the new DC of Fedora is Amazon-owned and Amazon-controlled, there may be something to conclude. Depending on one’s opinion regarding Amazon and AWS, which turns Free/libre things into proprietary.

This message from last year says the hosting of various services is moving to the city/area in VA where AWS has many datacentres that already route some Fedora traffic. While we don’t know for sure what’s going on, as we need to better understand what plans are made in the boardroom (behind closed doors), it seems likely there’s a move to Amazon.

Four years ago The Atlantic published “Where Are Amazon’s Data Centers?” and it took note of Ashburn and Sterling. About a year later Wikileaks revealed that both places have lots of datacentres in them, mostly Ashburn (shown above is our screenshot of these).

Going back to Kevin, less than a year ago he wrote: “Fedora Infrastructure currently has the majority of its hardware in a datacenter in Arizona, USA. Red Hat leases this space for use by a number of teams, including Fedora.”

“This message from last year says the hosting of various services is moving to the city/area in VA where AWS has many datacentres that already route some Fedora traffic.”We can confirm, based on lots of checks, that many services still link to that datacentre.

“However,” Kevin continued, “they’ve been seeking a more modern and cost effective location for some time and have decided on one: So, we will be migrating to a new datacenter located in Ashburn, Virginia in 2020.”

Oh, Ashburn. Where Amazon has a densely-distributed cluster of datacentres.

Is Red Hat moving to ‘the cloud’ and, if so, why not its own? Does IBM feel like it needs to outsource that to a competitor/partner?

The details on this are still not certain and I could not get anybody from Fedora (in IRC) to confirm or deny. Quite frankly, many of them may not be aware.

Share in other sites/networks: These icons link to social bookmarking sites where readers can share and discover new web pages.
  • Reddit
  • email

Decor ᶃ Gemini Space

Below is a Web proxy. We recommend getting a Gemini client/browser.

Black/white/grey bullet button This post is also available in Gemini over at this address (requires a Gemini client/browser to open).

Decor ✐ Cross-references

Black/white/grey bullet button Pages that cross-reference this one, if any exist, are listed below or will be listed below over time.

Decor ▢ Respond and Discuss

Black/white/grey bullet button If you liked this post, consider subscribing to the RSS feed or join us now at the IRC channels.

DecorWhat Else is New


  1. [Meme] Money Deducted in Payslips, But Nothing in Pensions

    Sirius ‘Open Source’ has stolen money from staff (in secret)



  2. IRC Proceedings: Friday, March 24, 2023

    IRC Proceedings: Friday, March 24, 2023



  3. The Corporate Media is Not Reporting Large-Scale Microsoft Layoffs (Too Busy With Chaffbot Puff Pieces), Leaks Required to Prove That More Layoffs Are Happening

    Just as we noted days ago, there are yet more Microsoft layoffs, but the mainstream media gets bribed to go “gaga” over vapourware and chaffbots (making chaff like “Bill Gates Says” pieces) instead of reporting actual news about Microsoft



  4. Sirius 'Open Source' Pensiongate: Time to Issue a Warrant of Arrest and Extradite the Fake 'Founder' of Sirius

    Sirius ‘Open Source’ is collapsing, but that does not mean that it can dodge accountability for crimes (e.g. money that it silently stole from its staff since at least 12 years ago)



  5. Links 24/03/2023: Microsoft's Fall on the Web and Many New Videos

    Links for the day



  6. IRC Proceedings: Thursday, March 23, 2023

    IRC logs for Thursday, March 23, 2023



  7. Links 24/03/2023: Social Control Media Bans Advancing

    Links for the day



  8. Links 24/03/2023: GNU Grep 3.10 and Microsoft Accenture in a Freefall

    Links for the day



  9. Links 23/03/2023: RSS Guard 4.3.3 and OpenBSD Webzine

    Links for the day



  10. Experiencing 15 Years of LibrePlanet Celebration Firsthand as a Volunteer: 2023 - Charting the Course

    Article by Marcia K Wilbur



  11. [Meme] Grabinski the Opportunity

    Reports of European Patents being invalidated (judges do not tolerate fake patents) have become so common that a kangaroo court becomes a matter of urgency for the EPO‘s Benoît Battistelli and António Campinos; will the EU and the EPO’s Administrative Council go along with it, helping to cover up more than a decade of profound corruption?



  12. Union Syndicale Fédérale Cautions the EPO's Administrative Council About Initiating an Illegal Kangaroo Court System for Patents (UPC) While EPO Breaks Laws and Sponsors the Ukraine Invasion

    Union Syndicale Fédérale (USF) is once again speaking out in support of the staff union of Europe's second-largest institution, which lacks oversight and governance because of profound corruption and regulatory capture



  13. Investigation Underway: Sirius 'Open Source' Embezzled/Stole Money, Robbed Its Own Staff

    In light of new developments and some progress in an investigation of Sirius ‘Open Source’ (for fraud!) we take stock of where things stand



  14. [Meme] Sirius 'Open Source' Pensions: Schemes or Scams? Giving a Bad Name to Open Source...

    What Sirius ‘Open Source’ did to its staff is rightly treated as a criminal matter; we know who the perpetrators are



  15. Sirius 'Open Source' Under Investigation for Pension Fraud, Several Pension Providers Examine the Facts

    2 pension providers are looking into Sirius ‘Open Source’, a company that defrauded its own staff; stay tuned as there’s lots more to come. Is this good representation for “Open Source”? From a company that had many high-profile clients in the public sector?



  16. Links 23/03/2023: Sparky 2023.03 Special Editions and SUSE Changes CEO (Dirk-Peter van Leeuwen)

    Links for the day



  17. Links 23/03/2023: Linux 6.2.8 and XWayland 23.1.0

    Links for the day



  18. IRC Proceedings: Wednesday, March 22, 2023

    IRC logs for Wednesday, March 22, 2023



  19. Apple 'Porn' Filter

    Guest post by Ryan Farmer: Apple and US State Governments Developing System to Require People to Report Themselves for Watching Porn.



  20. 3.5 Years Later Gemini Protocol and Geminispace Are Still 100% Community-Controlled

    Community-centric alternatives to the World Wide Web have gained traction; one of them, Gemini Protocol, continues to grow in 2023 and we're pleased to report progress and expansion



  21. Windows Falls to 16% Market Share in India (It was 97% in 2009), Microsoft Layoffs Reach India Too

    This month’s picture from the world’s most populous nation does not look good for Microsoft (it looks good for GNU/Linux); anonymous rumour mills online say that Microsoft isn’t moving to India but is actually firing staff based in India, so it’s a case of shrinking, not offshoring. When even low-paid (much lower salaries) staff is discarded it means things are very gloomy.



  22. Links 22/03/2023: GNOME 44 “Kuala Lumpur”

    Links for the day



  23. Microsoft Has Also Infiltrated the OSI's Board of Directors After Rigged Elections

    Weeks ago we warned that this would happen and for the third or fourth time in 2 years the OSI’s election process broke down; today the Open Source Initiative (OSI) writes: “The polls just closed, the results are in. Congratulations to the returning directors Aeva Black…” (Microsoft employee)



  24. Links 22/03/2023: Official Thunderbird Podcast Starts

    Links for the day



  25. IRC Proceedings: Tuesday, March 21, 2023

    IRC logs for Tuesday, March 21, 2023



  26. Many More Microsoft Layoffs Later Today

    Yesterday we shared rumours about Microsoft layoffs being planned for later today (there were 3 waves of layoffs so far this year). There are several more people here who say the same. How much noise will Microsoft make in the “media” in order to distract? Will the chaffbot "ChatGPT" help create enough chaff?



  27. Links 21/03/2023: JDK 20 and GNOME 43.5

    Links for the day



  28. Germany's Lobbyists-Infested Government Sponsors the War on Ukraine via the European Patent Office (EPO)

    The chief UPC ‘judge’ is basically seeking to break the law (and violate constitutions, conventions etc.) to start a kangaroo court while dodging real courts, just like Vladimir Putin does



  29. [Meme] The Meme That Team UPC (the Collusion to Break the European Laws, for Profit) Threats to Sue Us For

    António Campinos and Team UPC are intimidating people who simply point out that the Unified Patent Court (UPC) is illegal and Klaus Grabinksi, shown above, strives to head a de facto kangaroo court in violation of constitutions and conventions (the UK does not and cannot ratify; Ireland hasn’t even held a referendum on the matter)



  30. Microsoft is Sacking People Every Month This Year, Even Managers (While Sponsored Media Produces Endless Chatbot Chaff)

    Lots of Microsoft layoffs lately and so-called ‘journalists’ aren’t reporting these; they’re too busy running sponsored puff pieces for Microsoft, usually fluff along the “hey hi” (AI) theme


RSS 64x64RSS Feed: subscribe to the RSS feed for regular updates

Home iconSite Wiki: You can improve this site by helping the extension of the site's content

Home iconSite Home: Background about the site and some key features in the front page

Chat iconIRC Channel: Come and chat with us in real time

Recent Posts