Bonum Certa Men Certa

Links 19/11/2019: HPC Focus and LibreOffice 6.4 Beta



  • GNU/Linux

    • Desktop/Firmware

      • Google To Require "Designed For Chromebook" Devices Support Fwupd Firmware Updates

        Hughes shared the anecdote about the Fwupd requirement in this blog post while out of frustration also outlining how device manufacturers should work with him in Fwupd support for their products. That includes either specification or code access under a compatible license and without NDAs, the need for hardware access, understanding of device versioning, and other requirements.

      • Google and fwupd sitting in a tree

        I’ve been told by several sources (but not by Google directly, heh) that from Christmas onwards the “Designed for ChromeBook” sticker requires hardware vendors to use fwupd rather than random non-free binaries. This does make a lot of sense for Google, as all the firmware flash tools I’ve seen the source for are often decades old, contain layer-on-layers of abstractions, have dubious input sanitisation and are quite horrible to use. Many are setuid, which doesn’t make me sleep well at night, and I suspect the security team at Google also. Most vendor binaries are built for the specific ODM hardware device, and all of them but one doesn’t use any kind of source control or formal review process.

        The requirement from Google has caused mild panic among silicon suppliers and ODMs, as they’re having to actually interact with an open source upstream project and a slightly grumpy maintainer that wants to know lots of details about hardware that doesn’t implement one of the dozens of existing protocols that fwupd supports. These are companies that have never had to deal with working with “outside” people to develop software, and it probably comes as quite a shock to the system. To avoid repeating myself these are my basic rules when adding support for a device with a custom protocol in fwupd:

        I can give you advice on how to write the plugin if you give me the specifications without signing an NDA, and/or the existing code under a LGPLv2+ license. From experience, we’ll probably not end up using any of your old code in fwupd but the error defines and function names might be similar, and I don’t anyone to get “tainted” from looking at non-free code, so it’s safest all round if we have some reference code marked with the right license that actually compiles on Fedora 31. Yes, I know asking the legal team about releasing previously-nonfree code with a GPLish licence is difficult.

      • Matthew Garrett: Extending proprietary PC embedded controller firmware

        I'm still playing with my X210, a device that just keeps coming up with new ways to teach me things. I'm now running Coreboot full time, so the majority of the runtime platform firmware is free software. Unfortunately, the firmware that's running on the embedded controller (a separate chip that's awake even when the rest of the system is asleep and which handles stuff like fan control, battery charging, transitioning into different power states and so on) is proprietary and the manufacturer of the chip won't release data sheets for it. This was disappointing, because the stock EC firmware is kind of annoying (there's no hysteresis on the fan control, so it hits a threshold, speeds up, drops below the threshold, turns off, and repeats every few seconds - also, a bunch of the Thinkpad hotkeys don't do anything) and it would be nice to be able to improve it.

        A few months ago someone posted a bunch of fixes, a Ghidra project and a kernel patch that lets you overwrite the EC's code at runtime for purposes of experimentation. This seemed promising. Some amount of playing later and I'd produced a patch that generated keyboard scancodes for all the missing hotkeys, and I could then use udev to map those scancodes to the keycodes that the thinkpad_acpi driver would generate. I finally had a hotkey to tell me how much battery I had left.

        But something else included in that post was a list of the GPIO mappings on the EC. A whole bunch of hardware on the board is connected to the EC in ways that allow it to control them, including things like disabling the backlight or switching the wifi card to airplane mode. Unfortunately the ACPI spec doesn't cover how to control GPIO lines attached to the embedded controller - the only real way we have to communicate is via a set of registers that the EC firmware interprets and does stuff with.

    • Server

      • Cumulus Networks unveils updates to its Linux OS and NetQ

        Cumulus Networks announced on Monday that it has released Cumulus Linux 4.0, which is its network operating system (OS), and version 2.4 of its NetQ network operations toolset.

        Cumulus Networks' Partho Mishra, president and chief product officer, said Cumulus Linux 4.0 and NetQ 2.4 are key elements in the company's ongoing efforts to enable its customers' automation efforts across data centers and campus networks.

        "From a solutions standpoint, our focus has been on developing automation and the capabilities that our customers are going after to make their data centers run like an AWS or Google," Mishra said. "The biggest thing they focus on is automation and they've made big strides working with us and using their own resources."

      • Cumulus Linux 4.0 and NetQ 2.4 Announced

        Cumulus Networks has announced the most recent version of Cumulus Linux 4.0 and NetQ 2.4, promising its most feature-rich release to date. Last month, the company extended this software stack to campus networks. The latest version includes new support for the switch silicon, EVPN implementation for L2/L3 connectivity, increased visibility and troubleshooting with a NetQ cloud-based deployment model, comprehensive end-to-end network automation, and a single fabric across data center and campus networking environments. Last month, Cumulus Linux and NetQ for enterprises were extended to Campus Networks.

      • The world's fastest supercomputers hit higher speeds than ever with Linux

        Yes, there's a lot of talk now about how quantum computers can do jobs in 200 seconds that would take the world's fastest supercomputers 10,000 years. That's nice. But the simple truth is, for almost all jobs, supercomputers are faster than anything else on the planet. And, in the latest Top 500 supercomputer ratings, the average speed of these Linux-powered racers is now an astonishing 1.14 petaflops.

        The fastest of the fast machines haven't changed since the June 2019 Top 500 supercomputer list. Leading the way is Oak Ridge National Laboratory's Summit system, which holds top honors with an HPL result of 148.6 petaflops. This is an IBM-built supercomputer using Power9 CPUs and NVIDIA Tesla V100 GPUs.

      • Exploring AMD’s Ambitious ROCm Initiative

        Three years ago, AMD released the innovative ROCm hardware-accelerated, parallel-computing environment [1] [2]. Since then, the company has continued to refine its bold vision for an open source, multiplatform, high-performance computing (HPC) environment. Over the past three years, ROCm developers have contributed many new features and components to the ROCm open software platform.

        ROCm is a universal platform for GPU-accelerated computing. A modular design lets any hardware vendor build drivers that support the ROCm stack [3]. ROCm also integrates multiple programming languages and makes it easy to add support for other languages. ROCm even provides tools for porting vendor-specific CUDA code into a vendor-neutral ROCm format, which makes the massive body of source code written for CUDA available to AMD hardware and other hardware environments.

      • High-Performance Python – GPUs

        When GPUs became available, C code via CUDA, a parallel computing platform and programming model developed by Nvidia for GPUs, was the logical language of choice. Since then, Python has become the tool of choice for machine learning, deep learning, and, to some degree, scientific code in general.

        Not long after the release of CUDA, the Python world quickly created tools for use with GPUs. As with new technologies, a plethora of tools emerged to integrate Python with GPUs. For some time, the tools and libraries were adequate, but soon they started to show their age. The biggest problem was incompatibility.

        If you used a tool to write code for the GPU, no other tools could read or use the data on the GPU. After making computations on the GPU with one tool, the data had to be copied back to the CPU. Then a second tool had to copy the data from the CPU to the GPU before commencing its computations. The data movement between the CPU and the GPU really affected overall performance. However, these tools and libraries allowed people to write functions that worked with Python.

        In this article, I discuss the Python GPU tools that are being actively developed and, more importantly, likely to interoperate. Some tools don’t need to know CUDA for GPU code, and other tools do need to know CUDA for custom Python kernels.

      • Porting CUDA to HIP

        You’ve invested money and time in writing GPU-optimized software with CUDA, and you’re wondering if your efforts will have a life beyond the narrow, proprietary hardware environment supported by the CUDA language.

        Welcome to the world of HIP, the HPC-ready universal language at the core of AMD’s all-open ROCm platform [1]. You can use HIP to write code once and compile it for either the Nvidia or AMD hardware environment. HIP is the native format for AMD’s ROCm platform, and you can compile it seamlessly using the open source HIP/​Clang compiler. Just add CUDA header files, and you can also build the program with CUDA and the NVCC compiler stack (Figure 1).

      • OpenMP – Coding Habits and GPUs

        When first using a new programming tool or programming language, it’s always good to develop some good general habits. Everyone who codes with OpenMP directives develops their own habits – some good and some perhaps not so good. As this three-part OpenMP series finishes, I highlight best practices from the previous articles that can lead to good habits.

        Enamored with new things, especially those that drive performance and scalability, I can’t resist throwing a couple more new directives and clauses into the mix. After covering these new directives and clauses, I will briefly discuss OpenMP and GPUs. This pairing is fairly recent, and compilers are still catching up to the newer OpenMP standards, but it is important for you to understand that you can run OpenMP code on targeted offload devices (e.g., GPUs).

      • News and views on the GPU revolution in HPC and Big Data:

        Exploring AMD's Ambitious ROCm Initiative Porting CUDA to HIP Python with GPUs OpenMP – Coding Habits and GPUs

      • Exascale meets hyperscale: How high-performance computing is transitioning to cloud-like environments

        Twice a year the high-performance computing (HPC) community anxiously awaits the announcement of the latest edition of the Top500 list, cataloging the most powerful computers on the planet. The excitement of a supercomputer breaking the coveted exascale barrier and moving into the top position typically overshadows the question of which country will hold the record. As it turned out, the top 10 systems on the November 2019 Top500 list are unchanged from the previous revision with Summit and Sierra still holding #1 and #2 positions, respectively. Despite the natural uncertainty around the composition of the Top500 list, there is little doubt about software technologies that are helping to reshape the HPC landscape. Starting at the International Supercomputing conference earlier this year, one of the technologies leading this charge is containerization, lending further credence to how traditional enterprise technologies are influencing the next generation of supercomputing applications.

        Containers are borne out of Linux, the operating system underpinning Top500 systems. Because of that, the adoption of container technologies has gained momentum and many supercomputing sites already have some portion of their workflows containerized. As more supercomputers are being used to run artificial intelligence (AI) and machine learning (ML) applications to solve complex problems in science-- including disciplines like astrophysics, materials science, systems biology, weather modeling and cancer research, the focus of the research is transitioning from using purely computational methods to AI-accelerated approaches. This often requires the repackaging of applications and restaging the data for easier consumption, where containerized deployments are becoming more and more important.

      • AMD Announces Radeon Open Compute ROCm 3.0

        AMD just sent out their press release for SuperComputing 19 week in Denver. It turns out being released for SC19 is the latest major iteration of Radeon Open Compute, ROCm 3.0.

        AMD's press release mentions ROCm 3.0 being released though as of writing it has yet to appear via the ROCm repositories on GitHub. Once the actual drop happens, I'll certainly be writing about it and digging deeper into the other changes in full.

      • NVIDIA Releasing Reference Design For Stuffing Their GPUs Into Arm Servers

        NVIDIA CEO Jensen Huang announced from SC19 today in Denver that they are releasing a "reference design" of hardware and software to help in deployments of their graphics processors within Arm-based servers focused on HPC and AI.

        This isn't too surprising considering NVIDIA's past forays into ARM-based servers for HPC/AI and it was just a few months ago NVIDIA said they would be supporting CUDA on ARM Linux for HPC servers. NVIDIA has already been supporting their software for ARM-based SoCs for years as well considering their Tegra platform and Linux 4 Tegra (L4T).

      • HPE launches container platform, aims to be 100% open source Kubernetes

        Hewlett Packard Enterprise launched its HPE Container Platform, a Kubernetes container system designed to run both cloud and on-premises applications.

        On the surface, HPE Container Platform will face an uphill climb as all the top cloud providers have Kubernetes management tools and instances and IBM with Red Hat has a big foothold for hybrid cloud deployments and the container management that goes with it.

        HPE, which recently outlined a plan to make everything a service, is betting that the HPE Container Platform can differentiate itself based on two themes. First, HPE is pledging that its container platform will be 100% open source Kubernetes compared to other systems that have altered Kubernetes. In addition, HPE Container Platform will be able to run across multiple environments and provide one management layer.

      • IBM

        • Virtio-networking: first series finale and plans for 2020

          Let's take a short recap of the Virtio-networking series that we've been running the past few months. We've covered a lot of ground! Looking at this series from a high level, let's revisit some of the topics we covered:

          [...]

          For those who didn't crack and made it all the way here, we hope this series helped you clarify the dark magic of virtio and low-level networking both in the Linux kernel and in DPDK.

        • Inside the Book of Red Hat

          Shared stories are the cornerstone of community. And in open organizations like Red Hat—where community is paramount—shared stories are especially important to the collective identity that binds participants together.

          At Red Hat, we're quite fond of the stories that inform our shared history, purpose, and culture. We've just collected some of them in a new version of the Book of Red Hat, which is available now.

          Here are just three of the community-defining moments the book recounts.

    • Audiocasts/Shows

      • The System76 Superfan III Event: Gardiner and Jay Chat About Their AWESOME Experience There

        The System76 Superfan III event occurred on November 16th, 2019 and it was a ton of fun! Gardiner Bryant and I talk about our experience there, some of the things they revealed, and other geeky topics around System76 and their computers

      • 2019-11-18 | Linux Headlines

        The Oracle vs. Google copyright case goes to the Supreme Court, NextCry attacks Nextcloud servers, Chromebooks prepare to use LVFS, and Debian takes the systemd debate to the next level.

      • Things are Looking Pod-tastic | Fall Time Blathering

        I started to produce some video content on YouTube and this site to enhance some of my content and later, I thought I would cut my teeth on a podcast of my own to talk about the nerdy things I enjoy. My reoccurring topics consist of my additional thoughts about a subject or two of the last BDLL show and an openSUSE corner but truth be told, openSUSE weaves itself throughout my “noodlings”.

        In September of 2019, the formation of Destination Linux Network was announced where these well established content creators have pooled their resources to draw together their somewhat discrete communities and provide a forum for interaction in greater depth than what Telegram, Discord or YouTube can provide on their own.

      • Test and Code: 94: The real 11 reasons I don't hire you - Charity Majors

        If you get the job, and you enjoy the work, awesome, congratulations.

        If you don't get the job, it'd be really great to know why.

        Sometimes it isn't because you aren't a skilled engineer.

        What other reasons are there?

        Well, that's what we're talking about today.

        Charity Majors is the cofounder and CTO of Honeycomb.io, and we're going to talk about reasons for not hiring someone.

        This is a very informative episode both for people who job hunt in the future and for hiring managers and people on the interview team.

    • Kernel Space

      • Linux 4.9.202

        I'm announcing the release of the 4.9.202 kernel.

        All users of the 4.9 kernel series must upgrade.

        The updated 4.9.y git tree can be found at:

        git://git.kernel.org/pub/scm/linux/kernel/git/stable/linux-stable.git linux-4.9.y

        and can be browsed at the normal kernel.org git web browser:

        https://git.kernel.org/?p=linux/kernel/git/stable/linux-s...

      • Linux 4.4.202
      • Linux Kernel 5.4 to Arrive on November 24th as Linus Torvalds Releases Last RC

        Last week, Linus Torvalds was considering if there's need for an eighth Release Candidate (RC) for the upcoming Linux 5.4 kernel series, which is only needed on very busy development cycles, but while things were quite calm he still released the RC8 milestone just to make sure everything is in place and working out-of-the-box because more testing never hurts.

        "I'm not entirely sure we need an rc8, because last week was pretty calm despite the Intel hw workarounds landing. So I considered just making a final 5.4 and be done with it, but decided that there's no real downside to just doing the rc8 after having a release cycle that took a while to calm down," said Linus Torvalds in a mailing list announcement.

      • HP Linux Imaging & Printing Drivers Now Supported on Ubuntu 19.10 and Fedora 31

        The HP Linux Imaging and Printing 3.19.11 software stack is now available to download and it brings support for several new HP printers and scanners, including HP Color LaserJet MFP M776dn, HP Color LaserJet Flow MFP M776z, HP Color LaserJet Flow MFP M776zs, HP Color LaserJet M856dn, HP Color LaserJet M856x, and HP Color LaserJet E85055dn.

        But what's more important in this new HPLIP release is the fact that users can now install the drivers for their HP printers and scanners on several new GNU/Linux distributions, such as Canonical's Ubuntu 19.10 (Eoan Ermine), Fedora Project's Fedora 31, and Manjaro Linux 18.1.0. Of course, the drivers are only supported on 64-bit versions of these operating systems.

      • Summaries of Some Microconferences Released

        We know everyone is still waiting for the videos. Unfortunately, we?re having a small production glitch, so until we can release them, several MC leads have now sent us written summaries of their MCs which you can see here:

        Tracing microconference You, Me and IoT microconference Live Patching microconference Open Printing microconference Databases microconference Scheduler microconference VFIO/IOMMU/PCI microconference Power Management and Thermal Control microconference

      • Graphics Stack

        • Vulkan 1.1.128 Released With Performance Query Extension

          Vulkan 1.1.128 is out with various corrections and clarifications to this graphics/compute API specification but it also comes with one exciting new extension.

          The new extension that is quite notable for Vulkan 1.1.128 is VK_KHR_performance_query. This KHR-ratified extension is the first cross-vendor extension in Vulkan for the querying of any performance counters on the hardware. We are used to seeing various performance counter extensions within Vulkan (and other APIs like OpenGL) but they tend to be vendor-specific extensions tailored towards their own individual needs.

    • Benchmarks

      • A Linux Performance Look At AMD's 16-core Ryzen 9 3950X

        We’ve expanded our Linux test suite since the last CPU performance review. Every scenario, from compiling to rendering, has been expanded to help us get a fuller look at performance overall. Not all workloads are built alike, and the same can be said about the CPUs themselves. It’s never safe to take the opinion of a single test and expect everything to scale the same way.

        AMD’s Ryzen 9 3950X is the first mid-range desktop processor to offer sixteen cores, something that comes at a price-point that definitely doesn’t feel mid-range: $749. With the last generation, both AMD and Intel offered no larger than an eight-core chip in their respective mid-range lineups. AMD shook things up this generation with the release of the 12-core Ryzen 9 3900X in the summer. We’ve known about the 3950X since then, so to say that it’s been a long time coming would be an understatement.

        When the first Zen 2 chips released, their Linux experience left a lot to be desired. The popular systemd service manager failed to boot until a workaround patch was released. On Windows, Destiny 2 fans on Zen 2 fell victim to the same bug. Fortunately, that was many months ago, and with the most up-to-date AGESA firmware, issues like those are hopefully a thing of the past.

    • Applications

      • nuclear – desktop music player focused on free streaming

        Linux has an abundance of mouthwatering array of excellent open source music players. But I’m always on the look out for fresh and newfangled music players.

        nuclear lets you stream music over the internet. It offers easy access to YouTube, SoundCloud, and Jamendo (the latter with partial support), and there’s a plugin system to add additional services. It therefore seeks to offer a unified music environment for managing music content.

        nuclear is an Electron based application written in the JavaScript programming language. It features hardware acceleration (using your GPU).

      • Kodi 18.5 'Leia' available to download now!

        It’s been a couple of months since the Kodi Foundation released a new version of its hugely popular home theater software.

        After spending some time in the pre-release section, Kodi 18.5 is now finally deemed ready for all.

        As you might expect from a point release, there are no new features here, but rather the focus is on squashing bugs and improving stability and performance.

        The team hasn’t published a list of the main changes yet, but multiple issues have been addressed in this new build, with fixes for interface problems, tweaks to the PVR component, and more.

      • Kodi 19 'Matrix' with Python 3 now available to download, but be warned
      • forgit: Interactive Git Commands With Previews Powered By fzf Fuzzy Finder

        forgit is a command line utility that takes advantage of the popular fzf fuzzy finder to provide interactive git commands, with previews.

        New to fzf? This is a command line fuzzy finder that can be used for a wide range of purposes. This interactive command line filter can be used with various lists, like files, command history, processes, hostnames, git commits, and more.

        fzf is very fast, it's portable with no dependencies (it's written in Go), it has a flexible layout, and includes various extra features like the ability to preview highlighted files in a split window inside fzf (which is what forgit uses for its previewing files), a Vim/Neovim plugin, a script for launching fzf in a tmux pane, key bindings and fuzzy auto-completion.

      • QuiteRSS: 15.11.2019 - 8 years. Happy Birthday!

        Congratulations to those who spent days and nights to help us making application better. Congratulations to those who use it!

      • App Highlight: Flameshot for Taking and Editing Screenshots

        A lot of users want to simply upload their screenshots directly to the cloud in order to easily share it with others.

        You can do that by syncing your saved files to a cloud storage solution and share them later. But, that’s quite a few steps to follow in order to share your screenshot, right?

        So, here, Flameshot lets you upload your image directly to Imgur with a single click. All you have to do is share the URL.

        Do note that these uploads will not be associated with your Imgur account (if you have one) and will be only accessible to the ones with the link.

      • Proprietary

        • Overview of ycrash – finding the source of your problem

          Take a tour of ycrash in this article by Ram Lakshmanan. ycrash helps capture critical artifacts, including garbage collection logs, thread dumps, core dumps, heap dumps, disk usage, and more when the problem happens. It applies machine learning algorithms and generates a report which gives you a complete view of the problem, down to the lines of code that caused it. The industry has seen cutting edge application performance monitoring tools (AppDynamics, NewRelic, Dynatrace…), log analysis tools (DataDog, Splunk,…). These are great tools for detecting problems. i.e. they can detect CPU spiked by x%, memory degraded by y%, response time shot up by z seconds. But they don’t answer the question: Why has the CPU spiked up? Why has memory degraded? Why has the response time increased? You still need to engage developers/architects/vendors to troubleshoot the problem and identify the root cause of the problem.

          ycrash captures critical artifacts (GC logs, thread dumps, core dumps, heap dumps, netstat, vmstat, lsof, iostat, top, disk usage….) when the problem happens, applies machine learning algorithms, and generates one unified root cause analysis report. This report gives you a 360-degree view of the problem. The report points out the exact class, method, and line of code that caused the problem.

        • SAP HANA is now supported on SUSE Linux Enterprise Server for SAP Applications 15 SP1
    • Instructionals/Technical

    • Wine or Emulation

      • CrossOver 19 Enters Beta With Better Microsoft Office Support On Linux

        CodeWeavers' Jeremy White has announced that CrossOver 19 is now in beta for existing customers of this Wine-based software for running Windows programs on Linux and macOS.

        The biggest benefactor of CrossOver 19 is Apple macOS users with there being initial support for macOS Catalina. CrossOver/Wine needed a lot of changes to enable support for this newest version of macOS particularly for 32-bit Windows programs with Apple aiming to end 32-bit application support on their operating system.

      • CROSSOVER 19 IS PROGRESSING WELL AND IS NOW IN BETA!

        It's been two weeks; we feel we owe everyone an update on our efforts to support 32 bit Windows applications on macOS Catalina, despite Apple's decision to terminate support for 32 bit applications.

        I'm happy to announce that we have released the first beta version of CrossOver 19 on Friday, November 15, 2019 to our community of advocates and beta testers. Further, our alpha testing and other internal testing has gone well, so I am confident that we will have a final product ready before the end of the year.

    • Games

      • Valve Announcing Half-Life: Alyx VR Game On Thursday

        Valve has confirmed recent rumors around one of their new virtual reality games in development being Half-Life: Alyx.

        Valve tweeted out a short time ago that Half-Life: Alyx will be announced on Thursday. However, the VR game isn't expected to ship until sometime in 2020.

      • Valve has now confirmed Half-Life: Alyx, their new VR flagship title

        Well, that was a little sooner than expected. Valve have now officially confirmed Half-Life is back with their VR title Half-Life: Alyx.

      • Counter-Strike: Global Offensive releases the huge Operation Shattered Web update

        Not content with just announcing Half-Life: Alyx, their new VR flagship title, Valve also updated Counter-Strike: Global Offensive with a big new operation called Shattered Web.

        I have to admit, I'm really loving the humour from whoever has been running the CS:GO Twitter account lately. Earlier today they put up a poll on Twitter, asking what people preferred between a new Operation and a weapon nerf. They then quickly replied with "Loud and clear, Twitter. We'll get started." and then minutes later "OK, we're done"—brilliant. Not great for me mind you, being in the UK the timings are never great with it now gone midnight but here I am…

      • Free indie RTS game The Fertile Crescent adds team game support for online play

        While you've been able to play 1 on 1 in single-player against the AI and online multi-player for a while with The Fertile Crescent, it was missing team games which have now been added.

        This currently free indie retro Age of Empires-like game is an absolute gem, already quite polished too. The addition of team games is awesome, although only currently available for online play. You can pick all sorts of combinations too like a free for all, 2on2 and 3v1 across a new larger map. Online play is easy though, as they already have a nice working lobby system.

      • Half-Life: Alyx is rumoured to be the name of Valve's new VR game, apparently being unveiled soon

        Valve did say they were working on three VR titles some time ago and it looks like one is almost ready to be shown off, with Half-Life: Alyx.

        The information is all speculation and rumours right now though, so take it all with a heavy truckload of salt. We know a Half-Life VR game is pretty much a thing though, ValveNewsNetwork even had a video going over various details on it back in October so it's not like this is suddenly coming out of nowhere.

        Now though, we have more apparent leaks. Spotted by PC Gamer, they linked to a pastebin and in a later update a Google Document (update: contents now removed, so link removed) apparently showing snippets from an interview between Geoff Keighley of The Game Awards, Robin Walker and someone else they presume to be Gabe Newell.

      • The surprisingly good deck-builder Fate Hunters has arrived on GOG

        Releasing originally back in July, Fate Hunters is another deck-building roguelike filled with random encounters and just recently it was released DRM-free on GOG.

        As a massive fan of Slay the Spire, I'm always in the mood for more deck-builders like this. While it is another game where you build a deck and battle through random locations, the actual gameplay feels nothing really like Slay the Spire.

      • Start your week off with a new game, here's a few for Linux going cheap

        Another week, another sale of course. Let's have a little look over what's going cheap for Linux gamers across this week.

        First up on Steam we have Company of Heroes 2. Although the time to grab it free has ended, they've decided to now give it a big discount for anyone who missed it. You can get it on Steam with 75% off until November 24, plus a bunch of the DLC is also on sale.

      • Confessing my continued love for Jupiter Hell, the super slick roguelike

        Regular readers won't be surprised by my love for Jupiter Hell, I've written about it a few times now and the latest update just continues to allow me to gush about it. Note: I personally supported it during the Kickstarter.

        It's a roguelike, it's turn-based like the classics and depends upon tiles yet it feels so vastly different to anything else it's crazy. It feels like a proper action game, complete with a thick atmosphere and all. Absolutely gorgeous too, the lighting is absolutely fantastic and the barrel explosions sending everything flying looks excellent as well.

      • Stellaris is getting some big changes to empire customization and creation with Origins

        Paradox Interactive and Paradox Development Studio are mixing things up in Stellaris again with the upcoming update and the Federations expansion.

        As usual for Paradox games, when a big DLC is released it will come with a massive feature patch for everyone to access. With Federations, which still has no release date, Paradox will be changing a big part of how you pick your empire with an Origins system.

      • Relaxing flying sim with a morphing bird 'Fugl' has a nice big update

        Moving you away from the blood, the bullets and whatever else all these actions games have with Fugl, a relaxing sim about flying around and appreciating life.

        No set goals, no timers, none of that. Just you and your choice of bird, flying around different biomes to find other creatures to interact with them. Once you do manage to find others, you can then unlock their special form for you to fly around with. It's strange but very calming.

      • Google have now expanded the launch titles for Stadia up to 22

        Launching tomorrow for people who picked up the Founder and Premier editions, Stadia was originally only going to launch with 12 titles. Now this has expanded to 22!

        Announced by Google Vice President and GM, Phil Harrison, on Twitter with him then retweeting a list from Geoff Keighley to show the new titles. Now, it's actually quite a bit more impressive and it seems Stadia Pro gains an extra game too with both Destiny 2 and Samurai Shodown.

      • 23-Way Graphics Card Comparison With Shadow of the Tomb Raider On Linux

        The Linux port of Shadow of the Tomb Raider basically recommends at least an AMD GCN 1.2 or newer graphics card or GeForce GTX 680 or newer, basically the bare requirements on Linux for having a Vulkan driver out-of-the-box. It should also be possible getting a GCN 1.0/1.1 graphics card working if opting to use the AMDGPU DRM driver rather than Radeon DRM as needed for Vulkan driver support. But Feral recommends at least a Radeon RX 480 Polaris graphics card for decent performance. Current Intel graphics are not fast enough to run this game on Linux.

    • Desktop Environments/WMs

    • Distributions

      • IPFire Open-Source Linux Firewall Gets Improved and Faster QoS, Latest Updates

        IPFire 2.23 Core Update 137 is now available for download with improved Quality of Service (QoS), which allows the firewall to pass even more traffic on smaller systems, as well as reduce packet latency on faster machines, thus creating a faster and more responsive network. To take full advantage of the improved and faster QoS, the IPFire project recommends you reboot your systems after installing the new update.

        "Development around the Quality of Service and tackling some of the bugs required an exceptional amount of team effort in very short time and I am very happy that we are now able to deliver the result to you to improve your networks," said Michael Tremer in the announcement. "It allows to pass a lot more traffic on smaller systems as well as reduces packet latency on faster ones to create a more responsive and faster network."

      • IPFire 2.23 - Core Update 138 released

        Just days after the last one, we are releasing IPFire 2.23 - Core Update 138. It addresses and mitigates recently announced vulnerabilities in Intel processors.

      • Screenshots/Screencasts

      • Fedora Family

        • Fedora Toolbox. Unprivileged development environment at maximum

          Fedora Toolbox is a tool for developing and debugging software that basically is a frontend to the Podman container system. A simple way to test applications without getting billions of dependencies and cluttering up your operating system.

          First, Podman (Pod Manager tool) is a daemon less container engine for developing, managing, and running OCI Containers on your Linux System. With Podman, you can manage pods, containers, and container images. You can consult (Podman.io) the official website to learn more about Podman and container tooling.

          Fedora Toolbox gives you a quick frontend to Podman and it also creates an interactive container based on your current system. Toolbox (actually, Fedora Toolbox is now just Toolbox) use is particularly useful for the development and testing environment.

        • Building Successful Products

          Building a new product is hard. Building a successful new product is even harder. And building a profitable new product is the greatest challenge! To make things even more interesting, the fundamental customer requirements for a product change as the product and market mature. The very things that are required for success in an early stage product will hinder or even prevent success later on.

          Markets, technologies and products go through a series of predictable stages. Understanding this evolution – and understanding what to do at each stage! – is vital for navigating the shoals of building a successful and profitable product.

        • Fedora Developers Looking To Change The Default Text Editor From Vi To Nano

          Fedora will be adding the Nano text editor to their default Fedora Workstation installs as complementary to Vi but their stakeholders intend to submit a system-wide proposal that would change the default installed editor from Vi to Nano.

          The Fedora Workstation flavor can add the Nano text editor by default to their spins without replacing it as the default terminal-based text editor, which is currently held by Vi. At today's Fedora Workstation meeting they refrained from trying to change the default text editor just for Fedora Workstation and instead will issue a system-wide proposal to change it to Nano for all of Fedora's spins.

        • Fedora shirts and sweatshirts from HELLOTUX

          Linux clothes specialist HELLOTUX from Europe recently signed an agreement with Red Hat to make embroidered Fedora t-shirts, polo shirts and sweatshirts. They have been making Debian, Ubuntu, openSUSE, and other Linux shirts for more than a decade and now the collection is extended to Fedora.

      • Debian Family

        • Debian 10.2 Buster Linux distribution releases with the latest security and bug fixes

          Last week, the Debian team released Debian 10.2 as the latest point release to the “Buster” series. This release includes a number of bug fixes and security updates. In addition, starting this release Firefox ESR (Extended Support Release) is no longer supported on the ARMEL variant of Debian.

        • Russell Coker: 4K Monitors

          I like having lots of terminal windows on my desktop. For common tasks I might need a few terminals open at a time and if I get interrupted in a task I like to leave the terminal windows for it open so I can easily go back to it. Having more 80*25 terminal windows on screen increases my productivity. My previous monitor was 2560*1440 which for years had allowed me to have a 4*4 array of non-overlapping terminal windows as well as another 8 or 9 overlapping ones if I needed more. 16 terminals allows me to ssh to lots of systems and edit lots of files in vi. Earlier this year I had found it difficult to read the font size that previously worked well for me so I had to use a larger font that meant that only 3*3 terminals would fit on my screen. Going from 16 non-overlapping windows and an optional 8 overlapping to 9 non-overlapping and an optional 6 overlapping is a significant difference. I could get a second monitor, and I won’t rule out doing so at some future time. But it’s not ideal.

        • SCP Foundation needs you!

          SCP is a mind-blowing, diverse, high-quality collection of writings and illustrations, all released under the CC-BY-SA free license. If you never read horror stories written with scientific style -- have a try :)

          [obviously this has nothing to do with OpenSSH Secure CoPy ;)]

      • Canonical/Ubuntu Family

        • Ubuntu Weekly Newsletter 605

          Welcome to the Ubuntu Weekly Newsletter, Issue 605 for the week of November 10 – 16, 2019. The full version of this issue is available here.

    • Devices/Embedded

    • Free, Libre, and Open Source Software

      • Putting to Rest the Free Software/FOSS Divide

        After Richard Stallman’s resignation from key positions in free software, many are suggesting that we have entered the post-Stallman age. It is still too early to understand what that might mean, if anything. Still, one question keeps reoccurring to me: without Stallman to constantly reinforce the habit, will the preference for the terms “GNU/Linux” and “free software” survive? And, either way, does the answer matter any more? Or will trends that have existed for over a decade simply continue, or maybe accelerate?

        Before you start lecturing me on points I’ve known for twenty years, I know all the arguments in favor of GNU/Linux and free software. I even agree with most of them. Yes, how a subject is framed matters. Yes, without contributions from the GNU Project the free operating system known as Linux would not have happened, or at least would have been seriously delayed. You’re right, too, that Stallman’s preferred terms highlighted politics and philosophy. But all this is old history. I am not writing about the past, nor even what should be. I am wondering what might happen in the next few years.

      • Events

        • Linux Applications Summit

          had the pleasure of going to the Linux Applications Summit last week in Barcelona. A week of talks and discussion about getting Linux apps onto people’s computers. It’s the third one of these summits but the first ones started out with a smaller scope (and located in the US) being more focused on Gnome tech, while this renamed summit was true cross-project collaboration.

          Oor Aleix here opening the conference (Gnome had a rep there too of course).

          It was great to meet with Heather here from Canonical’s desktop team who does Gnome Snaps, catching up with Alan and Igor from Canonical too was good to do.

        • [Adriaan de Groot] Linux Applications Summit

          The change-over from 17 degrees in Barcelona to 6 and gloomy in Amsterdam is considerable. This past week I was in Catalunya to participate in the Linux App Summit, a new gathering of applications developers looking to deliver applications on Linux to end-users.

          Of course I handed out Run BSD stickers.

          To a large extent the conference was filled with people from the KDE community and GNOME – but people don’t have to be put in one single category, so we had FreeBSD people, Linux people, Elementary people, openSUSE people, coders, translators, designers and communicators.

          I’d like to give a special shout-out to Nuritzi and Kristi for organizational things and Regina and Shola for communications and Katarina and Emel Elvin for coding. To Heather for schooling me, Muriel for hearing me out and Yuliya for making me eat flan. To Hannah and Hannah for reminding me to update some packaging stuff.

        • LibreOffice localisation sprint (and other events) in Albania

          The Albanian LibreOffice community has been super active in recent years, organising the LibreOffice Conference 2018 in Tirana, and regularly contributing with translation and marketing efforts.

      • Web Browsers

        • Mozilla

          • Firefox 71 Beta 12 Testday – November 22nd

            We are happy to let you know that Friday, November 22nd, we are organizing Firefox 71 Beta 12 Testday. We’ll be focusing our testing on: Inactive CSS.

            Check out the detailed instructions via this gdoc.

            *Note that this events are no longer held on etherpad docs since public.etherpad-mozilla.org was disabled.

          • Mozilla Privacy Blog: Mozilla Mornings on the future of openness and data access in the EU

            On 10 December, Mozilla will host the next installment of our Mozilla Mornings series – regular breakfast meetings where we bring together policy experts, policymakers and practitioners for insight and discussion on the latest EU digital policy developments.

            The next installment will focus on openness and data access in the European Union. We’re bringing together an expert panel to discuss how the European Commission should approach a potential framework on data access, sharing and re-use.

      • Productivity Software/LibreOffice/Calligra

        • LibreOffice 6.4 Enters Beta with Native GTK Dialogs, QR Code Generator, and More

          The upcoming LibreOffice 6.4 office suite has been in development for a few months now, but it's currently entered public beta testing, which means that more reliable builds are available for early adopters to try on the new features and improvements, among which we can mention native GTK dialogs on GNU/Linux systems, a QR code generator, and improved Microsoft Office interoperability.

          Among other noteworthy enhancements, LibreOffice 6.4 will also add a new option in Writer to mark comments as resolved, along with faster table and table row/column moving and deletion, better selection of cells that contain hyperlinks in Calc, the ability to export Calc sheets to PDF with all pages in one PDF, as well as improved scalability of formula-groups computation on multi-core CPUs.

        • LibreOffice 6.4 Beta1 is ready for testing

          The LibreOffice Quality Assurance ( QA ) Team is happy to announce LibreOffice 6.4 Beta1 is ready for testing!

          LibreOffice 6.4 will be released as final at the beginning of February, 2020 ( Check the Release Plan ) being LibreOffice 6.4 Beta1 the second pre-release since the development of version 6.4 started in the beginning of June, 2019. Since then, 5677 commits have been submitted to the code repository and more than 850 bugs have been set to FIXED in Bugzilla. Check the release notes to find the new features included in this version of LibreOffice.

          LibreOffice 6.4 Beta1 can be downloaded from here for Linux, MacOS and Windows, and it can be installed alongside the standard version.

      • BSD

      • FSF/FSFE/GNU/SFLC

        • Prague launches mobile app to make its budget more transparent

          CityVizor was developed by the Ministry of Finance of the Czech Republic and published as an open-source under the GNU GPL license - free to use. The operation for non-Prague town halls is provided by the Open Cities Association and the Czech.digital community.

        • Photoshop for free? The best free alternatives

          We’re starting off with a big dog here. GIMP, which stands for Gnu Image Manipulation Program is the most fully formed and arguably most well-known Photoshop alternative there is. GIMP is like an open source Photoshop developed by a global team of volunteer developers to work on Microsoft Windows, Linux, and Apple Mac. It has an extensive set of features to rival what even Photoshop has to offer and can edit a wide range of file formats including RAW files. This means GIMP is a pro-friendly alternative to Photoshop with features like layer masks and filters enabling photographers and graphic designers to get their work done. GIMP is also a customizable photo editing software as users can download add-on packs to add the extra features they need.

      • Programming/Development

        • Introducing DjangoCon Africa

          Following the huge success of PyCon Africa, the Django community in Africa is ready to bring a new major software event to the continent - the very first DjangoCon Africa! The Django Software Foundation is excited to endorse and support this initiative.

          Plans are already in motion for a DjangoCon Africa to be held in Addis Ababa, Ethiopia in November 2020. Actual dates to be announced as soon as key details are in place.

          DjangoCon Africa will include 3 days of single-track talks, 1 day of workshops and sprints, and another day for touring for international visitors.

          The event will also include a Django Girls workshop to be held the weekend before DjangoCon Africa. To make the conference as inclusive as possible, the event will offer financial aid to members of under-represented communities in software to ensure they can also attend.

        • Django 3.0 release candidate 1 released

          Django 3.0 release candidate 1 is the final opportunity for you to try out the raft of new features before Django 3.0 is released.

          The release candidate stage marks the string freeze and the call for translators to submit translations. Provided no major bugs are discovered that can't be solved in the next two weeks, Django 3.0 will be released on or around December 2. Any delays will be communicated on the django-developers mailing list thread.

        • Cyber Discovery - What it is all about

          Cyber Discovery is made of 4 rounds. The first one being CyberStart Assess. It ran from the 3rd September to the 25th October 2019. There are 10 challenges starting easy, getting much harder. The aim for most of the challenges are to use 'Inspect Element' to get into the website and find the flag. I completed all of these challenges and was invited onto the next round: CyberStart Game. CyberStart Game is much more about finding things out yourself. A useful tip if you are stuck is to search for help on Google. CyberStart Game has 3 'Bases': Headquarters where you get to take part in lots of varied challenges, Moon Base where you learn the basics of Python and Internet Tools that can be run in python e.g. FTP... You also learn how to use python to Brute Force password protected ZIP files and other securities. The Forensics Base is, well you can guess: Forensics. It teaches you about Cryptography and other hiding methods.

        • PyDev of the Week: Martin Uribe

          While taking some college courses I learned Java, but I didn’t like it much. I know enough of the following to get things done: HTML, CSS, JavaScript, Perl, SQL, and BASH. Python is my favorite; I use it pretty much every day even though my job doesn’t require me to code.

        • You can now hone your testing / pytest skills on our platform

          Writing test code is an essential skill. As PyBites we believe writing code is the only solution to becoming a master (Ninja) at programming. The same applies to test code. For that reason we extended our regular exercises with Test Bites.

          In this article you will read about the feature showcasing it on our first ever Test Bite. We also share some details around implementation and a challenge we hit getting it to work. Enjoy and start honing your testing skills today!

        • unu – Using Qt on embedded Linux

          Right from the start, unu wanted to add a stylish, first-class embedded high-res display to their second generation electric scooter. Like many top-class engineering companies, unu didn’t have in-house expertise for building a modern UI, so they decided to partner with KDAB to build a modern UI based on Qt. In this video you learn more about the development process in this project and why unu chose KDAB as a partner.

        • Quicksort in Python

          Quicksort is a popular sorting algorithm and is often used, right alongside Merge Sort. It's a good example of an efficient sorting algorithm, with an average complexity of O(n logn). Part of its popularity also derives from the ease of implementation.

          We will use simple integers in the first part of this article, but we'll give an example of how to change this algorithm to sort objects of a custom class.

          Quicksort is a representative of three types of sorting algorithms: divide and conquer, in-place, and unstable.

        • GCC 10.0 Status Report (2019-11-18), Stage 3 in effect now
        • GCC 10 Feature Development Is Over - Now The Focus Turns To Bug Fixing

          GCC 10 has moved to its next stage of development that shifts away from feature work to instead general bug fixing with hopes of shipping the GNU Compiler Collection 10 release in the months ahead.

          GCC 10 release manager Richard Biener of SUSE announced this Monday morning that "stage three" development phase.

        • ttdo 0.0.4: Extension

          A first update release to the still very new (and still very small) ttdo package arrived on CRAN today. Introduced about two months ago in September, the ttdo package extends the most excellent (and very minimal / zero depends) unit testing package tinytest by Mark van der Loo with the very clever and well-done diffobj package by Brodie Gaslam.

        • Adventures optimizing a bytecode based scripting language
        • Keeping a simple markdown work-log, via emacs

          For the past few years I've been keeping a work-log of everything I do. I don't often share these, though it is sometimes interesting to be able to paste into a chat-channel "Oh on the 17th March I changed that .."

        • Implement C++ coroutines.
          This patch series is an initial implementation of a coroutine feature,
          expected to be standardised in C++20.
          
          

          Standardisation status (and potential impact on this implementation): ----------------------

          The facility was accepted into the working draft for C++20 by WG21 in February 2019. During two following WG21 meetings, design and national body comments have been reviewed, with no significant change resulting.

          Mature implementations (several years) of this exist in MSVC, clang and EDG with some experience using the clang one in production - so that the underlying principles are thought to be sound.

          At this stage, the remaining potential for change comes from two areas of national body comments that were not resolved during the last WG21 meeting: (a) handling of the situation where aligned allocation is available. (b) handling of the situation where a user wants coroutines, but does not want exceptions (e.g. a GPU).

          It is not expected that the resolution to either of these will produce any major change.

          The current GCC implementation is against n4835 [1].

          ABI ---

          The various compiler developers have discussed a minimal ABI to allow one implementation to call coroutines compiled by another; this amounts to:

          1. The layout of a public portion of the coroutine frame. 2. A number of compiler builtins that the standard library might use.

          The eventual home for the ABI is not decided yet, I will put a draft onto the wiki this week.

          The ABI has currently no target-specific content (a given psABI might elect to mandate alignment, but the common ABI does not do this).

          There is not need to add any new mangling, since the components of this are regular functions with manipulation of the coroutine via a type-erased handle.

          Standard Library impact -----------------------

          The current implementations require addition of only a single header to the standard library (no change to the runtime). This header is part of the patch series.

          GCC Implementation outline --------------------------

          The standard's design for coroutines does not decorate the definition of a coroutine in any way, so that a function is only known to be a coroutine when one of the keywords (co_await, co_yield, co_return) is encountered.

          This means that we cannot special-case such functions from the outset, but must process them differently when they are finalised - which we do from "finish_function ()".

          At a high level, this design of coroutine produces four pieces from the original user's function:

          1. A coroutine state frame (taking the logical place of the activation record for a regular function). One item stored in that state is the index of the current suspend point. 2. A "ramp" function This is what the user calls to construct the coroutine frame and start the coroutine execution. This will return some object representing the coroutine's eventual return value (or means to continue it when it it suspended). 3. A "resume" function. This is what gets called when a the coroutine is resumed when suspended. 4. A "destroy" function. This is what gets called when the coroutine state should be destroyed and its memory returned.

          The standard's coroutines involve cooperation of the user's authored function with a provided "promise" class, which includes mandatory methods for handling the state transitions and providing output values. Most realistic coroutines will also have one or more 'awaiter' classes that implement the user's actions for each suspend point. As we parse (or during template expansion) the types of the promise and awaiter classes become known, and can then be verified against the signatures expected by the standard.

          Once the function is parsed (and templates expanded) we are able to make the transformation into the four pieces noted above.

          The implementation here takes the approach of a series of AST transforms. The state machine suspend points are encoded in three internal functions (one of which represents an exit from scope without cleanups). These three IFNs are lowered early in the middle end, such that the majority of GCC's optimisers can be run on the resulting output.

          As a design choice, we have carried out the outlining of the user's function in the front end, and taken advantage of the existing middle end's abilities to inline and DCE where that is profitable.

          Since the state machine is actually common to both resumer and destroyer functions, we make only a single function "actor" that contains both the resume and destroy paths. The destroy function is represented by a small stub that sets a value to signal the use of the destroy path and calls the actor. The idea is that optimisation of the state machine need only be done once - and then the resume and destroy paths can be identified allowing the middle end's inline and DCE machinery to optimise as profitable as noted above.

          The middle end components for this implementation are: 1. Lower the coroutine builtins that allow the standard library header to interact with the coroutine frame (these fairly simple logical or numerical substitution of values given a coroutine frame pointer). 2. Lower the IFN that represents the exit from state without cleanup. Essentially, this becomes a gimple goto. 3. Lower the IFNs that represent the state machine paths for the resume and destroy cases. 4. A very late pass that is able to re-size the coroutine frame when there are unused entries and therefore choose the minimum allocation for it.

          There are no back-end implications to this current design.

          GCC Implementation Status -------------------------

          The current implementation should be considered somewhat experimental and is guarded by a "-fcoroutines" flag. I have set out to minimise impact on the compiler (such that with the switch off, coroutines should be a NOP).

          The branch has been feature-complete for a few weeks and published on Compiler Explorer since late September. I have been keeping a copy of the branch on my github page, and some bug reports have been filed there (and dealt with).

          The only common resource taken is a single bit in the function decl to flag that this function is determined to be a coroutine.

          Patch Series ------------

          The patch series is against r278049 (Mon 11th Nov).

          There are 6 pieces to try an localise the reviewer interest areas. However it would not make sense to commit except as possibly two (main and testsuite). I have not tested that the compiler would even build part-way through this series.

          1) Common code and base definitions.

          This is the background content, defining the gating flag, keywords etc.

          2) Builtins and internal functions.

          Definitions of the builtins used by the standard library header and the internal functions used to implement the state machine.

          3) Front end parsing and AST transforms.

          This is the largest part of the code, and has essentially two phases 1. parse (and template expansion) 2. analysis and transformation, which does the code generation for the state machine.

          4) Middle end expanders and transforms

          As per the description above.

          5) Standard library header.

          This is mostly mandated by the standard, although (of course) the decision to implement the interaction with the coroutine frame by inline builtin calls is pertinent.

          There is no runtime addition for this (the builtins are expanded directly).

          6) Testsuite.

          There are two chunks of tests. 1. those that check for correct error handling 2. those that check for the correct lowering of the state machine

          Since the second set are checking code-gen, they are run as 'torture' tests with the default options list.

          ======

          I will put this patch series onto a git branch for those that would prefer to view it in that form.

          thanks Iain
        • Initial Patches Wire In C++20 Coroutines For The GCC Compiler

          The GNU Compiler Collection continues picking up new features aligned for the upcoming C++20 standard. The latest are patches pending on the mailing list for implementing coroutines in C++.

          C++20 is expected to have coroutines per the pending technical specification. Coroutines allow a function to have its execution stopped/suspended and then to be resumed later.

        • Faster Winter 1: Vectors
        • Faster Winter 2: SPECIALIZE
        • Attention! Attention! Tutorial Proposal Deadline Approaching
        • Python Software Foundation Fellow Members for Q3 2019

          Congratulations! Thank you for your continued contributions. We have added you to our Fellow roster online.

          The above members have contributed to the Python ecosystem by teaching Python, maintaining popular libraries/tools such as cryptography and pytest, helping document on packaging.python.org, organizing Python events, starting Python communities in their home countries, and overall being great mentors in our community. Each of them continues to help make Python more accessible around the world. To learn more about the new Fellow members, check out their links above.

        • Why Sponsor PyCon 2020?

          Sponsors help keep PyCon affordable and accessible to the widest possible audience. Sponsors are what make this conference possible. From low ticket prices to financial aid, to video recording, the organizations who step forward to support PyCon, in turn, support the entire Python community. They make it possible for so many to attend, for so many to be presenters, and for the people at home to watch along.

        • Pandas GroupBy: Your Guide to Grouping Data in Python

          Whether you’ve just started working with Pandas and want to master one of its core facilities, or you’re looking to fill in some gaps in your understanding about .groupby(), this tutorial will help you to break down and visualize a Pandas GroupBy operation from start to finish.

          This tutorial is meant to complement the official documentation, where you’ll see self-contained, bite-sized examples. Here, however, you’ll focus on three more involved walk-throughs that use real-world datasets.

        • Solving CAPTCHA with Web automation

          CAPTCHA is no longer an alien term to the users. An acronym for Completely Automated Public Turing test to tell Computers and Humans Apart. CAPTCHA is basically a computer program built to distinguish between the human and machine to prevent any type of spam or data extraction from websites. The entire concept of CAPTCHA is based on the assumption that only a human would pass this test and bot or automated scripts would fail.

  • Leftovers

    • Distributed Teams: Why I Don’t Go to the Office More Often

      Total time to get home: 3h30min. Total distance traveled: 103km. Total cost: $3.10 for the subway token, $46 PRESTO ($6 for the card, $20 for the fare, $20 for the surprise fare), $2.86 for the LRT.

      At this point I’ve been awake for over 20 hours.

      Is it worth it? Hard to say. Every time I plan one of these trips I look forward to it. Conversations with office folks, eating office lunch, absconding with office snacks… and this time I even got to go out to dinner with a bunch of data people I work with all the time!

      But every time I do this, as I’m doing it, or as I’m recently back from doing it… I don’t feel great about it. It’s essentially a full work day (nearly eight full hours!) just in travel to spend 5 hours in the office, and (this time) a couple hours afterwards in a restaurant.

      Ultimately this — the share of my brain I need to devote purely to logistics, the manifold ways things can go wrong, the sheer _time_ it all takes — is why I don’t go into the office more often.

    • Science

      • The Early History of Usenet, Part III: File Format

        When we set out to design the over-the-wire file format, we were certain of one thing: we wouldn't get it perfectly right. That led to our first decision: the very first character of the transmitted file would be the letter "A" for the version. Why not a number on the first line, including perhaps a decimal point? If we ever considered that, I have no recollection of it. A more interesting question is why we didn't use email-style headers, a style later adopted for HTTP. The answer, I think, is that few, if any, of us had any experience with those protocols at that time. My own personal awareness of them started when I requested and received a copy of the Internet Protocol Transition Workbook a couple of years later — but I was only aware of it because of Usenet. (A few years earlier, I gained a fair amount of knowledge of the ARPANET from the user level, but I concentrated more on learning Multics.)

        Instead, we opted for the minimalist style epitomized by 7th Edition Unix. In fact, even if we had known of the Internet (in those days, ARPANET) style, we may have eschewed it anyway. Per a later discussion of implementation, the very first version of our code was a shell script. Dealing with entire lines as single units, and not trying to parse headers that allowed arbitrary case, optional white space, and continuation lines was certainly simpler!

        [...]

        Sending a date and an article title were obvious enough that these didn't even merit much discussion. The date and time line used the format generated by the ctime() or asctime() library routines. I do not recall if we normalized the date and time to UTC or just ignored the question; clearly, the former would have been the proper choice. (There is an interesting discrepancy here. A reproduction of the original announcement clearly shows a time zone. Neither the RFC nor the ctime() routine had one. I suspect that announcement was correct.) The most interesting question, though, was about what came to be called newsgroups.

        We decided, from the beginning, that we needed multiple categories of articles — newsgroups. For local use, there might be one for academic matters ("Doctoral orals start two weeks from tomorrow"), social activities ("Reminder: the spring picnic is Sunday!"), and more. But what about remote sites? The original design had one relayed newsgroup: NET. That is, there would be no distinction between different categories of non-local articles.

      • From humble Unix sysadmin to brutal separatist suppressor to president of Sri Lanka

        A former Unix sysadmin has been elected the new president of Sri Lanka, giving hope to all those IT workers who fear they are trapped in a role where the smallest of decisions can have catastrophic consequences if it goes wrong.

        Gotabaya Rajapaksa, younger brother of former president Mahindra, won the popular vote in an election held on Saturday (16 November). He is notable to The Register's readership for his stint working in America as a Solaris system integrator and later as a Unix sysadmin for a Los Angeles university.

    • Security (Confidentiality/Integrity/Availabilitiy)

    • Transparency/Investigative Reporting

      • The son of Julian Assange’s judge is linked to an anti-data leak company created by the UK intelligence establishment

        The son of Julian Assange’s senior judge is linked to an anti-data leak company created by the UK intelligence establishment and staffed by officials recruited from US intelligence agencies behind that country’s prosecution of the WikiLeaks founder.

        The son of Lady Emma Arbuthnot, the Westminster chief magistrate overseeing the extradition proceedings of Julian Assange, is the vice-president and cyber-security adviser of a firm heavily invested in a company founded by GCHQ and MI5 which seeks to stop data leaks, it can be revealed.

    • Environment

      • [Older] Soil in the Arctic Is Now Releasing More Carbon Dioxide Than 189 Countries

        The Arctic is now releasing more carbon dioxide in the winter than it can absorb in the summer, according to a new report. Now that heat waves are occurring in the winter, and the Arctic is warming three times faster than the global average because of human activity, greenhouse gases that would have normally remained frozen in the ground are being released into the atmosphere, according to a study published in the journal Nature Climate Change. The study indicates that more than 1.7 billion tons of carbon dioxide are being released from Arctic soil annually because of warming temperatures — but plant growth in the region can only draw around 1.1 billion tons of carbon dioxide into the soil during warmer months.

        [...]

        For years, scientists had suspected that an important shift was underway in the Arctic as global temperatures increased, but no data had been collected on emissions in the region. A team of scientists from 12 countries set out to change this by placing monitoring devices in more than 100 sites in the Arctic and gathered more than 1,000 readings, according to CBC. After extrapolating the data, they calculated annual emissions and compared them with the annual absorption level, and found a significant gap. The scientists estimate that emissions from the region will increase by 40% by the end of the century if countries continue on a “business-as-usual” trajectory and make no major efforts to halt the use of fossil fuels. Even if meaningful climate action occurs, emissions from the Arctic will rise by an estimated 17% by the end of the century.

    • Finance

      • ECB hawk Lautenschlaeger resigns amid policy backlash

        Germany’s appointee to the board of the European Central Bank, outspoken policy hawk Sabine Lautenschlaeger, has decided to step down, the ECB said on Wednesday, in what was likely to be seen as a further backlash against the latest stimulus measures.

        The ECB did not give a reason for Lautenschlaeger’s resignation, which will take effect at the end of next month -more than two years before the end of her term as one of the six members of the Executive Board that runs the central bank.

        “Sabine Lautenschlaeger, Member of the Executive Board and Governing Council of the European Central Bank (ECB), informed President Mario Draghi that she will resign from her position on 31 October 2019,” the ECB said.

      • The End of Neoliberalism and the Rebirth of History

        For 40 years, elites in rich and poor countries alike promised that neoliberal policies would lead to faster economic growth, and that the benefits would trickle down so that everyone, including the poorest, would be better off. Now that the evidence is in, is it any wonder that trust in elites and confidence in democracy have plummeted?

        [...]

        The form of globalization prescribed by neoliberalism left individuals and entire societies unable to control an important part of their own destiny, as Dani Rodrik of Harvard University has explained so clearly, and as I argue in my recent books Globalization and Its Discontents Revisited and People, Power, and Profits. The effects of capital-market liberalization were particularly odious: If a leading presidential candidate in an emerging market lost favor with Wall Street, the banks would pull their money out of the country. Voters then faced a stark choice: Give in to Wall Street or face a severe financial crisis. It was as if Wall Street had more political power than the country’s citizens. Even in rich countries, ordinary citizens were told, “You can’t pursue the policies you want” – whether adequate social protection, decent wages, progressive taxation, or a well-regulated financial system – “because the country will lose competitiveness, jobs will disappear, and you will suffer.” In rich and poor countries alike, elites promised that neoliberal policies would lead to faster economic growth, and that the benefits would trickle down so that everyone, including the poorest, would be better off. To get there, though, workers would have to accept lower wages, and all citizens would have to accept cutbacks in important government programs.

    • Freedom of Information / Freedom of the Press

      • A massive scandal: how Assange, his doctors, lawyers and visitors were all spied on for the U.S.

        It sounds like a James Bond movie, but it really happened. Julian Assange, the WikiLeaks journalists and every single lawyer, reporter, politician, artist and physician who visited the founder of WikiLeaks at the Ecuadorian embassy over the last seven years was subjected to systematic espionage. Meetings and conversations were recorded and filmed, and all the information was sent to US intelligence. Sometimes the espionage operations were truly off the wall: at one point spies even planned to steal the diaper of a baby brought to visit Assange inside the embassy. The purpose? To gather the baby's feces and perform a DNA test to establish whether the newborn was a secret son of Julian Assange.

        Repubblica has had access to some of the videos, audios and photos. Meetings between the founder of WikiLeaks and his lawyers, medical examinations of Julian Assange, diplomatic encounters of the Ecuadorian ambassador Carlos Abad Ortiz, meetings between Assange and journalists. Everything was spied on.

        The author of this article found that she was not just filmed, but her phones were screwed open, presumably to obtain the IMEI code that allows uniquely identifying the phone in order to intercept it. Spies also had access to our USB sticks, though at this stage it is not clear if they managed to break the encryption protecting the information stored in the USB flash drives inside our backpacks. These are very serious violations of the confidentiality of journalistic sources, given that our meetings inside the embassy were entirely professional and, as frequent visitors, we were repeatedly registered as "journalists".

    • Monopolies

      • Patents and Software Patents

        • German patent reform discussed at Brussels conference: automatic injunctions contravene EU law

          Apologies for being a bit slow to report on my own conference (held last week). On Friday I published all seven slide decks and an abstract. But there's so much going on that I have quite a backlog, which besides the component-level licensing parts of last week's conference also includes a DOJ amicus brief and a couple of SEP-related position papers as well as the Supreme Court's grant of certiorari in Oracle v. Google.

          Not only have I received plenty of positive feedback to last Tuesday's conference from attendees but there was also an unmistakable sign: even though we were running one hour behind schedule, the room was still almost full at the end of the day. That speaks to the way the conference as a whole was received, and also to the popularity of the "bonus session" (as I called it because it wasn't exactly about component-level licensing, and not even SEP-specific, apart from Bram Nijhof's highly informative overview of the evolution of case law since Huawei v. ZTE) at the end of a long day.

          Edmund Mangold, patent counsel at BMW and personally very much involved with the German patent reform debate at the level of industry bodies, provided an introduction. After Taylor Wessing's Bram Nijhof's case law overview, Cleary Gottlieb's Maurits Dolmans made an unequivocal statement in the title of his presentation: "We Need Proportionality Review for Patent Injunctions under German Law"

          It's no secret that I couldn't agree more. Having observed well over 100 German patent trials this decade (SEP and non-SEPs alike), I've seen too many situations in which injunctions came down that probably--and sometimes most probably--wouldn't have been granted in other jurisdictions. All too often it's easy for an experienced litigation watcher to figure that some plaintiffs are playing the lottery by asserting a bunch of mostly dubious patents in Germany (typically, the most questionable ones are selected for complaints brought in Munich) in hopes of a lucky punch that allows the winner to take it all: a favorable settlement.

        • PAIR UPDATES

          Tough 1-2 punch from the PTO: Private PAIR can no longer be used to access file histories not associated with your customer number; Public PAIR doesn’t work. On Friday, USPTO Chief Information Officer Jamie Holcombe provided a report to the PPAC but did not include this important practice change.

          Note: Public file histories are also available via the USPTO’s Global Dossier: https://globaldossier.uspto.gov/#/

        • Officers of the United States Shall be Appointed by the President

          The basic framework is that the Constitution requires all “Officers of the United States” to be nominated by the President with “Consent of the Senate.” U.S. Const. Article II, Section 2. The so-called appointments clause has a caveat that “inferior Officers” may be appointed by Courts or Heads of Departments if Congress so allows. Currently PTAB Judges (Administrative Patent Judges) are treated as inferior officers by statute — appointed by the Secretary of Commerce. However, the increased responsibility of AIA Trials has pushed their role into the territory of Principal Officers that must be appointed by the President.

        • National Patent Application Drafting Competition

          The program is an expansion of the International Patent Drafting Competition and is the brain child of Damian Porcari – Director of the Midwest Regional Patent Office in Detroit and longtime in-house IP counsel at Ford.

      • Copyrights

        • US court to hear long-running Google vs. Oracle case

          The complex case pitting two Silicon Valley giants against each other has raged on since 2010, and already saw many twists and turns before a jury found in favor of Google only to have that decision reversed by a circuit court. That prompted Google's appeal to the nation's highest court this past March.

          Oracle at the time asked the US Supreme Court to not review an appellate court's decision finding Google violated Oracle's copyright of the Java platform when building the Android mobile operating system.

          In that opposition brief, Oracle's attorneys said Google's copyright violation shut Oracle, the Java platform owner, out of the emerging smartphone market, causing incalculable harm to its business.

          Oracle noted Google had previously asked for a writ of certiorari—the legal term for review by the high court—in 2015 without success in an earlier phase of the case, and the company argues nothing has changed in the time since.



Recent Techrights' Posts

Free Software Foundation Subpoenaed by Serial GPL Infringers
These attacks on software freedom are subsidised by serial GPL infringers
Publicly Posting in Social Control Media About Oneself Makes It Public Information
sheer hypocrisy on privacy is evident in the Debian mailing lists
 
Links 01/05/2024: Surveillance and Hadopi, Russia Clones Wikipedia
Links for the day
Links 01/05/2024: FCC Takes on Illegal Data Sharing, Google Layoffs Expand
Links for the day
Links 01/05/2024: Calendaring, Spring Idleness, and Ads
Links for the day
Paul Tagliamonte & Debian: White House, Pentagon, USDS and anti-RMS mob ringleader
Reprinted with permission from disguised.work
Jacob Appelbaum character assassination was pushed from the White House
Reprinted with permission from disguised.work
Why We Revisit the Jacob Appelbaum Story (Demonised and Punished Behind the Scenes by Pentagon Contractor Inside Debian)
If people who got raped are reporting to Twitter instead of reporting to cops, then there's something deeply flawed
Red Hat's Official Web Site is Promoting Microsoft
we're seeing similar things at Canonical's Ubuntu.com
Enrico Zini & Debian: falsified harassment claims
Reprinted with permission from disguised.work
European Parliament Elections 2024: Daniel Pocock Running as an Independent Candidate
I became aware that Daniel Pocock had decided to enter politics
Over at Tux Machines...
GNU/Linux news for the past day
IRC Proceedings: Tuesday, April 30, 2024
IRC logs for Tuesday, April 30, 2024
[Meme] Sometimes Torvalds and RMS Agree on Things
hype around chatbots
[Video] Linus Torvalds on 'Hilarious' AI Hype: "I Hate the Hype" and "I Don't Want to be Part of the Hype", "You Need to Be a Bit Cynical About This Whole Hype Cycle"
Linus Torvalds on LLMs
Colin Watson, Steve McIntyre & Debian, Ubuntu cover-up mission after Frans Pop suicide
Reprinted with permission from disguised.work
Links 30/04/2024: Wireless Carriers Selling Customer Location Data, Facebook Posts Causing Trouble
Links for the day
Frans Pop suicide and Ubuntu grievances
Reprinted with permission from disguised.work
Links 30/04/2024: More Google Layoffs (Wide-Ranging)
Links for the day
Fresh Rumours of Impending Mass Layoffs at IBM Red Hat
"IBM filed a W.A.R.N with the state of North Carolina. That only means one thing."
Workers' Right to Disconnect Won't Matter If Such a Right Isn't Properly Enforced
I was always "on-call" and my main role or function was being "on-call" in case of incidents
Mark Shuttleworth's (MS's) Canonical is Promoting Microsoft This Week (Surveillance Slanted as 'Confidential')
Who runs Canonical these days? Why does Canonical help sell Windows?
A Discussion About Suicides in Science and Technology (Including Debian and the European Patent Office)
In Debian, there is a long history of deaths, suicides, and mysterious disappearances
Federal News Network is Corrupt, It Runs Propaganda Pieces for Microsoft
Federal News Network used to be OK some years ago
What Mark Shuttleworth and Canonical Can to Remedy the Damage Done to Frans Pop's Family
Mr. Shuttleworth and Canonical as a company can at the very least apologise for putting undue pressure
Amnesty International & Debian Day suicides comparison
Reprinted with permission from disguised.work
[Meme] A Way to Get No Real Work Done
Walter White looking at phone: Your changes could not be saved to device
Modern Measures of 'Productivity' Boil Down to Time Wasting and Misguided Measurements/Yardsticks
People are forgetting the value of nature and other human beings
Countries That Beat the United States at RSF's World Press Freedom Index (After US Plunged Some More)
The United States (US) was 17 when these rankings started in 2002
Record Productivity and Preserving People's Past on the Net
We're very productive these days, partly owing to online news slowing down (less time spent on curating Daily Links)
Over at Tux Machines...
GNU/Linux news for the past day
IRC Proceedings: Monday, April 29, 2024
IRC logs for Monday, April 29, 2024
Links 30/04/2024: Malaysian and Russian Governments Crack Down on Journalists
Links for the day
Frans Pop Debian Day suicide, Ubuntu, Google and the DEP-5 machine-readable copyright file
Reprinted with permission from disguised.work
Axel Beckert (ETH Zurich), the mentality of sexual violence on campus
Reprinted with permission from Daniel Pocock
[Meme] Russian Reversal
Mark Shuttleworth: In Soviet Russia's spacecraft... Man exploits peasants
Frans Pop & Debian suicide denial
Reprinted with permission from disguised.work
Hard Evidence Reinforces Suspicion That Mark Shuttleworth May Have Worked Volunteers to Death
Today we start re-publishing articles that contain unaltered E-mails
The Real Threats to Society Include Software Patents and the Corporations That Promote Them
The OIN issue isn't a new one and many recognise this by now
Links 30/04/2024: OpenBSD and Enterprise Cloaking Device
Links for the day
Microsoft Still Owes Over 100 Billion Dollars and It Cannot be Paid Back Using 'Goodwill'
Meanwhile, Microsoft's cash at hand (in the bank) nearly halved in the past year.
[Teaser] Ubuntu Cover-up After Death
Attack the messenger
The Cyber Show Explains What CCTV is About
CCTV does not typically resolve crime
[Video] Ignore Buzzwords and Pay Attention to Attacks on Software Developers
AI in the Machine Learning sense is nothing new
Outline of Themes to Cover in the Coming Weeks
We're accelerating coverage and increasing focus on suppressed topics
[Video] Not Everyone Claiming to Protect the Vulnerable is Being Honest
"Diversity" bursaries aren't always what they seem to be
[Video] Enshittification of the Media, of the Web, and of Computing in General
It manifests itself in altered conditions and expectations
[Meme] Write Code 100% of the Time
IBM: Produce code for us till we buy the community... And never use "bad words" like "master" and "slave" (pioneered by IBM itself in the computing context)
[Video] How Much Will It Take for Most People to Realise "Open Source" Became Just Openwashing (Proprietary Giants Exploiting Cost-Free or Unpaid 'Human Resources')?
turning "Open Source" into proprietary software
Freedom of Speech... Let's Ban All Software Freedom Speeches?
There's a moral panic over people trying to actually control their computing
Richard Stallman's Talk in Spain Canceled (at Short Notice)
So it seems to have been canceled very fast
Links 29/04/2024: "AI" Hype Deflated, Economies Slow Down Further
Links for the day
Gemini Links 29/04/2024: Gopher Experiment and Profectus Alpha 0.9
Links for the day
[Video] Why Microsoft is by Far the Biggest Foe of Computer Security (Clue: It Profits From Security Failings)
Microsoft is infiltrating policy-making bodies, ensuring real security is never pursued
Debian 'Cabal' (via SPI) Tried to Silence or 'Cancel' Daniel Pocock at DNS Level. It Didn't Work. It Backfired as the Material Received Even More Visibility.
know the truth about modern slavery
Lucas Nussbaum & Debian attempted exploit of OVH Hosting insider
Reprinted with permission from disguised.work
Software in the Public Interest (SPI) is Not a Friend of Freedom
We'll shortly reproduce two older articles from disguised.work
Harassment Against My Wife Continues
Drug addict versus family of Techrights authors
Syria, John Lennon & Debian WIPO panel appointed
Reprinted with permission from disguised.work
Over at Tux Machines...
GNU/Linux news for the past day
IRC Proceedings: Sunday, April 28, 2024
IRC logs for Sunday, April 28, 2024
[Video] GNU and Linux Everywhere (Except by Name)
In a sense, Linux already has over 50% of the world's "OS" market