11.12.19

Maintaining the ‘Delete Github’ page

Posted in Microsoft, Site News at 11:31 am by Dr. Roy Schestowitz

By figosdev

Summary: “This list really is a starting point, which can hopefully increase awareness about the issue of concern.”

THE primary goal of the Delete Github page is an early-warning system for projects that Microsoft has undue control over — they can restrict access, delete projects, or (perhaps in a worst-case scenario) either deliberately modify a project download as SourceForge once did, or simply act a vector for malicious activity — as has already happened to Gentoo and Canonical.

“At this stage there is an effort to be as comprehensive as possible — to include as many mainstream Free software projects as we can, to give people a better idea of how many projects are potentially threatened by a monopoly.”As with higher quality warning systems, including all anti-malware software, false positives are a known possibility. Some care is taken to avoid them, simply to increase the quality of the list itself. At this stage there is an effort to be as comprehensive as possible — to include as many mainstream Free software projects as we can, to give people a better idea of how many projects are potentially threatened by a monopoly.

Admittedly, there is a lower threshold of evidence to get on the list than to be removed. While people may decide to make decisions based on this list, we hope they would double-check our findings first. This is research, it is not law — the list includes IceWM, which I use routinely, and JWM, which I have running as a process as we speak. It includes Leafpad, which I am using to type this very line of text. If we find evidence that WordPress is using Github (and there is a repo for it, I haven’t checked it yet and it isn’t on the list yet) it is unlikely that Roy will decide to remove it from the Techrights servers.

“To some of us, this list is like an endangered species list — we are concerned about the future of projects that are hosted on Github.”If a clerical error places an allergy you don’t have on your medical records, it is very possible that when this error is found, a single line will be drawn through the note. That is the sort of response that will most likely come of discovering errors in the list. A prominent note (perhaps in bold text) is the most likely correction.

To some of us, this list is like an endangered species list — we are concerned about the future of projects that are hosted on Github. And we are most eager to update the status of each one (even possibly Systemd, not necessarily for any wishes of success.)

But how can we know that we are in error? Certainly there is some level of assurance we can be given, most likely we will try to re-evaluate items that are called into question as our priorities allow. But we can check on the first one during the writing of this article:

“We will also probably use this lowercase convention for projects that abandon Github for other repos — if it’s all lowercase, that means we no longer believe the project is relying on Github.”“OpenBSD does not use Github for development at all Roy. They do not even use git!”

“I’m sure there are others in the list that also do not use github.”

Hopefully so. And by no means is Wikipedia a facts-only source of information, but if we simply grep https://en.wikipedia.org/wiki/OpenBSD:

$ leafpad <(wget -O- https://en.wikipedia.org/wiki/OpenBSD | tr ">" "\n" | grep ithub)

We get this:

<a rel="nofollow" class="external text" href="https://github.com/openbsd"

“When this list started, it was a completely manual process.”Now, it’s possible that OpenBSD doesn’t use this repo. If so, the article should be corrected. Let’s increase the quality of this check and visit the actual article: https://en.wikipedia.org/wiki/OpenBSD

Ah, now — one of the not-yet-implemented (though already considered) quality checks is to verify if the Github link is in the summary box (or whatever it’s called, I’m not a Wikipedian) on the top right. Selecting it and right-clicking, “View Selection Source” we find this is probably called an “infobox” and that it begins with:

<table class="infobox vevent"

It completes with "</tbody></table>" which we can use in the future to parse infoboxes.

Doing a further search for our Github link, we find the non-url text of our link is in the “External links” section, titled “Github mirror”.

“There are still checks in place, but in the writing of this article we have found things we can do to refine the process.”What have we learned:

1. The Github link isn’t in the infobox section, which decreases the significance of the link.

2. The official repo is https://cvsweb.openbsd.org/cgi-bin/cvsweb/ this is actually the most pertinent information.

3. The text of the very link that put OpenBSD on our list was labeled “Github mirror” which isn’t necessarily criteria we want or need to add projects to this list.

“Obviously people who are more intimately familiar with these projects are encouraged to help us, as arm did.”I would say this warrants a correction, which I will issue now before the article is finished. Since the first letter of every list entry is capitalised, we will make it easier to parse / automatically filter corrected entries by making the title all lowercase: OpenBSD -> openbsd. And… updated. Thanks to arm for setting this straight.

We will also probably use this lowercase convention for projects that abandon Github for other repos — if it’s all lowercase, that means we no longer believe the project is relying on Github.

When this list started, it was a completely manual process. Since then, (only) some aspects have drifted towards rudimentary automation. This has produced a more complete list, with a slightly higher tendency towards false positives. There are still checks in place, but in the writing of this article we have found things we can do to refine the process.

“For the moment, the highest priority is still adding to the list, so we may not double check every existing entry right away.”Even after items are added, we can use this new information to double-check the entries (even all at once.) That doesn’t mean this will happen today, but it may ultimately make it easier to discover when projects abandon Github, which would be a bonus. Obviously people who are more intimately familiar with these projects are encouraged to help us, as arm did.

For the moment, the highest priority is still adding to the list, so we may not double check every existing entry right away. This is the stage where the list is built as large as possible, within reason and utility.

At no point has the process become entirely automated, though I did just parse all of English Wikipedia for Github entries. Wikipedia will not be (is not) our only source, entries do not automatically go from discovery to inclusion (they are checked further, and also chosen manually for relevance, but please feel very free to suggest items you think are worth adding) but we will now try to slightly refine the process of adding new entries, first.

“We also want to encourage projects to leave Github.”We want everybody to realise that Github holds more mainstream projects than many of us thought. We also want to encourage projects to leave Github. We already know that GNOME and KDE mostly have their own repositories, though at the moment it appears that Kate for example, may use Github for something (that is still being looked into.) It’s surprising to discover gnuradio on Github — we are also aware that some of these (like OpenBSD) are just mirrors.

This list really is a starting point, which can hopefully increase awareness about the issue of concern. As awareness increases, we fully expect the quality of this information to be further refined — not unlike the software projects listed here as talented people review them and contribute to the sources.

Thanks again, you’ve helped make this list better than it was when we started. █

Licence: Creative Commons CC0 1.0 (public domain)

Permalink  Send this to a friend