EditorsAbout the SiteComes vs. MicrosoftUsing This Web SiteSite ArchivesCredibility IndexOOXMLOpenDocumentPatentsNovellNews DigestSite NewsRSS

08.19.19

Speaking Truth to Monopolies (or How to Write Guest Posts in Techrights)

Posted in Site News at 4:16 am by Dr. Roy Schestowitz

“The jaws of power are always open to devour, and her arm is always stretched out, if possible, to destroy the freedom of thinking, speaking, and writing.”

John Adams

Summary: We need to have more articles tackling the passage of all power — especially when it comes to software — to few large monopolies that disregard human rights or actively participate in their abolishment in the digital realm

I HAVE spent much of my adult life writing about (and against) software monopolies. I had done that before this site even existed. Seeing that the reach of Techrights is growing and more people get involved in various capacity levels, we openly — and freely — welcome more articles from more people.

“We’re at the point now where both Free software and GNU/Linux are in a peculiar and precarious position.”The topics we cover aren’t hard to see; we do not, for example, publish HowTos; instead we just link to many. We ‘specialise’ in tackling attacks on Software Freedom, be these attacks technical or legal (e.g. acquisition or patents). We’re at the point now where both Free software and GNU/Linux are in a peculiar and precarious position. Their “livery” — so to speak — is being swapped by companies like Microsoft. It’s an attack on the very identity of one’s idealogical opposition. It’s designed to confuse, to obfuscate, to disorientate. We need to fight back as narratives are being distorted, not only in the media but mostly in the media. The demise of several big publishers contributes to this.

We invite readers to contribute posts. We’re very liberal when it comes to format and substance. Articles can be sent to bytesmedia@bytesmedia.co.uk which our core people read on a daily basis.

There’s a good chance Techrights will have posted its 26,000th post before the site turns 13 (middle of November), i.e. average at over 2,000 posts per year. We’re becoming more productive this year because more people have become involved.

08.17.19

Caturdays and Sundays at Techrights Will Get Busier

Posted in Site News at 4:12 am by Dr. Roy Schestowitz

Not cat photos but analysis of issues pertaining (or puuurtaining) to Software Freedom

Cat

Summary: Our plan to spend the weekends writing more articles about Software Freedom; it seems like a high-priority issue

THE growth of openwashing recently necessitated more and more responses (some of which too long for editorial comments in daily links). So last weekend we started the “Openwashing Report” — a series we intend to continue this weekend. This does not mean that we will focus any less on the EPO, Campinos, Battistelli and so on.

“There’s a certain urgency as windows seem to be closing on digital freedom…”Having recently (about 3 weeks ago) 'quit' Twitter I now have more time to spend writing articles. Weekends in particular will be spent writing about Free/libre software, GNU/Linux and technology rights (as per this site’s name). We will try to publish more articles per day (at our peak about a decade ago we averaged at more than 10 articles a day). As I’m working full time (job unrelated to this site) it’s more likely that weekends as opposed to weekdays will have more articles produced (counterintuitive as paid writers publish throughout the week and barely during weekends). There’s a certain urgency as windows seem to be closing on digital freedom; we now have listening devices out there; Microsoft admits recording and retaining it all (it’s a GAFAM thing). This is considered almost ‘normal’ now. How did we get to this point? How do we get out of it? Expect a bit of a focus shift. Let’s hope that each Caturday can make a positive difference by means of reporting the ills. We’re watching Free software adversaries.

Cat

Why Techrights Doesn’t Do Social Control Media

Posted in Site News at 3:36 am by Dr. Roy Schestowitz

More about Social Control than about Media. Standing on one’s own means more freedom of speech and no self-censorship.

One-tree hill

Summary: Being managed and censored by platform owners (sometimes their shareholders) isn’t an alluring proposition when a site challenges conformist norms and the status quo; Techrights belongs in a platform of its own

AFTER posting more than 670,000 tweets and having coined the term "Social Control Media" (which even Julian Assange and Wikileaks adopted) I decided to no longer use Twitter except to check replies once a day. It had become a massive productivity drain and usually an utter waste of time. I still have stuff posted there, albeit only as copies exported from decentralised and Free software platforms such as Diaspora and Pleroma (Mastodon-compatible).

“It seems pretty clear where Twitter is going with this; it wants to eventually become another Facebook.”A decade ago Techrights, Boycott Novell and TechBytes had active accounts in Identi.ca (and two of these in Twitter as well). At some stage it seemed clear that this kind of activity was detrimental to — not contributory towards — actual journalism. Techrights never had a Twitter account and character length is still a major limitation. Over the years surveillance and bloat got a lot worse; almost exactly a year ago Twitter also killed third-party tools by deprecating key APIs. It seems pretty clear where Twitter is going with this; it wants to eventually become another Facebook. We probably don’t have to explain why Facebook is so bad (many aspects to that).

There’s nothing to regret here overall; we didn’t participate in these sites and we probably lost nothing by staying out of these. I have personal accounts there, but these express my personal views (on politics) rather than the site’s.

“There’s nothing to regret here overall; we didn’t participate in these sites and we probably lost nothing by staying out of these.”Social Control Media (so-called ‘social’ ‘media’) is neither social nor media; when people socialise they don’t get managed by the billions by one single company/shareholders and media has generally (historically) checked claims/facts. Twitter lacks that.

Social Control Media has sadly ‘replaced’ the “long form” writings in a lot of blogs. That’s a shame really. Quality is being compromised for the sake of speed and concision. When we’re trying to actually find/syndicate reliable blogs we nowadays come to realise that many are inactive/dormant. Instead of sites they become “accounts” (on someone else’s platform, complete with throttling, censorship and ads); what used to be a site/blog is just some Twitter account that posts and reposts unverified nonsense. Techrights doesn’t wish to ever become something so shallow.

08.15.19

Links Are Not Endorsements

Posted in Site News at 2:10 am by Dr. Roy Schestowitz

No rollerblades

Summary: If the only alternative is to say nothing and link to nothing, then we have a problem; a lot of people still assume that because someone links to something it therefore implies agreement and consent

WE recently wrote about how Twitter cheapens if not ruins fact-finding/fact-checking. Weeks later it turned out that someone who had decided to declare a person dead was in fact wrong. That person is still alive. The Web is a fascinating maze of fabrications, hearsay, bad reporting, but also good investigative journalism. It’s not always easy to tell apart one from the other and it’s something we work on. Sometimes, including last night, we get feedback; a reader made contact with us to tell us something that we linked to (in daily links) was wrong. It doesn’t happen very often, maybe a couple of times per year and usually when it’s about more sensitive and divisive subjects like Kashmir.

“We don’t link to social control media; we only link to blogs and news sites. That’s still prone to mistakes/issues. But most importantly when we link to something that does not imply we endorse the message.”Our daily links are an effort to make sense of a lot of information. (Mis)information overload is a big problem and the ‘cheapening’ of it, especially because of social control media, means that any random person can utter false words. These things get rated and shared (passed on) based on emotion rather than adherence to facts, so that’s another problem. The scoring is all wrong. Nowadays a lot of so-called ‘articles’ are based on nothing but a “tweet”; that’s another problem.

Sometimes a Web site or an article we link to may offend someone. Sometimes the article is wrong. We don’t link to social control media; we only link to blogs and news sites. That’s still prone to mistakes/issues. But most importantly when we link to something that does not imply we endorse the message. Sometimes we link to sites we do not agree with simply because we need to highlight something that was said, done, or happened. In social control media it’s common to specify in profiles/disclaimers that “links (or other) are not endorsements”; the same applies here.

What we argue in our own (original) articles is another matter altogether. At least those can be judged based on more than a link alone.

08.10.19

Techrights Coding Projects: Making the Web Light Again

Posted in Site News at 11:05 am by Dr. Roy Schestowitz

A boatload of bytes that serve no purpose at all (99% of all the traffic sent from some Web sites)

Very bloated boat

Summary: Ongoing technical projects that improve access to information and better organise credible information preceded by a depressing overview regarding the health of the Web (it’s unbelievably bloated)

OVER the past few months (since spring) we’ve been working hard on coding automation and improving the back end in various ways. More than 100 hours were spent on this and it puts us in a better position to grow in the long run and also improve uptime. Last year we left behind most US/USPTO coverage to better focus on the European Patent Office (EPO) and GNU/Linux — a subject neglected here for nearly half a decade (more so after we had begun coverage of EPO scandals).

As readers may have noticed, in recent months we were able to produce more daily links (and more per day). About a month ago we reduced the volume of political coverage in these links. Journalism is waning and the quality of reporting — not to mention sites — is rapidly declining.

“As readers may have noticed, in recent months we were able to produce more daily links (and more per day).”To quote one of our guys, “looking at the insides of today’s web sites has been one of the most depressing things I have experienced in recent decades. I underestimated the cruft in an earlier message. Probably 95% of the bytes transmitted between client and server have nothing to do with content. That’s a truly rotten infrastructure upon which society is tottering.”

We typically gather and curate news using RSS feed readers. These keep sites light and tidy. They help us survey the news without wrestling with clickbait, ads, and spam. It’s the only way to keep up with quality while leaving out cruft and FUD (and Microsoft's googlebombing). A huge amount of effort goes into this and it takes a lot of time. It’s all done manually.

“We typically gather and curate news using RSS feed readers. These keep sites light and tidy. They help us survey the news without wrestling with clickbait, ads, and spam.”“I’ve been letting wget below run while I am mostly outside painting part of the house,” said that guy, having chosen to survey/assess the above-stated problem. “It turns out that the idea that 95% of what web severs send is crap was too optimistic. I spidered the latest URL from each one of the unique sites sent in the links from January through July and measured the raw size for the individual pages and their prerequisites. Each article, including any duds and 404 messages, averaged 42 objects [3] per article. The median, however, was 22 objects. Many had hundreds of objects, not counting cookies or scripts that call in scripts.

“I measured disk space for each article, then I ran lynx over the same URLs to get the approximate size of the content. If one counts everything as content then the lynx output is on average 1% the size of the raw material. If I estimate that only 75% or 50% of the text rendered is actual content then that number obviously goes down proportionally.

“I suppose that means that 99% of the electricity used to push those bits around is wasted as well. By extension, it could also mean that 99% of the greenhouse gases produced by that electricity is produced for no reason.

“The results are not scientifically sound but satisfy my curiosity on the topic, for now.

“Eliminating the dud URLs will produce a much higher object count.

“The results are not scientifically sound but satisfy my curiosity on the topic, for now.”
      –Anonymous
“Using more mainstream sites and fewer tech blogs will drive up the article sizes greatly.

“The work is not peer reviewed or even properly planned. I just tried some spur of the minute checks on article sizes in the first way I could think of,” said the guy. We covered this subject before in relation to JavaScript bloat and sites' simplicity, but here we have actual numbers to present.

“The numbers depend on the quality of the data,” the guy added, “that is to say the selection of links and the culling the results of 404′s, paywall messages, and cookie warnings and so on.

“As mentioned I just took the latest link from each of the sites I have bookmarked this year. That skews it towards lean tech blogs. Though some publishers which should know very much better are real pigs:


$ wget --continue --page-requisites --timeout=30
--directory-prefix=./test.a/

https://www.technologyreview.com/s/614079/what-is-geoengineering-and-why-should-you-care-climate-change-harvard/

. . .

$ lynx --dump

https://www.technologyreview.com/s/614079/what-is-geoengineering-and-why-should-you-care-climate-change-harvard/

> test.b

$ du -bs ./test.?
2485779	./test.a
35109	./test.b

“Trimming some of the lines of cruft from the text version for that article, I get close to two orders of magnitude difference between the original edition versus the trimmed text edition:

$ du -bs ./test.?
2485779	./test.a
35109	./test.b
27147	./test.c

“Also the trimmed text edition is close to 75% the size of the automated text edition. So, at least for that article, the guess of 75% content may be about right. However, given the quick and dirty approach, of this survey, not much can be said conclusively except 1) there is a lot of waste, 2) there is an opportunity for someone to do an easy piece of research.”

Based on links from 2019-08-08 and 2019-08-09, we get one set of results (extracted all URLs saved from January 2019 through July 2019; http and https only, eliminated PDF and other links to obviously non-html material). Technical appendices and footnotes are below for those wishing to explore further and reproduce.



+ this only retrieves the first layer of javascript, far from all of it
+ some site gave wget trouble, should have fiddled the agent string,
	--user-agent=""
+ too many sites respond without proper HTTP response headers,
	slows collection down intolerably
+ the pages themselves often contain many dead links
+ serial fetching is slow and because the sites are unique

$ find . -mindepth 1 -maxdepth 1 -type d -print | wc -l
91
$ find . -mindepth 1 -type f -print | wc -l
4171
which is an average of 78 objects per "article"

+ some sites were tech blogs with lean, hand-crafted HTML,
	mainstream sites are much heavier,
	so the above average is skewed towards being too light

Quantity and size of objects associated with articles,
does not count cookies nor secondary scripts:

$ find . -mindepth 1 -type f -printf '%s\t%p\n' \
| sort -k1,1n -k2,2 \
| awk '$1>10{
		sum+=$1;
		c++;
		s[c]=$1;
		n[c]=$2
	}
	END{
		printf "%10s\t%10s\n","Bytes","Measurement";
		printf "%10d\tSMALLEST\n",s[1];
		for (i in s){
			if(i==int(c/2)){
				printf "%10d\tMEDIAN SIZE\n",s[i];
			}
		};
		printf "%10d\tLARGEST\n",s[c];
		printf "%10d\tAVG SIZE\n",sum/c;
		printf "%10d\tCOUNT\n",c;
	}'

     Bytes      File Size
        13      SMALLEST
     10056      MEDIAN SIZE
  32035328      LARGEST
     53643      AVG SIZE
     38164      COUNT

     


Overall article size [1] including only the first layer of scripts,

     Bytes      Article Size
      8442      SMALLEST
    995476      MEDIAN
  61097209      LARGEST
   2319854      AVG
       921      COUNT

Estimated content [2] size including links, headers, navigation text, etc:

+ deleted files with errors or warnings,
	probably a mistake as that skews the results for lynx higher

     Bytes      Article Size
       929      SMALLEST
     18782      MEDIAN
    244311      LARGEST
     23997      AVG
       889      COUNT

+ lynx returns all text within the document not just the main content,
	at 75% content the figures are more realistic for some sites:

     Bytes      Measurement
       697	SMALLEST
     14087	MEDIAN
    183233	LARGEST
     17998	AVG
       889	COUNT

	at 50% content the figures are more realistic for other sites:

       465	SMALLEST
      9391	MEDIAN
    122156	LARGEST
     11999	AVG
       889	COUNT


       
       
$ du -bs * \
| sort -k1,1n -k2,2 \
| awk '$2!="l" && $1 {
		c++;
		s[c]=$1;
		n[c]=$2;
		sum+=$1
	}
	END {
		for (i in s){
			if(i==int(c/2)){
				m=i
			};
			printf "% 10d\t%s\n", s[i],n[i]
		};
		printf "% 10s\tArticle Size\n","Bytes";
		printf "% 10d\tSMALLEST %s\n",s[1],n[1];
		printf "% 10d\tMEDIAN %s\n",s[m],n[m];
		printf "% 10d\tLARGEST  %s\n",s[c],n[c];
		printf "% 10d\tAVG\n", sum/c;
		printf "% 10d\tCOUNT\n",c;
	}' OFS=$'\t'



[1]

$ time bash -c 'count=0;
shuf l \
| while read u; do
	echo $u;
	wget --continue --page-requisites --timeout=30 "$u" &
	echo $((count++));
	if ((count % 5 == 0)); then
		wait;
	fi;
	done;'
	


[2]

$ count=0;
time for i in $(cat l); do
	echo;echo $i;
	lynx -dump "$i" > $count;
	echo $((count++));
	done;


[3]

$ find . -mindepth 1 -maxdepth 1 -type d -print | wc -l
921

$ find . -mindepth 1 -type f -print | wc -l
38249



[4]

$ find . -mindepth 1 -type f -print \
| awk '{sub("\./","");sub("/.*","");print;}' | uniq -c | sort -k1,1n
-k2,2 | awk '$1{c++;s[c]=$1;sum+=$1;} END{for(i in s){if(i ==
int(c/2)){m=s[i];}}; print "MEDIAN: ",m; print "AVG", sum/c; print
"Quantity",c; }'



[5] 

$ find . -mindepth 1 -type f -name '*.js' -exec du -sh {} \; | sort
-k1,1rh | head
16M     ./www.icij.org/app/themes/icij/dist/scripts/main_8707d181.js
3.4M
./europeanconservative.com/wp-content/themes/Generations/assets/scripts/fontawesome-all.min.js
1.8M    ./www.9news.com.au/assets/main.f7ba1448.js
1.8M
./www.technologyreview.com/_next/static/chunks/commons.7eed6fd0fd49f117e780.js
1.8M    ./www.thetimes.co.uk/d/js/app-7a9b7f4da3.js
1.5M    ./www.crossfit.com/main.997a9d1e71cdc5056c64.js
1.4M
./www.icann.org/assets/application-4366ce9f0552171ee2c82c9421d286b7ae8141d4c034a005c1ac3d7409eb118b.js
1.3M
./www.digitalhealth.net/wp-content/plugins/event-espresso-core-reg/assets/dist/ee-vendor.e12aca2f149e71e409e8.dist.js
1.2M
./www.fresnobee.com/wps/build/webpack/videoStory.bundle-69dae9d5d577db8a7bb4.js
1.2M    ./www.ft.lk/assets/libs/angular/angular/angular.js


[6] About page bloat, one can pick just about any page and find from one to close to two orders of magnitude difference between the lynx dump and the full web page. For example,


$ wget --continue --page-requisites --timeout=30 \
    --directory-prefix=./test.a/ \

https://www.newsweek.com/saudi-uae-war-themselves-yemen-1453371

. . .

$ lynx --dump \
    https://www.newsweek.com/saudi-uae-war-themselves-yemen-1453371 \
    > test.b

$ du -bs ./test.?
250793	./test.a
 15385	./test.b

Guest Post: Why I Wrote Fig (the Computer Language That Helps Techrights Operations)

Posted in Free/Libre Software, Site News at 8:38 am by Dr. Roy Schestowitz

By figosdev

Fig project

Summary: Fig project introduced; its developer wrote some code for Techrights and has just decided to explain why he uses his own programming language

SOME helper code we recently published for the site [1, 2] had been written in Fig, whose developer explained the language as follows.


Since Techrights is now publishing Fig code, I thought it best to write a couple of articles about it — one explaining how to use it, and the other explaining why.

Stallman didn’t start the GNU project just to have another operating system, and I didn’t write Fig just to have another language. Rather I grew up with fun languages like Basic and Logo, and wanted to use them to teach coding to friends and others.

Logo lives on in modern adaptations, and so does Basic. A lot of people think Basic is bad for you, but that’s due to stuff written about the versions of Basic in the 1960s and 70s, before it gained better commands for structural programming. Both Linus Torvalds and I first learned Basic as our introduction to programming. It’s a fun, simple language when it’s done right.

“Both Linus Torvalds and I first learned Basic as our introduction to programming. It’s a fun, simple language when it’s done right.”Python is a good language for beginners, and it has modern advantages that even the most modern versions of Basic do not — it’s also a good shell language, a decent language for making games, as well as applications. But Python is definitely not as simple as Basic, and I wanted to create an even simpler language if possible.

A simple language benefits from having few rules, but it does not need to be completely consistent. If your language only has 2 rules, it will be very consistent but it could be tedious to work within that level of consistency. Fig tries to find a balance between having as few rules as possible, and having enough to be friendly.

The original Basic had very few commands, which made it trivial to learn “the whole thing.” Fig was not my first attempt to create a fun, simple language — in fact it is named after the logo for a previous language project of mine — originally fig was called “Fig Basic”. Like its predecessor, Fig was a experiment in lightweight syntax.

I had a couple of rules for developing Fig — one, punctuation would only be added as needed, within reason. This produced a language that uses “quotes for strings” and # hashes for comments. Decimal points work the same way in fig that they do in Python and Basic.

“Since Fig compiles to Python code, and has an inline Python feature, it was (and remains) possible to cheat and just use Python pretty much wherever it is needed.”As with punctuation for syntax, the other rule was to only add new statements or functions as needed, when it became too tedious to do without them. This resulted in a language with fewer than 100 commands.

Since Fig compiles to Python code, and has an inline Python feature, it was (and remains) possible to cheat and just use Python pretty much wherever it is needed. You can create a Fig function that is just a wrapper around a Python import, and that’s pretty nice. Then you can call that function using Fig’s lightweight syntax.

Fig can also do shell code, so it can interact with other programs on the computer. But I wrote an extensive (20 short chapters) tutorial for Basic more than 12 years ago, started adapting it to Python, and eventually decided to teach coding in terms of the following near-standard features:

1. variables
2. input
3. output
4. basic math
5. loops
6. conditionals
7. functions

I realise that in Python, “functions” are really method calls, but that’s not very important to most people learning programming for the first time. I see a lot of people working their way up to “rock paper scissors” programs when first learning Python, and that’s typical of how they taught Basic as well.

“While Python is case-sensitive and indented, Fig uses Basic-like (Pascal-like, Bash-like) command pairs and is case-insensitive.”I started creating simple Python tutorials aimed at Basic coders, and those gradually turned into a simple library of Basic-like Python routines, which eventually turned into Fig Basic. And Fig Basic is different than most dialects of Basic, because it took into account a lot of things I learned while trying to teach Basic and Python — namely, the things people had the most problems with in terms of syntax.

While Python is case-sensitive and indented, Fig uses Basic-like (Pascal-like, Bash-like) command pairs and is case-insensitive. Today even most versions of Basic are case-sensitive, but during its peak as a language it was not.

Fig is not parenthetical and has no operator order, other than left to right. It inherits Python’s (and QBasic’s) conventions about newlines — after an if statement, you start a newline (QBasic had a cheat about that, but Fig is consistent.)

So for example, a for loop from 10 to 2 with a step of -1 might look like this:

    for p = (10, 2, -2) 
        now = p ; print
        next

Many things about this are optional — both equals signs, the parentheses, the commas and the indentation. Here is the code with those removed:

    for p 10 2 -2 
    now p print
    next

This is based on the inspiration Fig takes from logo (specifically turtle graphics) which tends to require less punctuation in its syntax than almost any popular language — certainly among the educational languages.

The inline Python feature naturally requires indentation.

“The inline Python feature naturally requires indentation.”But I’ve created languages before and since, and Fig is only the best of those. What I really want is for people to learn programming and for people to learn how to create simple programming languages. The latter is something I’ve taught as a sort of “next step” after coding basics.

I strongly feel that we need a more computer-literate society, and code is really the shortest route to computer literacy. A programming language is basically a text-based interface to the computer itself (built on a number of abstractions… so is a command line, but a command line works like a programming language all on its own.) We need to make it easy for teachers to learn, so they can make it easy for students to learn.

A more computer-literate society would be better able to make political decisions regarding computers, and it would be better able to contribute to Free software, so creating more tools to teach coding to everyone would help immensely in my opinion.

“I strongly feel that we need a more computer-literate society, and code is really the shortest route to computer literacy.”And I feel if more people worked to create their own language, we would learn a lot more about what sorts of features (and omissions) would best suit the task of creating an educational language for everyone. Sure, Basic did very well and Logo has done very well; Python is doing well. As to whether we can make it so fewer people struggle, or explaining what makes coding truly easy or truly difficult in the initial steps, there is only so much data we have on this. We could have a lot more.

Years ago, I wanted to create a piece of software as a “kit” for making it much, much easier to design your own language. Instead of creating a formal specification and writing/generating a parser, you would choose a parser based on the style of your language and tweak it to your needs. There would be a giant collection of features, and a way of turning that into a catalog, from which you would “select” the routines you wanted your language to have.

Meanwhile, Fig is possible to take apart and change. version 4.6 is a single 58k Python script, with 1,154 unique lines of code. This includes a simple help system, which allows you to search for any command by name (even part of the name) and that gives you the syntax and a quick description of the command and what it does.

I would be just as happy for people to adapt Fig or be inspired to create their own version or own language, as I would be for them to adopt Fig for their own use. I would still like to make it easier to teach coding, so that more people are capable, earlier on, with less intimation — just like in the days when Logo was really, really simple.

“I would be just as happy for people to adapt Fig or be inspired to create their own version or own language, as I would be for them to adopt Fig for their own use.”I now use Fig for most tasks, and let me apologise for the code that Roy has published so far. I wrote that as code for internal use and took every shortcut, and he was totally free to publish it, but I don’t use the same care (recommended with good reason) when naming variables that I do when naming commands for a programming language. I actually put a lot of thought into that sort of thing — but when I name variables, I’m very sloppy. It’s a beginner’s mistake, and I still do that more than a quarter of a century later.

I will write a simple Fig tutorial or two next, but I will try to use better variable names. One convention I use a lot though — is if I need a “throwaway” variable, I will just use “p” or “now.” If the variable is going to be referenced a lot, these are definitely not good variables names at all. Writing Fig has made me a better coder and designing a language will make you a better coder too, but the variable names thing — sorry about that.

Fig puts almost 100% of variables on the left of each line (almost like creating a named pipe) so they’re easy to find. For loops and Forin loops put the variable a little more to the right, but every “standard” command in Fig begins with a variable:

    howmuch = 2
    p = "hello" ; left howmuch ; ucase ; len ; print ; print ; print

You can download the latest version of fig (4.6 is very stable!) here from PyGame. You can also find the language featured in the first issue of DistroWatch Weekly of the new 2017 year:

[download fig 4.6] SHA256: 4762871cf854cc8c1b35304a26d7e22c5f73b19a223b699d7946accea98d2783

Whatever language you love most, happy coding!

Licence: Creative Commons cc0 1.0 (public domain)

08.09.19

Techrights Blog Archive and Word Analysis

Posted in Site News at 2:21 am by Dr. Roy Schestowitz

Old library

Summary: Summing up almost 13 years of Techrights and making archival simpler just in case something bad happens some time in the future

IN LIGHT of Linux Journal's shutdown, which was sudden and announced in the middle of the summer holidays (may have been strategic timing), we’ve already published a list of Techrights blog URLs and our full Wiki as XML files (or compressed archive of them all). That’s not because we expect anything bad to happen; it’s just better to be prepared in advance rather than when it’s too late. I’ve poured my entire adult life into this site and the analysis/research it takes to write/run it. It’s good to have backups, including elsewhere. Mirroring does no harm, it improves overall resilience when things are decentralised. During the archiving process figosdev wrote some code that we have not published until today. It’s used to generate a keyword index with merges for similar ones. So for example, for the latter it comes up with (number of occurrences on the left):


14191 chrome 9611:chrome. 233:chromebook 1754:chromebook. 121:chromebooks 1513:chromebooks. 156:chromebox 74:chromecast 396:chromeos 333:
14195 e.u. 109:eu 13203:eu-us 107:eu-wide 144:eu. 632:
14271 process 11848:process. 2423:
14278 china 13069:china-based 95:china. 1114:
14371 receive 3588:received 7239:received. 141:receives 1405:receiving 1998:
14411 firefox 14411:
14556 download 9003:download. 1015:downloadable 220:downloaded 1417:downloader 101:downloaders 97:downloading 1100:downloads 1454:downloads. 149:
14739 gates 14158:gates-backed 114:gates-funded 141:gates. 326:
14850 digital 14577:digital. 86:digitally 187:
14894 employed 938:employee 2773:employee. 154:employees 6620:employees. 704:employer 989:employer. 168:employers 644:employing 284:employment 1158:employment. 93:employs 369:
15026 package 6369:package. 512:packaged 533:packagers 85:packages 5762:packages. 767:packaging 998:
15170 menace 148:menacing 100:menos 418:mental 1124:mentality 273:mentally 216:mention 4183:mention. 103:mentioned 6743:mentioned. 155:mentioning 720:mentions 987:
15196 games 14158:games. 1038:
15231 event 7653:event. 954:events 6020:events. 604:
15350 corporate 8067:corporates 77:corporation 2759:corporation. 240:corporations 3790:corporations. 417:
15419 battistelli 14513:battistelli. 906:
15560 browser 9915:browser. 1333:browsers 4024:browsers. 288:
15709 privacy 13613:privacy-focused 124:privacy. 992:privacy/surveillance 980:
15749 press 14852:press. 897:
15759 require 3825:required 4340:required. 380:requirement 1182:requirement. 139:requirements 2226:requirements. 480:requires 3187:
15932 intel 15517:intel-based 101:intel. 232:intel/amd 82:
16050 police 15513:police. 537:
16695 mobile 16432:mobile. 263:
16877 troll 4535:troll. 434:trolles 205:trolling 1127:trolling. 111:trolls 9276:trolls. 1189:
16880 enterprise 13684:enterprise-class 132:enterprise-grade 221:enterprise-ready 131:enterprise. 424:enterprises 2072:enterprises. 216:
16985 hardware 15321:hardware. 1664:
17492 policies 4017:policies. 638:policy 11524:policy. 1313:
17832 reported 9524:reported. 848:reportedly 3159:reporter 1927:reporter. 94:reporters 2141:reporters. 139:
17865 user 16688:user. 826:userbase 82:username 174:usernames 95:
18106 federal 18106:
18157 money 16399:money. 1758:
18170 result 6842:result. 452:resulted 1431:resulting 1346:results 7119:results. 980:
18209 management 17099:management. 1110:
18362 research 11250:research. 727:researched 153:researcher 1346:researchers 4435:researchers. 196:researching 255:
18746 review 12483:review. 723:reviewed 1304:reviewer 149:reviewers 230:reviewing 721:reviews 2931:reviews. 205:
18993 censor 1588:censored 952:censored. 74:censoring 859:censors 823:censorship 12500:censorship. 701:censorship/free 972:censorship/privacy/civil 404:censorship/web 120:
19233 debconf 276:debian 17855:debian-based 626:debian. 476:
19233 name 10460:name. 764:named 4402:names 3607:
19371 facebook 18648:facebook. 723:
19681 question 9688:question. 879:questioned 808:questioning 729:questions 6938:questions. 639:
19804 order 14028:order. 692:ordered 2399:ordering 431:orders 2047:orders. 207:
19861 organisation 3331:organisation. 393:organisational 97:organisations 1789:organisations. 194:organization 7031:organization. 717:organizational 290:organizations 5484:organizations. 535:
20396 national 20321:national-security 75:
20636 member 7943:member. 320:members 11588:members. 785:
21621 copyright 16583:copyright. 470:copyrighted 710:copyrights 3676:copyrights. 182:
21704 official 11777:official. 211:officials 9084:officials. 632:
21809 rights 18893:rights. 1936:rights/policing 980:
21944 cloud 20228:cloud-based 661:cloud. 1055:
21978 nsa 21533:nsa. 445:
22096 industrial 2497:industries 1720:industries. 357:industry 15436:industry. 2086:
22146 kde 21049:kde. 488:kde3 72:kde4 537:
22170 fedora 21663:fedora. 507:
22411 standard 8863:standard. 889:standardisation 232:standardised 77:standardization 422:standardize 182:standardized 329:standardizing 80:standards 9113:standards-based 123:standards. 1265:standards/consortia 836:
22416 market 16524:market. 2932:markets 2361:markets. 599:
22428 protect 6965:protected 2043:protected. 142:protecting 2235:protection 8152:protection. 633:protections 1715:protections. 307:protective 236:
22526 political 13646:politician 758:politicians 3860:politicians. 247:politico 350:politics 3118:politics. 547:
22664 product 8764:product. 1060:products 10698:products. 2142:
22994 publish 2200:published 13513:published. 392:publisher 1348:publisher. 70:publishers 1654:publishers. 127:publishes 825:publishing 2757:publishing. 108:
23532 next 23174:next. 358:
23594 permalink 23594:
23726 foundation 22066:foundation. 1660:
23765 opensource 268:open-source 22369:open-source. 110:open-sourced 387:open-sources 402:open-sourcing 229:
24419 gnome 23574:gnome-based 84:gnome-shell 123:gnome. 497:gnome3 141:
24981 video 18925:video. 596:video> 271:videolan 113:videos 4794:videos. 282:
25022 icons 24892:icons. 130:
25227 plan 8738:plan. 578:planned 3084:planned. 168:planning 3385:planning. 77:plans 8758:plans. 439:
25671 attack 10039:attack. 819:attacked 1573:attacker 739:attackers 1018:attacking 1986:attacks 8438:attacks. 1059:
26054 network 14441:network. 1261:networked 234:networking 4706:networking. 186:networks 4378:networks. 848:
26394 countries 9065:countries. 1438:country 13370:country. 2521:
26827 information 24642:information. 2185:
27074 phone 16072:phone. 891:phones 9315:phones. 796:
28926 http 28821:http/2 105:
29542 posted 28779:posted. 94:poster 429:posters 240:
29681 server-side 196:server 19956:server. 1406:servers 6949:servers. 1174:
29914 office 27490:office. 2424:
30007 platform 19050:platform. 3365:platformer 484:platforms 5649:platforms. 1459:
30132 development 26183:development. 2005:developments 1744:developments. 200:
30137 code 25179:code. 3191:coder 167:coders 349:coding 1168:coding. 83:
30291 media 28467:media. 1824:
30445 feature 9058:feature. 481:features 18736:features. 2170:
30791 kernel 27263:kernel. 2250:kernel-based 284:kernels 870:kernels. 124:
31184 communities 3334:communities. 737:community 23348:community-based 105:community-driven 193:community. 3467:
31828 u.s 201:u.s. 28527:u.s.-backed 113:u.s.-based 132:u.s.-led 93:us-backed 91:us-based 401:us-led 135:usa 1960:usa. 175:
32042 used 31372:used. 670:
32990 program 13901:program. 2001:programmed 179:programmer 803:programmers 1432:programmers. 143:programmes 381:programming 5917:programming. 237:programming/development 818:programs 6212:programs. 966:
33607 reader 4579:reader. 278:readers 28308:readers. 347:readership 95:
34045 internet 30644:internet. 2458:internet/net 943:
34523 america 11784:america. 1032:american 14414:american. 80:americans 6414:americans. 589:americas 210:
34826 page 6591:page. 1001:pages 3250:pages. 23984:
34864 technological 1251:technologically 152:technologies 7327:technologies. 1290:technology 22387:technology. 2457:
35836 presidency 1256:presidency. 275:president 27787:president. 1084:presidential 4834:presidents 600:
37154 distribution 12395:distribution. 1967:distributions 12171:distributions. 1095:distro 5812:distro. 652:distros 2743:distros. 319:
37446 epo 35039:epo. 2075:epo.org 332:
37687 public 35751:public. 1936:
37723 social 37600:social-media 123:
37846 gnu 6426:gnu. 93:gnu/hurd 81:gnu/kfreebsd 85:gnu/linux 29520:gnu/linux-based 91:gnu/linux. 1550:
38367 novell 36653:novell. 1714:
39579 developer 13599:developer. 373:developers 23843:developers. 1764:
40375 case 27904:case. 2515:caselaw 108:cases 8853:cases. 995:
41238 version 33068:version. 1440:versioned 146:versioning 164:versions 5822:versions. 598:
41601 users 37295:users. 4306:
42802 europe 16585:europe. 1902:european 22948:europeans 444:euros 923:
42826 court 34888:court. 1976:courts 5330:courts. 632:
43149 photon 97:photos 2662:photos. 191:photoshop 483:php 2278:php-fpm 105:php5 164:phpmyadmin 164:phrase 911:phrases 272:physical 2586:physically 388:physician 130:physicians 160:physicist 95:physics 466:physx 70:pi 7518:pi. 361:piana 111:pic 72:pichai 173:pick 1939:picked 1183:picket 100:picking 722:picks 900:pico 71:pico-itx 156:pics 99:picture 2355:picture. 321:pictured 311:pictures 1450:pictures. 80:pidgin 230:pie 1637:pie. 72:piece 5026:piece. 171:pieces 2059:pieces. 132:pierre 180:pierre-yves 94:pieter 113:pig 220:pigs 193:pike 97:pile 578:piled 78:piles 129:pilger 118:piling 112:pill 149:pillar 135:pillars 217:pilot 988:pilots 320:pim 201:pimping 70:pin 272:pinch 120:pine 176:pine64 118:
43445 reporting 4831:reporting. 240:report 26127:report. 1583:reports 9493:reports. 1171:
43555 update 20736:update. 1135:updated 8817:updated. 210:updatedx2 85:updater 93:updates 10442:updates. 999:updating 1038:
43583 desktop 36980:desktop. 2291:desktop/gtk 1351:desktops 2468:desktops. 493:
49940 win32 90:windows 46655:windows-based 204:windows-only 192:windows. 2799:
50033 project 32819:project. 3734:projects 11447:projects. 2033:
51865 government 43803:government. 2416:governmental 467:governments 4822:governments. 357:
54220 web 52026:web-based 864:web-browser 150:web. 1180:
54986 goog 218:google 53003:google+ 410:google. 1355:
55737 secure 7203:secure. 417:security 46115:security-focused 77:security. 1925:
60088 support 49775:support! 74:support. 3530:supported 6428:supported. 281:
64412 android 60980:android-based 684:android-powered 403:android-x86 196:android. 1650:android/linux 499:
66673 patenting 1475:patenting. 120:patents 58590:patents. 6408:patents] 80:
68988 people 65485:people. 3115:peoples 388:
71735 ubucon 83:ubuntu 68622:ubuntu! 86:ubuntu-based 643:ubuntu-powered 183:ubuntu. 1791:ubuntu/linux 327:
82416 patent 79808:patent. 1178:patented 1245:patented. 185:
111622 release 54907:release! 153:release. 5083:released 36637:released! 844:released. 1621:releases 11390:releases. 987:
117419 who 86708:why 30100:why. 611:
123764 open 123225:open. 539:
128149 software 111593:software-as-a-service 94:software-based 127:software-defined 636:software-like 535:software-related 106:software. 8488:software/open 6570:
165377 msft 784:microsoft 157420:microsoft-connected 384:microsoft-friendly 239:microsoft-funded 276:microsoft-influenced 73:microsoft-novell 147:microsoft-sponsored 129:microsoft-taxed 102:microsoft. 5451:microsoft/novell 227:microsoft] 145:
199787 linux 189841:linux! 329:linux- 121:linux. 8892:linux-related 104:linux/android 266:linux/unix 150:linux] 84:

The underlying code, trc.fig (in Fig):

#### license: creative commons cc0 1.0 (public domain) 
#### http://creativecommons.org/publicdomain/zero/1.0/ 
# load log.txt (db of entries) and search from trc.txt (output from coll.fig)

python
    reload(sys)
    sys.setdefaultencoding('utf8')
    fig

nl 10 chr
now "" arr
nowlen arr

function getlink p
    eachr p ltrim
    pg split eachr nl mid 1 1
    pr split pg "ignorethis " join pr "" return pr
    fig

function rplace p c t
    now split p c join now t return now
    fig

function ctext t
    quot 34 chr
    nl 10 chr
    tab 9 chr
    now t lcase rplace now quot nl swap now t
    now t rplace now " " nl swap now t
    now t rplace now "&" nl swap now t
    now t rplace now ";" nl swap now t

    now t rplace now ")" nl swap now t
    now t rplace now "(" nl swap now t
    now t rplace now "," nl swap now t
    now t rplace now "'" nl swap now t
    now t rplace now ":" nl swap now t
    now t rplace now "?" nl swap now t
python
    t = t.replace(unichr(8220), chr(10))
    t = t.replace(unichr(8221), chr(10))
    fig
    now t rplace now tab nl swap now t return t
    fig

function nohtml p
    buf ""
    intag 0 
    forin each p
        ifequal each "<"
            intag 1
            fig
        ifequal intag 0
            now buf plus each swap now buf
            fig
        ifequal each ">" 
            intag 0
            fig
        next
    now return buf
    fig

function lookfor findthese db dbcopy origfind
    nl 10 chr
    dblen db len
    buf 1
    for eachlen 1 dblen 1
        each arrget db eachlen
        copyeach arrget dbcopy eachlen

        forin eachofthese findthese
            iftrue eachofthese
                #now eachofthese plus "-" print
                now each split now nl
                found 0
                found instr now eachofthese
                iftrue found

                    iftrue buf
                        now origfind print # print current list of searchwords
                        buf 0
                        fig

                    now "*[[" prints
                    now copyeach split now "\n" mid 6 2 
                    eurl getlink copyeach prints "|" prints
                    etitle now mid 2 1 ltrim rtrim
                    edate now mid 1 1 ltrim rtrim plus " " plus etitle plus "]]" print
                    iftrue 0
                        now eachofthese print # print found keyword
                        fig
                    break
                    fig

                fig
            next

        next
    fig



db "" arr
dbcopy "" arr
dblen 0
python 
    db = open("log.txt").read().split("If you liked this post, consider")
    dbcopy = db[:]
    dblen = len(db)
    fig

now "processing entries... " prints
prevcount 0
for eachdb dblen 1 -1
    dbcount eachdb divby dblen times 1000 int divby 10 minus 100 times -1

    ifmore dbcount prevcount
        now dbcount str plus "% " prints
        prevcount dbcount
        fig 
    p arrget db eachdb 
    now nohtml p ctext now
    db arrset eachdb now

    p arrget dbcopy eachdb 
    now nohtml p
    dbcopy arrset eachdb now

    next
now "" print

pg arropen "trc.txt" 
nowlen pg len

for eachloop nowlen 1 -1
    each pg mid eachloop 1   
    eaches split each " " 
    eacheslen eaches len minus 1
    now "" print
    reaches eaches right eacheslen join reaches " " split reaches ":" 
python
    reaches.remove('')
    fig

    iftrue reaches
        findthese arr mid 1 0 
        forin eachreaches reaches
            onereach split eachreaches " " mid 1 1 
            findthese plus onereach 
            next
        iftrue findthese
            now lookfor findthese db dbcopy each
            fig
        fig

   next

And coll.fig


#### license: creative commons cc0 1.0 (public domain) 
#### http://creativecommons.org/publicdomain/zero/1.0/ 
# combine trranks into trc.txt (with > trc.txt)
fl arropen "trranks"
buf "" arr mid 1 0
buftext ""
bufsum 0
forin each fl
    lentr each rtrim len
    iftrue lentr
        buf plus each
    else
        iftrue buf
            forin beach buf
                now buftext plus beach plus ":" swap now buftext
                teg split beach " " mid 1 1
                rank split beach " " mid 2 1 val
                now bufsum plus rank swap now bufsum
                next
            now bufsum prints " " prints
            now buftext print 
            buf "" arr mid 1 0
            buftext ""
            bufsum 0
            fig
    fig
    next

There’s more code we’ve yet to share. Just needs more tidying up. We want everything to be open data, Free/libre software, Open Access etc. It takes time.

08.06.19

Techrights Available for Download as XML Files

Posted in Site News at 3:09 am by Dr. Roy Schestowitz

Open data, Free/libre software, free society

tar gz file

Summary: Archiving of the site in progress; scripts and data are being made available for all to download

IN ADDITION to Drupal pages and almost 26,000 WordPress blog posts (and videos and podcasts, not to mention loads of IRC logs) we also have nearly 700 Wiki pages and a reader decided to organise an archive or an index for an archive.

Now available for download as a compressed archive (lots of XML files) is last week’s pile. It’s also available for download from the Internet Archive. Those are all the Wiki pages.

The code in Fig:


# < fig code >
# comments | built-in functions | user functions | inline python

#### license: creative commons cc0 1.0 (public domain) 
#### http://creativecommons.org/publicdomain/zero/1.0/ 
proginf = "wikidump 0.1 jul 2019 mn"
# dump xml for each wiki page on techrights (no edit hist)
# create each actual file
function dumppage url
    urltext = arrcurl url
    title = ""
    forin each urltext
        findtitle = instr each "<title>"
        iftrue = findtitle
            title = each ltrim rtrim split title "<title>" join title "" split title "</title>" join title ""
            break
            fig
        next
    iftrue title
        outfilename = "techrights_" plus title plus ".xml" split outfilename "/" join outfilename ":slash:" open "w"
        forin each urltext
            now = each fprint outfilename
            next
        now = outfilename close
        fig
    fig

# get list of pages
allpages = arrcurl "http://techrights.org/wiki/index.php/Special:AllPages"
allpageslen = allpages len
longest = 0
longestindex = 0
for each 1 allpageslen 1
    eachlen = allpages mid each 1 len
    ifmore eachlen longest
        longest = eachlen
        longestindex = each
        fig
    next

# process list of pages and call dumppage for each
quot = 34 chr
pages = allpages mid longestindex 1 split pages quot
forin each pages
    iswiki = instr each "/wiki/index.php/"
    ifequal each "/wiki/index.php/Special:AllPages"
        ignoreit
    else
        iftrue iswiki
            now = "http://techrights.org" plus each split now "http://techrights.org/wiki/index.php/" join now "http://techrights.org/wiki/index.php/Special:Export/" dumppage now
            fig
        fig
    next

# create tgz archive
pos = 0
python
    if figosname != "nt": pos = 1
    fig    
iftrue pos
    tm = time split tm ":" join tm "."
    dt = date split dt "/" join dt "-"
    tgzname = "techrightswiki_" plus dt plus "_" plus tm plus ".tar.gz"
    now = "tar -cvzf " plus tgzname plus " techrights_*" shell
    fig



Or alternatively in Python (Fig-derived):


#!/usr/bin/env python2
# encoding: utf-8
# fig translator version: fig 4.6
import sys, os
from sys import stdin, stdout
from sys import argv as figargv
try: 
    from colorama import init
    init()
except: 
    pass # (only) windows users want colorama installed or ansi.sys enabled

try: 
    from sys import exit as quit
except: 
    pass

from random import randint
from time import sleep

from os import chdir as figoch
from os import popen as figpo
from os import system as figsh
from os import name as figosname
figsysteme = 0
figfilehandles = {}
figfilecounters = {}
from sys import stdout
def figlocate(x, l = "ignore", c = "ignore"):    
    import sys 
    if l == "ignore" and c == "ignore": pass 
    # do nothing. want it to return an error? 

    elif l < 1 and c != "ignore": 
        sys.stdout.write("[" + str(c) + "G") # not ansi.sys compatible 
    elif l != "ignore" and c == "ignore": 
        sys.stdout.write("[" + str(l) + ";" + str(1) + "H") 
    else: sys.stdout.write("[" + str(l) + ";" + str(c) + "H") 

import time

def fignonz(p, n=None):
    if n==None:
        if p == 0: return 1
    else:
        if p == 0: return n
    return p

def fignot(p):
    if p: return 0
    return -1

figbac = None
figprsbac = None
sub = None
def fignone(p, figbac):
    if p == None: return figbac
    return p
    return -1

def stopgraphics():
    global yourscreen
    global figraphics
    figraphics = 0
    try: pygame.quit()
    except: pass


def figcolortext(x, f):
    b = 0
    if f == None: f = 0
    if b == None: b = 0
    n = "0"
    if f > 7: 
        n = "1"
        f = f - 8

    if   f == 1: f = 4 ## switch ansi colours for qb colours
    elif f == 4: f = 1 ## 1 = blue not red, 4 = red not blue, etc.

    if   f == 3: f = 6
    elif f == 6: f = 3

    if b > 7: b = b - 8

    if   b == 1: b = 4
    elif b == 4: b = 1

    if   b == 3: b = 6
    elif b == 6: b = 3

    stdout.write("\x1b[" + n + ";" + str(30+f) + "m")
    return "\x1b[" + n + ";" + str(30+f) + ";" + str(40+b) + "m"

def figcolourtext(x, f):
    b = 0
    if f == None: f = 0
    if b == None: b = 0
    n = "0"
    if f > 7: 
        n = "1"
        f = f - 8

    if   f == 1: f = 4 ## switch ansi colours for qb colours
    elif f == 4: f = 1 ## 1 = blue not red, 4 = red not blue, etc.

    if   f == 3: f = 6
    elif f == 6: f = 3

    if b > 7: b = b - 8

    if   b == 1: b = 4
    elif b == 4: b = 1

    if   b == 3: b = 6
    elif b == 6: b = 3

    stdout.write("\x1b[" + n + ";" + str(30+f) + "m")
    return "\x1b[" + n + ";" + str(30+f) + ";" + str(40+b) + "m"

figcgapal =   [(0, 0, 0),      (0, 0, 170),    (0, 170, 0),     (0, 170, 170),
(170, 0, 0),   (170, 0, 170),  (170, 85, 0),   (170, 170, 170), 
(85, 85, 85),  (85, 85, 255),  (85, 255, 85),  (85, 255, 255), 
(255, 85, 85), (255, 85, 255), (255, 255, 85), (255, 255, 255)]

def figget(p, s): 
    return s

def fighighlight(x, b):
    f = None
    if f == None: f = 0
    if b == None: b = 0
    n = "0"
    if f > 7: 
        n = "1" 
        f = f - 8

    if   f == 1: f = 4 ## switch ansi colours for qb colours
    elif f == 4: f = 1 ## 1 = blue not red, 4 = red not blue, etc.

    if   f == 3: f = 6
    elif f == 6: f = 3

    if b > 7: b = b - 8

    if   b == 1: b = 4
    elif b == 4: b = 1

    if   b == 3: b = 6
    elif b == 6: b = 3

    stdout.write("\x1b[" + n + str(40+b) + "m")
    return "\x1b[" + n + str(40+b) + "m"

def figinstr(x, p, e):
    try: 
        return p.index(e) + 1
    except: 
        return 0

def figchdir(p):
    try: 
        figoch(p)
    except: 
        print "no such file or directory: " + str(p)
        figend(1)

def figshell(p):
    global figsysteme
    try: 
        figsysteme = figsh(p)
    except:
        print "error running shell command: " + chr(34) + str(p) + chr(34)
        figend(1)

def figarrshell(c):
    global figsysteme
    try:
        figsysteme = 0
        sh = figpo(c)
        ps = sh.read().replace(chr(13) + chr(10), 
        chr(10)).replace(chr(13), chr(10)).split(chr(10))
        figsysteme = sh.close()
    except:
        print "error running arrshell command: " + chr(34) + str(c) + chr(34) 
        figend(1)
    return ps[:]

def figsgn(p):
    p = float(p)
    if p > 0: 
        return 1
    if p < 0: 
        return -1
    return 0

def figstr(p): 
    return str(p)

def figprint(p): 
    print p

def figchr(p): 
    if type(p) == str:
        if len(p) > 0:
            return p[0]
    return chr(p)

def figprints(p): 
    stdout.write(str(p))
    sys.stdout.flush()

def figleft(p, s): 
    return p[:s]

def figmid(p, s, x):
    arr = 0
    if type(p) == list or type(p) == tuple: 
        arr = 1
    rt = p[s - 1:
        x + s - 1]
    if arr and len(rt) == 1: 
        rt = rt[0]
    return rt

def figright(p, s): 
    return p[-s:]

def figrandint(x, s, f):
    return randint(s, f)

def figlcase(p): 
    return p.lower()

def figucase(p): 
    return p.upper()

def figint(p): 
    return int(p)

def figarrset(x, p, s): 
    if type(s) == tuple:
        if len(s) == 1: fas = s[0]
    elif type(s) == list:
        if len(s) == 1: fas = s[0]
    else:
        fas = s
    x[p - 1] = s

def figopen(x, s):
    import fileinput
    if s.lower() == "w":
        if (x) not in figfilehandles.keys(): 
            figfilehandles[x] = open(x[:], s.lower())
    elif s.lower() == "r":
        if (x) not in figfilehandles.keys():
            figfilehandles[x] = fileinput.input(x[:])
            figfilecounters[x] = 0
    else:
        if (x) not in figfilehandles.keys(): figfilehandles[x] = open(x[:], s[:])

def figfprint(x, s):
    fon = figosname
    sep = chr(10)
    if fon == "nt": 
        sep = chr(13) + chr(10)
    figfilehandles[s].write(str(x) + sep)

def figflineinput(x, s):
    try:
        p = figfilehandles[s][figfilecounters[s]].replace(chr(13), 
        "").replace(chr(10), "")
        figfilecounters[s] += 1
    except:
        p = chr(10)
    return p

def figclose(x):
    if (x) in figfilehandles.keys():
        figfilehandles[x].close()
        del figfilehandles[x]
        try: 
            del figfilecounters[x]
        except: 
            pass

def figcls(x):
    if figosname == "nt": 
        cls = figsh("cls") 
    else: 
        stdout.write("[2J[1;1H")
        sys.stdout.flush()

def figarropen(x, s):
    x = open(s).read().replace(chr(13) + chr(10), chr(10)).replace(chr(13), 
    chr(10)).split(chr(10))
    return x[:]

def figarrcurl(x, s):
    from urllib import urlopen
    x = str(urlopen(s).read())
    x = x.replace(chr(13) + chr(10), 
    chr(10)).replace(chr(13), chr(10)).split(chr(10))
    return x[:]

def figarrstdin(x):
    ps = []
    for p in stdin: 
        ps += [p[:-1]]
    return ps[:]

def figarrget(x, p, s): 
    if 1:
        return p[s - 1]

def figplus(p, s): 
    if type(p) in (float, int):
        if type(s) in (float, int):
            p = p + s
        else:
            p = p + s # float(s) if you want it easier
        if p == float(int(p)): 
            p = int(p)
    else:
        if type(p) == str: p = p + s # str(s) if you want it easier
        if type(p) == list: 
            if type(s) == tuple:
                p = p + list(s)
            elif type(s) == list:
                p = p + s[:]
            else:
                p = p + [s]
        if type(p) == tuple: 
            if type(s) == tuple:
                p = tuple(list(p) + list(s))
            elif type(s) == list:
                p = tuple(list(p) + s[:])
            else:
                p = tuple(list(p) + [s])
    return p

def figjoin(p, x, s):
    t = ""
    if len(x): 
        t = str(x[0])
    for c in range(len(x)):
        if c > 0: t += str(s) + str(x[c]) 
    return t # s.join(x)

def figarr(p):
    if type(p) in (float, int, str): 
        p = [p]
    else:
        p = list(p)
    return p

def figsplit(p, x, s):
    return x.split(s)

def figval(n): 
    n = float(n) 
    if float(int(n)) == float(n): 
        n = int(n) 
    return n
    
def figtimes(p, s):
    if type(p) in (float, int):
        p = p * s # float(s) if you want it easier
        if p == float(int(p)): 
            p = int(p)
    else:
        if type(p) == list:
            p = p[:] * s # figval(s)
        else:
            p = p * s # figval(s) if you want it easer
    return p

def figdivby(p, s):
    p = float(p) / s
    if p == float(int(p)): 
        p = int(p)
    return p
def figminus(p, s): 
    return p - s

def figtopwr(p, s): 
    p = p ** s
    if p == float(int(p)): 
        p = int(p)
    return p
def figmod(p, s): 
    return p % s

def figcos(p): 
    from math import cos
    p = cos(p)
    if p == float(int(p)): 
        p = int(p)
    return p

def figsin(p): 
    from math import sin 
    p = sin(p)
    if p == float(int(p)): 
        p = int(p)
    return p
def figsqr(p): 
    from math import sqrt
    p = sqrt(p)
    if p == float(int(p)): 
        p = int(p)
    return p

def figltrim(p): 
    return p.lstrip() 
def figlineinput(p): 
    return raw_input() 
def figlen(p): 
    return len(p) 
def figasc(p): 
    return ord(p[0])
def figatn(p):
    from math import atan
    p = atan(p)
    if p == float(int(p)): 
        p = int(p)
    return p

def fighex(p): 
    return hex(p)
def figrtrim(p): 
    return p.rstrip() 
def figstring(x, p, n): 
    if type(n) == str: 
        return n * p 
    return chr(n) * p 
def figtimer(p):
    from time import strftime
    return int(strftime("%H"))*60*60+int(strftime("%M"))*60+int(strftime("%S"))

def figtime(p): 
    from time import strftime
    return strftime("%H:%M:%S")

def figdate(p): 
    from time import strftime
    return strftime("%m/%d/%Y")

def figcommand(p): 
    return figargv[1:]

def figtan(p): 
    from math import tan 
    p = tan(p)
    if p == float(int(p)): 
        p = int(p)
    return p

def figoct(p): return oct(p)

def figsleep(p, s): 
    sleep(s)
def figarrsort(p): 
    p.sort()

def figdisplay(x): 
    global figraphics, figrupd
    figrupd = 0
    if figraphics == 1:
        pygame.display.update()

def figreverse(p): 
    if type(p) == list: 
        p.reverse() 
        return p
    elif type(p) == str:
        p = map(str, p) 
        p.reverse()
        p = "".join(p)
        return p

def figarreverse(p): 
    p.reverse()

def figfunction(p, s): 
    return p

def figend(x): 
    quit()

def figif(p, s): 
    return p

def figthen(p, s): 
    return p

def figsystem(x): 
    quit()

#### http://creativecommons.org/publicdomain/zero/1.0/ 


figlist = 0
try: figlist = int(type(proginf) == list)
except NameError: pass
if not figlist: proginf = 0
proginf = "wikidump 0.1 jul 2019 mn"

# dump xml for each wiki page on techrights (no edit hist)

# create each actual file
def dumppage(url):

    figlist = 0
    try: figlist = int(type(urltext) == list)
    except NameError: pass
    if not figlist: urltext = 0
    urltext = figarrcurl(urltext, url)


    figlist = 0
    try: figlist = int(type(title) == list)
    except NameError: pass
    if not figlist: title = 0
    title = ""

    for each in urltext:

        figlist = 0
        try: figlist = int(type(findtitle) == list)
        except NameError: pass
        if not figlist: findtitle = 0
        findtitle = figinstr(findtitle, each, "<title>")

        if findtitle:

            figlist = 0
            try: figlist = int(type(title) == list)
            except NameError: pass
            if not figlist: title = 0
            title = each
            title = figltrim(title)
            title = figrtrim(title)
            title = figsplit(title, title, "<title>")
            title = figjoin(title, title, "")
            title = figsplit(title, title, "</title>")
            title = figjoin(title, title, "")

            break
    if title:

        figlist = 0
        try: figlist = int(type(outfilename) == list)
        except NameError: pass
        if not figlist: outfilename = 0
        outfilename = "techrights_"
        outfilename = figplus(outfilename, title)
        outfilename = figplus(outfilename, ".xml")
        outfilename = figsplit(outfilename, outfilename, "/")
        outfilename = figjoin(outfilename, outfilename, ":slash:")
        figopen(outfilename, "w")

        for each in urltext:

            figlist = 0
            try: figlist = int(type(now) == list)
            except NameError: pass
            if not figlist: now = 0
            now = each
            figfprint(now, outfilename)


        figlist = 0
        try: figlist = int(type(now) == list)
        except NameError: pass
        if not figlist: now = 0
        now = outfilename
        figclose(now)


# get list of pages

figlist = 0
try: figlist = int(type(allpages) == list)
except NameError: pass
if not figlist: allpages = 0
allpages = figarrcurl(allpages, "http://techrights.org/wiki/index.php/Special:AllPages")


figlist = 0
try: figlist = int(type(allpageslen) == list)
except NameError: pass
if not figlist: allpageslen = 0
allpageslen = allpages
allpageslen = figlen(allpageslen)


figlist = 0
try: figlist = int(type(longest) == list)
except NameError: pass
if not figlist: longest = 0
longest = 0


figlist = 0
try: figlist = int(type(longestindex) == list)
except NameError: pass
if not figlist: longestindex = 0
longestindex = 0

for each in range(int(float(1)), int(float(allpageslen)) + figsgn(1), fignonz(int(float(1)))):

    figlist = 0
    try: figlist = int(type(eachlen) == list)
    except NameError: pass
    if not figlist: eachlen = 0
    eachlen = allpages
    eachlen = figmid(eachlen, each, 1)
    eachlen = figlen(eachlen)

    if eachlen > longest:

        figlist = 0
        try: figlist = int(type(longest) == list)
        except NameError: pass
        if not figlist: longest = 0
        longest = eachlen


        figlist = 0
        try: figlist = int(type(longestindex) == list)
        except NameError: pass
        if not figlist: longestindex = 0
        longestindex = each


# process list of pages and call dumppage for each

figlist = 0
try: figlist = int(type(quot) == list)
except NameError: pass
if not figlist: quot = 0
quot = 34
quot = figchr(quot)


figlist = 0
try: figlist = int(type(pages) == list)
except NameError: pass
if not figlist: pages = 0
pages = allpages
pages = figmid(pages, longestindex, 1)
pages = figsplit(pages, pages, quot)

for each in pages:

    figlist = 0
    try: figlist = int(type(iswiki) == list)
    except NameError: pass
    if not figlist: iswiki = 0
    iswiki = figinstr(iswiki, each, "/wiki/index.php/")

    if each == "/wiki/index.php/Special:AllPages":

        figlist = 0
        try: figlist = int(type(ignoreit) == list)
        except NameError: pass
        if not figlist: ignoreit = 0

    else:
        if iswiki:

            figlist = 0
            try: figlist = int(type(now) == list)
            except NameError: pass
            if not figlist: now = 0
            now = "http://techrights.org"
            now = figplus(now, each)
            now = figsplit(now, now, "http://techrights.org/wiki/index.php/")
            now = figjoin(now, now, "http://techrights.org/wiki/index.php/Special:Export/")
            figbac = now
            now = dumppage(now) ; now = fignone(now, figbac) ; 

# create tgz archive

figlist = 0
try: figlist = int(type(pos) == list)
except NameError: pass
if not figlist: pos = 0
pos = 0

if figosname != "nt": pos = 1
if pos:

    figlist = 0
    try: figlist = int(type(tm) == list)
    except NameError: pass
    if not figlist: tm = 0
    tm = figtime(tm)
    tm = figsplit(tm, tm, ":")
    tm = figjoin(tm, tm, ".")


    figlist = 0
    try: figlist = int(type(dt) == list)
    except NameError: pass
    if not figlist: dt = 0
    dt = figdate(dt)
    dt = figsplit(dt, dt, "/")
    dt = figjoin(dt, dt, "-")


    figlist = 0
    try: figlist = int(type(tgzname) == list)
    except NameError: pass
    if not figlist: tgzname = 0
    tgzname = "techrightswiki_"
    tgzname = figplus(tgzname, dt)
    tgzname = figplus(tgzname, "_")
    tgzname = figplus(tgzname, tm)
    tgzname = figplus(tgzname, ".tar.gz")


    figlist = 0
    try: figlist = int(type(now) == list)
    except NameError: pass
    if not figlist: now = 0
    now = "tar -cvzf "
    now = figplus(now, tgzname)
    now = figplus(now, " techrights_*")
    figshell(now)

We also have a complete set of pages and associated scripts/metadata. We might post these separately some other day. We encourage people to make backups or copies. No site lasts forever and decentralisation is key to longterm preservation.

« Previous entries Next Page » Next Page »

RSS 64x64RSS Feed: subscribe to the RSS feed for regular updates

Home iconSite Wiki: You can improve this site by helping the extension of the site's content

Home iconSite Home: Background about the site and some key features in the front page

Chat iconIRC Channels: Come and chat with us in real time

New to This Site? Here Are Some Introductory Resources

No

Mono

ODF

Samba logo






We support

End software patents

GPLv3

GNU project

BLAG

EFF bloggers

Comcast is Blocktastic? SavetheInternet.com



Recent Posts