Techrights logo

IRC: #boycottnovell @ FreeNode: Monday, February 15, 2021

(ℹ) Join us now at the IRC channel | ䷉ Find the plain text version at this address.

Techrights-secgemini://gemini.techrights.org/feed/Feb 15 07:37
schestowitz__excellent, will pass on to callum !!Feb 15 07:38
*rianne has quit (Quit: Konversation terminated!)Feb 15 07:48
*rianne_ (~rianne@host81-154-169-167.range81-154.btcentralplus.com) has joined #boycottnovellFeb 15 07:48
schestowitz__some people told me they're reading now from gemini first, can it be made to update more regularly, e.g. every 3 hours for latest articles to be added?Feb 15 08:04
Techrights-secg/feed/Feb 15 08:18
Techrights-sec# m h  dom mon dow   commandFeb 15 08:18
Techrights-sec06 */3 * * * /home/gemini/bin/gemini-cron-updater.shFeb 15 08:18
Techrights-secIs that too often now?Feb 15 08:18
Techrights-secSee the few lines above  :)Feb 15 08:18
schestowitz__checking the site... :-)Feb 15 08:18
schestowitz__I have just checked runtime duration for /home/gemini/bin/gemini-cron-updater.sh and it is under 50 secondsFeb 15 08:20
Techrights-secSee the few lines above  :)Feb 15 08:21
Techrights-secThat only updates every third hour, next time will be at 1200 UTCFeb 15 08:21
Techrights-secTHe change went in after 0900 UTCFeb 15 08:21
Techrights-secHmm.  That is long anyway.  The slow part though is fetching the              Feb 15 08:22
Techrights-secRSS feed.  That takes many 10s of seconds.Feb 15 08:22
schestowitz__I am asking and checking also because I want to link to gemini:// in social control media, for the latest posts :-)Feb 15 08:22
Techrights-secIt can be run more often but then I would have it a little more  narrowed inFeb 15 08:26
Techrights-secscope first.Feb 15 08:26
Techrights-secBut as mentioned the slow part is waiting for the response to the Feb 15 08:26
Techrights-secHTTP request in fetching the RSS feed...Feb 15 08:26
Techrights-secSomething is not quite right there on the server side.Feb 15 08:26
schestowitz__response speed? RSS was always a tad slow and more so when the client is 'thin' because daily links mean a large large xml downloadFeb 15 08:27
Techrights-sec$ time wget -q -S -O /dev/null http://techrights.org/feed/Feb 15 08:29
Techrights-sec  HTTP/1.1 200 OKFeb 15 08:29
Techrights-sec  Date: Mon, 15 Feb 2021 08:28:14 GMTFeb 15 08:29
Techrights-sec  Server: Apache/2.4.6 (CentOS)Feb 15 08:29
Techrights-sec  X-Powered-By: PHP/5.4.16Feb 15 08:29
Techrights-sec  Vary: CookieFeb 15 08:29
Techrights-sec  X-Pingback: http://techrights.org/xmlrpc.phpFeb 15 08:29
Techrights-sec  Last-Modified: Mon, 15 Feb 2021 07:51:48 GMTFeb 15 08:29
Techrights-sec  ETag: "8fad75f5a8c43b67f8b185864a38d32b"Feb 15 08:29
Techrights-sec  Connection: closeFeb 15 08:30
Techrights-sec  Transfer-Encoding: chunkedFeb 15 08:30
Techrights-sec  Content-Type: text/xml; charset=UTF-8Feb 15 08:30
Techrights-secreal    0m2.561sFeb 15 08:30
Techrights-secuser    0m0.000sFeb 15 08:30
Techrights-secsys     0m0.022sFeb 15 08:30
Techrights-secHmm its faster at the moment, but can be quite slow.Feb 15 08:30
Techrights-secEach article in the feed is fetched as well, regardless. There's notFeb 15 08:30
Techrights-seca resonable way to avoid some refetching.Feb 15 08:30
schestowitz__I think it's o(n) over time, as the feed does not increase in size, only the capsule does, and afaik the thing that will scale less nicely is building index for bulletin and irc, though not a noticeable slowdown until we have lots of days. Same issue for ipfs BTW.Feb 15 08:30
Techrights-secOnce the data is on the local file system, processing goes very quickly even on the RPi.  The wait is the multiple HTTP requestsFeb 15 08:32
Techrights-sec done in serial.Feb 15 08:32
Techrights-secSince the target is a single server and TR at that it would not be usefulFeb 15 08:32
Techrights-secto parallelize it.Feb 15 08:32
Techrights-sec2019 is 75M as Gemini.  Shall I add it to the RPi?Feb 15 10:45
Techrights-sec1200 UTC in about 2 hours, time to check the cron output thenFeb 15 10:45
schestowitz__I ran it manually before we did exercise and the output seemed OK all in all. I wanted to link to gemini:// along with http://, as I shall probably do from now on 50% of the time (helps raise awareness, at least)Feb 15 10:45
-TechrightsBN/#boycottnovell-Could not resolve host: ,; Unknown error ( status 0 @ http://, )Feb 15 10:45
Techrights-secI've added the scripts to Git on TR nowFeb 15 10:58
schestowitz__excellent, adds a rollback mechanismFeb 15 10:58
Techrights-sec2019 is 75M as Gemini.  Shall I add it to the RPi?Feb 15 11:01
schestowitz__yes, I don't see why not :-)Feb 15 11:01
Techrights-secadded and the inventory updatedFeb 15 11:03
Techrights-sec2019 is in placeFeb 15 11:05
Techrights-secI'll add the other years then?Feb 15 11:05
Techrights-secProbably one per day, as it takes about 7 to 8 hours with throttling toFeb 15 11:05
Techrights-secfetch and extract the URLs.  Then the parsing takes a few minutes afterFeb 15 11:05
Techrights-secthat.Feb 15 11:05
schestowitz__It's a one-time thing and each year will be 'fatter' than prior ones as over time we become more efficient and produce moreFeb 15 11:05
Techrights-secAs long as the cron scripts function, the growth forward will be automatic.Feb 15 11:07
Techrights-secThe adding of the archives has to be done manually though.Feb 15 11:07
Techrights-secThere is a slight risk of a link not getting made Dec 31 after 2100 UTC, but I am not sure yet.Feb 15 11:07
Techrights-secFrom 0000 on Jan 1 each year it should be good though.Feb 15 11:07
schestowitz__Assuming the "master" copy of the DB and WWW, we don't need to worry about imperfections. Due to gopher/gemini limitations, regardless, the conversions are inherently lossy.Feb 15 11:08
schestowitz__*is the DBFeb 15 11:08
Techrights-secThe reason being that if an article is added after the last indexing, theFeb 15 11:10
Techrights-secof the year, the next year does not automatically go and reindex the previousFeb 15 11:10
Techrights-secyear, even though it was a few minutes prior.Feb 15 11:10
Techrights-secI'm just worried about major changes to the CMS layout / structureFeb 15 11:10
schestowitz__Unlikely to happen any time soon. That's the reason I insist on LTS and don't go for fancy/shiny crap without proper justification. Some that go for "latest" doom their own sites. Also, bulletins and other stuff depend on consistency in the CMS.Feb 15 11:11
Techrights-sec$ ls -1 | sort -t . -k2,2r -k1,1Feb 15 11:11
Techrights-secgemini-scripts-README.txtFeb 15 11:11
Techrights-secgemini-bulletin-irc-update.shFeb 15 11:11
Techrights-secgemini-cron-updater.shFeb 15 11:11
Techrights-secgemini-main-index-template.shFeb 15 11:11
Techrights-secgemini-fetch-urls-from-rss.plFeb 15 11:11
Techrights-secgemini-fetch-web-page.plFeb 15 11:11
Techrights-secgemini-inventory.plFeb 15 11:11
Techrights-secgemini-parse-html-to-gemini.plFeb 15 11:12
Techrights-secThat a lot.  Though most are very simple.Feb 15 11:12
schestowitz__I made a full pi (/home) backup this past weekend.Feb 15 11:12
Techrights-secok goodFeb 15 11:54
Techrights-secThe microSD cards wear out eventuallyFeb 15 11:54
schestowitz__That is the scenario I prepare for...Feb 15 11:55
schestowitz__Right now there's one IP that either DDOSes the gemini capsule or indexes itFeb 15 11:57
schestowitz__gemini@raspberrypi:~ $ grep radia.bortzmeyer.org log.txt | wcFeb 15 11:59
schestowitz__  31515  189086 2930850Feb 15 11:59
Techrights-secBTW the update at noon seemed to go ok.  The "feed" link got updated as Feb 15 12:16
Techrights-secpart of that, all automatically.Feb 15 12:16
schestowitz__I was about to do a video about subsFeb 15 12:17
Techrights-secturning-sites-into-capsules.webm has no video after some minutes, Feb 15 12:44
Techrights-secthe audio continuesFeb 15 12:44
schestowitz__will try to reproduce hereFeb 15 12:45
schestowitz__works on locallyFeb 15 12:50
schestowitz__will msd sumcheck wrt serverFeb 15 12:50
schestowitz__mdsFeb 15 12:50
Techrights-secin VLC there is no video after a short while into the videoFeb 15 12:51
schestowitz__maybe it got corrupted during download rather than upload?Feb 15 12:51
schestowitz__I just did a 20-min video about subscriptionFeb 15 12:52
schestowitz__checksum okFeb 15 12:52
Techrights-sec$ sha256sum turning-sites-into-capsules.webmFeb 15 12:53
Techrights-sec7d53d77870401bf1e971bc7c3743d605e18e7b64548eb93f6f923365d305ce2b  turning-sites-into-capsules.webmFeb 15 12:53
schestowitz__$ sha256sum turning-sites-into-capsules.webmFeb 15 12:53
schestowitz__7d53d77870401bf1e971bc7c3743d605e18e7b64548eb93f6f923365d305ce2b  turning-sites-into-capsules.webmFeb 15 12:53
schestowitz__I get the sameFeb 15 12:53
schestowitz__I open the file in ParoleFeb 15 12:54
Techrights-secThe server has the same checksumFeb 15 12:54
schestowitz__https://twitter.com/jamesabernard/status/1361285000646127616Feb 15 13:47
-TechrightsBN/#boycottnovell-@jamesabernard: @schestowitz Must be doing something right!Feb 15 13:47
schestowitz__https://twitter.com/jasonsnitker/status/1361131811640905728Feb 15 13:47
-TechrightsBN/#boycottnovell-@jasonsnitker: <3 from Debian! https://t.co/25ChJctPAtFeb 15 13:47
-TechrightsBN/#boycottnovell-@schestowitz: "Debian sends all of our love and a giant “Thank you” to the upstream and downstream creators and maintainers, host… https://t.co/LapStCCbyYFeb 15 13:47
schestowitz__https://www.bortzmeyer.org/ seems like a legit domain, but 100,000+ hits in a few hours is odd. Maybe some research being done. It takes up about 100KB/sec, so tolerable given our capacity/throughput.Feb 15 13:51
-TechrightsBN/#boycottnovell-www.bortzmeyer.org | Blog Stéphane Bortzmeyer: AccueilFeb 15 13:51
schestowitz__it seems to be rate-limited or paces, about a dozen connections every 10 seconds, with pauses in betweenFeb 15 13:53
schestowitz__My session got around after "The server has the same checksum"Feb 15 16:14
Techrights-secI'm not familiar with SBFeb 15 16:15
Techrights-secor his background, though he is apparently famousFeb 15 16:15
Techrights-sec(still looking at traffic shaping for Linux)Feb 15 16:15
schestowitz__Did I miss something after "The server has the same checksum"?Feb 15 16:15
Techrights-secProbably not.Feb 15 16:25
Techrights-sec'Feb 15 16:25
Techrights-secBTW what bandwidth do you wish to limit the RPi to?Feb 15 16:25
schestowitz__I am relieved to know I lost nothing during the hang-up (those happened more lately), knowing it's not ideal comms tool but a good bridge that let's me redact things is needed. The buffers can be wiped without notice and it's not easy to find what was missed in any log files, maybe some pipes down the system (i/o). Regarding pi b/w, I see no reason for it to use more than 100kb (800mbit) per second as we have other machines Feb 15 16:27
schestowitz__plugged in and the pi does not stream videos or anythingFeb 15 16:27
Techrights-secOk. I will work around a model of 800Mb/s topsFeb 15 16:29
schestowitz__:) Kbit would be betterFeb 15 16:29
schestowitz__I am not a TEMPORA outpost :-)Feb 15 16:29
Techrights-secSorry I mend 800kb/sFeb 15 16:31
Techrights-secOnce a salesman was making a mistake in a contract proposal writing B insteadFeb 15 16:31
Techrights-secof b and I almost let him write it out.  Feb 15 16:31
schestowitz__sometimes people give away how little they understand about computingFeb 15 16:31
schestowitz__> "Although such advances make it possible for everyone to have access to advanced computer systems, Thompson believes #programming skills are being eroded." https://www.computerweekly.com/news/252496341/ENIAC-anniversary-What-75-years-of-computer-technology-have-deliveredFeb 15 16:32
-TechrightsBN/#boycottnovell-www.computerweekly.com | ENIAC anniversary: What 75 years of computer technology have deliveredFeb 15 16:32
schestowitz__author noted, Microsoft boosterFeb 15 16:32
schestowitz__I will post Daily Links shortly, then will do work on the gemini thing. We've attracted attention of some high-level folks, I have some unread emails in my box about it. radia.bortzmeyer.org reqs stopped some hours ago and all along it seemed like research-related, maybe even plan to index itFeb 15 16:34
schestowitz__they were spread about 10 secs apart, I think, so rated-limited so as to not hammer too much on the back endFeb 15 16:35
Techrights-seclater today I might be able to add 2018, it depends on other things firstFeb 15 16:35
schestowitz__excellent. In recent years we had many exclusive stories and some had massive impact, e.g. Lenovo, RasPi, EPO...Feb 15 16:36
schestowitz__BN was out first "success" story, but it took lots of energy, albeit it gave us credibility (never mind the anti-BN sites, these were a sign of us being effective, given who was pissed off)Feb 15 16:37
schestowitz__gemini@raspberrypi:~ $ grep radia.bortzmeyer.org log.txt | wcFeb 15 16:40
schestowitz__ 134932  809594 12548723Feb 15 16:40
schestowitz__gemini@raspberrypi:~ $ cat log.txt | wcFeb 15 16:40
schestowitz__ 187773  998034 15134720Feb 15 16:40
schestowitz__I did not even turn on the logging all the time, so the real number would be a lot higher, maybe quarter million, but it's not what matters. We need to get gemini noticed, not just our capsule. I sometimes run updates manually just so that I can link to the gemini:.// url of something just published.Feb 15 16:41
Techrights-secIf things go well, I plan to add 1 year per day.  That's aboutFeb 15 16:42
Techrights-secthe rate which is feasible.  It takes about 8 hours of unattended, rate-limiteFeb 15 16:42
Techrights-secdowloads to get the URLs and the and hour of hand work to process them and Feb 15 16:42
Techrights-secupload to the RPi.Feb 15 16:42
Techrights-secTry wc -lFeb 15 16:42
Techrights-secwc -l log.txtFeb 15 16:42
schestowitz__I am familiar with that option, but was lazy...Feb 15 16:42
schestowitz__re the story about Microsoft recruiters, the only safe way to shed light on it is to bring it up routinely and repeatedly, quoting selective bits each time but never the whole lotFeb 15 16:44
schestowitz__as part of tidying, should we remove the broken sept. 2020 link for bulletins? Is it part of a sliding window of 6 months?Feb 15 16:50
Techrights-secYes, a sliding 6 months.Feb 15 16:57
Techrights-secI can make it a sliding  months.Feb 15 16:57
Techrights-secAll setFeb 15 16:57
schestowitz__excellent, thanks... I thought it had been coded somewhere without yet checking. And it seems to be spidered/indexed, so good to be sure all is in tactFeb 15 16:57
schestowitz__ot: back in oct. when I looked into gemini i did not know if it was really 'going places', but now it seems to have gathered momentum/inertia, so my only regret is settling with ipfs alone and neglecting the rest, inc. gnunetFeb 15 16:59
schestowitz__I know of nobody who uses gnunet alreayFeb 15 16:59
schestowitz__todo: check how truly secure a connection over gemini is, in case we wish to do doc drops (or url/tip drops) through thereFeb 15 17:01
schestowitz__this is where microsoft spying on github downloads and projects being on github (NSA) poses a risk to wbersFeb 15 17:02
Techrights-secSecure in the context depends on how the certs are verified or not.  If you Feb 15 17:03
Techrights-sechave Gemini with TLS, but as an Onnion service, that would be an idea.Feb 15 17:03
Techrights-secThe anonymity is important even if it is not absolute. Feb 15 17:03
schestowitz__dns lookup is one issueFeb 15 17:03
schestowitz__gemini attack surface is lower than a www browser or worse, a whole distro like tails with systemdFeb 15 17:03
Techrights-secyesFeb 15 17:04
Techrights-secbut the clients are not available in any repositories yet...Feb 15 17:04
Techrights-secpeople can't be expected to maitntain their own Git clone and compileFeb 15 17:04
Techrights-secregularly.Feb 15 17:04
schestowitz__if we host in our site a precompiled set of downloads, that would be gpl-compatible, I thinkFeb 15 17:05
schestowitz__then http and not https is a potential riskFeb 15 17:05
Techrights-secIf Tails starts to support Gemini, then that would be a very good thing.Feb 15 17:06
Techrights-secYes, but that's overhead and not relate to producing articles.  OtherFeb 15 17:06
Techrights-secproject have the prioritization for that.Feb 15 17:06
schestowitz__correctFeb 15 17:06
schestowitz__port 1965 is probably not in the "toolset" of the NSAs and GCHQs of the world, YET.Feb 15 17:06
schestowitz__they will call it something like "dark web" and insist that it's not racist to call it that :-)Feb 15 17:07
Techrights-secThey'll adapt quickly enough I expect.Feb 15 17:08
Techrights-secTLS is probably in the bag.Feb 15 17:08
Techrights-secProbablyFeb 15 17:08
schestowitz__http://ipkitten.blogspot.com/2021/02/highlights-from-new-epo-guidelines-for.html?showComment=1613403745122#c8380797494215912787Feb 15 17:36
-TechrightsBN/#boycottnovell-ipkitten.blogspot.com | Highlights from the new EPO guidelines for examination 2021: ViCo oral proceedings, description amendments and antibodies - The IPKatFeb 15 17:36
schestowitz__"Feb 15 17:36
schestowitz__To the great dismay of Kant, I would like to draw the attention to an interesting document about the legality of virtual oral proceedings.Feb 15 17:36
schestowitz__https://www.meissnerbolte.de/de/news/legality-of-virtual-oral-proceedings-under-art-116-epc/Feb 15 17:36
schestowitz__Please do not come with the argument that the attorney firm is sitting in Munich, but make the effort to reply reasonably and in substance to the arguments brought forward in this paper. Feb 15 17:36
-TechrightsBN/#boycottnovell-www.meissnerbolte.de | Legality of Virtual Oral Proceedings under Art. 116 EPC – Pros and ConsFeb 15 17:36
schestowitz__"Feb 15 17:36
*schestowitz__ has quit (Read error: Connection reset by peer)Feb 15 21:53
*schestowitz__ (~schestowi@unaffiliated/schestowitz) has joined #boycottnovellFeb 15 21:54
*ChanServ gives channel operator status to schestowitz__Feb 15 21:54
*gry has quit (Ping timeout: 240 seconds)Feb 15 22:17
*gry (~test@unaffiliated/gryllida) has joined #boycottnovellFeb 15 22:37
*randomgry (~test@unaffiliated/gryllida) has joined #boycottnovellFeb 15 22:43
*gry has quit (Read error: Connection reset by peer)Feb 15 22:43
*randomgry is now known as gryFeb 15 22:47
*gry has quit (Ping timeout: 240 seconds)Feb 15 22:56
*gry (~test@unaffiliated/gryllida) has joined #boycottnovellFeb 15 23:05
*gry has quit (Read error: Connection reset by peer)Feb 15 23:07
*gry (~test@unaffiliated/gryllida) has joined #boycottnovellFeb 15 23:11

Generated by irclog2html.py 2.6 | ䷉ find the plain text version at this address.