●● IRC: #boycottnovell @ FreeNode: Monday, February 15, 2021 ●● ● Feb 15 [07:37] Techrights-sec gemini://gemini.techrights.org/feed/ [07:38] schestowitz__ excellent, will pass on to callum !! [07:48] *rianne has quit (Quit: Konversation terminated!) [07:48] *rianne_ (~rianne@host81-154-169-167.range81-154.btcentralplus.com) has joined #boycottnovell ● Feb 15 [08:04] schestowitz__ some people told me they're reading now from gemini first, can it be made to update more regularly, e.g. every 3 hours for latest articles to be added? [08:18] Techrights-sec g/feed/ [08:18] Techrights-sec # m h dom mon dow command [08:18] Techrights-sec 06 */3 * * * /home/gemini/bin/gemini-cron-updater.sh [08:18] Techrights-sec Is that too often now? [08:18] Techrights-sec See the few lines above :) [08:18] schestowitz__ checking the site... :-) [08:20] schestowitz__ I have just checked runtime duration for /home/gemini/bin/gemini-cron-updater.sh and it is under 50 seconds [08:21] Techrights-sec See the few lines above :) [08:21] Techrights-sec That only updates every third hour, next time will be at 1200 UTC [08:21] Techrights-sec THe change went in after 0900 UTC [08:22] Techrights-sec Hmm. That is long anyway. The slow part though is fetching the [08:22] Techrights-sec RSS feed. That takes many 10s of seconds. [08:22] schestowitz__ I am asking and checking also because I want to link to gemini:// in social control media, for the latest posts :-) [08:26] Techrights-sec It can be run more often but then I would have it a little more narrowed in [08:26] Techrights-sec scope first. [08:26] Techrights-sec But as mentioned the slow part is waiting for the response to the [08:26] Techrights-sec HTTP request in fetching the RSS feed... [08:26] Techrights-sec Something is not quite right there on the server side. [08:27] schestowitz__ response speed? RSS was always a tad slow and more so when the client is 'thin' because daily links mean a large large xml download [08:29] Techrights-sec $ time wget -q -S -O /dev/null http://techrights.org/feed/ [08:29] Techrights-sec HTTP/1.1 200 OK [08:29] Techrights-sec Date: Mon, 15 Feb 2021 08:28:14 GMT [08:29] Techrights-sec Server: Apache/2.4.6 (CentOS) [08:29] Techrights-sec X-Powered-By: PHP/5.4.16 [08:29] Techrights-sec Vary: Cookie [08:29] Techrights-sec X-Pingback: http://techrights.org/xmlrpc.php [08:29] Techrights-sec Last-Modified: Mon, 15 Feb 2021 07:51:48 GMT [08:29] Techrights-sec ETag: "8fad75f5a8c43b67f8b185864a38d32b" [08:30] Techrights-sec Connection: close [08:30] Techrights-sec Transfer-Encoding: chunked [08:30] Techrights-sec Content-Type: text/xml; charset=UTF-8 [08:30] Techrights-sec real 0m2.561s [08:30] Techrights-sec user 0m0.000s [08:30] Techrights-sec sys 0m0.022s [08:30] Techrights-sec Hmm its faster at the moment, but can be quite slow. [08:30] Techrights-sec Each article in the feed is fetched as well, regardless. There's not [08:30] Techrights-sec a resonable way to avoid some refetching. [08:30] schestowitz__ I think it's o(n) over time, as the feed does not increase in size, only the capsule does, and afaik the thing that will scale less nicely is building index for bulletin and irc, though not a noticeable slowdown until we have lots of days. Same issue for ipfs BTW. [08:32] Techrights-sec Once the data is on the local file system, processing goes very quickly even on the RPi. The wait is the multiple HTTP requests [08:32] Techrights-sec done in serial. [08:32] Techrights-sec Since the target is a single server and TR at that it would not be useful [08:32] Techrights-sec to parallelize it. ● Feb 15 [10:45] Techrights-sec 2019 is 75M as Gemini. Shall I add it to the RPi? [10:45] Techrights-sec 1200 UTC in about 2 hours, time to check the cron output then [10:45] schestowitz__ I ran it manually before we did exercise and the output seemed OK all in all. I wanted to link to gemini:// along with http://, as I shall probably do from now on 50% of the time (helps raise awareness, at least) [10:45] -TechrightsBN/#boycottnovell-Could not resolve host: ,; Unknown error ( status 0 @ http://, ) [10:58] Techrights-sec I've added the scripts to Git on TR now [10:58] schestowitz__ excellent, adds a rollback mechanism ● Feb 15 [11:01] Techrights-sec 2019 is 75M as Gemini. Shall I add it to the RPi? [11:01] schestowitz__ yes, I don't see why not :-) [11:03] Techrights-sec added and the inventory updated [11:05] Techrights-sec 2019 is in place [11:05] Techrights-sec I'll add the other years then? [11:05] Techrights-sec Probably one per day, as it takes about 7 to 8 hours with throttling to [11:05] Techrights-sec fetch and extract the URLs. Then the parsing takes a few minutes after [11:05] Techrights-sec that. [11:05] schestowitz__ It's a one-time thing and each year will be 'fatter' than prior ones as over time we become more efficient and produce more [11:07] Techrights-sec As long as the cron scripts function, the growth forward will be automatic. [11:07] Techrights-sec The adding of the archives has to be done manually though. [11:07] Techrights-sec There is a slight risk of a link not getting made Dec 31 after 2100 UTC, but I am not sure yet. [11:07] Techrights-sec From 0000 on Jan 1 each year it should be good though. [11:08] schestowitz__ Assuming the "master" copy of the DB and WWW, we don't need to worry about imperfections. Due to gopher/gemini limitations, regardless, the conversions are inherently lossy. [11:08] schestowitz__ *is the DB [11:10] Techrights-sec The reason being that if an article is added after the last indexing, the [11:10] Techrights-sec of the year, the next year does not automatically go and reindex the previous [11:10] Techrights-sec year, even though it was a few minutes prior. [11:10] Techrights-sec I'm just worried about major changes to the CMS layout / structure [11:11] schestowitz__ Unlikely to happen any time soon. That's the reason I insist on LTS and don't go for fancy/shiny crap without proper justification. Some that go for "latest" doom their own sites. Also, bulletins and other stuff depend on consistency in the CMS. [11:11] Techrights-sec $ ls -1 | sort -t . -k2,2r -k1,1 [11:11] Techrights-sec gemini-scripts-README.txt [11:11] Techrights-sec gemini-bulletin-irc-update.sh [11:11] Techrights-sec gemini-cron-updater.sh [11:11] Techrights-sec gemini-main-index-template.sh [11:11] Techrights-sec gemini-fetch-urls-from-rss.pl [11:11] Techrights-sec gemini-fetch-web-page.pl [11:11] Techrights-sec gemini-inventory.pl [11:12] Techrights-sec gemini-parse-html-to-gemini.pl [11:12] Techrights-sec That a lot. Though most are very simple. [11:12] schestowitz__ I made a full pi (/home) backup this past weekend. [11:54] Techrights-sec ok good [11:54] Techrights-sec The microSD cards wear out eventually [11:55] schestowitz__ That is the scenario I prepare for... [11:57] schestowitz__ Right now there's one IP that either DDOSes the gemini capsule or indexes it [11:59] schestowitz__ gemini@raspberrypi:~ $ grep radia.bortzmeyer.org log.txt | wc [11:59] schestowitz__ 31515 189086 2930850 ● Feb 15 [12:16] Techrights-sec BTW the update at noon seemed to go ok. The "feed" link got updated as [12:16] Techrights-sec part of that, all automatically. [12:17] schestowitz__ I was about to do a video about subs [12:44] Techrights-sec turning-sites-into-capsules.webm has no video after some minutes, [12:44] Techrights-sec the audio continues [12:45] schestowitz__ will try to reproduce here [12:50] schestowitz__ works on locally [12:50] schestowitz__ will msd sumcheck wrt server [12:50] schestowitz__ mds [12:51] Techrights-sec in VLC there is no video after a short while into the video [12:51] schestowitz__ maybe it got corrupted during download rather than upload? [12:52] schestowitz__ I just did a 20-min video about subscription [12:52] schestowitz__ checksum ok [12:53] Techrights-sec $ sha256sum turning-sites-into-capsules.webm [12:53] Techrights-sec 7d53d77870401bf1e971bc7c3743d605e18e7b64548eb93f6f923365d305ce2b turning-sites-into-capsules.webm [12:53] schestowitz__ $ sha256sum turning-sites-into-capsules.webm [12:53] schestowitz__ 7d53d77870401bf1e971bc7c3743d605e18e7b64548eb93f6f923365d305ce2b turning-sites-into-capsules.webm [12:53] schestowitz__ I get the same [12:54] schestowitz__ I open the file in Parole [12:54] Techrights-sec The server has the same checksum ● Feb 15 [13:47] schestowitz__ https://twitter.com/jamesabernard/status/1361285000646127616 [13:47] -TechrightsBN/#boycottnovell-@jamesabernard: @schestowitz Must be doing something right! [13:47] schestowitz__ https://twitter.com/jasonsnitker/status/1361131811640905728 [13:47] -TechrightsBN/#boycottnovell-@jasonsnitker: <3 from Debian! https://t.co/25ChJctPAt [13:47] -TechrightsBN/#boycottnovell-@schestowitz: "Debian sends all of our love and a giant Thank you to the upstream and downstream creators and maintainers, host https://t.co/LapStCCbyY [13:51] schestowitz__ https://www.bortzmeyer.org/ seems like a legit domain, but 100,000+ hits in a few hours is odd. Maybe some research being done. It takes up about 100KB/sec, so tolerable given our capacity/throughput. [13:51] -TechrightsBN/#boycottnovell-www.bortzmeyer.org | Blog Stphane Bortzmeyer: Accueil [13:53] schestowitz__ it seems to be rate-limited or paces, about a dozen connections every 10 seconds, with pauses in between ● Feb 15 [16:14] schestowitz__ My session got around after "The server has the same checksum" [16:15] Techrights-sec I'm not familiar with SB [16:15] Techrights-sec or his background, though he is apparently famous [16:15] Techrights-sec (still looking at traffic shaping for Linux) [16:15] schestowitz__ Did I miss something after "The server has the same checksum"? [16:25] Techrights-sec Probably not. [16:25] Techrights-sec ' [16:25] Techrights-sec BTW what bandwidth do you wish to limit the RPi to? [16:27] schestowitz__ I am relieved to know I lost nothing during the hang-up (those happened more lately), knowing it's not ideal comms tool but a good bridge that let's me redact things is needed. The buffers can be wiped without notice and it's not easy to find what was missed in any log files, maybe some pipes down the system (i/o). Regarding pi b/w, I see no reason for it to use more than 100kb (800mbit) per second as we have other machines [16:27] schestowitz__ plugged in and the pi does not stream videos or anything [16:29] Techrights-sec Ok. I will work around a model of 800Mb/s tops [16:29] schestowitz__ :) Kbit would be better [16:29] schestowitz__ I am not a TEMPORA outpost :-) [16:31] Techrights-sec Sorry I mend 800kb/s [16:31] Techrights-sec Once a salesman was making a mistake in a contract proposal writing B instead [16:31] Techrights-sec of b and I almost let him write it out. [16:31] schestowitz__ sometimes people give away how little they understand about computing [16:32] schestowitz__ > "Although such advances make it possible for everyone to have access to advanced computer systems, Thompson believes #programming skills are being eroded." https://www.computerweekly.com/news/252496341/ENIAC-anniversary-What-75-years-of-computer-technology-have-delivered [16:32] -TechrightsBN/#boycottnovell-www.computerweekly.com | ENIAC anniversary: What 75 years of computer technology have delivered [16:32] schestowitz__ author noted, Microsoft booster [16:34] schestowitz__ I will post Daily Links shortly, then will do work on the gemini thing. We've attracted attention of some high-level folks, I have some unread emails in my box about it. radia.bortzmeyer.org reqs stopped some hours ago and all along it seemed like research-related, maybe even plan to index it [16:35] schestowitz__ they were spread about 10 secs apart, I think, so rated-limited so as to not hammer too much on the back end [16:35] Techrights-sec later today I might be able to add 2018, it depends on other things first [16:36] schestowitz__ excellent. In recent years we had many exclusive stories and some had massive impact, e.g. Lenovo, RasPi, EPO... [16:37] schestowitz__ BN was out first "success" story, but it took lots of energy, albeit it gave us credibility (never mind the anti-BN sites, these were a sign of us being effective, given who was pissed off) [16:40] schestowitz__ gemini@raspberrypi:~ $ grep radia.bortzmeyer.org log.txt | wc [16:40] schestowitz__ 134932 809594 12548723 [16:40] schestowitz__ gemini@raspberrypi:~ $ cat log.txt | wc [16:40] schestowitz__ 187773 998034 15134720 [16:41] schestowitz__ I did not even turn on the logging all the time, so the real number would be a lot higher, maybe quarter million, but it's not what matters. We need to get gemini noticed, not just our capsule. I sometimes run updates manually just so that I can link to the gemini:.// url of something just published. [16:42] Techrights-sec If things go well, I plan to add 1 year per day. That's about [16:42] Techrights-sec the rate which is feasible. It takes about 8 hours of unattended, rate-limite [16:42] Techrights-sec dowloads to get the URLs and the and hour of hand work to process them and [16:42] Techrights-sec upload to the RPi. [16:42] Techrights-sec Try wc -l [16:42] Techrights-sec wc -l log.txt [16:42] schestowitz__ I am familiar with that option, but was lazy... [16:44] schestowitz__ re the story about Microsoft recruiters, the only safe way to shed light on it is to bring it up routinely and repeatedly, quoting selective bits each time but never the whole lot [16:50] schestowitz__ as part of tidying, should we remove the broken sept. 2020 link for bulletins? Is it part of a sliding window of 6 months? [16:57] Techrights-sec Yes, a sliding 6 months. [16:57] Techrights-sec I can make it a sliding months. [16:57] Techrights-sec All set [16:57] schestowitz__ excellent, thanks... I thought it had been coded somewhere without yet checking. And it seems to be spidered/indexed, so good to be sure all is in tact [16:59] schestowitz__ ot: back in oct. when I looked into gemini i did not know if it was really 'going places', but now it seems to have gathered momentum/inertia, so my only regret is settling with ipfs alone and neglecting the rest, inc. gnunet [16:59] schestowitz__ I know of nobody who uses gnunet alreay ● Feb 15 [17:01] schestowitz__ todo: check how truly secure a connection over gemini is, in case we wish to do doc drops (or url/tip drops) through there [17:02] schestowitz__ this is where microsoft spying on github downloads and projects being on github (NSA) poses a risk to wbers [17:03] Techrights-sec Secure in the context depends on how the certs are verified or not. If you [17:03] Techrights-sec have Gemini with TLS, but as an Onnion service, that would be an idea. [17:03] Techrights-sec The anonymity is important even if it is not absolute. [17:03] schestowitz__ dns lookup is one issue [17:03] schestowitz__ gemini attack surface is lower than a www browser or worse, a whole distro like tails with systemd [17:04] Techrights-sec yes [17:04] Techrights-sec but the clients are not available in any repositories yet... [17:04] Techrights-sec people can't be expected to maitntain their own Git clone and compile [17:04] Techrights-sec regularly. [17:05] schestowitz__ if we host in our site a precompiled set of downloads, that would be gpl-compatible, I think [17:05] schestowitz__ then http and not https is a potential risk [17:06] Techrights-sec If Tails starts to support Gemini, then that would be a very good thing. [17:06] Techrights-sec Yes, but that's overhead and not relate to producing articles. Other [17:06] Techrights-sec project have the prioritization for that. [17:06] schestowitz__ correct [17:06] schestowitz__ port 1965 is probably not in the "toolset" of the NSAs and GCHQs of the world, YET. [17:07] schestowitz__ they will call it something like "dark web" and insist that it's not racist to call it that :-) [17:08] Techrights-sec They'll adapt quickly enough I expect. [17:08] Techrights-sec TLS is probably in the bag. [17:08] Techrights-sec Probably [17:36] schestowitz__ http://ipkitten.blogspot.com/2021/02/highlights-from-new-epo-guidelines-for.html?showComment=1613403745122#c8380797494215912787 [17:36] -TechrightsBN/#boycottnovell-ipkitten.blogspot.com | Highlights from the new EPO guidelines for examination 2021: ViCo oral proceedings, description amendments and antibodies - The IPKat [17:36] schestowitz__ " [17:36] schestowitz__ To the great dismay of Kant, I would like to draw the attention to an interesting document about the legality of virtual oral proceedings. [17:36] schestowitz__ https://www.meissnerbolte.de/de/news/legality-of-virtual-oral-proceedings-under-art-116-epc/ [17:36] schestowitz__ Please do not come with the argument that the attorney firm is sitting in Munich, but make the effort to reply reasonably and in substance to the arguments brought forward in this paper. [17:36] -TechrightsBN/#boycottnovell-www.meissnerbolte.de | Legality of Virtual Oral Proceedings under Art. 116 EPC Pros and Cons [17:36] schestowitz__ " ● Feb 15 [21:53] *schestowitz__ has quit (Read error: Connection reset by peer) [21:54] *schestowitz__ (~schestowi@unaffiliated/schestowitz) has joined #boycottnovell [21:54] *ChanServ gives channel operator status to schestowitz__ ● Feb 15 [22:17] *gry has quit (Ping timeout: 240 seconds) [22:37] *gry (~test@unaffiliated/gryllida) has joined #boycottnovell [22:43] *randomgry (~test@unaffiliated/gryllida) has joined #boycottnovell [22:43] *gry has quit (Read error: Connection reset by peer) [22:47] *randomgry is now known as gry [22:56] *gry has quit (Ping timeout: 240 seconds) ● Feb 15 [23:05] *gry (~test@unaffiliated/gryllida) has joined #boycottnovell [23:07] *gry has quit (Read error: Connection reset by peer) [23:11] *gry (~test@unaffiliated/gryllida) has joined #boycottnovell