(ℹ) Join us now at the IRC channel | ䷉ Find the plain text version at this address (HTTP) or in Gemini (how to use Gemini) with a full GemText version.
| *u-amarsh04 has quit (Connection closed) | Apr 16 00:13 | |
| *u-amarsh04 has quit (connection closed) | Apr 16 00:13 | |
| *DaemonFC has quit (Quit: Leaving) | Apr 16 06:33 | |
| schestowitz-TR | restarted gewmini proxy | Apr 16 07:43 |
|---|---|---|
| schestowitz-TR | lately it has been having issues after 2 months of smooth sailing | Apr 16 07:43 |
| schestowitz-TR | this predates any code changes | Apr 16 07:43 |
| Techrights-sec | any log records regardingthe freezes? systemd should have restarted it if | Apr 16 08:26 |
| Techrights-sec | it had stopped running and exited | Apr 16 08:26 |
| Techrights-sec | /var/log/nginx/* | Apr 16 08:26 |
| schestowitz-TR | it seems to sort of hang rather than crash. It has generally been running mow slowly (longer load times) recently, but didn't completely fail until about a week ago. I did not apply any system updates or anything... | Apr 16 08:27 |
| schestowitz | " | Apr 16 08:39 |
| schestowitz | 2022/04/16 07:40:22 [error] 29508#29508: *11165 connect() to unix:/var/run/nginx/nginx.gemini.proxy.0.sock failed (11: Resource temporarily unavailable) while connecting to upstream, client: 83.202.186.108, server: _, request: "GET /proxy?url=gemini://gemini.techrights.org/2020/05/27/canonical-helps-microsoft/ HTTP/1.1", upstream: "fastcgi://unix:/var/run/nginx/nginx.gemini.proxy.0.sock:", host: "gemini.techrights.org", referrer: "http:// | Apr 16 08:39 |
| schestowitz | techrights.org/" | Apr 16 08:39 |
| Techrights-sec | hmm. it will be hard to figure out what makes it hang without specific | Apr 16 08:40 |
| Techrights-sec | log entries from /var/log/nginx/ and maybe even not then | Apr 16 08:40 |
| Techrights-sec | my guess though is that the http response from TR is tripping it up | Apr 16 08:40 |
| Techrights-sec | since that is not reliable and most recent software including python | Apr 16 08:40 |
| Techrights-sec | expet the network to both be 100% availabe and 100% perfectly reliable | Apr 16 08:40 |
| schestowitz-TR | checking... | Apr 16 08:40 |
| schestowitz-TR | of course one lusy workaround is to restart the service every hour | Apr 16 08:43 |
| schestowitz-TR | one thing I nopticed is that it slowed down a lot | Apr 16 08:43 |
| schestowitz-TR | also, to restart it that may take like 2 minutes | Apr 16 08:43 |
| schestowitz-TR | which seems far too long | Apr 16 08:43 |
| schestowitz-TR | I saw no indication of memory going low | Apr 16 08:43 |
| Techrights-sec | based on the shell script problems, python is probably waiting for | Apr 16 08:49 |
| Techrights-sec | a dropped HTTP transaction which will never finish to finish; maybe | Apr 16 08:49 |
| Techrights-sec | there is a way to put in a timeout | Apr 16 08:49 |
| Techrights-sec | scratch that, the question would be Gemini protocol and Perl not HTTP and python | Apr 16 08:49 |
| schestowitz-TR | what intrigued me is that it ran OK for 2 months, no reboots or updates, and then suddenly the problems started | Apr 16 08:51 |
| schestowitz-TR | afaik, we did not change anythiing | Apr 16 08:51 |
| schestowitz-TR | except maybe adding the tc stuff and changing sudoers | Apr 16 08:51 |
| schestowitz-TR | maybe when tc gets run and there is an open nginx connection it gets it worked out | Apr 16 08:51 |
| schestowitz-TR | like stopping a packet? | Apr 16 08:51 |
| Techrights-sec | try git now | Apr 16 08:56 |
| schestowitz-TR | manually saw changelog.changes, applying now... will restart | Apr 16 08:56 |
| schestowitz-TR | done | Apr 16 08:59 |
| schestowitz-TR | I am going to tail -f error.log | Apr 16 08:59 |
| schestowitz-TR | as times there correspond with those cases of proxy downtimes | Apr 16 08:59 |
| schestowitz-TR | for now it is empty | Apr 16 08:59 |
| schestowitz-TR | btw, did you see the LF article? | Apr 16 08:59 |
| schestowitz-TR | it is the very latest post in TR | Apr 16 09:02 |
| schestowitz-TR | it kept me up until 2am, way beyond my sleep time, as I wanted the upload to finish and be verified with md5 | Apr 16 09:02 |
| schestowitz-TR | I think the timing of the report (easter friday) says how LF itself feels about it | Apr 16 09:02 |
| schestowitz-TR | this has already been discussed a lot in IRC, where I floated a first draft of it | Apr 16 09:02 |
| Techrights-sec | Apr 16 09:05 | |
| Techrights-sec | tahnks | Apr 16 09:05 |
| Techrights-sec | ^thanks | Apr 16 09:05 |
| Techrights-sec | I'll need to refresh my memory on the LF article, can you send the URL again? | Apr 16 09:05 |
| Techrights-sec | THe one about the CoC? Yes, I read it. It looks likely that | Apr 16 09:05 |
| Techrights-sec | LF is using it to silence anyone not toeing the line, especially in regards | Apr 16 09:05 |
| Techrights-sec | to M$ insurgency | Apr 16 09:05 |
| Techrights-sec | Yes, releasing on not just late on Friday but a very major holiday for most | Apr 16 09:05 |
| Techrights-sec | of the world is a strong indication they wish to bury such awareness | Apr 16 09:05 |
| Techrights-sec | Not that there are any journalists left to cover it, there are so few and | Apr 16 09:05 |
| Techrights-sec | those few are already so busy | Apr 16 09:05 |
| schestowitz-TR | remember that now to criticise criminals is the moral equivalent of kicking little innocent puppies | Apr 16 09:05 |
| Techrights-sec | yes they are pushing that narrative hard nowadays | Apr 16 09:05 |
| Techrights-sec | It's "unprofessional". | Apr 16 09:05 |
| schestowitz-TR | we agree 100% on this | Apr 16 09:09 |
| schestowitz-TR | no point debating this further | Apr 16 09:09 |
| schestowitz-TR | I don't suppose any "news" site will pick up the topic | Apr 16 09:09 |
| schestowitz-TR | that's x-ist | Apr 16 09:09 |
| schestowitz-TR | "it helps the [some bad group]" | Apr 16 09:09 |
| schestowitz-TR | there sre sying along the lines of | Apr 16 09:13 |
| schestowitz-TR | "if you don't offer legal options, then ./.. piracy" | Apr 16 09:13 |
| schestowitz-TR | "if you suppress subject x... then [some consequences" | Apr 16 09:13 |
| schestowitz-TR | BN and TR did well over the years because they do offer an outlet for suppressed views | Apr 16 09:13 |
| schestowitz-TR | but the perpetual risk is being taken out of context and presented as "Extremists" | Apr 16 09:13 |
| schestowitz-TR | as happend to FSF, Pocock and others | Apr 16 09:13 |
| schestowitz-TR | it's a "delicate dance" | Apr 16 09:13 |
| schestowitz-TR | all in all I think we did alright | Apr 16 09:13 |
| schestowitz-TR | it's very hard to argue that we're extreme | Apr 16 09:13 |
| schestowitz-TR | gulagtroll is still trying | Apr 16 09:13 |
| schestowitz-TR | like provoking DFC with certain subjects, then linking to that in Musk Social | Apr 16 09:13 |
| Techrights-sec | ack | Apr 16 09:17 |
| Techrights-sec | yes he is still trolling | Apr 16 09:17 |
| schestowitz-TR | btw, irc is suitable for discussions like sheela zemlin | Apr 16 09:24 |
| schestowitz-TR | because that is very informal | Apr 16 09:24 |
| schestowitz-TR | hardly counts as "stalking" or "harassment" | Apr 16 09:24 |
| schestowitz-TR | the subject is VERY relevant | Apr 16 09:24 |
| schestowitz-TR | but you would not want to talk about it much in articles | Apr 16 09:24 |
| schestowitz-TR | nor will you wish to mention anything race--related | Apr 16 09:24 |
| schestowitz-TR | or cultural differences | Apr 16 09:24 |
| schestowitz-TR | it might be OK foirm people from rich nations | Apr 16 09:24 |
| schestowitz-TR | I still think many CEO appoinments in the US | Apr 16 09:24 |
| schestowitz-TR | are based not on skills | Apr 16 09:24 |
| schestowitz-TR | but based on a baggage associated with public opinion | Apr 16 09:24 |
| schestowitz-TR | seeing that some of them (accent aside) cannot speak coherently | Apr 16 09:24 |
| Techrights-sec | :( | Apr 16 09:25 |
| Techrights-sec | yes if even that. same for nearly all other CxO appointements | Apr 16 09:25 |
| Techrights-sec | The C-suite as it is called just seems to shuffle between failures at | Apr 16 09:25 |
| Techrights-sec | various businesses | Apr 16 09:25 |
| schestowitz-TR | usually they do not do actual work | Apr 16 09:25 |
| schestowitz-TR | they just pass 'email' (Outlook) and tell peopple to do things | Apr 16 09:25 |
| Techrights-sec | Outlook/Exchange is used for plausible deniability | Apr 16 09:29 |
| Techrights-sec | never a real mail service | Apr 16 09:29 |
| schestowitz-TR | microsoft <3 outlook | Apr 16 09:29 |
| schestowitz-TR | russia <3 outlook | Apr 16 09:29 |
| schestowitz-TR | nsa <3 outlook | Apr 16 09:29 |
| schestowitz-TR | stupid MBAs <3 stupid magazines that tell them criminals from microsoft = "professionalism" | Apr 16 09:29 |
| Techrights-sec | yep | Apr 16 09:37 |
| Techrights-sec | probably the main reason that putin can't attack the computers is because | Apr 16 09:37 |
| Techrights-sec | russia itself is open to a devastating counter attack due to its failure | Apr 16 09:37 |
| Techrights-sec | 4x to eliminate M$ products from within its borders. At least some of the | Apr 16 09:37 |
| Techrights-sec | time that was due to out of control corruption which then resulted in firing | Apr 16 09:37 |
| Techrights-sec | of many who knew computers and relaced them with microsofters | Apr 16 09:37 |
| schestowitz-TR | at around 9:01am the error logs started filling up again | Apr 16 10:07 |
| schestowitz-TR | I've just restarted the service, about an hour late | Apr 16 10:07 |
| schestowitz | 2022/04/16 10:05:51 [error] 29508#29508: *11484 connect() to unix:/var/run/nginx/nginx.gemini.proxy.0.sock failed (111: Connection refused) while connecting to upstream, client: 201.65.94.22, server: _, request: "GET /proxy?url=gemini://gemini.techrights.org/2021/04/04/microsoft-nationalism/ HTTP/1.1", upstream: "fastcgi://unix:/var/run/nginx/nginx.gemini.proxy.0.sock:", host: "gemini.techrights.org", referrer: "http://techrights.org/" | Apr 16 10:08 |
| schestowitz | 2022/04/16 10:06:49 [error] 29508#29508: *11488 connect() to unix:/var/run/nginx/nginx.gemini.proxy.0.sock failed (111: Connection refused) while connecting to upstream, client: 81.154.169.249, server: _, request: "GET /proxy?url=gemini://gemini.techrights.org/git/tr-git/IPFS/ HTTP/1.1", upstream: "fastcgi://unix:/var/run/nginx/nginx.gemini.proxy.0.sock:", host: "gemini.techrights.org" | Apr 16 10:08 |
| -TechrightsBN/#boycottnovell-techrights.org | Welcome to Techrights | Apr 16 10:08 | |
| Techrights-sec | ack | Apr 16 10:12 |
| Techrights-sec | thanks | Apr 16 10:12 |
| *u-amarsh04 (~amarsh04@joseon-rmogvn.g0d7.dtdf.mc4289.IP) has joined #boycottnovell | Apr 16 10:16 | |
| *u-amarsh04 (~amarsh04@v6xmmrhxmbafc.irc) has joined #boycottnovell | Apr 16 10:16 | |
| schestowitz | -rw-r----- 1 www-data adm 114391 Apr 16 10:28 error.log | Apr 16 10:29 |
| schestowitz | -rw-r----- 1 www-data adm 59570 Apr 15 23:57 error.log.1 | Apr 16 10:29 |
| schestowitz | -rw-r----- 1 www-data adm 2756 Apr 6 23:53 error.log.10.gz | Apr 16 10:29 |
| schestowitz | -rw-r----- 1 www-data adm 2141 Apr 5 23:35 error.log.11.gz | Apr 16 10:29 |
| schestowitz | -rw-r----- 1 www-data adm 2764 Apr 4 23:27 error.log.12.gz | Apr 16 10:29 |
| schestowitz | -rw-r----- 1 www-data adm 3249 Apr 3 23:52 error.log.13.gz | Apr 16 10:29 |
| schestowitz | -rw-r----- 1 www-data adm 4320 Apr 2 23:29 error.log.14.gz | Apr 16 10:29 |
| schestowitz | -rw-r----- 1 www-data adm 3319 Apr 14 23:58 error.log.2.gz | Apr 16 10:29 |
| schestowitz | -rw-r----- 1 www-data adm 7667 Apr 13 23:45 error.log.3.gz | Apr 16 10:29 |
| schestowitz | -rw-r----- 1 www-data adm 23349 Apr 12 23:25 error.log.4.gz | Apr 16 10:29 |
| schestowitz | -rw-r----- 1 www-data adm 2112 Apr 11 23:15 error.log.5.gz | Apr 16 10:29 |
| schestowitz | -rw-r----- 1 www-data adm 1885 Apr 10 22:33 error.log.6.gz | Apr 16 10:29 |
| schestowitz | -rw-r----- 1 www-data adm 1613 Apr 9 23:14 error.log.7.gz | Apr 16 10:29 |
| schestowitz | -rw-r----- 1 www-data adm 3112 Apr 8 23:21 error.log.8.gz | Apr 16 10:29 |
| schestowitz | -rw-r----- 1 www-data adm 2657 Apr 7 23:28 error.log.9.gz | Apr 16 10:29 |
| schestowitz-TR | now it barely lasts a minute eveven if or when I restart the services | Apr 16 10:29 |
| schestowitz-TR | looking into it | Apr 16 10:29 |
| schestowitz | systemctl status gemini-proxy.service | Apr 16 10:34 |
| schestowitz | ● gemini-proxy.service - Gemini Proxy Service | Apr 16 10:34 |
| schestowitz | Loaded: loaded (/etc/systemd/system/gemini-proxy.service; enabled; vendor preset: enabled) | Apr 16 10:34 |
| schestowitz | Active: inactive (dead) since Sat 2022-04-16 10:33:28 BST; 1min 3s ago | Apr 16 10:34 |
| schestowitz-TR | it looks like with the latest git change is became more sensitive | Apr 16 10:38 |
| schestowitz-TR | it also looks like systemd does try to restart it a few times | Apr 16 10:38 |
| Techrights-sec | ok some more changes in the works | Apr 16 10:59 |
| Techrights-sec | ok sevral changes in Git now | Apr 16 10:59 |
| schestowitz-TR | running for 2 mins now, no errors yet | Apr 16 10:59 |
| schestowitz-TR | so far so good | Apr 16 11:03 |
| schestowitz | 2022/04/16 11:03:17 [error] 11909#11909: *58 FastCGI sent in stderr: "[Sat Apr 16 11:03:17 2022] gemini-proxy.pl: Failed to connect: Resource temporarily unavailable, 'SSL wants a read first'" while reading response header from upstream, client: 93.218.25.16, server: _, request: "GET /proxy?url=gemini://gemini.techrights.org/2022/04/14/spurning-science/ HTTP/1.1", upstream: "fastcgi://unix:/var/run/nginx/nginx.gemini.proxy.0.sock:", host: " | Apr 16 11:03 |
| schestowitz | gemini.techrights.org", referrer: "http://techrights.org/ | Apr 16 11:03 |
| -TechrightsBN/#boycottnovell-techrights.org | Welcome to Techrights | Apr 16 11:03 | |
| Techrights-sec | ok | Apr 16 11:12 |
| Techrights-sec | checking | Apr 16 11:12 |
| Techrights-sec | (fwiw it works here in the local test set up) | Apr 16 11:12 |
| Techrights-sec | still checking | Apr 16 11:12 |
| Techrights-sec | strange only the path is different | Apr 16 11:12 |
| Techrights-sec | that is the matter, | Apr 16 11:12 |
| Techrights-sec | where would be a good place to cache a copy of the public cert for | Apr 16 11:12 |
| Techrights-sec | gemini.techrihgts.org ? Somewhere in /etc? | Apr 16 11:12 |
| schestowitz-TR | maybe userspace? | Apr 16 11:13 |
| schestowitz | 2022/04/16 11:07:55 [error] 11909#11909: *75 connect() to unix:/var/run/nginx/nginx.gemini.proxy.0.sock failed (111: Connection refused) while connecting to upstream, client: 93.23.18.203, server: _, request: "GET /proxy?url=gemini://gemini.techrights.org/2021/11/10/firefox-esr-91-issues/ HTTP/1.1", upstream: "fastcgi://unix:/var/run/nginx/nginx.gemini.proxy.0.sock:", host: "gemini.techrights.org", referrer: "http://techrights.org/" | Apr 16 11:14 |
| schestowitz | 2022/04/16 11:08:27 [error] 11909#11909: *77 connect() to unix:/var/run/nginx/nginx.gemini.proxy.0.sock failed (111: Connection refused) while connecting to upstream, client: 84.249.218.214, server: _, request: "GET /proxy?url=gemini://gemini.techrights.org/2020/08/31/linux-should-reject-github/ HTTP/1.1", upstream: "fastcgi://unix:/var/run/nginx/nginx.gemini.proxy.0.sock:", host: "gemini.techrights.org", referrer: "http://techrights.org/" | Apr 16 11:14 |
| schestowitz | 2022/04/16 11:09:07 [crit] 11909#11909: *89 SSL_do_handshake() failed (SSL: error:141CF06C:SSL routines:tls_parse_ctos_key_share:bad key share) while SSL handshaking, client: 51.159.99.253, server: 0.0.0.0:443 | Apr 16 11:14 |
| schestowitz | 2022/04/16 11:09:35 [error] 11909#11909: *90 connect() to unix:/var/run/nginx/nginx.gemini.proxy.0.sock failed (111: Connection refused) while connecting to upstream, client: 95.115.120.75, server: _, request: "GET /proxy?url=gemini://gemini.techrights.org/2019/04/19/pypy-7-1-1/ HTTP/1.1", upstream: "fastcgi://unix:/var/run/nginx/nginx.gemini.proxy.0.sock:", host: "gemini.techrights.org", referrer: "http://techrights.org/" | Apr 16 11:14 |
| schestowitz | 2022/04/16 11:10:44 [error] 11909#11909: *93 connect() to unix:/var/run/nginx/nginx.gemini.proxy.0.sock failed (111: Connection refused) while connecting to upstream, client: 84.249.218.214, server: _, request: "GET /proxy?url=gemini://gemini.techrights.org/2020/06/18/rusty-censorship/ HTTP/1.1", upstream: "fastcgi://unix:/var/run/nginx/nginx.gemini.proxy.0.sock:", host: "gemini.techrights.org", referrer: "http://techrights.org/" | Apr 16 11:14 |
| Techrights-sec | /home/gemini/bin/Gemini-Proxy/Cert-Cache/gemini.tecrights.org.pem ? | Apr 16 11:15 |
| Techrights-sec | $ git diff | Apr 16 11:17 |
| Techrights-sec | diff --git a/Gemini/Gemini-Proxy/gemini-proxy.pl b/Gemini/Gemini-Proxy/gemini-proxy.pl | Apr 16 11:17 |
| Techrights-sec | index d861b5e..5c8b4ce 100755 | Apr 16 11:17 |
| Techrights-sec | --- a/Gemini/Gemini-Proxy/gemini-proxy.pl | Apr 16 11:17 |
| Techrights-sec | +++ b/Gemini/Gemini-Proxy/gemini-proxy.pl | Apr 16 11:17 |
| Techrights-sec | @@ -108,10 +108,10 @@ sub fetch_gemini { | Apr 16 11:17 |
| Techrights-sec | Timeout=>2, | Apr 16 11:17 |
| Techrights-sec | Proto => 'tcp', | Apr 16 11:17 |
| Techrights-sec | # SSL_ca_path => '/home/pi/Certs/', | Apr 16 11:17 |
| Techrights-sec | - SSL_ca_file => '/home/gemini/bin/Gemini-Proxy/Cert-Cache/gemini.tecrights.org.pem', | Apr 16 11:17 |
| Techrights-sec | + # SSL_ca_file => '/home/gemini/bin/Gemini-Proxy/Cert-Cache/gemini.tecrights.org.pem', | Apr 16 11:18 |
| Techrights-sec | SSL_hostname => "gemini.techrights.org", | Apr 16 11:18 |
| Techrights-sec | - SSL_verify_mode => SSL_VERIFY_PEER, # overriden by next line ... | Apr 16 11:18 |
| Techrights-sec | - # SSL_verify_mode => SSL_VERIFY_NONE, # unsafe kludge | Apr 16 11:18 |
| Techrights-sec | + # SSL_verify_mode => SSL_VERIFY_PEER, # overriden by next line ... | Apr 16 11:18 |
| Techrights-sec | + SSL_verify_mode => SSL_VERIFY_NONE, # unsafe kludge | Apr 16 11:18 |
| Techrights-sec | ) | Apr 16 11:18 |
| Techrights-sec | or die("Failed to connect: $!, '$SSL_ERROR'\n"); | Apr 16 11:18 |
| Techrights-sec | maybe the NAT hairpinning (NAT loopback) is causing trouble with the | Apr 16 11:20 |
| Techrights-sec | host IP + name + SSL combination? | Apr 16 11:20 |
| Techrights-sec | Anyway, try the above diff | Apr 16 11:20 |
| schestowitz-TR | it does not appear to be in git, doy ou want me to apply the change manually? | Apr 16 11:20 |
| Techrights-sec | Apr 16 11:27 | |
| Techrights-sec | if it's possible | Apr 16 11:27 |
| Techrights-sec | thanks | Apr 16 11:27 |
| schestowitz-TR | changing, will restart | Apr 16 11:27 |
| schestowitz-TR | 111 access forbidden by rule... | Apr 16 11:27 |
| schestowitz | 2022/04/16 11:23:42 [error] 11909#11909: *111 access forbidden by rule, client: 23.100.232.233, server: _, request: "GET /proxy?url=gemini://gemini.techrights.org/2012/05/12/antisocial-element-at-microsoft/ HTTP/1.1", host: "gemini.techrights.org", referrer: "http://techrights.org/2012/05/12/antisocial-element-at-microsoft/" | Apr 16 11:28 |
| schestowitz | 2022/04/16 11:24:06 [error] 11909#11909: *112 access forbidden by rule, client: 52.162.211.179, server: _, request: "GET /proxy?url=gemini://gemini.techrights.org/2018/07/08/striving-to-improve-patent-quality/ HTTP/1.1", host: "gemini.techrights.org", referrer: "http://techrights.org/2018/07/08/striving-to-improve-patent-quality/" | Apr 16 11:28 |
| -TechrightsBN/#boycottnovell-techrights.org | Charles Manson, the Unabomber, and Microsoft | Techrights | Apr 16 11:28 | |
| schestowitz | 2022/04/16 11:24:52 [error] 11909#11909: *113 FastCGI sent in stderr: "[Sat Apr 16 11:24:52 2022] gemini-proxy.pl: Failed to connect: Resource temporarily unavailable, 'SSL wants a read first'" while reading response header from upstream, client: 81.154.169.249, server: _, request: "GET /proxy?url=gemini://gemini.techrights.org/git/files/ HTTP/1.1", upstream: "fastcgi://unix:/var/run/nginx/nginx.gemini.proxy.0.sock:", host: "gemini.techrights. | Apr 16 11:28 |
| -TechrightsBN/#boycottnovell-techrights.org | In Spite of Resistance From the Patent Microcosm the USPTO Strives to Improve Patent Quality | Techrights | Apr 16 11:28 | |
| schestowitz | org", referrer: "http://gemini.techrights.org/proxy?url=gemini://gemini.techrights.org/git/" | Apr 16 11:28 |
| schestowitz | 2022/04/16 11:26:47 [error] 11909#11909: *117 FastCGI sent in stderr: "[Sat Apr 16 11:26:47 2022] gemini-proxy.pl: Failed to connect: Resource temporarily unavailable, 'SSL wants a read first'" while reading response header from upstream, client: 84.146.40.202, server: _, request: "GET /proxy?url=gemini://gemini.techrights.org/2021/03/15/duckduckgo-in-2021/ HTTP/1.1", upstream: "fastcgi://unix:/var/run/nginx/nginx.gemini.proxy.0.sock:", host: " | Apr 16 11:28 |
| schestowitz | gemini.techrights.org", referrer: "http://techrights.org/" | Apr 16 11:28 |
| -TechrightsBN/#boycottnovell-gemini.techrights.org | Gemini Proxy for Techrights | Apr 16 11:28 | |
| -TechrightsBN/#boycottnovell-techrights.org | Welcome to Techrights | Apr 16 11:28 | |
| *DaemonFC (~daemonfc@k5ghkuun6w7re.irc) has joined #boycottnovell | Apr 16 11:30 | |
| Techrights-sec | is that with the diff? | Apr 16 11:38 |
| schestowitz | 2022/04/16 11:39:50 [warn] 19740#19740: "ssl_stapling" ignored, issuer certificate not found for certificate "/etc/ssl/certs/nginx-selfsigned.crt" | Apr 16 11:40 |
| schestowitz | 2022/04/16 11:39:50 [warn] 19741#19741: "ssl_stapling" ignored, issuer certificate not found for certificate "/etc/ssl/certs/nginx-selfsigned.crt" | Apr 16 11:40 |
| schestowitz | 2022/04/16 11:40:15 [error] 19743#19743: *1 access forbidden by rule, client: 23.100.232.233, server: _, request: "GET /proxy?url=gemini://gemini.techrights.org/2010/09/16/microsoft-latin-america-vs-freedom-sw/ HTTP/1.1", host: "gemini.techrights.org", referrer: "http://techrights.org/2010/09/16/microsoft-latin-america-vs-freedom-sw/" | Apr 16 11:40 |
| -TechrightsBN/#boycottnovell-techrights.org | Microsoft ‘Pulls a Russia’ in Brazil | Techrights | Apr 16 11:40 | |
| schestowitz-TR | yes, but let me check again | Apr 16 11:40 |
| schestowitz-TR | ok, seems like I edited this correctly | Apr 16 11:40 |
| schestowitz-TR | btu I've just restarted nginx as well | Apr 16 11:40 |
| Techrights-sec | try /tmp/gemini-proxy.pl that should be the old version | Apr 16 11:52 |
| schestowitz | 2022/04/16 11:52:38 [error] 20752#20752: *5 recv() failed (104: Connection reset by peer) while reading response header from upstream, client: 81.154.169.249, server: _, request: "GET /proxy?url=gemini://gemini.techrights.org/git/sloc.gmi HTTP/1.1", upstream: "fastcgi://unix:/var/run/nginx/nginx.gemini.proxy.0.sock:", host: "gemini.techrights.org", referrer: "http://gemini.techrights.org/proxy?url=gemini://gemini.techrights.org/git/" | Apr 16 11:53 |
| schestowitz | 2022/04/16 11:52:56 [error] 20752#20752: *1 FastCGI sent in stderr: "[Sat Apr 16 11:52:56 2022] gemini-proxy.pl: Failed to connect: Resource temporarily unavailable, 'SSL wants a read first'" while reading response header from upstream, client: 80.220.101.127, server: _, request: "GET /proxy?url=gemini://gemini.techrights.org/ HTTP/1.1", upstream: "fastcgi://unix:/var/run/nginx/nginx.gemini.proxy.0.sock:", host: "gemini.techrights.org" | Apr 16 11:53 |
| schestowitz | 2022/04/16 11:53:01 [error] 20752#20752: *5 FastCGI sent in stderr: "[Sat Apr 16 11:53:01 2022] gemini-proxy.pl: Failed to connect: Resource temporarily unavailable, 'SSL wants a read first'" while reading response header from upstream, client: 81.154.169.249, server: _, request: "GET /proxy?url=gemini://gemini.techrights.org/git/sloc.gmi HTTP/1.1", upstream: "fastcgi://unix:/var/run/nginx/nginx.gemini.proxy.0.sock:", host: "gemini.techrights. | Apr 16 11:53 |
| schestowitz | org", referrer: "http://gemini.techrights.org/proxy?url=gemini://gemini.techrights.org/git/" | Apr 16 11:53 |
| schestowitz | 2022/04/16 11:53:10 [error] 20752#20752: *9 FastCGI sent in stderr: "[Sat Apr 16 11:53:10 2022] gemini-proxy.pl: Failed to connect: Resource temporarily unavailable, 'SSL wants a read first'" while reading response header from upstream, client: 80.220.101.127, server: _, request: "GET /proxy?url=gemini://gemini.techrights.org/ HTTP/1.1", upstream: "fastcgi://unix:/var/run/nginx/nginx.gemini.proxy.0.sock:", host: "gemini.techrights.org" | Apr 16 11:53 |
| Techrights-sec | seems to work in place now | Apr 16 12:19 |
| Techrights-sec | hmm now it shows "502 Bad Gateway" | Apr 16 12:19 |
| Techrights-sec | over at gemini.techrights.org | Apr 16 12:19 |
| Techrights-sec | it appears to not be running, according to ps and pgrep | Apr 16 12:19 |
| schestowitz-TR | it died after a while | Apr 16 12:19 |
| Techrights-sec | I am not sure at all what is killing it | Apr 16 12:25 |
| schestowitz-TR | nothing has changed on the server aroudund then except sudoers and tc in scripts | Apr 16 12:25 |
| schestowitz-TR | it also became a lot slower to respond | Apr 16 12:29 |
| schestowitz-TR | it used to be very snappy | Apr 16 12:30 |
| schestowitz-TR | now, even when it works, it can take several seconds | Apr 16 12:30 |
| schestowitz-TR | lxer has just posted some github crap | Apr 16 12:30 |
| schestowitz-TR | they don't like their readers | Apr 16 12:30 |
| *Noisytoot has quit (Ping timeout: 2m30s) | Apr 16 12:33 | |
| Techrights-sec | I presume something has changed in the environment? | Apr 16 12:41 |
| Techrights-sec | techrights.org | Apr 16 12:44 |
| Techrights-sec | gemini.techrights.org is an alias for home.techrights.org. | Apr 16 12:44 |
| Techrights-sec | home.techrights.org has address 81.154.169.249 | Apr 16 12:44 |
| Techrights-sec | real 0m1.503s | Apr 16 12:44 |
| Techrights-sec | user 0m0.027s | Apr 16 12:44 |
| Techrights-sec | sys 0m0.027s | Apr 16 12:44 |
| Techrights-sec | 1.5 seconds :( | Apr 16 12:44 |
| Techrights-sec | thinking | Apr 16 12:44 |
| Techrights-sec | $ time host gemini.techrights.org | Apr 16 12:44 |
| Techrights-sec | gemini.techrights.org is an alias for home.techrights.org. | Apr 16 12:44 |
| Techrights-sec | home.techrights.org has address 81.154.169.249 | Apr 16 12:44 |
| Techrights-sec | real 0m1.503s | Apr 16 12:44 |
| Techrights-sec | user 0m0.027s | Apr 16 12:44 |
| Techrights-sec | sys 0m0.027s | Apr 16 12:44 |
| schestowitz-TR | afaik, i could only see issues after 2 months of smooth sailing around the time we added c to the video update script and | Apr 16 12:44 |
| schestowitz-TR | the capsule update script | Apr 16 12:44 |
| schestowitz-TR | some days later I also killed and restarted ipfs | Apr 16 12:45 |
| schestowitz-TR | because of the connection issues | Apr 16 12:45 |
| schestowitz-TR | which left me using the wifi on another channel | Apr 16 12:45 |
| schestowitz-TR | but that was days after | Apr 16 12:45 |
| schestowitz-TR | I think that was days after the proxy isues | Apr 16 12:45 |
| Techrights-sec | thinking | Apr 16 12:45 |
| Techrights-sec | It's an intermittent problem | Apr 16 12:45 |
| schestowitz | gemini pts/8 tmux(5296).%12 Sat Apr 16 10:08 gone - no logout | Apr 16 12:59 |
| schestowitz-TR | is this you? | Apr 16 12:59 |
| Techrights-sec | probably | Apr 16 13:21 |
| Techrights-sec | I have gemini with tmux atm | Apr 16 13:21 |
| schestowitz-TR | I think that around the 10th was the first time I needed to restart ngnix but then I realised | Apr 16 13:23 |
| schestowitz-TR | the proxy server needed to be restarted after 1 montk, 30 days | Apr 16 13:23 |
| schestowitz-TR | sometimes it takes a couple of minutes just to restart | Apr 16 13:23 |
| schestowitz-TR | but pages were then slow to load | Apr 16 13:23 |
| schestowitz-TR | and it would not be the last restart needed, I think another was needed later in the same day | Apr 16 13:23 |
| schestowitz-TR | evewn if I got it to self-heal | Apr 16 13:27 |
| schestowitz-TR | that would still not explain why it got so much slower | Apr 16 13:27 |
| schestowitz-TR | and I suspect that the slowness is the cause of the crashes or is related to it | Apr 16 13:27 |
| schestowitz-TR | the capsule, gemini://, is super fact from here | Apr 16 13:27 |
| schestowitz-TR | the proxy is not | Apr 16 13:27 |
| Techrights-sec | ack | Apr 16 13:30 |
| Techrights-sec | the capsule does not use DNS though, the proxy does at least one lookup per | Apr 16 13:30 |
| Techrights-sec | request | Apr 16 13:30 |
| Techrights-sec | the requests seem to take well oveer 1 second for DNS | Apr 16 13:30 |
| Techrights-sec | so that might be part of the problem or just one symptom | Apr 16 13:30 |
| Techrights-sec | $ time host gemini.techrights.org | Apr 16 13:30 |
| Techrights-sec | gemini.techrights.org is an alias for home.techrights.org. | Apr 16 13:30 |
| Techrights-sec | home.techrights.org has address 81.154.169.249 | Apr 16 13:30 |
| Techrights-sec | real 0m5.109s | Apr 16 13:30 |
| Techrights-sec | user 0m0.021s | Apr 16 13:30 |
| Techrights-sec | sys 0m0.051s | Apr 16 13:30 |
| *DaemonFC has quit (Quit: Leaving) | Apr 16 13:30 | |
| Techrights-sec | that's from your RPi | Apr 16 13:36 |
| Techrights-sec | [ | Apr 16 13:36 |
| Techrights-sec | perhaps, if they overlap just right; yes it varies a lot but is always slow | Apr 16 13:36 |
| schestowitz-TR | if I run this from my machine or root at pi it's a lot faster\ | Apr 16 13:37 |
| schestowitz-TR | oh, actually it varies from the pi | Apr 16 13:37 |
| schestowitz-TR | even over a second sometimes | Apr 16 13:37 |
| schestowitz-TR | OK, so I think it narrows down the issue | Apr 16 13:37 |
| schestowitz-TR | maybe a TC thing | Apr 16 13:37 |
| schestowitz-TR | is it possible that at one point two scripts called the tc script at the same time, resulting in a bad state? | Apr 16 13:37 |
| schestowitz-TR | i have just run tc-0shaper-06 | Apr 16 13:37 |
| schestowitz-TR | and the response is faster | Apr 16 13:37 |
| schestowitz-TR | wait, it is no faster, it varies a lot | Apr 16 13:37 |
| schestowitz-TR | even over 6 secs at one point | Apr 16 13:37 |
| schestowitz-TR | so I think this is def. the source of the problem | Apr 16 13:37 |
| Techrights-sec | likely | Apr 16 13:40 |
| schestowitz-TR | video updates run tc swapper | Apr 16 13:42 |
| schestowitz-TR | they start every three hours past 20 | Apr 16 13:42 |
| schestowitz-TR | planet every 3 hours but no tc there | Apr 16 13:42 |
| schestowitz-TR | it looks like you commented out the global rrss updater | Apr 16 13:42 |
| schestowitz-TR | so it runs every hour, past 6 | Apr 16 13:43 |
| schestowitz-TR | the commented out bit says every 3 hours | Apr 16 13:43 |
| schestowitz-TR | was this change recent? | Apr 16 13:43 |
| schestowitz-TR | crontab -l | grep updater | Apr 16 13:45 |
| schestowitz-TR | this would reduce reliability as it increases the chance of a stale lock file | Apr 16 13:45 |
| schestowitz-TR | sometimes I run a ls comment to check the time of the latest lock | Apr 16 13:45 |
| schestowitz-TR | and manually remove it if it's old | Apr 16 13:45 |
| Techrights-sec | not that recent. I mentioned it but did not update the text as I was waiting | Apr 16 13:45 |
| Techrights-sec | to see if it caused an increase in reliable updates. Too often the updates | Apr 16 13:45 |
| Techrights-sec | hang and then fail. | Apr 16 13:45 |
| Techrights-sec | That's another problem to track down, the scripts are supposed to remove the | Apr 16 13:45 |
| Techrights-sec | their lockfiles upon exit regardless of the reason for exiting | Apr 16 13:45 |
| schestowitz-TR | i think what we have here is another 'blessing in disguise' (like with obs after noisetorch issue) | Apr 16 13:48 |
| schestowitz-TR | as the proxy issue is def. connected to lookup latecy, which it can eventualyl choke on | Apr 16 13:48 |
| schestowitz-TR | and to tackle this issue we can also ackle the lockfile issue and tc | Apr 16 13:48 |
| schestowitz-TR | overlapping scripts can contribute to the issue, I think | Apr 16 13:48 |
| Techrights-sec | I've already relabled the lockfiles so that each script uses a unique name | Apr 16 13:48 |
| schestowitz-TR | one thing i've noticed is, sometimes the scripts do not terminate | Apr 16 13:50 |
| schestowitz-TR | they leave ffprobe behind | Apr 16 13:50 |
| schestowitz-TR | and also leave themselves behind | Apr 16 13:50 |
| schestowitz-TR | an hour ago I killed manually several "updaters" | Apr 16 13:50 |
| schestowitz-TR | they might be the reason lock files are left behind | Apr 16 13:50 |
| schestowitz-TR | so in a sense our pileup of scripts with system calls maybe made its way to the proxy | Apr 16 13:50 |
| schestowitz-TR | and the proxy issue is just a symptom | Apr 16 13:50 |
| schestowitz-TR | hence not predictable | Apr 16 13:50 |
| schestowitz-TR | but I suppose more robust after some code changes | Apr 16 13:50 |
| Techrights-sec | yes, I was just thinking of that too | Apr 16 13:50 |
| schestowitz-TR | Yes, I got the same, but it's OK now | Apr 16 13:53 |
| schestowitz-TR | I was about to ask if you ran some command | Apr 16 13:53 |
| Techrights-sec | DNS is not supposed to take 6 seconds | Apr 16 13:53 |
| Techrights-sec | Failed to connect to the server: dial tcp 81.154. 169.249:1965: i/o timeout. | Apr 16 13:53 |
| Techrights-sec | ^gemini | Apr 16 13:53 |
| Techrights-sec | I tried lsof but don't have any use of it | Apr 16 13:57 |
| schestowitz-TR | maybe reset tc if such a concept exists? | Apr 16 14:00 |
| schestowitz-TR | until we get saner dns again? | Apr 16 14:00 |
| Techrights-sec | I tried lsof but don't have any use of it | Apr 16 14:00 |
| Techrights-sec | the tc scripts should clear the queues | Apr 16 14:00 |
| schestowitz-TR | is it you running tcpdump? | Apr 16 14:09 |
| schestowitz-TR | process 15593 | Apr 16 14:09 |
| schestowitz-TR | the pi is clocking more than 1MB/sec sometimes | Apr 16 14:09 |
| schestowitz-TR | so I think that might choke all traffic inc. DNS | Apr 16 14:09 |
| schestowitz-TR | not sure, let me shrottle it more | Apr 16 14:09 |
| Techrights-sec | IPFS? | Apr 16 14:09 |
| schestowitz-TR | shutting down ipfs to test | Apr 16 14:18 |
| schestowitz-TR | now the proxy seems snappier | Apr 16 14:18 |
| schestowitz-TR | I wonder if ipfs itself became more b/w-heavy | Apr 16 14:18 |
| schestowitz-TR | I did not notice anything different on other machines | Apr 16 14:18 |
| schestowitz-TR | restarting ipfs | Apr 16 14:18 |
| schestowitz-TR | its traffic level are saner, for now | Apr 16 14:18 |
| schestowitz-TR | and the proxy is responsive and fast | Apr 16 14:18 |
| Techrights-sec | ack | Apr 16 14:18 |
| schestowitz-TR | i suppose the fuller effect will be known when scripts run with tc in them but I think we at least have a far better | Apr 16 14:20 |
| schestowitz-TR | understanding now of what goes on | Apr 16 14:20 |
| schestowitz-TR | according to my router the pi down about 10gb a day up+down | Apr 16 14:21 |
| Techrights-sec | which ports is that distributed oveer? | Apr 16 14:21 |
| Techrights-sec | Is most IPFS? | Apr 16 14:21 |
| schestowitz-TR | for sure, but there is no breakdown by port numbers etc. | Apr 16 14:22 |
| schestowitz-TR | I'm going to monitor the services and things for the next few hours | Apr 16 14:22 |
| schestowitz-TR | it seems like the proxy is at least fast again and does not fall over, for now... | Apr 16 14:22 |
| schestowitz-TR | afaik, in technical terms we're doing many things other capsules do not do | Apr 16 14:22 |
| schestowitz-TR | so we are in the area of "undocumented" | Apr 16 14:22 |
| *Noisytoot (~noisytoot@y56qsb6r6yg94.irc) has joined #boycottnovell | Apr 16 14:24 | |
| Techrights-sec | ack | Apr 16 14:25 |
| *u-amarsh04 has quit (Quit: Konversation terminated!) | Apr 16 17:15 | |
| *u-amarsh04 has quit (Quit: Konversation terminated!) | Apr 16 17:15 | |
| schestowitz-TR | https://nitter.eu/FOSSForce/status/1515352822946086914#m | Apr 16 17:20 |
| -TechrightsBN/#boycottnovell-nitter.eu | FOSS Force (@FOSSForce): "GNU Core Utilities: coreutils-9.1 released | Tux Machines https://buff.ly/3Og74pB" | nitter | Apr 16 17:20 | |
| schestowitz-TR | https://nitter.eu/geeknik/status/1515345773856182278#m | Apr 16 17:21 |
| -TechrightsBN/#boycottnovell-nitter.eu | Geeknik`s {🌤}; Farm (@geeknik): "What’s New in Fedora 36 http://www.tuxmachines.org/node/163768" | nitter | Apr 16 17:21 | |
| schestowitz-TR | https://nitter.eu/BrideOfLinux/status/1515329239632986119#m | Apr 16 17:21 |
| -TechrightsBN/#boycottnovell-nitter.eu | Christine Hall (@BrideOfLinux): "Still kicking after all these years: GNU Core Utilities: coreutils-9.1 released | Tux Machines https://buff.ly/3Og74pB" | nitter | Apr 16 17:21 | |
| schestowitz-TR | backing upo pi:/home to external usb at the moment | Apr 16 18:25 |
| schestowitz-TR | the usb backup holds the whole system inclluding OS | Apr 16 18:25 |
| schestowitz-TR | we have not made many changes at that level | Apr 16 18:25 |
| schestowitz-TR | regarding the proxy, I need to manually restart it every now and then every 30 minutes or so | Apr 16 18:25 |
| schestowitz-TR | so we did not fix the issue, but we better understand what the issue is | Apr 16 18:25 |
| schestowitz-TR | and when it runs, the lag is at least minimal | Apr 16 18:25 |
| schestowitz-TR | as before | Apr 16 18:25 |
| schestowitz-TR | I've now cleaned ALL my RSS feeds except the patent stuff! | Apr 16 18:25 |
| schestowitz-TR | backup during the holiday is a priority | Apr 16 18:25 |
| Techrights-sec | ack | Apr 16 18:25 |
| Techrights-sec | maybe a cron job. I suppose it should be rewritten to have several parallel | Apr 16 18:25 |
| Techrights-sec | processes going. | Apr 16 18:25 |
| Techrights-sec | So that if one slows down or has to wait, other connections can still happen. | Apr 16 18:25 |
| schestowitz-TR | yes, before cronn ing everything I just want to better understand the uptime | Apr 16 18:25 |
| schestowitz-TR | it did run perfectly OK for 2 months | Apr 16 18:25 |
| schestowitz-TR | the pi is immensely valuable to me | Apr 16 18:32 |
| schestowitz-TR | with it, my "working notes" get replicated to 3 places | Apr 16 18:32 |
| schestowitz-TR | external usb, pi, and desktop over ssh with cache | Apr 16 18:32 |
| schestowitz-TR | it's now in another floor, upstairs | Apr 16 18:32 |
| schestowitz-TR | if ipfs goes down and I'm afak, I will know | Apr 16 18:32 |
| schestowitz-TR | blinking | Apr 16 18:32 |
| schestowitz-TR | over gemini it has served about 4 million pages | Apr 16 18:32 |
| schestowitz-TR | and over ipfs TBs of data | Apr 16 18:32 |
| schestowitz-TR | rianne uses hers for work, with a 28-inch screen and lots of monitoring, 2 or 3 VPNs | Apr 16 18:32 |
| schestowitz-TR | it is used even when I'm on shift (rpi400) | Apr 16 18:32 |
| Techrights-sec | excellent! | Apr 16 18:32 |
| Techrights-sec | ack | Apr 16 18:32 |
| Techrights-sec | I suppose the soldering will never be convenient to try | Apr 16 18:32 |
| Techrights-sec | yay! | Apr 16 18:32 |
| schestowitz-TR | at the moment I cannot think of a use for buttons on the device as even a reboot button would | Apr 16 18:33 |
| schestowitz-TR | never get used | Apr 16 18:33 |
| schestowitz-TR | so from a practical POV, it's a case of "wait, I'll find use for it later on... maybe another | Apr 16 18:33 |
| schestowitz-TR | device" | Apr 16 18:33 |
| Techrights-sec | She has a model 400? | Apr 16 18:34 |
| Techrights-sec | yes there are other "HAT"s which can do the button thing without needing | Apr 16 18:34 |
| Techrights-sec | to be soldered | Apr 16 18:34 |
| schestowitz-TR | so far since the morning no lock fiels got 'jammed' | Apr 16 18:35 |
| schestowitz-TR | i think at one point I'll run some script to 'clean up' orphaned and incomplete runs | Apr 16 18:35 |
| schestowitz-TR | ffmpeg probes etc. | Apr 16 18:35 |
| Techrights-sec | great | Apr 16 18:36 |
| schestowitz-TR | recently I noticed we get about 20-30% more unique IPs in gemini then lastt year | Apr 16 18:38 |
| schestowitz-TR | takern on a daily basis | Apr 16 18:38 |
| schestowitz-TR | drew is taking a break for gemini to spend more time smokign weed :-) | Apr 16 18:38 |
| schestowitz-TR | *from gemini | Apr 16 18:38 |
| schestowitz-TR | That's fine, he made some good tools for ir and it's still growing, I put more nad more | Apr 16 18:38 |
| schestowitz-TR | in Daily Links | Apr 16 18:38 |
| Techrights-sec | :/ | Apr 16 18:38 |
| schestowitz-TR | I've finally sorted out the microphone or sound quality on the new laptop | Apr 16 21:59 |
| schestowitz-TR | but only after I had recoreded a couple with terrible quality | Apr 16 21:59 |
| *u-amarsh04 (~amarsh04@joseon-rmogvn.g0d7.dtdf.mc4289.IP) has joined #boycottnovell | Apr 16 22:17 | |
| *u-amarsh04 (~amarsh04@v6xmmrhxmbafc.irc) has joined #boycottnovell | Apr 16 22:17 | |
| *u-amarsh04 has quit (connection closed) | Apr 16 22:40 | |
| *u-amarsh04 has quit (Connection closed) | Apr 16 22:40 | |
| schestowitz-TR | I sepent about an our trying to hack obs into doing webm as output | Apr 16 23:16 |
| schestowitz-TR | I think due to a lack of CPU/GPU capacity this won't be possible | Apr 16 23:16 |
| schestowitz-TR | as it cannot keep up | Apr 16 23:16 |
| schestowitz-TR | unless it is sound only | Apr 16 23:16 |
| psydruid | Intel/AMD won't work for the GPU offloading part? | Apr 16 23:57 |
Generated by irclog2html.py 2.6 | ䷉ find the plain text version at this address (HTTP) or in Gemini (how to use Gemini) with a full GemText version.