IRC: #boycottnovell @ Techrights IRC Network: Monday, August 15, 2022

Techrights-sec2

add-and-refresh-from-db.sh

Techrights-sec2

update-and-refresh-from-db.sh

Techrights-sec2

The perl scripts never need to be touched directly, under almost all

Techrights-sec2

circumstances. The update-and-refresh-from-db.sh takes a URL as an option.

Techrights-sec2

So I guess the site is in alpha still. I hope the main site is being updated

Techrights-sec2

in parallel still.

schestowitz-TR2

I've put up a notice about the changes

Aug 15 04:14

schestowitz-TR2

I've just updated, successfully, two pages to include followup stories

Aug 15 04:22

schestowitz-TR2

this is going better than I expected

Aug 15 04:22

schestowitz-TR2

also, thanks for reminding me of the wrapper's name

Aug 15 04:22

*britney (~britney@muarsy2vret2q.irc) has joined #boycottnovell

Aug 15 04:39

schestowitz-TR2

"TechRepublic Academy" = diploma mill spam, symptom of the site giving up on anything that resembles reporting.

Aug 15 04:47

Techrights-sec2

no problem

Aug 15 04:48

Techrights-sec2

found some more bugs in the process today

Aug 15 04:48

Techrights-sec2

checking

Aug 15 04:48

schestowitz-TR2

ing up on anything that resembles reporting.

schestowitz-TR2

bugs are to be expected, this is OK

schestowitz-TR2

so long as it's not like a "wipe the whole DB" bug, along with static pages that this DB made

schestowitz-TR2

one thing I love about this appoach is that even the DB broke, unlike with wordpress and drupal,

schestowitz-TR2

the site would still serve pages

Techrights-sec2

ack

Aug 15 04:50

schestowitz-TR2

what tm is: a place to find the latest news, curated and organised

schestowitz-TR2

what it is not: an original news source (used to do a few reviews a week)

schestowitz-TR2

maybe next I will do some original articles there

schestowitz-TR2

testing the ground for something like techrights later

Techrights-sec2

excellent, original content is always good

Aug 15 04:59

schestowitz-TR2

not sure if this is appreciated, but in new TM I sometimes add short one-line summaries, esp. for clusters of links

schestowitz-TR2

so they give an outline of what's in there

schestowitz-TR2

the improved speed and less clutter are a plus

schestowitz-TR2

which geeks will likely appreciate

schestowitz-TR2

and the same is then inherited by Gemini too

schestowitz-TR2

2400+ views in the capsule now

schestowitz-TR2

at some point the topic can be inherited from "description" for push notifications

schestowitz-TR2

or maybe that and the title

schestowitz-TR2

since you've used temp files I've had no issues of temporaily invalid rss feed

schestowitz-TR2

so maybe the irc stuff can be better streamlined (I do it manually in case of issues, which can lead to channel flooding)

Techrights-sec2

Techrights-sec2

the topic would be dc.subject

Techrights-sec2

that can be added in easily

schestowitz-TR2

oh, just a small suggestion

schestowitz-TR2

index.shtml is today's posts or latest 50, which ever is longer

schestowitz-TR2

if a node gets updates

schestowitz-TR2

and its dc.updated-time (or whatever is called) is today, can that too be included in front page?

schestowitz-TR2

sometimes I add "(UPDATED)" and it can help if those updates get noticed

Techrights-sec2

dc.date.created

Techrights-sec2

just the date, no timestamp

Techrights-sec2

it sorts only by date, so the updates ought to be there but they won't

Techrights-sec2

necessarily be at the top

Techrights-sec2

it sounds like we may have to add a timestamp and redo the database though

schestowitz-TR2

oh, just a small suggestion

schestowitz-TR2

index.shtml is today's posts or latest 50, which ever is longer

schestowitz-TR2

if a node gets updates

schestowitz-TR2

and its dc.updated-time (or whatever is called) is today, can that too be included in front page?

schestowitz-TR2

sometimes I add "(UPDATED)" and it can help if those updates get noticed

*schestowitz-TR2 has quit (Quit: Konversation term)

*schestowitz-TR2 has quit (Quit: Konversation term)

*schestowitz-TR (~acer-box@x6gnqyt8r46u8.irc) has joined #boycottnovell

*schestowitz-TR (~acer-box@freenode/user/schestowitz) has joined #boycottnovell

schestowitz-TR

oh, just a small suggestion

schestowitz-TR

index.shtml is today's posts or latest 50, which ever is longer

schestowitz-TR

if a node gets updates

schestowitz-TR

and its dc.updated-time (or whatever is called) is today, can that too be included in front page?

schestowitz-TR

sometimes I add "(UPDATED)" and it can help if those updates get noticed

schestowitz-TR

a) there are two examples right now in front page: 1) Linux RC 2) SparkyLinux

schestowitz-TR

b) iirc, all posts have an "updated" field, by default same as created

schestowitz-TR

in that case, index.shtml criterion can be updated instead of created?

schestowitz-TR

then, later on, maybe we can scan for "(Update" and then add some emphasis there, as if to say, there's something new here

Techrights-sec2

ack

Aug 15 05:17

Techrights-sec2

yes as far as I know it goes by dc.date.modified, but I can double-check

Aug 15 05:17

schestowitz-TR

thanks, there are at least 2 examples there in the front page right now

Aug 15 05:18

schestowitz-TR

prioritising them or drawing more attention to them can help

Aug 15 05:18

schestowitz-TR

the idea is to editorialise a bit the pool of articles

Aug 15 05:18

schestowitz-TR

with drupal we used comments for that -- hardly the correct approach

Aug 15 05:19

schestowitz-TR

but that helped "bump" up the nodes

Aug 15 05:19

Techrights-sec2

I can probably add a timestamp field this week and use that for the sorting

Aug 15 05:23

Techrights-sec2

since it seems to be a recurring request

Aug 15 05:23

schestowitz-TR

in rss validotors it is a non-critical warnings

schestowitz-TR

but tbh seeing that the front page already gets the order right

schestowitz-TR

maybe due to order of records (primary key?) I don't think such a field

schestowitz-TR

would be needed

schestowitz-TR

(btw, ot, rianne is already acing some mock exams... depending on what questions come up... can vary a bit)

Aug 15 05:28

schestowitz-TR

(so she has been super-busy with this... no 9-to-5 job and kids to make it hard to study)

Aug 15 05:28

Techrights-sec2

great, I hope the actual test goes smoothly. Is it online or in-person?

Aug 15 05:28

schestowitz-TR

better in person, I think, as I can imagine some truly horrible s/w would be neeced for "online"

Aug 15 05:29

schestowitz-TR

rootkits and all

Aug 15 05:29

schestowitz-TR

thanks for the git push, I can see you're made the sql query more witty with date.modified

Aug 15 05:31

Techrights-sec2

yes the reason is so that when there are skips in sequence, say when a post

Aug 15 05:38

Techrights-sec2

from many days ago gets modified, the navigation menus in the connected posts

Aug 15 05:38

Techrights-sec2

also get updated

Aug 15 05:38

*Despatche has quit (Quit: Read error: Connection reset by deer)

Aug 15 05:39

*DaemonFC has quit (Quit: Leaving)

Aug 15 05:55

schestowitz-TR

marius has just posted a page successfully

Aug 15 06:13

Techrights-sec2

excellent

Aug 15 06:15

schestowitz-TR

so the breaking news will still land on the site if rianne and I are both away

schestowitz-TR

he typically covers what's important very fast

schestowitz-TR

the other posters in old TM... were... of varying value... some more harm than good

schestowitz-TR

webspam, wsl, vscode... always promoting their own sites/businesses

schestowitz-TR

and sometimes even marketing campaigns for LF (itsfoss did this in TM)... I unpublished these

Techrights-sec2

wsl is spam too, so is pretty much any aspect of M$ these days

schestowitz-TR

the idea is open posting (digg, /., reddit) seems fine on the surface

schestowitz-TR

but sooner or later it's just lowering the quality

schestowitz-TR

in conde nast, it's about censoring for sponsors' agenda

schestowitz-TR

so that's a whole mdifferent modality

schestowitz-TR

like deciding that not condonisng Microsoft cimes is "hatred"

schestowitz-TR

and then partnering with Microisoft to promote Gates, IE9 etc.

schestowitz-TR

marketing campaigns

schestowitz-TR

in "community" or "AMA" clothing (and the moderators anything... see what comes through)

Techrights-sec2

and too many sites are owned by conde nast without their visitors realising it

Aug 15 06:20

schestowitz-TR

yes, conde nast is a MARKETING firm

schestowitz-TR

many "publishers" are actually PR outfits

schestowitz-TR

they do "news" sites

schestowitz-TR

regarding WSL, covering ius dying down

schestowitz-TR

they keep RE-announcing WSL with new gimmicks and names

schestowitz-TR

give it 2-3 years and it'll be abandoned

schestowitz-TR

too expensive to maintain for very few adopters

schestowitz-TR

"at least we TRIED!" - Microsoft

schestowitz-TR

"let's keep saying Vista 12 some more... and next summer Vista 13..."

schestowitz-TR

ME3, ME4, ME5

Techrights-sec2

ack

Aug 15 06:26

schestowitz-TR

did wget come along ok btw?

schestowitz-TR

i'm going to nice the pi external backups

schestowitz-TR

seeing they can lead gemini to timeouts, due to load

Techrights-sec2

haven't checked wget yet

Techrights-sec2

just checked, it's till running and only up to node 167895 so far

schestowitz-TR

ok, so the requests are well spaced in time

schestowitz-TR

there will be about 4 in that range that can return 404

schestowitz-TR

like duplicate nodes we later unpublished

schestowitz-TR

but the rest contain lots of stuff, plus comments (below the main body)

Techrights-sec2

there is work on the back end as it processes the results and looks for

Techrights-sec2

unexplored links

schestowitz-TR

it should just download those pages alone

schestowitz-TR

no spindering needed

schestowitz-TR

*spidiering

schestowitz-TR

and reference to files like pages or images made absolute url

schestowitz-TR

so that they work on in any context

*britney has quit (Ping timeout: 2m30s)

Aug 15 07:12

schestowitz-TR

<techrights-news> "Discover" now has "Get Lagrange!" Excellent, making Gemini clients even easier and faster to install, update...

Aug 15 07:15

schestowitz-TR

Geopard has also been added to repos. Getting Gemini clients on GNU/Linux is getting a lot easier.

Aug 15 07:17

Techrights-sec2

ack

Aug 15 07:34

Techrights-sec2

later today or tomorrow I will have to redo part of the metadata table

Aug 15 07:45

Techrights-sec2

before deploying the recent Git commits; more work is still needed first

Aug 15 07:45

Techrights-sec2

on the how the feed generators handle the new date-time stamp

Aug 15 07:45

schestowitz-TR

I've just changed drupal permissions to prevent poosting of new nodes

Aug 15 09:25

schestowitz-TR

will have to redo after re revert back to backup

Aug 15 09:25

schestowitz-TR

so that it's staying unchanged

Aug 15 09:25

Techrights-sec2

I think the table can be redone without having to wipe the contents and start

Techrights-sec2

over.

Techrights-sec2

The preliminary changes for the requisite scripts are in Git now but not rolled

Techrights-sec2

out to the site.

Techrights-sec2

ack

Techrights-sec2

> I've just changed drupal permissions

Techrights-sec2

hmm that throws the new CMS from mid-alpha to production, skipping several

Techrights-sec2

stages and with only minimal testing and even less feedback from marius and

Techrights-sec2

rianne

schestowitz-TR

we cab revert back to the drupal site at any point

Aug 15 09:29

schestowitz-TR

at the moment the new stuff goes into new

Aug 15 09:29

schestowitz-TR

old is either temporarily or permanently frozen

Aug 15 09:29

Techrights-sec2

without loss of records?

Aug 15 09:30

Techrights-sec2

into the new database which, as alpha, can get wiped

Aug 15 09:30

schestowitz-TR

if we roll out changes to the DB, then we can make a complete backup of the files and, once locked, the DB too

Aug 15 09:32

schestowitz-TR

rianne and marius reported no new with the new site, marius already posted two stories in new today

Aug 15 09:32

schestowitz-TR

since techrights is not up and running yet we can call it alpha.tuxmachines.org, copy everything to there, then use that for testing

schestowitz-TR

later the same cms will likely run there too anyhow

schestowitz-TR

if the main change to the schema is addition of time field, then we can also roll that out for techrigfhts alpha, test it there, then come back later

schestowitz-TR

to tuxmachnes

Techrights-sec2

right be worrying about breaking things means lost time with 'unnecessary'

Techrights-sec2

preparations and reserved time for rolling back; these changes could have been

Techrights-sec2

tested in situ a few hours ago but for that factor;

Techrights-sec2

there are also some structural questions like cascading deletion and external

Techrights-sec2

references between tables that need working out

Techrights-sec2

one thing at a time please, we can get TM going first and then minimize the

Techrights-sec2

amount of work that is done in parallel on the scripts and the db structure

Techrights-sec2

the changes today do not need changing any field definitions or table definitions but only changing the /contents/ of a key field. Changing a table defini

Techrights-sec2

tion

Techrights-sec2

will be far more troublesome now that the db is live.

Techrights-sec2

As mentioned a few days ago, at our current pace, I figured we were about

Techrights-sec2

half a month to a month and a half away from live

Techrights-sec2

haste makes waste

*psydruid (~psydruid@jevhxkzmtrbww.irc) has left #boycottnovell

Aug 15 10:09

*psydruid (~psydruid@jevhxkzmtrbww.irc) has joined #boycottnovell

Aug 15 10:09

schestowitz-TR

<techrights-news> "Around 2005, the Organization for the Advancement of Structured Information Standards (OASIS) group defined an open format for office documents of all types" ☛ https://opensource.com/article/22/8/odt-files | Source: OpenSource.com

Aug 15 10:15

-TechrightsBN/#boycottnovell-How ODT files are structured | Opensource.com

Aug 15 10:15

*britney (~britney@muarsy2vret2q.irc) has joined #boycottnovell

Aug 15 10:37

*wallacer has quit (Ping timeout: 2m30s)

Aug 15 10:38

*wallacer (~quassel@6bsu33ajs4zs4.irc) has joined #boycottnovell

Aug 15 10:41

schestowitz-TR

rianne has just posted some, she also saw the video I had made to Marius

Aug 15 11:17

*psydruid (~psydruid@jevhxkzmtrbww.irc) has left #boycottnovell

Aug 15 11:34

*psydruid (~psydruid@jevhxkzmtrbww.irc) has joined #boycottnovell

Aug 15 11:34

*britney has quit (Ping timeout: 2m30s)

Aug 15 12:51

schestowitz-TR

I think ipfs updates are not fully automated with cron. I've started crond and tested, times it for 4:48am.

Aug 15 12:56

*Despatche (~desp@u3xy9z2ifjzci.irc) has joined #boycottnovell

Aug 15 12:58

*psydroid2 (~psydroid@memzbmehf99re.irc) has joined #boycottnovell

Aug 15 13:32

*britney (~britney@muarsy2vret2q.irc) has joined #boycottnovell

Aug 15 14:41

schestowitz

> How did you edit the entry? I didn't get that part. Which file do I need to edit when I make a mistake?

schestowitz

Editing is done as follows:

schestowitz

update-and-refresh-from-db.sh [the URL]

schestowitz

e.g.

schestowitz

update-and-refresh-from-db.sh https://news.tuxmachines.org/n/2022/08/15/today_s_howtos.1.shtml

-TechrightsBN/#boycottnovell-news.tuxmachines.org | Tux Machines — today's howtos

schestowitz

then you can edit the entry again.

schestowitz

Hopefully image uploader will work shortly, as at the moment, it's wget, then a reference to the downloaded file.

schestowitz

----------------------

schestowitz

Re: server move changing the rss feeds (feature request to restore old depth to the feed, or an alternative feed)

schestowitz

> Hi,

schestowitz

schestowitz

> I'm slightly unhappy with the change to the rss feeds handling of the

schestowitz

> "today's combined collection of XXX"-style articles.

schestowitz

Thanks for the feedback. It is still work in progress (alpha). My understanding is, you are extracting from the full HTML feed (the old one) a list of links and want this functionality back. Is that correct?

schestowitz

> On the old site, the rss feed offered also offered enough of the

schestowitz

> sections of the collection articles to massage the feed and extract a

schestowitz

> list of (most of?) the individual articles.

schestowitz

Yes, now the xml file is a lot smaller. So you want full content instead?

schestowitz

> Can you consider restoring this extra feed "depth" to the current feed?

schestowitz

schestowitz

> Or maybe create a feed that lists each article and EACH entry of the

schestowitz

> collection articles?

schestowitz

schestowitz

> (with the links pointing to the collection article, the individual

schestowitz

> entries in the collection or the source for the entry; but the

schestowitz

> link doesn't actually matter that much :-) ).

schestowitz

schestowitz

> [a not too long excerpt or just the first paragraph of the entry

schestowitz

> would be a bonus]

schestowitz

So basically a feed of

schestowitz

entry i

schestowitz

link j

schestowitz

link j+1

schestowitz

entry i+1

schestowitz

... ??

schestowitz

schestowitz

> Use case example:

schestowitz

schestowitz

> Get a feed listing all the howto's directly by their headlines. Consider

schestowitz

> https://news.tuxmachines.org/n/2022/08/15/today_s_howtos.1.shtml

schestowitz

schestowitz

schestowitz

> would ADD the entries to a (second?) feed:

schestowitz

-TechrightsBN/#boycottnovell-news.tuxmachines.org | Tux Machines — today's howtos

schestowitz

> - How to Install & Configure Redis 7 on Rocky Linux 9 | Mark Ai Code

schestowitz

schestowitz

> - My uncertainty over whether an URL format is actually legal

schestowitz

> ...

schestowitz

schestowitz

> - How to Install Redis on Rocky Linux 9

schestowitz

So, just to ensure I understand correctly, you want an alternative feed that is a "flattened" collection of all the links in all the entries? Not clustered or grouped?

schestowitz

> PS: Also thx for making gemini a bit more prominent: lagrange,

schestowitz

> clagrange and newswaffle seem to shape up quite nicely and distraction

schestowitz

> free.

schestowitz

Thanks for noticing. Few people know (yet!) what Gemini is, but we hope that will change...

schestowitz

Re: about new site

schestowitz

> Hi Roy

schestowitz

> How are you? I hope you are doing great.

schestowitz

schestowitz

> xxx here, from xxxxx.

schestowitz

schestowitz

> I saw I cannot post on the main tuxmachines website (showing access

schestowitz

> denied) since it is moving to news.tuxmachines.org

> <http://news.tuxmachines.org>.

schestowitz

-TechrightsBN/#boycottnovell-news.tuxmachines.org | Tux Machines

schestowitz

schestowitz

> a) So, how to submit to the new website? Is there any other place

schestowitz

> where I need to register again?

schestowitz

> b) Also, I have a new website dedicated to NEWS only

schestowitz

> xxxxx how to

schestowitz

> feature that in the Tuxmachines news portal?

schestowitz

schestowitz

> Please let me know.

schestowitz

schestowitz

> My current account is: xxxx

schestowitz

schestowitz

> Regards

schestowitz

Hi,

schestowitz

Yes, I did not forget. Jut doing this one step at a time. The new system is managed over SSH. Can you send me your public ssh key so we can create an account? I'd also need to check with a friend and send you some instructions.

*britney has quit (Ping timeout: 2m30s)

Aug 15 16:23

schestowitz-TR

I have issues reaching ipfs objects, is it possible the firewall isn't permissive enough for it? I turned off ipfs at home hours ago,

Aug 15 16:31

schestowitz-TR

but this has made no difference

Aug 15 16:31

*u-amarsh04 has quit (Quit: Konversation terminated!)

Aug 15 17:07

*u-amarsh04 has quit (Quit: Konversation terminated!)

Aug 15 17:07

*britney (~britney@muarsy2vret2q.irc) has joined #boycottnovell

Aug 15 17:12

Techrights-sec2

which ports are needed for IPFS?

Aug 15 17:13

schestowitz-TR

ha, good question... ipfs is apparently using several

Aug 15 17:13

schestowitz-TR

I'm not 100% sure the firewall is the issue, but I tried many other things and ran out of ideas

Aug 15 17:13

Techrights-sec2

yes I figured it would be a few

Techrights-sec2

there don't seem to be any authoritative answers:

https://discuss.ipfs.tech/t/how-should-i-configure-my-firewall/471

Techrights-sec2

Techrights-sec2

I'll log in and take a look with netstat

-TechrightsBN/#boycottnovell-How should I configure my firewall? - Old FAQ - discuss.ipfs.tech

Techrights-sec2

netstat says TCP 4001

schestowitz-TR

thanks!

Techrights-sec2

ok try now, the defaults have been changed now to allow 4001 TCP in

Techrights-sec2

see /etc/nftables.nft

Techrights-sec2

nft -f /etc/nftables.nft

schestowitz-TR

attempting 4 requests now

schestowitz-TR

retrying

Aug 15 17:17

schestowitz-TR

should that also be out? it's a two-way transmission platform like p2p

Aug 15 17:17

Techrights-sec2

no out is covered already in other parts of the ruleset

Techrights-sec2

in those rules out is allowed by default and in is blocked by default

Techrights-sec2

exceptions are there in the top third of the page

Techrights-sec2

ssh, http, https, gemini, mqtts, and now also ipfs

Techrights-sec2

maybe restart the ipfs daemon or send it SIGHUP to reload its configuration

*u-amarsh04 (~amarsh04@freenode-rmogvn.g0d7.dtdf.mc4289.IP) has joined #boycottnovell

Aug 15 17:21

*u-amarsh04 (~amarsh04@5tcc2vuaj9aks.irc) has joined #boycottnovell

Techrights-sec2

nope it did not tolerate SIGHUP :/

Techrights-sec2

furthermore there is something wonky with the setup:

Techrights-sec2

# rc-service ipfs restart

Techrights-sec2

* Starting ipfs ...

Techrights-sec2

* start-stop-daemon: /usr/bin/ipfs died

Techrights-sec2

* Failed to start ipfs [ !! ]

Techrights-sec2

* ERROR: ipfs failed to start

schestowitz-TR

yes, I did just wonder where thre process had gone, then saw you stopped it

schestowitz-TR

I can't remember when I last saw trhings working, I just randomly tested it today

Techrights-sec2

SIGHUP traditionally causes a daemon to reload its configuration file and

Techrights-sec2

restart itself

*psydroid2 has quit (connection closed)

schestowitz-TR

it is also weird like this on the pi

Aug 15 17:24

schestowitz-TR

I have to sometimes killall, not shutdown, due to a lock

Aug 15 17:24

schestowitz-TR

or even chomd 000 the binary to steop it respawning

Aug 15 17:24

Techrights-sec2

that would be the toxic effects of systemd perhaps

Aug 15 17:25

Techrights-sec2

sytemctl disable ...

Aug 15 17:25

Techrights-sec2

ipfs is running manually under the account ipfs now

Aug 15 17:26

Techrights-sec2

I don't know /anything/ about ipfs so perhaps that is all wrong and sending

Aug 15 17:26

Techrights-sec2

the wrong stuff. But at least it is aviable on port TCP 4001 now

Aug 15 17:26

*psydroid2 (~psydroid@memzbmehf99re.irc) has joined #boycottnovell

Aug 15 17:28

*psydruid (~psydruid@jevhxkzmtrbww.irc) has left #boycottnovell

Aug 15 17:28

schestowitz-TR

I keep trying to reach objects

schestowitz-TR

the config file is in ~/./ipfs iirc

schestowitz-TR

and that's a critical parameter

schestowitz-TR

which the default service in alpine would seek

schestowitz-TR

checked server, ~/.ipfs/config

Techrights-sec2

no configuration file was pointed to there when I launched it so you may want

Techrights-sec2

to kill that process and relaunch it with the right settings

Techrights-sec2

back in an hour or three

schestowitz-TR

ok, I'll have a knock

*u-amarsh04 has quit (Quit: Konversation terminated!)

Aug 15 17:30

*u-amarsh04 has quit (Quit: Konversation terminated!)

Aug 15 17:30

*u-amarsh04 (~amarsh04@5tcc2vuaj9aks.irc) has joined #boycottnovell

Aug 15 17:31

*u-amarsh04 (~amarsh04@freenode-rmogvn.g0d7.dtdf.mc4289.IP) has joined #boycottnovell

Aug 15 17:31

schestowitz-TR

it's back, and working as a "proper" service

schestowitz-TR

I bundled the permissions a bit

schestowitz-TR

my guess is that a mixture of account names denied access

schestowitz-TR

nothing in ipfs datastore is ssensitive

schestowitz-TR

anyway, I can get objects again

schestowitz-TR

service ipfs status/start/stop

schestowitz-TR

BTW, we got some nice feedback on the site

schestowitz-TR

marius now knows how to edit is needed

schestowitz-TR

some ACS guy has suggestions

schestowitz-TR

but the key thing is, the site works and all

schestowitz-TR

and can cope with loads

schestowitz-TR

he'sa CS guy and likes to scrape links off the rss feeds

schestowitz-TR

so wanted more "views" to latch onto, I asked for more clarifications

schestowitz-TR

told him it's an alpha for now

Techrights-sec2

back

Techrights-sec2

which improvements are needed, it's kind of too late to make changes since it

Techrights-sec2

was put into production a while ago

Techrights-sec2

what kind of views?

Techrights-sec2

it stopped being alpha when you turned off the editing of the main site :(

Techrights-sec2

it's been pushed into production a couple of weeks prematurely :(

Techrights-sec2

tasks that would have taken 5 to 10 minutes of undisciplined experimentation

Techrights-sec2

now take hours of planning and cautious execution, at least in regards to

Techrights-sec2

the modification of the database tables

schestowitz-TR

yes, I fucked that up and felt bad about it all day

Aug 15 18:14

Techrights-sec2

that's ok but things go slowly now and some will require long blocks of time

Aug 15 18:14

Techrights-sec2

just to be safe

Aug 15 18:14

schestowitz-TR

yes, I didn't envision this scenario

schestowitz-TR

too risky

schestowitz-TR

partly because it was a very slow sunday and seen as opportunity(ism)

Techrights-sec2

another week or two of testing would have been fine, and quite useful

Techrights-sec2

anyway, what feedback did the CS guy have or will he get back about that later

schestowitz-TR

I anonymised for transpaency in the channel, but I need more info

schestowitz-TR

from what I understand, he likes not just full html in rss as before but wanted more from it

schestowitz-TR

because of clusters of links

schestowitz-TR

he wanted a whole feed just for all the links

schestowitz-TR

this is doable, but I need to know what he does and needs

schestowitz-TR

I can even implement that myself

schestowitz-TR

maybe he does some ACS "data sceince" with it

Techrights-sec2

the feeds contain the recent URLs which point to the full post

schestowitz-TR

yes, he seems to have an "edge" case of needs

schestowitz-TR

but I need clarification

schestowitz-TR

I also think that me putting extra effort to write a short outline might be better than just making a copy of the whole page

schestowitz-TR

but he wants something else, it's not like pages take long to load, it's just a simple file transmission

schestowitz-TR

my guess is that he uses some scripts

schestowitz-TR

and he's very familar with gemini

Aug 15 18:22

schestowitz

Record added

schestowitz

Creating temporary directories and files

schestowitz

Creating static XHTML and GemText hierarchies

schestowitz

Date::Calc::PP::Date_to_Time(): Date::Calc::Date_to_Time(): date out of range at /usr/local/bin/tm-extract-posts-sql.pl line 862

schestowitz

Updating SSI files

schestowitz

Could not validate '20220815'

schestowitz

Issuing rollback() due to DESTROY without explicit disconnect() of DBD::SQLite::db handle dbname=/var/www/tuxmachines.org/db/tm-static-site-generator.sqlite3.

schestowitz

Writing the Atom feed for HTTP

schestowitz

Could not validate '20220815'

schestowitz

Issuing rollback() due to DESTROY without explicit disconnect() of DBD::SQLite::db handle dbname=/var/www/tuxmachines.org/db/tm-static-site-generator.sqlite3.

schestowitz

Writing the Atom feed for Gemini

schestowitz

Could not validate '20220815'

schestowitz

Issuing rollback() due to DESTROY without explicit disconnect() of DBD::SQLite::db handle dbname=/var/www/tuxmachines.org/db/tm-static-site-generator.sqlite3.

schestowitz

Writing the Gemini index

schestowitz

Could not validate '20220815'

schestowitz

Issuing rollback() due to DESTROY without explicit disconnect() of DBD::SQLite::db handle dbname=/var/www/tuxmachines.org/db/tm-static-site-generator.sqlite3.

schestowitz

Pinging via MQTT

schestowitz

Erasing temporary directories and their files.

schestowitz-TR

ok, so now we have a little db issue

schestowitz-TR

rianne and I posted around the same time, I think

*psydruid (~psydruid@jevhxkzmtrbww.irc) has joined #boycottnovell

Aug 15 18:39

*DaemonFC (~daemonfc@2wbkh9pt9z5du.irc) has joined #boycottnovell

Aug 15 19:00

schestowitz-TR

back

schestowitz-TR

showed rianne how to edit and add images

schestowitz-TR

which she did to the ipfire page

schestowitz-TR

also showed her the capsules using various http proxies

schestowitz-TR

seems the capsule part is appreciated by readers who hate web bloat

schestowitz-TR

now reading below what I missed...

Techrights-sec2

yes I'm working on the live system with the changes from this morning

Techrights-sec2

they should be in place now, the lock did not take and two people added

Techrights-sec2

records while I was working on the database; those had to be fixed manually

Techrights-sec2

but there were only two;

Techrights-sec2

try now, it should be back to similar behavior as before but with a date-time

Techrights-sec2

stamp instead of just the date

schestowitz-TR

yes, I think it was a triple conflict

schestowitz-TR

in the sense that rianne and I added something at the same time (no communication, she is upstairs)

schestowitz-TR

and you did the DB work

schestowitz-TR

am I safe to add another page now?

Techrights-sec2

It is safe to test it , update should work too now

DaemonFC

schestowitz-TR, Who are you talking to in here?

Aug 15 19:05

schestowitz-TR

DaemonFC: I am talking to Techrights-sec2

schestowitz-TR

good job, when rianne updated the page it went smoothly

schestowitz-TR

I'll try another one

schestowitz-TR

also, the capsule as updated to not have the test pages

schestowitz-TR

I've just given rianne the css-only button ("read on") as she was still using an old template

schestowitz-TR

I'll add the page about moz hiring an exec from FB

schestowitz-TR

(again)

schestowitz-TR

add successfully, no warnings shown

Techrights-sec2

ack

Techrights-sec2

there are some bigger changes needed in a few days but I have to work out more

Techrights-sec2

details off line before trying them, preferably at an hour when others are not

Techrights-sec2

actvive

schestowitz-TR

sooner or later more "bots" like baidu and gulag will reach the domain, google is definitely adding more pages

schestowitz-TR

there's only server load or temporary spike when adding or updading nodes

schestowitz-TR

which happened at most 50 times a day for about 5 secs, i.e. max 5 mins total

schestowitz-TR

the rest runs like a brute force varnish server

Techrights-sec2

ack

Aug 15 19:16

schestowitz-TR

thanks a lot again. I think ipfs iw working and will update automatically from now on, also export the hashes at 5am to gemini (I'll test again tomorrow)

Aug 15 19:18

schestowitz-TR

Back to work Weds

Aug 15 19:18

Techrights-sec2

Aug 15 19:38

*psydroid2 has quit (connection closed)

Aug 15 19:43

*psydroid2 (~psydroid@memzbmehf99re.irc) has joined #boycottnovell

Aug 15 19:46

*psydroid2 has quit (connection closed)

*psydroid2 (~psydroid@memzbmehf99re.irc) has joined #boycottnovell

Techrights-sec2

feeds are (probably) now properly sorted by date-time stamp based on time of

Techrights-sec2

modification so now the recently modified ones float to the top

schestowitz-TR

yes, I noticed that and it's great

Aug 15 19:48

schestowitz-TR

in more than just that

Aug 15 19:49

schestowitz-TR

prior to that it seemed arbitrary order

Aug 15 19:49

schestowitz-TR

today I updated linux rc1 page about 6 times

Aug 15 19:49

Techrights-sec2

it was only somewhat arbitrary before but from the outside it probably looked

Techrights-sec2

random

Techrights-sec2

the date now uses 2 x 16 characters for storage instead of 2 x 8

Techrights-sec2

back in the day, that could break the utility of a database by crimping

Techrights-sec2

the max number of records available on the partition / device severely

Techrights-sec2

nowadays it's not such a problem and the benefit is a standardized,

Techrights-sec2

human-readble format; though it'd probably make more sense just to store

Techrights-sec2

as an integer (unix epoch) and convert as needed

schestowitz-TR

some hours ago I had to look up all the options of gnu date (coreutils iirc)

schestowitz-TR

because I made some changes to how I observe changes other than notif for git commit

schestowitz-TR

I pushed some stuff back to git and it wll evolve over time

Techrights-sec2

yes

Techrights-sec2

GNU date has a long-standing bug in how it mishandles relative dates

schestowitz-TR

still more expressive than thje busybox impl

Aug 15 19:55

schestowitz-TR

I had to add gnu stuf to tr's alpine, as we had done in tm

Aug 15 19:55

schestowitz-TR

I saw thje maintainer's name and it rang a bell

Aug 15 19:55

Techrights-sec2

nah, I got the impression that was not an issue but that there may be some

Aug 15 20:10

Techrights-sec2

underlying structural problems that would entail a lot of work

Aug 15 20:10

schestowitz-TR

date is used so widely

Aug 15 20:11

schestowitz-TR

that maybe it's one of those bugs that, if fixed, would end up breaking the whole planet('s systems)

schestowitz-TR

I cannot remember many examples _offhand_, but in Microsoft's Windows and Office there are many bugs like these

schestowitz-TR

and the bugs have been spun as the desirable behaviour

schestowitz-TR

like names you cannot give files... but can... and trouble lurks

Techrights-sec2

the latest round of wget seems to have finished, it has counted from

Aug 15 20:31

Techrights-sec2

166500 168324 inclusive and fetch the numbered nodes. However, although

Aug 15 20:31

Techrights-sec2

I have not investigated, I have the strong feeling that most are orhpaned nodes

Aug 15 20:31

schestowitz-TR

i am going to check assuming they're in "old" path

schestowitz-TR

Oh, I see. it looks for relative path for css fiiles

schestowitz-TR

does wget have an option for absolute urls in downloaded pages?

schestowitz-TR

it's not just missingfiles, it also links to pages in the wrong context/domain

Techrights-sec2

/var/www/tuxmachines.org/htdocs/old/

Aug 15 20:35

Techrights-sec2

not sure, there are options to convert the contained URLs to local ones as

Aug 15 20:40

Techrights-sec2

well as ones to pull in page components.

Aug 15 20:40

Techrights-sec2

see -k or --convert-links maybe

Techrights-sec2

latest iteration : /var/www/tuxmachines.org/bin/mirror-old-tm.sh

schestowitz-TR

maybe converting to local ones is what we want, if that means "/something" becomes "domain/something"

schestowitz-TR

checking...

schestowitz-TR

<techrights-news> "Luckily, I was able to reconstruct most of Drew's capsule using saved content from older crawls from Kennedy, my Gemini search engine. I rewrote the internal hyperlinks to be relative links, so you can read the capsule online or off." gemini://gemi.dev/gemlog/2022-08-15-drew-devault-mirror.gmi

Aug 15 20:46

Techrights-sec2

There were a heck of a lot of '2022-08-15 08:23:55 ERROR 403: Forbidden' errors

Aug 15 21:02

Techrights-sec2

perhaps there is a built-in anti-scraping module that triggered?

Aug 15 21:02

schestowitz-TR

re "There were a heck of a lot of '2022-08-15 08:23:55 ERROR 403: Forbidden' errors

schestowitz-TR

schestowitz-TR

no, that's me being stupid

schestowitz-TR

for about 20 minutes I had the permissions set wrong

schestowitz-TR

so public could not access nodes

schestowitz-TR

I caught the mistake quickly enough owing to monitoring