Bonum Certa Men Certa

Improving Site Navigation and Discovery

posted by Roy Schestowitz on Dec 12, 2023

An open book

THE site is growing fast and people have a hard time searching for much older material. We fully recognise this limitation. It's a real peril. Many sites have the exact same limitation. This problem isn't limited to digital media, either (volumes of material, some of it outdated or unlinked).

In WordPress we used code that checks references in reverse; for any given article, it would (at the bottom) show later (future) articles that link to it. This was very CPU-intensive (at the database level), resulting in pages taking far longer to load. Unless properly cached, it would require scanning about 10 GB of text (or 40,000 blog posts' bodies, not counting drafts/revisions).

We needed to move on. Better sooner than later. Having a server screaming 24/7 to serve requests (whose growing proportion is rogue bots) is not a long-term strategy. Running a Web server on a machine with almost 100 CPU cores isn't cheap.

Before the very final post from Pamela Jones of Groklaw (just over 10 years ago) she wrote about the challenges of preserving old material. She had quit before, then came back, then retired. Fair enough, she wasn't getting young, but it was important for her to ensure the information remains accessible for many years to come (debunking lies about the GPL and origins of Linux). Some time later the site was converted into static pages (still hosted at ibiblio.org), but some material such as old comments disappeared in the process. Geeklog had its share of limitations and apparently it's still being maintained.

Anyway, unlike Groklaw we're still going. I'm 41 and in good health. I receive help from many people and we're good to go. Nothing can stop us, even though some extremists are trying. We won't let wackadoodles waste our time. They just validate what we wrote months ago and they try to attack my wife. Misogynists are like that; they love picking on women.

So what next for search? We've long envisioned this site having self-hosted search, not that lousy WordPress search our blog used to have (it's just some lousy WordPress database scan, which is notoriously weak at delivering relevant results).

No, we don't want to rely on third parties either. We don't want to hear, "how about Google?" or "why not ClownFlare?" (Wherever or whenever there are DDoS attacks)

Any third party means Outsourcing. Outsourcing does not solve the issue; it typically creates additional issues, even if they are temporarily not visible (ClownFlare does not make money yet, so a "big squeeze" is impending and Google is not search anymore).

Several of our articles this month got over 3,000 views and we do not depend on Google, social control media, Gulag Noise (Google News), "Hacker" "News" etc. We have our loyal readership, i.e. people who come back not because "Google told me to..." (so-called 'search')

Many people don't know this, but way back in 2006 we made a "download site" option available (our database was relatively small back then and a WordPress plugin existed to make a database available sans sensitive things like user accounts). For about a year this whole site was available for download, but the site grew too big and it was no longer feasible to generate the dump on the fly and serve requests. These requests were nightmarish. They caused PHP timeouts and MySQL strain.

So what next for data?

Well, we considered what we can install for self-hosted search, seeing what's available that is Free software and is also more potent than just a database scan (over fields like title and body).

Search can help, wiki pages can help even more, but ideally we may go back in time and turn the site into a kind of hierarchical 'book' (a big project! Big but still feasible). It's still debated in IRC.

I quit my job so that I can devote more of my time to promotion of Software Freedom, abolition of software patents etc.

While we continue to discuss the best way to organise information in this site (suggestions welcome, IRC would work best) we remind readers that we're actively seeking help with server bills. We want to keep going for more than a decade to come and help from readers enables us to spend more time researching, writing, tidying up existing material (lots of wiki refactoring to come over the Christmas period), maybe adding a self-hosted search facility. █

Dog Golden Retriever Card: Watercolor painting of a golden retriever dog holding a leash

Other Recent Techrights' Posts

Intercontinental Brigading - Part XXXXXIV - Hostility Towards Media, Coming From People Who Promote Microsoft Back Doors and Engage in Mass Plagiarism, Attack Women Etc.
The core issue or the seminal issue is Microsoft as the "master" of all computers
 
IBM Moved From Selling Technology to Selling Its Buildings
IBM is trying very hard to silence/crush critics
Staff Union of the European Patent Office (SUEPO) Looking to Extend Industrial Actions and Strikes to Almost 1.5 Years
It says a lot about the media that for 1.5 years staff is on strike in the second-largest organisation in Europe and the media pretends it's not happening
Richard Stallman's Talk in Quito This Week (At Least 5 Talks After That)
He's having a busy end of year
Gemini Links 18/11/2026: "Love For Old Devices" and New Software/Slop-ware Releases
Links for the day
Over at Tux Machines...
GNU/Linux news for the past day
IRC Proceedings: Tuesday, November 17, 2026
IRC logs for Tuesday, November 17, 2026
Gemini Links 17/11/2026: The RAM Question and Lagrange Sync
Links for the day
EPO Strikes to Continue Until Summer of 2027 (If Not Further) Based on Existing Plan
It'll never end until the dictatorship ends
Trading 212 UK Limited Acting Like a Scam, FCA Says It's Legitimate
If this was some coy prank or a scam, it is certainly not funny
Richard Stallman's Talk Earlier Today and Next Month's Appearance in Mexico
He seems to have managed to dodge the 'cancel mob' for quite some time now
The Cost of Losing Integrity at The Register MS
Recently we learned that they make a "ton of money" out of their Huawei spam
Links 17/11/2026: Jensen Huang's Price-Fixing Story, "Russian Attacks Aimed At 'Breaking The Spirit' Of Ukrainians"
Links for the day
NVIDIA Continues Diving and Press Reports Speak of Business Crimes of Jensen Huang (Not China or Sex)
The deeper they go...
Gemini Links 17/11/2026: Super Power, Knowember, Writing in Shorthand, Cooking over a Woodstove, Decoupling Code and Content
Links for the day
Links 17/11/2026: California Sues Slop Peddlers and Nvidia Was Already in Trouble Before the Jensen Huang Mess
Links for the day
An American and South Korean Cartel Artificially Inflating the Prices You Pay for Computers (or Any Service That Relies on Computers, I.e. Almost Everything)
As long as the principal tenant of the White House is a white-collar criminal we can expect only cover-up and participation, not a crackdown
NVIDIA Corp Was Grossly Overvalued, Lost 36.06% of Alleged "Market Value" in 5-6 Weeks
NVIDIA has many of "obligations" (to pay), not money
Over at Tux Machines...
GNU/Linux news for the past day
IRC Proceedings: Monday, November 16, 2026
IRC logs for Monday, November 16, 2026
The EPO Still on Strike. 141 Tickets Opened Regarding Lack of Toilet Paper (or Similar).
Yesterday staff representatives at the EPO in The Hague (LSCTH] spoke of a meeting which took place a month ago
Intercontinental Brigading - Part XXXXXIII - An Attack on the Dignity of Reporters and Their Family Members (by Proxy/Attack Dogs)
The Microsofters seem to be super-upset that I uncovered Microsoft fraud
IBM and OSI Said opensource.net Would Replace opensource.com, But It Has Been Dead for 1.5 Years and the Person Who Had Made Promises Left OSI
Over a year ago he left
Red Hat and IBM Meet Targets (of Pushing People Out)
now there are claims of security breach inside IBM, but no press reports related to that
Gemini Links 17/11/2026: Reading Clocks and StreetComplete
Links for the day
Buy Influence "Over Dinner With The Register."
The Register MS is the facilitator or 'pimp'
While the Slop Industry Wobbles The Register MS Bags Tens of Thousands of Pounds to Publish Propaganda With "AI" 23 Times in It
The Register MS is selling its reputation away
The Real News About the White House (Not the 'Exclusive' Rupert Murdoch Published for It at Midnight Monday) is Complicity in NVIDIA Fraud
They need a distraction from commercial malpractice
Links 16/11/2026: Microsoft Increases Indian Clown Computing Prices by 15%, Meta Fined $887 Million in Italy
Links for the day
Gemini Links 16/11/2026: Container Gardening, Laughing at an LLM, "EU 72-Hour Guidelines Through a Lawyer's Eyes"
Links for the day
I Tried Linux Again After 10 Years - Here's What Blew My Mind
It's like leaving your hometown as a teenager and coming back to find the weird kid from high school is now the mayor
Links 16/11/2026: Data Centers (White Elephants) as Tax Evasion Scam/Ploy, Social Control Media Harms Boomers, "Microsoft Outage Hindered Ticket Issuance in Greece"
Links for the day
The Register MS Seems to Have Moved to 'The Cloud' (WeWork) After Shutting Down the Office
The Register MS will have a hard time explaining why it so passionately participated in the "hey hi" pyramid scheme
Holiday Plans (We Need More IBM Whistleblowers)
in particular with possession of material in relation to PIPs, NDAs, and silent RAs
After Microsoft Outage GNU/Linux Usage Leaps to 9% in the UK (Where Public Services Were Notoriously Overdependent on Microsoft Disservices)
in the UK we are extremely dependent on Microsoft stuff in the NHS, courts, law firms, police etc.
Richard Stallman Was Right About the Corruption of the Term "Artificial intelligence"
This will do severe damage to the administration which participated in the scam
The Dictatorship Already Working to Distract From NVIDIA Corruption and Slop Bubble Reaching Its Limits
Circular financing is simply a form of financial fraud
IBM Selling Itself, Piecewise, to Raise Money (to Pay Salaries)
IBM is a zombie waiting to inhale one last time (last breath)
"Allies" of Disgraced NVIDIA CEO Aren't About Morality, They're Just Covering Up Their Own Backs (and Loans)
The next stage is examining the false accounting (fraud basically)
No Outage Since 2023
I did host from home between 2020 and 2023
Intercontinental Brigading - Part XXXXXII - Back Doors and Kill Switches From Microsoft Are Objectively Undesirable
"I dual-boot Omarchy and Windows. Be careful updating your BIOS. Mine made Omarchy unbootable."
Gemini Links 16/11/2026: Enjoyable Coding, Social Control Media Considered Harmful, and Recursive Rectangular Subdivision
Links for the day
Over at Tux Machines...
GNU/Linux news for the past day
IRC Proceedings: Sunday, November 15, 2026
IRC logs for Sunday, November 15, 2026