A Year Since the Big Switch - Part I - Why We Moved Away From WordPress, Drupal, and MediaWiki (PHP and Dynamic, Database-Driven)
One very considerable factor is, a lot has changed since 18 years ago; nowadays a lot of the Web's requests come from hostile bots that are a nuisance and aren't worth serving or working hard for
BETWEEN 2006 and 2023 this site ran on the formidable and nowadays (unlike in 2006) ubiquitous WordPress, at least for articles (Drupal and MediaWiki were also used, but not to the same extent). In 2022 we decided to make our own alternative to WordPress for various reasons we had already explained for a decade. As a WordPress contributor since 20 years ago it wasn't an easy decision for me to make, but our needs as a site are not compatible with a "one-size-fits-all" system, such as WordPress (which also introduces security risk, bloat etc.) and nothing else we've looked into really seemed suitable. In my job I saw a zillion systems and I saw that our clients too had issues with "one-size-fits-all" systems, not limited to Drupal and WordPress (the "big two"). Some of them had very complex and extensive catalogues of information.
Earlier today someone asked: "Is there a way to find the series of articles mentioned here?"
"It says stay tuned but there are no forward links."
With WordPress we had a system module that scans the database for 'forward links' and then lists them below the article at hand. Our new system lacks this sort of functionality, but it makes things much faster (leaner, i.e. lower latency). Going through a database with about 43,000 blog posts to find 'forward links' is computationally expensive, no matter how it's done. And if the resultant pages are served to obnoxious bots that scan pages that aren't cached (old and almost random pages), the RAM and CPU toll is unbelievable. Last year a lot of these bots were far more useless than googlebot. Some were just scraping pages for analytics or so-called "hey hi" (AI) - the hype wave du jour (money down the drain). Why should we max up 32 CPU cores for them? Why respect their requests 24/7? It's not cheap and it worsens the speed for everybody else. It's a yoke and carrying this yoke on a road to nowhere made no sense anymore.
There is lots to be said about how disorganised blogs are. Many bloggers explained (even decades ago) that the chronological and "flat" nature of blogs is problematic for newcomers, i.e. not people who follow a blog closely every day or over RSS. This is an old and widespread problem. A solution to this problem remains elusive. When writing a book, before finalising it, the author can modify (amend, extend, move around) every part. That does not work with blogs (bloggers publish as they go along and rarely go back) and even good outlining upfront is no panacea. Blogs that cover news/events also deal with dynamic stories and facts that seem to change (or be further elucidated over time); unlike science books, they deal with a fast-moving body of material and the topics too can change (e.g. companies get taken over, rename themselves).
Around 2009 we added a wiki to this site (using MediaWiki) because we wanted to make it easier to find information based on topics, including overviews. But that too turned out to be imperfect and spammers kept hitting the wiki pages. MediaWiki was eventually (around 2013) made access-restricted, but it continued to be a resource hog (due to bots mostly). Seeing how poorly structured the current blog of Cory Doctorow tends to be (walls of text covering many disparate topic), I think we've done relatively OK. Blogging isn't easy and blogging in a very organised fashion is almost impossible because you only go forward, not backward. Otherwise it's not really blogging.
Tomorrow marks exactly one year since we moved to our new system, so we're going to tell some behind-the-scenes stories. All changes are difficult and many changes turn out to be for the better (and better done even earlier, if not for procrastination tendencies we all suffer from).
This series should be lots of fun to work on. It also brings back memories. █