01.10.23
Gemini version available ♊︎When the Employer Doesn’t Understand or Grossly Underestimates AWS Capacity Issues (and a Story of Lost E-mail in Clown Computing)
Summary: The Sirius ‘Open Source’ management was dumb enough to replace the in-house infrastructure with overpriced (and outsourced) junk that did not even work as expected
THE report we deposited over a month ago already covered the fiasco of outsourcing (gradual) where I had worked for nearly 12 years. We don’t want to repeat what was already covered. I discussed this in person with the main individual responsible for the awful decision. He said they envisioned it would save money, but based on bills that I saw it was beyond insane to suggest so! Why would any sane company throw about 10,000 pounds down the drain every year? A modest second-hand server can be purchased for just 1,000 pounds and we didn’t need to buy any. We already had servers!!! We had an ISP, too.
“Who’s going to be held accountable?”When the company’s “cloud” (or “clown”) bills keep blowing upwards (upward to almost a thousand pounds a month), for something that started very small (the vendor lock-in relies on this sort of illusion, before exit barriers are raised), you have to wonder about the judgment of short-sighted decision-makers like Mr Kink. Who’s going to be held accountable? Or when?
As a reminder, AWS operated at a loss for years and Azure still seems to be operating at a loss (they just call everything “Azure” now). They are enticing people to enter the trap. Microsoft loses money and so does Google. Billions in losses! I brought this up over the phone, speaking to the CEO for about an hour almost a year ago! But they don’t want to listen!
“As a reminder, AWS operated at a loss for years and Azure still seems to be operating at a loss (they just call everything “Azure” now).”As a reminder, Microsoft is laying off staff, cancelling and shutting down datacentres, as they overprovisioned for something that never came (or resulted in massive losses). Microsoft basically misleads shareholders by rebranding many things “cloud” and/or “Azure”, so even if it’s not growing Microsoft can claim otherwise. There’s no proper definition of “cloud” or “Azure”.
On the phone about a year ago I suggested small self-hosted machines (the CEO called this “hobbyist”). It’s worth reminding ourselves that we lost staff that looked after our servers. That too was the fault of the management, for reasons we explained before.
It would be so much cheaper and safer to run our own infrastructure, as we already did for decades. And yes, we covered this in the report and earlier in this series. This is a no-brainer.
To give one example of what moving to AWS caused Sirius: OTRS, a ticketing system, needed us throwing more and more resources at it (partly because of bad design, partly due to workers sending megabytes of text in E-mails, as they top-post — the “Microsoft Way” basically — and don’t bother trimming/snipping what they respond to). Each time you add resources the bills go up by a lot! That’s the “magic” of “the clown”! It’s getting very expensive very fast!
“To give one example of what moving to AWS caused Sirius: OTRS, a ticketing system, needed us throwing more and more resources at it…”Remember that we used to self-host all the E-mail of the company; now the company uses phony encryption as a tenant on someone else’s servers (Amazon). I challenged my colleagues about this. I argued with management. They could not even defend their decision. They saw no need to defend what they had done! We’ve had arguments over this internally in 2022. Of course it was risky for me to bring this up, but at this stage it was the moral thing to do, even a moral obligation. At Sirius, colleagues felt like their efforts and contributions were ignored/discarded by the cabal (family), so they quit caring. This is how nepotism dooms companies. Some colleagues left, some remained but without much desire to go beyond the basics. And this aspect too we’ve covered here before.
Regarding E-mail hosting in “the clown”, here’s a 2020 story. To quote an Evening Shift handover: “Spent most of my evening tracking down missing emails. I was rather perturbed by xxxxx’s handover email disappearing and I’m guessing that because the server was underpowered it started to behave strangely and misclassified legitimate emails as viruses and deleted them. Fortunately each email is given an unique id by the system which is useful for searching the logs. Managed to get a list of deleted ones and sent it to xxxxx, xxxxx, and xxxxx suggesting that they identify their clients or ones they recognise and email them with the time + 1 hour asking to resend. I found one from xxxxx and emailed and xxxxx kindly sent his email again.”
Wonderful! What a mess.
“Ironically,” Ryan Farmer notes today, “”Cloud Hosting” only makes sense if your needs are so small that it’s hardly worth setting anything up yourself.”
In some cases useful virtual machines were turned off to “save money”. Even if they took little space and CPU. If self-hosted, they would cost almost nothing to leave on.
“Clown computing: it’s here today, but gone tomorrow. You’re not part of the decision!”Clown computing is a trap. To quote one new (days-old) cautionary tale (already in Daily Links): “Turns out that Revue is getting shut down. This means that I won’t be able to use it anymore (and I stopped using it because it wasn’t getting much traction vs the amount of work I put into it).”
So maybe outsourcing isn’t such a wise long-term strategy after all.
At one point by far our biggest client relied on VMware for clown hosting; of course VMware shut the whole thing down and in a hurry we needed to get all the servers out of there. Clown computing: it’s here today, but gone tomorrow. You’re not part of the decision! It does not matter if you have critical services on there and they give you a very short notice (to vacate). █
ᶃ Gemini Space
This post is also available in 
Content is available under CC-BY-SA