Comments

• twitter

  2010-07-23 22:01:28

  This should be a special occasion to Call Out Windows. I've read several headlines about "bypassed safety systems" but did not realize that the system in question was bypassed because it was Windows and suffered from the usual Microsoft problems. As a Gulf Coast resident, I'm personally offended by this but not particularly surprised. Many in the press might not think it significant because there were so many bad decisions that BP made, but things might have been different if the alarm system had been working. Thanks for documenting it.

• satipera

  2010-07-23 21:18:37

  Using Microsoft software for safety critical applications is criminal negligence.

• Dr. Roy Schestowitz

  2010-07-23 21:32:48

  I know someone who suggests criminal prosecution either for those who choose Windows or those who make/sell Windows, but I don't agree. Either way, I wrote about Microsoft's deliberate/willful negligence in [1, 2, 3].

• twitter

  2010-07-25 16:38:12

  It turns out that the unreliable system is directly responsible for the most of the Deepwater Horizon deaths and could have prevented the accident if it had worked properly. The New York Times article requires a login, and transcripts won't be available for three weeks. The New Orleans Times Picayune has this article describing the deaths caused by the alarm bypass. The relevant opinion and expert quotes are worth documenting here. The problem is not particular to the Deepwater Horizon, all of Transocean's rigs have the same system and, of course, anywhere people use Windows for mission critical work they wastefully risk worker's lives, public health and their own business.
  
  

  With the general alarm set to bypass, the rig's one danger alarm never sounded, Transocean chief electronics technician Mike Williams testified. If it had, he said workers in the drilling area -- the shaker room, the mud room, the pit and pump room -- would have immediately evacuated. Several of the 11 workers killed in the explosion worked in those areas. [no one from these areas survived]

  
  
  

  ... drilling area is extremely susceptible to fire if gas kicks up from the well, so the rooms are air tight, and control panels can be set to shut down if gas seeps in, but Williams testified that one such panel in the drilling shack was set to bypass.

  
  
  

  About five weeks before the accident, Williams was called to check on a computer system in the drill shack that was constantly on the fritz. Williams said the software was chronically bad, leaving a "blue screen of death" on the driller's interface and often causing the driller to lose crucial data about what was going on in the well. Once, when the Deepwater Horizon was drilling a different well, the computer froze up and the rig took a kick of natural gas while the driller was looking at "erroneous data," Williams said.

  
  
  

  the rig's general alarm and indicator lights were set to "inhibited," meaning they would record high gas levels or fire in a computer, but wouldn't trigger any warning signals. "When I discovered they were inhibited a year ago I inquired why, and the explanation I got was that from the OIM (the top Transocean official on the rig) on down, they did not want people woken up at 3 a.m. due to false alarms," said Williams ... Williams said an emergency shutdown system, which was supposed to shut off the engines, didn't trip, either. The engines ended up overspeeding by drawing power off the gas and Engine No. 3 exploded ... Mark Hay, the Transocean senior subsea supervisor, set the control panel system to bypass its gas shutdown function, and when Williams questioned him, Hay said there was no point in Williams fixing it because none of the Transocean rigs use the safety system. ... "Damn thing's been in bypass for five years. Matter of fact, the entire (Transocean) fleet runs them in bypass"

  
  
  It is clear from Williams testimony that Windows was not up to the task and that this directly lead to the accident. The first warning workers got of gas in the drilling room was a generator overspeed and explosion, when a properly functioning system would have activated a warning alarm and shut equipment down. The system was bypassed because it was not reliable. Transocean issued a lame excuse for this negligence, calling the bypass standard industry practice. It may be true that other drillers take similar risks but that does not make it a good practice. There were many other mistakes made as documented by this overlapping article that documents damage to underwater equipment and four failed safety tests, but the explosion and fire itself may have been prevented if the alarm and shutdown system had worked reliably.
  
  Industry should purge itself of this unreliable and costly software.

• Dr. Roy Schestowitz

  2010-07-25 16:53:11

  A long thread that I saw earlier (initiated in a newsgroup in response to one article I wrote) noted that the software runs on Windows only. They run it on a flaky foundation.
  
  BP still has many platforms that run the same software, i.e. they can suffer BSODs that would multiply the scale of the existing disaster.
  
  BP must look at the platform it uses (you can read that in more than one way).

• FactBknown

  2010-07-28 20:43:45

  First, I am not a big Microsoft lover. I have worked in the IT field for over 12 years and over that time, with 4 companies. Each company was predominantly Windows based. Grant you, none of them were in such an industry that could cause death if a system failed. However the fact is that Microsoft did not turn the Alarms off. They did not bypass the safety systems. Their software is dominated by errors, BSOD, hardware incompatibilities and etc… The fact is that though Microsoft is not the best and we would likely still have the issues if it had been a different OS. The fact is that someone at BP wanted the alarms off due to false alarms. That is when they need to have a strict on-call rotation that only one person is woke up and they can physically check the alarm. If it is needed then the others can be awaken. If it is in fact a false alarm, then reset it and go back to bed. Also, if they are getting BSODs, why is there not a backup alarm system or operating system? And why was the systems bypassed instead of being looked at by a tech or by Microsoft? It would be real sad if the BSOD was caused by a simple driver issue that could have been resolved easily. BSOD happens to everyone, but if it is a vital system then have a backup system. Sounds like someone took Out of Sight, Out of mind literally.

• Dr. Roy Schestowitz

  2010-07-28 20:56:50

  Based on some research I saw, the software in question only runs under Windows. It's irresponsible to run such crucial systems on an operating system that's largely rejected by stock exchanges/markets.
  
  I hope that lessons will be learned and weak links will be removed.

• twitter

  2010-07-29 17:45:01

  I like what Richard Stallman had to say about it,
  
  

  Managers careful about safety would have had the cause of the false alarms fixed. But that would have cost money, and they probably gave cost savings higher priority than safety.

  
  
  It is too bad that managers don't understand that free software is cheaper from start to finish and start the migration efforts sooner than later. People making these systems must understand things by now but that won't replace old systems that are still in the field. The cost of not replacing the system in this case was obviously higher.

• Dr. Roy Schestowitz

  2010-07-29 17:49:50

  I guess the question is, what did he call "BP" this time? "Big Polluter" is his most common joke, but there are variations.