from the cracked-some-eggs...-noggins-next? dept.
In a letter to nearly 1,000 patients, University Hospitals in Ohio says that a tank's remote alarm system, meant to alert an employee to temperature swings, was disabled for an unknown length of time. That led to the destruction of around 4,000 frozen eggs and embryos, double the original estimate:
Hospital officials say they doesn't know who turned the remote alarm off or how long it was disabled. They also said they were aware the tank in question needed preventative maintenance. Some of the eggs and embryos had been stored there since the 1980s. The hospital's investigation is ongoing.
"Right now we do not know whether it's mechanical or human or [a] combination," said James Liu, chairman of the department of obstetrics and gynecology at University Hospitals.
He says he doesn't think anyone intentionally disengaged the alarm. "Because it is a computer, we think it's unlikely that there was any kind of external force that was working to hack the computer or anything like that. We think it's unlikely," Liu said.
Previously:
Freezer Malfunction May Have Damaged Up to 2,000 Frozen Eggs and Embryos
Two Fertility Clinic Freezer Failures Occurred in a Single Day
Related Stories
University Hospitals has notified about 700 fertility patients and their families that the frozen eggs and embryos they had stored at one of its hospitals may have been damaged over the weekend when the temperature rose in a storage tank.
The problem, in one of two large freezers preserving specimens at the UH Fertility Center housed at the Ahuja Medical Center in Beachwood, was discovered on Sunday morning. It occurred some time after staff left the previous afternoon, according to Patti DePompei, president of UH Rainbow Babies & Children's Hospital and MacDonald Women's Hospital.
The liquid nitrogen freezer held about 2,000 egg and embryo specimens, according to Dr. James Liu, chairman of the department of Obstetrics and Gynecology at UH Cleveland Medical Center. Some patients had more than one sample stored, and some of the samples were provided as long ago as the 1980's.
Also at Newsweek.
A second fertility clinic has reported a liquid nitrogen cooling system failure:
A San Francisco fertility clinic says that a problem with the liquid nitrogen in one of its storage tanks may have damaged thousands of frozen eggs and embryos, triggering calls and letters to more than 400 concerned patients of the Pacific Fertility Center.
The nitrogen level in one tank fell very low, according to Dr. Carl Herbert, the fertility clinic's president. Herbert told ABC News that an "emergency filling" immediately took place, and that the tank's contents were then transferred to a fully functioning tank.
The problem struck on March 4 — the same day that a similar cryogenic tank failure was reported in Cleveland, where the University Hospital Fertility Clinic is investigating "an unexpected temperature fluctuation" that jeopardized its tissue storage bank, where liquid nitrogen preserves eggs and embryos. That incident reportedly affected some 700 patients.
One failure: accident. Two failures...?
- How cryostorage is supposed to protect embryos like those possibly damaged at 2 fertility clinics
- Proposed class action lawsuit filed against UH for egg, embryo freezer malfunction
Previously: Freezer Malfunction May Have Damaged Up to 2,000 Frozen Eggs and Embryos
(Score: 3, Funny) by c0lo on Wednesday March 28 2018, @11:11AM (3 children)
Wikipedia [wikipedia.org]
The cosmic perils of the IoT - I wonder why they need a computer to trigger an alarm? Is it because the were using Twitter for notification?
https://www.youtube.com/watch?v=aoFiw2jMy-0
(Score: 0) by Anonymous Coward on Wednesday March 28 2018, @11:14AM (2 children)
I wonder what the numbers are now.
I suppose we're mostly getting errors in non-essential data, such as video streams.
(Score: 3, Informative) by c0lo on Wednesday March 28 2018, @11:52AM
With the transistors getting smaller, I think is reasonable a cosmic ray incident will cause more extensive damage by flipping more transistors. The frequency may stay the same, just the damage is more intensive.
Now, I don't know how much of that study looked into the isotopic composition of the materials around. E.g the chips are covered in plastic and 14C (in 1 ppm abundance) is radioactive. 40K - potassium - will be present in the concrete that makes the building - half chances to decay in energetic gammas that penetrate the chips.
https://www.youtube.com/watch?v=aoFiw2jMy-0
(Score: 3, Informative) by c0lo on Wednesday March 28 2018, @01:11PM
A more recent (2003) article (PDF) [iaea.org]
256 MB is 2048 Mbit - thus apple-to-apple comparison would say 0.0172032 errors/hour at the level of 2003 memory. Or 1 soft error every 58h = 2.6 days in 2003
That was 2003. The today transistors are much easy to upset (lower charge thus incident energy required) thus I expect that the number upset events to be increased.
Supplementary, the amount of memory cells in the same silicon volume increased zillion-fold - Moore's law hold quite strong. I wouldn't be surprised to hear of multiple soft errors per hour today.
Other stories
The Invisible Neutron Threat [lanl.gov] - SEU - single event upset
Google: DRAM error rates vastly higher than previously thought [computerworld.com] 2009
https://www.youtube.com/watch?v=aoFiw2jMy-0
(Score: 1) by Spelli on Wednesday March 28 2018, @11:15AM (7 children)
You'd think a business that safeguards the potential future generation of their clients might try to diversify risk, as any capable business does. Splitting storage between facilities might have been a good idea, akin to good backup practices.
I can smell the putrid fragrance of salivating lawyers already.
(Score: 0) by Anonymous Coward on Wednesday March 28 2018, @11:29AM (3 children)
My guess, based on the numbers, is that they have multiple freezers. 4000 eggs and embryos could very well fit in one freezer. They probably have much more embryos and eggs stored in other freezers. These type of freezers are also a bit more expensive (in purchase and operation) than your household type of freezer, lowering the amount stored in one freezer would mean it becomes also more expensive to store per sample/egg/embryo.
(Score: 1) by anubi on Wednesday March 28 2018, @12:09PM (1 child)
I believe on something this significant, multiple heat exchangers ( running on redundant refrigeration loops ) would be called for. Many embryos could share the same physical cooling bin, but each bin may have, say, three heat exchangers in it, when any one of them is quite capable of maintaining temperature.
In order to assure redundancy, each refrigeration system powered from an independent source, and during normal operation all three are cycled into operation. Failure of any one of them gives time to repair it while the other two continue to function. Simultaneous failure of two is cause for great urgency to repair, as simultaneous failure of three will result in lots and lots of lawyer calls.
A lot of their customers trusted them with the continuation of the family bloodline. Now, that's not gonna happen, and their bloodline stopped in a malfunctioning fridge.
"Prove all things; hold fast that which is good." [KJV: I Thessalonians 5:21]
(Score: 5, Informative) by rleigh on Wednesday March 28 2018, @06:32PM
There is no refrigeration, and there are no heat exchangers, in liquid nitrogen storage tanks. They are nothing more than a large stainless steel dewar with a few inches of liquid nitrogen at the bottom. While the bottom box or two of a stack might be immersed in liquid, most are in nitrogen vapour above the liquid at -196℃. The top of the tank narrows to form a neck, which is sealed by a lid with a huge compressed syrofoam plug a foot or so thick.
There are only two ways that such a tank can fail. The first is the tank being physically damaged, resulting in a loss of vacuum in the dewar walls, a leak, or a damaged plug which results in loss of nitrogen or the insulation. The second is a failure to keep the nitrogen levels topped up. So long as there's even the smallest hint of liquid in the tank and the lid is left on, the vapour temperature will remain at ~-196, but as soon as it's gone the temperatures will rise quickly.
The failure here is not simply the alarm being disabled. It's the fact that the tank was allowed to go dry, and it wasn't picked up on by manual inspection. It takes just a few minutes to do a daily inspection of an entire roomful of tanks; all you have to do is take the lid off and shine a torch inside to check the level. It's a couple of minutes per tank at most. This isn't a technology failure; it's a process failure. The alarm is to alert when abnormal events happen like the tank leaking, or someone forgets to fill it up; it should never, ever, go off if a routine process is followed to maintain the tanks. Someone, or someones, didn't do their job, and no one else checked up that the job had been done. Which is both a failure on an individual level and a management level.
(Score: 3, Informative) by rleigh on Wednesday March 28 2018, @07:03PM
Can be way more than 4000, depending on the tank size. A typical cryovial storage box (https://assets.fishersci.com/TFS-Assets/LCD/product-images/F32007~p.eps-650.jpg) can store 100 vials. There are many sizes and designs; sperm and embryos are typically stored in straws (http://www.irvinesci.com/products/25292-high-security-embryo-sperm-straws?dpt=Assisted+Reproductive+Technology). I've seen stacks ranging from 4 to 10 trays, and tanks which store from 4 to 8 stacks. So a mini tank about 2.5 feet high and wide can store 4 racks of 4 boxes = 1600 vials. A big tank about 4 feet wide and 5 high can store 8 racks of 10 boxes = 8000 vials. A room of them could contain hundreds of thousands. Straws are a different shape, and are stored at a lower density, but you can still pack thousands of them in.
Regarding costs, they aren't as expensive as you might think. The steel tank is expensive; can be $50k for a decent sized one, and more if you want built-in electronics to monitor liquid levels and temperatures. But liquid nitrogen is quite cheap, and you order it by the road tanker load.
An actual freezer is likely more expensive to run overall; ultra-low temperature freezers go down to -80℃ but are in my experience more prone to failure, even with redundancies in place (I've experienced several). They are not generally used for long term storage because they aren't sufficiently reliable or cold enough. They are typically used to do the initial freezing under controlled conditions prior to transfer to nitrogen storage. Because it's just regular square shelves, the capacity is huge though; you could fit several hundred boxes in one freezer; my old lab had a whole bank of them.
(Score: 2) by fadrian on Wednesday March 28 2018, @01:00PM (1 child)
Well, the lawyers deserve to be salivating in this one. There was monetary loss - people paid a lot for those embryos (in the orders of thousands of dollars), as anyone who has had IVF procedures can tell you. The clinic was clearly negligible about storing the embryos - they've as much as admitted it in their letter.
I know some people have an irrational hatred of lawyers, but in this case, let them in to teach other medical facilities a lesson.
That is all.
(Score: 2, Informative) by Spelli on Thursday March 29 2018, @07:15AM
Also, since they had eggs & embryos frozen in the 80s, some women might not be able to take the offer of free compensatory IVF cycles any more.
Those women will now definitely never have children of their own.
(Score: 4, Funny) by SomeGuy on Wednesday March 28 2018, @01:19PM
Or at least they should have put an alarm on the alarm that would go off if the alarm failed. And then put an alarm on that alarm, and an alarm on THAT alarm, until it is alarms all the way down.
(Score: 0) by Anonymous Coward on Wednesday March 28 2018, @11:22AM (2 children)
So, the disabled alarm was the cause of the failure? Seriously, the cause of the failure is that the freezer broke down... not that a safety measure was not working. Makes you wonder how often they tested their safety measures.
(Score: 2, Interesting) by Anonymous Coward on Wednesday March 28 2018, @11:32AM (1 child)
and we still would like to know what happened to the other freezer that failed elsewhere the same day...
(Score: 2) by SomeGuy on Wednesday March 28 2018, @01:12PM
No, no, no, you are supposed to pay attention to THIS freezer. The other freezer was the one where they kept the alien-human hybrids.
(Score: 3, Interesting) by LoRdTAW on Wednesday March 28 2018, @01:41PM
One thing I have learned is that every shop, no matter the kind, always has some amount of dysfunction. In this case, as in many others, an alarm or safety was disabled. Happens a lot when something breaks down because of a minor issue that can easily be duct taped. Probably faulty sensor was causing the alarm to sound so someone unplugged it until it could be serviced. Then the maintenance guy is overworked or overbooked and cant get to it for a few days/weeks. Then something comes up, focus shifts, and the "minor" issue of the faulty alarm is forgotten about. Happens all the time. Big projects get in the way of little ones. Poor time management, poor planning, or possibly under staffing.
(Score: 2) by hemocyanin on Thursday March 29 2018, @03:43AM
It isn't like the world is running out of humans.