About an eighth of a University of Alberta collection of ice cores has melted due to a freezer malfunction:
A precious collection of ice cores from the Canadian Arctic has suffered a catastrophic meltdown. A freezer failure at a cold storage facility in Edmonton run by the University of Alberta (UA) caused 180 of the meter-long ice cylinders to melt, depriving scientists of some of the oldest records of climate change in Canada's far north.
The 2 April failure left "pools of water all over the floor and steam in the room," UA glaciologist Martin Sharp told ScienceInsider. "It was like a changing room in a swimming pool."
The melted cores represented 12.8% of the collection, which held 1408 samples taken from across the Canadian Arctic. The cores hold air bubbles, dust grains, pollen, and other evidence that can provide crucial information about past climates and environments, and inform predictions about the future.
(Score: 2) by kaszz on Tuesday April 11 2017, @03:29AM (19 children)
Almost like one can sense a suspicion of sabotage by some climate change is just a hoax..
But how can they flunk a such basic function?
Why no thermal battery? redundant compressor system? Use Einstein refrigerator that only needs heat?
Warning system?
(Score: 3, Funny) by Anonymous Coward on Tuesday April 11 2017, @03:54AM (5 children)
Almost like one can sense a suspicion of sabotage by some climate change is just a hoax..
Joke's on them! I say global warming has gotten so bad, even ice in the freezers is melting.
(Score: 2) by kaszz on Tuesday April 11 2017, @04:27AM (4 children)
Let's call it local warming. I think someones getting a hot seat too.. ;-)
(Score: 0) by Anonymous Coward on Tuesday April 11 2017, @04:45AM
It was Trump. He hasn't been mentioned yet.
He's the new godwin.
(Score: -1, Offtopic) by Anonymous Coward on Tuesday April 11 2017, @05:58AM (2 children)
I want to know where khallow was when this "accident" occurred! We have already had the Harperpocalypse with scientific records, it is too much to imagine that some oil interests in (((Calgary))) might have made a very (((lucrutive))) offer to a certain ((unemployed)) code monkey (no, not you TMB, you are too much of a security risk because of your rather large and unrestrained beak; you talk too much) to destroy the evidence that might actually prove Anthropogenic Global Warming? He who controls the refridgerators, controls the past. And he who controls the past, controls the future. Orwell, I believe. Where was khallow? Or maybe it was that Bradley#randomnumber guy? So many petro-shills on SoylentNews. I grow tired trying to sort them out. But this, this is a bad thing. Next I expect an attack on Dante's Death Mask, to hide the fact that CO2 levels in hell have gone down since the Republicans came back into power. Or was that a Dan Brown novel?
(Score: 2) by Azuma Hazuki on Tuesday April 11 2017, @11:18AM (1 child)
Just so you know, putting things in triple parens is something the alt-reich does when they're trying to point out some Jewish conspiracy. Unless you're attempting a form of Bayesian poisoning attack on the technique, it doesn't make sense to use it in this context. Though if you are, that's rather clever.
I am "that girl" your mother warned you about...
(Score: 0) by Anonymous Coward on Tuesday April 11 2017, @04:46PM
The triple parens thing is mostly useful as a sign that the poster can be safely ignored :)
I'd just like to know what are the double parens around "unemployed" supposed to mean. Was it a mistake, or is there some new signalling that I can disappointingly shake my head at?
(Score: 2) by Scruffy Beard 2 on Tuesday April 11 2017, @08:27AM (12 children)
I want to know how the temperature in the freezer got to 40C. The ambient temperature [weather.gc.ca] was warm last week, but never above 10C.
Best guess is the the heat came from the still functioning freezers. How that happened in the absence of compression, I don't know.
(Score: 0) by Anonymous Coward on Tuesday April 11 2017, @09:39AM
In Canada, even freezers need heating sometimes. Likely as not, someone jostled the control for the heat pump.
(Score: 2) by VLM on Tuesday April 11 2017, @01:32PM (9 children)
If you lose a little refrigerant the cold side runs a lower pressure so its actually colder than normal, see the old AC or dehumidifier or car AC icing up. If you lose almost all of the refrigerant until there's not condensation anymore you're basically cooling the compressors and the "cold" side will get pretty darn warm. A non-engineer might have noticed the fridge is 20 degrees cooler friday afternoon before going home and thought nothing of it.
They may have over-spec'd the cooling system for the contents. If the cooler is barely able to keep the contents cool then instead of converting 100 watts of electricity into pumping 500 watts of heat resulting in 100 watts of melting which will take a long time to melt all that ice, they probably installed 10000 watts of cooling capable of moving 50000 thermal watts of heat and the thermostat mostly stayed off, but once the pressure drops low enough they have 10000 watts of heat getting dumped somewhere such as the inside of the fridge and 10 KW will roast a pretty big pile of ice.
Another option is stuff like lights and defrosting automatic heaters don't do much when the fridge section is working, but when the fridge section fails they're just heaters inside well insulated boxes. Water stores a lot of heat, to cause this much damage would take too long unless melt water got into some electrical contacts. So probably most of the damage was caused by loss of refrigerant.
April 2nd was a Sunday but you'd still expect some form of monitoring would operate. In my own (non-climate change lab) experience stuff that never breaks tends to have distribution lists consisting of grad students that graduated, employees that quit or retired years ago, group distribution lists that were deleted several re-orgs ago... So ironically if the fridges burned out once a semester the response would have been practiced and fast. The lack of response to this event would strongly imply the cryostats or whatever achieved an extremely large MTBF and thus were totally ignored for years until the disaster.
A side effect of stuff never breaking is we can assume since it never breaks no one does any maintenance, so when it finally went its not surprising it really blew. They're lucky they didn't have a refrigerant oil fire or a dust fire.
(Score: 2) by kaszz on Tuesday April 11 2017, @03:05PM (2 children)
As I recall the thermal transfer for compression heat pumps is 1/3 input electricity with 2/3 thermal power pumped out to the hot side. So the 1:5 factor seems like a lot.
Is there enough refrigerant oil to burn and that can actually ignite easily? it must leak out of the tubing I assume?
Anyway.. there's one thing missing. A monitoring system. Sure it was a pain in the ass when all that was available were 4-bit CPU with memory that you could type out with pen an paper. But these days with super integrated microelectronics, worldwide digital and cheap telecomms and abundant knowhow. How could they miss this? Something is seriously missing in this story.
If( temperature > too_high ) run_compressor();
if( temperature_derive > positive ) kill_compressor();
if( temperature > really_bad) alert_all_staff();
Temperature shield + Arduino + GSM module = Done!
(Score: 2) by VLM on Tuesday April 11 2017, @04:34PM (1 child)
COP depends very strongly on outside air temperature. I checked the weather and its just below freezing at night so the COP should be super high.
Refrigerant oils optimistically are refined not to burn well or easily but here they are in a multi-KW pump generating heat and wooshing around making mist so yeah it happens but not a common problem. There are flammable refrigerants, in fact propane is a pretty good refrigerant other than the whole explosion thing.
The problem with your design is the GSM shield, accounting cancelled that SIM card to "save money" etc. Or they used an ethernet wifi shield but IT didn't want to share and won't let it on the LAN without a windows virus scanner (even though its an arduino) and root access.
Also I've worked at datacenters with big Liebert chillers and we're all assuming the ice core freezer has cruder monitoring than the datacenter which is unlikely. What is likely is it was never set up or that part of the testing was removed from the contract to save money.
(Score: 2) by kaszz on Tuesday April 11 2017, @06:23PM
With a GSM-adapter one can circumvent the whole IT-department. Any server would only need power and no department-LAN. I'll expect the managers of the freezer room to have enough clue about SIM card. There has to be some minimum standard. Or it's like instructing the heart surgeon as a patient on surgery 101 before sedation..
Contracts can be expensive. But they could at least done a simple monitoring setup by the local electronics club. I hope every decent university has one. Even a junk computerphone has all the necessary parts except the temperature probe.
Again, this story smells rat.
(Score: 2) by Scruffy Beard 2 on Tuesday April 11 2017, @05:26PM (5 children)
Another poster found more information [soylentnews.org]
The Freezer was brand new: having undergone about 6 months of testing prior to loading the ice cores.
The monitoring system apparently failed due to database corruption. They finally figured out something was wrong when the fire alarm went off.
(Score: 2) by kaszz on Tuesday April 11 2017, @06:55PM (2 children)
The article author Colette Derworiz wrote yesterday
There was a temperature alarm, but it failed too. A computer system on the freezer was sending out alarm signals but they never made to the university's control centre. Investigation is ongoing. Several contingencies now...
So we have a overly complex monitoring system without redundancy.
(which makes my point elsewhere about a homebrew DIY self sufficient monitor even more relevant)
I hope it did run Microsoft embedded so I can say.. I told you so! (tm) ;-)
(Score: 2) by VLM on Tuesday April 11 2017, @08:51PM (1 child)
So we have a overly complex monitoring system without redundancy.
I have a couple decades experience in telecom... The point being I'm not sure if there is any other kind of enterprise sized monitoring system.
I seen some stuff...
Honest to god I worked at (name redacted but it was a big name) and a freshly promoted admin was provisioning "whats up" ping monitoring of a production system using 127.0.0.1 as the monitored device IP addresses because they didn't want to deal with 2am phone calls about red alerts on telecom gear thats not even been shipped to us much less carrying prod traffic, because the MS Project gnatt chart said monitoring system had to be operational before installation could proceed.
My guess is looking at SCADA type stuff "alarm condition" stuff probably is at best 50% effective if its non-hazardous to life. I hope refineries and chem plants and nukes where lives depend on monitoring, take it more seriously, but if there's no dead bodies then 50% savings of enormous damages will still fund the monitoring project even if it fails 50% of the time.
I mean you have to be realistic. Nobody died. Nobody drowned. You can't expect NASA/NRC level monitoring. Its a great excuse to dig up some new cores...
(Score: 2) by kaszz on Tuesday April 11 2017, @11:43PM
I think that considering the expense of the cores. And the possibility that they possible can't be replaced or at least risk contaminating the source. Other research groups may depend on the availability of the ice cores. When funds are tight and global warming skeptics are abound it's necessary to be able to prove the case. And if one fails a lot of lives may be destroyed indirectly.
Do you believe it's a SCADA system that has been used for this freezer?
Still if the monitoring system depends on a database. That adds something that can go wrong and which isn't strictly needed. The second weak point is the need to interact with a complex campus wide network. Control rooms are also abound with complexity. On top of all this there's no little redundant 8-bit MCU doing any just-in-case-fubar alert. On a systematic level there seems to been a blindspot for the case where if the temperature is too high. Turning on the heat pump will actually increase the temperature. That is tricky, but should be something professional freeze room designers and users should take into account in my opinion.
Still curious about this incident. There's many missing pieces of information and there' s probably something to be learned.
I think a found a suitable word for this incident inspired by the original article.. Incorepetence ;-)
(Score: 2) by kaszz on Tuesday April 11 2017, @11:30PM (1 child)
I checked University of Alberta [ualberta.ca] own news ticker. Nothing..
Embarrassing? ;)
(Score: 1) by Scruffy Beard 2 on Wednesday April 12 2017, @12:49AM
Looks like the story fell off that page already (published April 6th, most recent story is April 7th).
(Score: 2) by ilsa on Monday April 17 2017, @03:27PM
Even then equipment that cools things, generate their own heat. The monitoring equipment... basically all electronics and mechanical devices, generate heat. eg: Switch an AC unit from A/C to De-humidifier if you can. The temperature of the air it blows out is surprisingly warm.
If something happened to break the actual cooling (coolant leak, compressor failure, etc) the rest of the equipment will still run. And if the environment is closed (as it would be if it was a refrigerated area), then that heat can build up surprisingly high and surprisingly quickly.
(Score: 0) by Anonymous Coward on Tuesday April 11 2017, @06:25AM (5 children)
Maybe global warming is fake and they know trump, unlike obama is going to grill them on it. So instead of admitting that they made convenient "adjustments" to the data, they melt the data.
(Score: 2, Insightful) by aristarchus on Tuesday April 11 2017, @06:29AM (4 children)
Yep, that is what they want us to think! It's just like that increase in violet stroms, tornados and such, when it is only that now we have "sensors" that can sense much more activity, and we know damn well that there used to be at least as much activity in the past, because we do not have any data on how much activity there was in the past. This is Jeff Foxworthy territory! You might be a redneck, if you are not smarter than a Fifth Grader!
(Score: -1, Troll) by Anonymous Coward on Tuesday April 11 2017, @01:48PM (1 child)
Why do you insist on inserting insults, faggotry, or both, into every post you make?
(Score: -1, Flamebait) by Anonymous Coward on Tuesday April 11 2017, @06:01PM
He's Greek, where do you expect him to insert his faggotry?
(Score: 0) by Anonymous Coward on Tuesday April 11 2017, @03:57PM (1 child)
It's just like that increase in violet stroms
violet stroms are much worse than blue stroms
(Score: 3, Funny) by aristarchus on Tuesday April 11 2017, @08:24PM
violet stroms are much worse than blue stroms
Just wait until the Global Warming gets really severe, and we start having Ultra-violet stroms!!!
(Score: 2) by Zyx Abacab on Tuesday April 11 2017, @08:22AM
It was like a changing room in a swimming pool.
While atmospheric CO2 levels have been rising non-stop, at least we know that Cl levels have been going down....
(Score: 2) by zocalo on Tuesday April 11 2017, @08:45AM (6 children)
To add insult to injury, according to the UAlberta article, this is apparently a modern, custom-designed, $4.6m (Canadian, I assume) facility that was only opened earlier this year, after several months of testing and commissioning, with the cores being moved from the Geological Survey of Canada’s Ice Core Research Laboratory in Ottawa mid-January. Serious embarrasment all round, I'm thinking...
UNIX? They're not even circumcised! Savages!
(Score: 4, Insightful) by PiMuNu on Tuesday April 11 2017, @09:07AM
> that was only opened earlier this year
> It was like a changing room in a swimming pool.
I guess they are finding out about the bathtub curve
(Score: 2) by Azuma Hazuki on Tuesday April 11 2017, @11:21AM (1 child)
That is bizarre. I mean "cold" is Canada's number one natural resource, jeez.
I am "that girl" your mother warned you about...
(Score: 2) by takyon on Tuesday April 11 2017, @08:01PM
Not for long. Pretty soon it will be corn.
[SIG] 10/28/2017: Soylent Upgrade v14 [soylentnews.org]
(Score: 2) by Scruffy Beard 2 on Tuesday April 11 2017, @05:30PM (2 children)
Sounds like they tested the freezer during the coldest months of the year instead of the warmest months of the year. Isn't that Bass-ackwards?
(Score: 2) by kaszz on Wednesday April 12 2017, @01:26AM (1 child)
If it fails when it's really hot outside you got trouble. If it fails when it's cold there's some time before it starts to be critical. Thus test when it's cold outside.
(Score: 2) by Scruffy Beard 2 on Wednesday April 12 2017, @02:48AM
They tested with water buckets they did not particularly care about.
(Score: 2) by Dunbal on Tuesday April 11 2017, @10:00AM (2 children)
Definitive proof that warming is man made...
(Score: 0) by Anonymous Coward on Tuesday April 11 2017, @01:47PM (1 child)
I've experienced more woman-made warming than man-made . . . .
(Score: 2) by Dunbal on Tuesday April 11 2017, @10:01PM
You've obviously never run into those women who have permanent blocks of ice for feet and insist on cuddling and sticking them against you during the middle of the night...
(Score: 0) by Anonymous Coward on Tuesday April 11 2017, @02:39PM (3 children)
it was aliens. not fucking trump. or fake news. or anti global warming cabals. aliens. repeat after me. ALIENS.
(Score: 2) by AthanasiusKircher on Tuesday April 11 2017, @02:54PM
it was aliens. not fucking trump. or fake news. or anti global warming cabals. aliens. repeat after me. ALIENS.
I know all you Trump supporters want to blame EVERYTHING on "hot-blooded" Mexicans migrating illegally across borders, but this goes too far....
(Score: 0) by Anonymous Coward on Tuesday April 11 2017, @03:15PM
It was someone making a drink that needed some ice.
(Score: 0) by Anonymous Coward on Tuesday April 11 2017, @03:28PM
You assume those are mutually exclusive.
(Score: 2) by wonkey_monkey on Tuesday April 11 2017, @06:47PM
They could always refreeze 'em, but they just won't taste the same.
systemd is Roko's Basilisk
(Score: 2) by kaszz on Wednesday April 12 2017, @02:47AM (3 children)
[Scruffy Beard 2]
Temperature 2017-04-02 varied between 0.8 - 9.9 ⁰C [weather.gc.ca]
Elevation is 671 meters. Position 53.524444, -113.524444 [wmflabs.org]
Wind speed 1.7 - 7.0 m/s.
[VLM]
Refrigerant oils optimistically are refined not to burn well or easily but here they are in a multi-KW pump generating heat and wooshing around making mist so yeah it happens but not a common problem.
[zocalo]
Freezer failure results in damage to ice core collection [ualberta.ca]:
Monitoring system failed as a result of a database corruption. Temperature went from -37 to +40 ⁰C.
Modern, custom-designed, $4.6 million (assumed CAD) facility that was only opened earlier this year, after several months of testing and commissioning.
Affecting 12.8% of the collection.
1400 metres of ice in total.
South Academic Building, contains two freezers, a storage unit chilled to -37 ⁰C and an adjacent working unit cooled to -25 ⁰C.
The ice core samples arrived at University of Alberta north campus on 2017-01-15.
Late in the afternoon of Sunday, 2017-04-02, Protective Services and Edmonton Fire Service responded to a high-heat alarm in CICA. Facilities and operations found the temperature in the storage freezer had reached +40 ⁰C, resulting in the damage to the ice core samples.
All affected ice core samples were immediately moved into the working freezer, which was functioning properly and where the majority of the collection was being stored.
Refrigeration chillers shut down due to “high head pressure” conditions. Essentially, the chillers were not able to reject their heat through the condenser water system—heat instead of cold circulated through the freezer.
Compounding matters, the system monitoring the freezer temperatures failed due to a database corruption. The freezer’s computer system was actually sending out alarm signals that the temperature was rising, but those signals never made it to the university’s service provider or the on-campus control centre.
In the short term, refrigeration technicians are monitoring the freezers through twice-daily checks, Sharman said. The computer database corruption was resolved by adding a second monitoring controller, which is now issuing real-time messaging updates every eight hours.
Over the next few weeks, a second, independent path of alarming will be installed. The refrigeration system will also be modified to improve performance during failures, such as shutting down the evaporators in the event of a condenser failure.
University of Alberta’s Canadian Ice Core Archive [ualberta.ca]
Canadian Ice Core Archive at the University of Alberta's Faculty of Science [youtube.com] (video)
[Scruffy Beard 2]
src [soylentnews.org]
The Freezer was brand new: having undergone about 6 months of testing prior to loading the ice cores.
[kaszz]
The article author Colette Derworiz wrote yesterday [sciencemag.org]:
There was a temperature alarm, but it failed too. A computer system on the freezer was sending out alarm signals but they never made to the university's control centre. Investigation is ongoing. Several contingencies now...
Other stuff found on internet related to the matter:
Technical Design Requirements for Alberta Infrastructure Facilities [alberta.ca]
Unique Canadian ice core collection suffers catastrophic meltdown [wattsupwiththat.com]
## Summary ##
A new freezer facility in the South Academic Building at a cost of $4.6 million (assumed CAD) and 6 months of testing. Containing two freezers, a working unit at -25 ⁰C and a adjacent storage unit at -37 ⁰C. The ice core samples arrived at University of Alberta north campus on 2017-01-15.
Late afternoon on 2017-04-02 the outdoor temperature varied between 0.8 - 9.9 ⁰C with a wind speed 1.7 - 7.0 m/s. Thus the condensers should have no problem to dump the heat. But the refrigeration chillers shut down due to “high head pressure” conditions. Essentially, the chillers were not able to reject their heat through the condenser water system—heat instead of cold circulated through the freezer.
The temperature monitoring system for the freezer failed due to a database corruption. The freezer’s computer system was actually sending out alarm signals that the temperature was rising, but those signals never made it to the university’s service provider or the on-campus control center.
Protective Services and Edmonton Fire Service got a high-heat alarm in CICA that they responded to. Facilities and operations found the temperature in the storage freezer had reached +40 ⁰C.
Measures that will be carried out in the short term is that refrigeration technicians will be monitoring the freezers through two checks per day. The computer database corruption was resolved by adding a second monitoring controller, which is now issuing real-time messaging updates every eight hours.
Over the next few weeks, a second, a independent path of alarming will be installed. The refrigeration system will also be modified to improve performance during failures, such as shutting down the evaporators in the event of a condenser failure.
## Lessons ##
* Redundancy.
* Keep things simple.
* Fail gracefully.
* Have automated verification that sensors actually have a connection.
* Independent systems and communication paths.
* Monitor that equipment behave as expected. Watch out for sensor data that move in the wrong direction.
* Don't assume turning on the cooling equipment will result in a lower temperature. Presumed action and result.
* Verify with actual humans regularly.
* Test that systems work. And simulate faults.
Any comments?
(Score: 2) by Scruffy Beard 2 on Wednesday April 12 2017, @02:53AM (2 children)
The new cooler at work warns that the refrigerant is an explosion hazard. I can only presume it is butane or something.
The refrigerant catching fire would explain the rise in temperature with the fire alarm going off.
(Score: 2) by kaszz on Wednesday April 12 2017, @03:30AM
I think the heat rise above the outdoor temperature is simpler than that. The heat pump simply converted all electrical energy into heat inside the freezer room which is well insulated. Hence the +40 ⁰C temperature.
Of course without better sources it's hard to know for sure.
(Score: 0) by Anonymous Coward on Thursday April 13 2017, @08:33PM
Technically all refrigerants in use are explosion hazard even if the refrigerant itself is non-flammable,
since it is mixed with an oil aerosol of whichever type oil is miscible with that refrigerant.