Ferguson's Model and Its Implications

posted by Fnord666 on Monday May 11 2020, @08:06AM

from the garbage-in-garbage-out dept.

We have had several submissions in the past couple of days about the Ferguson code.

For those that don't know - the Ferguson code, also known as the Ferguson Model or the Imperial College Model, is the epidemiology prediction software and the underlying model upon which the UK government is basing all its decisions relating to combating CV-19.

It appears that there is some question about the accuracy of the model and the repeatability of its predictions.

Thanks to NPC-131072, FatPhil, and nutherguy for their submissions. Details begin below the fold.

Is the Chilling Truth That the Decision to Impose Lockdown Was Based on Crude Mathematical Guesswork

upstart writes in with an IRC submission for nutherguy:

Is the chilling truth that the decision to impose lockdown was based on crude mathematical guesswork? -- Sott.net:

Details of the model [Ferguson's] team built to predict the epidemic are emerging and they are not pretty. In the respective words of four experienced modellers, the code is "deeply riddled" with bugs, "a fairly arbitrary Heath Robinson machine", has "huge blocks of code - bad practice" and is "quite possibly the worst production code I have ever seen".
When ministers make statements about coronavirus policy they invariably say that they are "following the science". But cutting-edge science is messy and unclear, a contest of ideas arbitrated by facts, a process of conjecture and refutation. This is not new. Almost two centuries ago Thomas Huxley described the "great tragedy of science - the slaying of a beautiful hypothesis by an ugly fact."
In this case, that phrase "the science" effectively means the Imperial College model, forecasting potentially hundreds of thousands of deaths, on the output of which the Government instituted the lockdown in March. Sage's advice has a huge impact on the lives of millions. Yet the committee meets in private, publishes no minutes, and until it was put under pressure did not even release the names of its members. We were making decisions based on the output of a black box, and a locked one at that.
It has become commonplace among financial forecasters, the Treasury, climate scientists, and epidemiologists to cite the output of mathematical models as if it was "evidence". The proper use of models is to test theories of complex systems against facts. If instead we are going to use models for forecasting and policy, we must be able to check that they are accurate, particularly when they drive life and death decisions. This has not been the case with the Imperial College model.
At the time of the lockdown, the model had not been released to the scientific community. When Ferguson finally released his code last week, it was a reorganised program different from the version run on March 16.
[...] We now know that the model's software is a 13-year-old, 15,000-line program that simulates homes, offices, schools, people and movements. According to a team at Edinburgh University which ran the model, the same inputs give different outputs, and the program gives different results if it is run on different machines, and even if it is run on the same machine using different numbers of central-processing units.
Worse, the code does not allow for large variations among groups of people with respect to their susceptibility to the virus and their social connections. An infected nurse in a hospital is likely to transmit the virus to many more people than an asymptomatic child. Introducing such heterogeneity shows that the threshold to achieve herd immunity with modest social distancing is much lower than the 50-60 per cent implied by the Ferguson model. One experienced modeller tells us that "my own modelling suggests that somewhere between 10 per cent and 30 per cent would suffice, depending on what assumptions one makes."

Code Review of Ferguson's Model – Lockdown Sceptics

upstart writes in with an IRC submission for FatPhil:

Code Review of Ferguson's Model – Lockdown Sceptics:

by Sue Denim (not the author's real name)
Imperial finally released a derivative of Ferguson's code. I figured I'd do a review of it and send you some of the things I noticed. I don't know your background so apologies if some of this is pitched at the wrong level.
[...] It isn't the code Ferguson ran to produce his famous Report 9. What's been released on GitHub is a heavily modified derivative of it, after having been upgraded for over a month by a team from Microsoft and others. This revised codebase is split into multiple files for legibility and written in C++, whereas the original program was "a single 15,000 line file that had been worked on for a decade" (this is considered extremely poor practice). A request for the original code was made 8 days ago but ignored, and it will probably take some kind of legal compulsion to make them release it. Clearly, Imperial are too embarrassed by the state of it ever to release it of their own free will, which is unacceptable given that it was paid for by the taxpayer and belongs to them.
What it's doing is best described as "SimCity without the graphics". It attempts to simulate households, schools, offices, people and their movements, etc. I won't go further into the underlying assumptions, since that's well explored elsewhere.
Due to bugs, the code can produce very different results given identical inputs. They routinely act as if this is unimportant.
This problem makes the code unusable for scientific purposes, given that a key part of the scientific method is the ability to replicate results. Without replication, the findings might not be real at all – as the field of psychology has been finding out to its cost. Even if their original code was released, it's apparent that the same numbers as in Report 9 might not come out of it.

We Now Know Far More About COVID-19 - the Lockdown Should End

upstart writes in with an IRC submission for nutherguy:

We Now Know Far More About COVID-19 - The Lockdown Should End:

On March 23rd, when Boris Johnson declared a lockdown in the UK, it was a beyond surreal moment for me. With no debate, our freedoms, social life and jobs were gone.
The reasons given for the lockdown were to try and save lives, slow the spread of this virus and limit the impact on the NHS. It sounds good until you start to pose searching questions. Confining people to their homes and a complete loss of social life comes with its own set of serious problems. Focusing on Covid-19 means other people needing operations are postponed for months.
We had heard about other so-called Pandemics that had turned out to be nothing of the sort, Swine flu being one example. What was different about Covid-19? Johnson had seemed to be going the way of putting in some mitigation recommendations, like social distancing, hand washing and isolating of the elderly. Then he changed his mind.
The reason were the numbers of possible deaths that could occur if a full lockdown was not implemented. The numbers came from a Prof Neil Ferguson of Imperial College, London.
Ferguson had told the government that according to his computer model, over 500,000 people would die in the UK if they did nothing, 250,000 people would die if he continued with lesser mitigation in place, but allowing businesses to stay open as usual. With a full lockdown, deaths would be 20,000 or less, and the impact to the NHS would be kept to a minimum.
What immediately struck me was that Ferguson's computer model is just that, it's an estimate based on certain data. His projections could be totally wrong, we've all heard the expression, garbage in, garbage out. Why on earth would Johnson decide to implement such drastic measures based on a theoretical computer model?
It was also disturbing to find out that Ferguson has a lot of form for making highly exaggerated claims with his computer models.

Code Review of Ferguson's Model

NPC-131072 writes:

Wonder why you're in lockdown? Wonder no more, the code review is in:

Imperial finally released a derivative of Ferguson's code. I figured I'd do a review of it and send you some of the things I noticed. I don't know your background so apologies if some of this is pitched at the wrong level.
It isn't the code Ferguson ran to produce his famous Report 9. What's been released on GitHub is a heavily modified derivative of it, after having been upgraded for over a month by a team from Microsoft and others. This revised codebase is split into multiple files for legibility and written in C++, whereas the original program was "a single 15,000 line file that had been worked on for a decade" (this is considered extremely poor practice). A request for the original code was made 8 days ago but ignored, and it will probably take some kind of legal compulsion to make them release it. Clearly, Imperial are too embarrassed by the state of it ever to release it of their own free will, which is unacceptable given that it was paid for by the taxpayer and belongs to them.

I predict this story will be better commented than the original code.

Original Submission #1 Original Submission #2 Original Submission #3 Original Submission #4

SoylentNews

SoylentNews is people

Navigation

Sections

SoylentNews

Log In

Related Links