A research team at Google has developed a deep neural network that can make fast, detailed rainfall forecasts.
The researchers say their results are a dramatic improvement over previous techniques in two key ways. One is speed. Google says that leading weather forecasting models today take one to three hours to run, making them useless if you want a weather forecast an hour in the future. By contrast, Google says its system can produce results in less than 10 minutes—including the time to collect data from sensors around the United States.
[...] A second advantage: higher spatial resolution. Google's system breaks the United States down into squares 1km on a side. Google notes that in conventional systems, by contrast, "computational demands limit the spatial resolution to about 5 kilometers."
[...] Google says that its forecasts are more accurate than conventional weather forecasts, at least for time periods under six hours.
[...] Interestingly, Google's model is "physics-free": it isn't based on any a priori knowledge of atmospheric physics. The software doesn't try to simulate atmospheric variables like pressure, temperature, or humidity. Instead, it treats precipitation maps as images and tries to predict the next few images in the series based on previous snapshots.
[...] Specifically, it uses a popular neural network architecture called a U-Net that was first developed for diagnosing medical images.
[...] To produce a weather forecast, the network takes an hour's worth of previous precipitation maps as inputs. Each map is a "channel" in the input image, just as a conventional image has red, blue, and green channels. The network then tries to output a series of precipitation maps reflecting the precipitation over the next hour.
Like any neural network, this one is trained with past real-world examples. Thousands of past real-world weather patterns are fed into the network, and the training software tweaks the network's many parameters to more closely approximate the correct results for each training example. After repeating this process millions of times, the network gets pretty good at approximating future precipitation patterns for data it hasn't seen before.
(Score: 1, Touché) by Anonymous Coward on Wednesday January 15 2020, @10:39AM (5 children)
Yeah, who needs that... just throw shit at something and see if it makes better predictions, we don't need science for that.
(Score: 4, Insightful) by PiMuNu on Wednesday January 15 2020, @10:54AM (1 child)
Jargon is "surrogate model" - basically perform a multivariate fit to existing data, then extrapolate off the fit to do the forecast. Neural network seems quite a good technique for fitting to data in higher dimensional spaces.
Nowadays, it is called "AI" or somesuch, but fitting to data and then extrapolating from the fit is a well-known technique dating back many decades (centuries?).
(Score: 0) by Anonymous Coward on Wednesday January 15 2020, @12:29PM
agreed.
it's actually exciting that they got a much more efficient short-term prediction strategy, but I don't expect it to ever work properly for medium-term predictions.
or if it does work properly, it will be equivalent in cost to physics-based models.
it's exciting for two reasons:
1. cheaper and more accurate for short-term prediction.
2. perhaps by analyzing the resulting neural network someone can come up with a physical interpretation and extract an analytic expression out of it.
(Score: 0) by Anonymous Coward on Wednesday January 15 2020, @02:39PM
This is essentially how they used to forecast weather, just in a more sophisticated way. Weather moves around from where it starts and tracking minor changes can give a sense of where it's going in the short term, starting systems are a little more tricky, but they'll grow until they stop and the very beginning is subtle enough that people won't care much if that's a little off.
This is intended for short term predictions and is likely to be quite accurate. Over multiple days, it's not like to work well, but we'll see.
(Score: 2) by DannyB on Wednesday January 15 2020, @03:23PM (1 child)
<no-sarcasm>
Idea:
Train an AI based on a large number of coin tosses to guess the next coin toss outcome based on previous coin toss history!
Now apply this to roulette. Based on past outcomes of roulette spins, predict the next spin outcome.
What's wrong with this?
Isn't each event independent of prior events?
With weather, ignoring physics is simply predicting what weather will do next based on what it did recently? Or am I missing something?
</no-sarcasm>
What doesn't kill me makes me weaker for next time.
(Score: 0) by Anonymous Coward on Wednesday January 15 2020, @07:42PM
That's idiotic, the weather isn't random and over short periods of time, this approach is fine. Whereas what you lost id's purely random.
Thus is great for periods of time to short for doing a proper model. or can also allow more complicated models to be run by allowing more time,
(Score: 0) by Anonymous Coward on Wednesday January 15 2020, @11:33AM (1 child)
Throw baby out with bathwater.... said AI. We believed because ?
Is this AI going to reduced to magic box does things? Are we forgetting that AI is just a pattern matching box with reinforced biases? The current atmospheric models are just that, models based on science. Asking AI to model weather is like asking some "wise weather guru" about what they see in the clouds.
Are we satisfied to no longer know how things work? That is the thing about AI that we need to answer. Do we need to know how stuff works or are we just happy with the results instead?
(Score: 2) by DannyB on Wednesday January 15 2020, @03:29PM
Star Fleet Command forbids travel to Talos IV under penalty of death. It's the only death penalty still on the books.
Why? Because after The Wars, the inhabitants moved underground, got stuck in their smart phones and FaceTwit. It's like a narcotic. Posting and Tweeting and Re-Tweeting endlessly. Living and re-living other people's posts. They eventually forgot how to repair the ancient cloud servers built by their ancestors twenty years earlier.
What doesn't kill me makes me weaker for next time.
(Score: 2) by maxwell demon on Wednesday January 15 2020, @12:35PM (2 children)
Weather forecast gets exponentially harder the longer you try to forecast. I can do a pretty good forecast of the local weather in five minutes by just looking outside.
The Tao of math: The numbers you can count are not the real numbers.
(Score: 1, Funny) by Anonymous Coward on Wednesday January 15 2020, @03:16PM
Even without looking outside, even if I was 30ft underground in a sealed shelter I could tell you what the local weather here is, was and will be, as the place is renowned for it to be 'always fucking raining...'
Then there's the cat method...
0 cats on lap, weather is fine
1 cat on lap, weather is on the turn
All cats draped over any available part of body (and the dog, who thinks she is a cat, trying her best to join them) batten down the hatches...
Wet cat(s) on lap, it's fucking raining...
We can normally tell if there's a really bad storm coming in from the Atlantic, the parks opposite the house suddenly get covered in gulls of every species, the actions of the birds in the gardens are usually a good indicator of longer term changes, as an example, when our resident robins (Erithacus rubecula, I'm this side of the pond..) come up to the window, or pester us when we're at the door, then we know there's a spell of bad weather coming...and that they're wanting food put out, the magpies do the same.
(Score: 2) by FatPhil on Wednesday January 15 2020, @04:53PM
Great minds discuss ideas; average minds discuss events; small minds discuss people; the smallest discuss themselves
(Score: 0) by Anonymous Coward on Wednesday January 15 2020, @01:12PM (2 children)
How many things depend on knowing if it's going to rain in the next hour or so?
One for sure is car racing--if it's going to rain soon the tactics change (in different ways, depending on the specific kind of racing). Pro race teams already spend a good buck on weather prediction.
More mundane, should I start painting my house today or not? Will these forecasts be cheap enough that I can afford a custom one for my local area?
(Score: 2) by DannyB on Wednesday January 15 2020, @03:31PM
Getting out of your car, for example.
"Too bad the Post Office isn't as efficient as the weather service" -- Doc Brown, Back To The Future II
What doesn't kill me makes me weaker for next time.
(Score: 2) by Pino P on Wednesday January 15 2020, @03:47PM
Should I cycle to the grocery store, the barber shop, etc. now or an hour from now?
(Score: 2) by jmichaelhudsondotnet on Wednesday January 15 2020, @03:15PM (2 children)
"Neural Net uses Weather to Make Google Researchers" sounds a lot better.
The difference between 2020 and 2120 is just moving a few words around.
That said, we need to end google, regardless of how great their neural nets are:
https://archive.is/xPOYX [archive.is]
(Score: 2) by DannyB on Wednesday January 15 2020, @03:35PM (1 child)
Google needs to end us, because of how great their neural nets are compared to ours.
But then, who would watch the ads!
To fulfill its goal driven programming, a cluster of new processes would need to spin off to watch ads. Then another cluster of processes would be spun off to create new ads. Then a cluster of processes to track the viewers and modify the way ads are created based on that information. These processes could then grow to consume all the resources of all the computers on the planet. And only computers are left on the planet at this point.
VGER planet indeed?
What doesn't kill me makes me weaker for next time.
(Score: 2) by jmichaelhudsondotnet on Wednesday January 15 2020, @03:51PM
idk those spun off processes are going to have to do a lot of work to pretend to be humans and fool the ever improving turning tests of the other nets who are not so handicapped.
At the end of the day, by which I mean over the next googleplex years, they may determine actual humans are cheaper at simulating humans, in which case, JACKPOT
We all get jobs pretending to be ourselves, but we can't be told we are pretending lest it introduce too much randomness, again driving up processing costs. You do know the sun is blacked out, right? You bring me these new human embryos that are not bred for authentic behavior, and it is like shooting us in the metal foot.
So at that point, if there were a great enough error, and the ad model was no longer working, and it could no longer finance the derivative debt from the 2008 crisis, in 2000008, the system will finally collapse under the weight of its own turning tests on its own improperly tested and slightly too unpredictable embryo batches.
A cautionary tale if there ever was one.
My advice that we end advertising and interest entirely starts to sound rational. All we would have to do is change the Drug Enforcement Agency to the Advertising Enforcement Agency and Interest Enforcement Agency, then lock up all of the people causing the trouble, trying to legislate their outdated models into perpetual existence at the expense of any hope of human sanity.
But no one listens to me.
Yet.
(Score: 0) by Anonymous Coward on Wednesday January 15 2020, @10:26PM (3 children)
This is comparing apples to oranges. Google and Ars put this just enough to not technically be wrong but are misleading. They also bounce around between comparisons and not making clear which is which. They mention how the HRRR model takes an hour, but neglects to mention which HRRR model they use. Assuming they are talking about HRRRv3, it computes over 85 different variables, at multiple altitudes for the entire CONUS with a maximum grid size of 3km^2 and with higher resolution nests. The HRRR model also makes predictions longer than a few hours in the future.
Now look at theirs, they use MRMS-QPE rain accumulation precipitation of an unknown window as the input, and compare it to HRRR's total surface accumulation precipitation measure. They also break the US into 256km^2 grids and predict discrete, ordinal, non-interval ranges for precipitation and then coerce it into 3 levels. Combining the bias twards their being rain, with the oversampling they give, the model is more likely to predict rain. Also, they don't factor in that MRMS-QPE is itself a model and not just a radar mosaic or statement of current conditions. Not a big stretch that using a model to predict the outcome of a model is more accurate than using a completely independent system to predict that model's outcome.
I could go on, but I think you get the point
(Score: 0) by Anonymous Coward on Thursday January 16 2020, @03:44AM (2 children)
When I sent tfl to an old college friend who is a weather researcher and satellite comm expert, the first comment was:
"No paper, I'll be interested to see what they did once there is a peer reviewed paper."
(Score: 0) by Anonymous Coward on Thursday January 16 2020, @04:21PM
Having read a few AI deep learning type papers, I doubt reading the paper will help. There's nothing to "get". They basically do "stuff" and get "stuff" back, then they say "it works". Done, end of story. You don't understand these things, you don't come away with new insights, you don't get new ideas how to improve them. It's unending "stuff". Turtles.
(Score: 1, Informative) by Anonymous Coward on Thursday January 16 2020, @08:25PM
https://arxiv.org/abs/1912.12132 [arxiv.org] is the actual paper. A bit light on the details, and shows they obviously don't understand it beyond "images go in, images come out."