Follow this blog:
RSS

Why algorithms need humans to predict the weather

By | September 11, 2012, 3:10 AM PDT

History is rich with intellectuals who have revered theories of determinism; ideas that suggest if we could only know every facet of a situation, every molecule of the landscape, we could predict and even shape future political, economic, and cultural outcomes.

But when it comes to the weather, forecasters long ago gave up any hope of cataloging all of the variables that could impact rainfall in Seattle, or the arrival of a cold front in New York. At least that’s what Nate Silver reports in his new book, The Signal and the Noise: Why So Many Predictions Fail — but Some Don’t, an excerpt of which was adapted for a recent article in The New York Times Magazine.

If you go by Silver’s account, weather forecasting is something of a dark art. Despite all of the measurements, modeling, and statistical analyses, the weather business relies as much on human insight as it does on computer programming. This is best evidenced by the National Weather Service’s own historical records. According to the agency’s data, a combination of human and computing power creates the most accurate weather forecasts. People improve accuracy levels for precipitation and temperature forecasts by about 25 percent and 10 percent respectively over forecasts done by computers alone.

In other words, the algorithms haven’t bested us yet.

Even as modern futurists envision a time when computers will out-think people, it turns out that there may always be a role for the human mind. In weather forecasting, even the most sophisticated computer modeling systems disagree with each other all the time. It’s up to the people studying those models to illuminate nuance and apply additional context; whether that means knowing how best to weight the variables that determine where a storm is headed, or that morning fog in the northeast tends to dissipate quickly when the wind is blowing in a particular direction.

As powerful as computers are, they can’t “see” everything. And they’re not necessarily as good as humans at knowing when and where to look for more information. Our obsession with big data, and the quantification of industries - finance, advertising, space - can sometimes blind us to the fact that human perception and insight, fuzzy and imprecise though they may be, are still critical to society’s progress. Maybe in the future they’ll laugh at our fixation with numbers. Or maybe they’ll simply recognize better than we seem to that numbers are only part of the equation.

Image credit: Sam Solomon on Flickr

Related on SmartPlanet:

Start your week smarter with our weekly e-mail newsletter. It's your cheat sheet for good ideas. Get it.

Mari Silbey

About Mari Silbey

Mari Silbey is a contributing editor for SmartPlanet.

Mari Silbey

Mari Silbey

Contributing Editor

Mari Silbey is an independent tech writer based in Washington, D.C. With a background in cable and telecom, she's a contributor to several trade publications, and part of the GigaOM analyst network. She also writes for the long-running digital media blog Zatz Not Funny, and has written for both corporate and association clients focused on broadband networks, mobile apps, and video delivery. She's a graduate of Duke University.

Follow her on Twitter.

Mari Silbey

Mari Silbey

Mari Silbey does not hold any investments in the technology companies she covers.

She writes for SmartPlanet and is not an employee of CBS.

If you liked this, don't miss...
23
Comments

Join the conversation!

Follow via:
RSS
0 Votes
+ -
A good example --
..is the Accu-Weather app available on the Kindle Fire. The near-term forecast mostly agrees with what A/W is saying, but it has a 15-day outlook that I'd guess is the raw output of some model, and can have wild and improbable figures. It'll show a day in that period with, say, a 20-degree departure from the previous day, and with the next forecast cycle, that variance may go away or swing the other way. Or it'll show 5 days of cold rain coming up, then a day later all traces of that are gone.

Here in the Northeast, even carefully constructed forecasts have trouble with 'back door' cold fronts and wind flow off the ocean -- the models usually have the marine air moving out at some point, but in reality it frequently takes much longer to break the pattern. Possibly because the models don't have enough data points over the ocean, or the algorithms underestimate the marine influence.
Posted by ProfQuill
Updated - 11th Sep
+1 Vote
+ -
Weather predictions
An experienced and knowledgeable weather man, who knows the area in which he lives, and has observed the weather patterns for years, and who has a good working knowledge of weather, can almost always beat a computer model.

A computer can only report on what it knows, and it can't know everything that a 50-something weatherman has seen living in the same place all his live. Many computer models also seem to rely too heavily on history, i.e. this one will be the same as a previoius one.

Follow the iso bars... the weather does.
Posted by bb_apptix
Updated - 11th Sep
0 Votes
+ -
But...
...isn't "morning fog in the northeast tends to dissipate quickly when the wind is blowing in a particular direction" precisely the kind of data on which a computer excels??? I think you picked a pretty bad example to illustrate the point.

The only difference between a computer and a human brain considering variables to predict an outcome is that they've yet to make a machine with the same capacity and resolution as the human organ.

That the brain can consider, often intuitively, causes and effects beyond what the computer model considers is only a statement on the lack of complexity of the machine and/or its programming.

Give it time. At this point it's like expecting supersonic speeds from a Model T.
Posted by omb00900@...
11th Sep
0 Votes
+ -
Data points
omb- I considered your point about the dissipation of fog as a data point even as I wrote that sentence. However, it was an example that Nate Silver used, and it struck me that, as simple a data point as it might seem, that fact about wind could easily still be neglected among the billions of other pieces of information that a computer has to collect - not because a computer couldn't handle the volume, but because someone would have to tell it to consider that specific variable.

As to your point about a computer and a brain only differing in capacity and resolution, even if that is true, it doesn't mean we'll be able to replicate what the mind can do in silicon. Like the weather, there are so many variables, that the idea of uncovering them all is virtually unfathomable. I don't know that we have enough imagination to change that.
Posted by msilbey
11th Sep
+2 Votes
+ -
Man vs machine
I am a computer programmer, with 15 years of experience. Computers, nor machines, will be able to do certain things better than humans, not now, not never. Human intelligence, creativity, intuition, and other aspects of our thinking are amazing. Computers just crunch numbers, and find patterns. They can't create new knowledge based on previous knowledge, experience, intuition, and imagination. There are certain things that are not fully comprehended, and may never be, about the human mind. Horray for humans, we are still, and will always be better than machines, or any other animal!
Posted by ahpitre
11th Sep
0 Votes
+ -
Or maybe it's the program design
One thing I learned long ago as a programmer working with realtime data, and recovery of data, is that if I can arrive at a correct answer, I should be able to analyze how I did it and write my software in such a way as to do the same thing. In a sense, it's like fudging, but in truth you are simply replicating a human's perfectly acceptable methods of performing certain tasks.

What I find really annoying about the forecasts is that they publish longer and longer ranges, when there's no history of accuracy with short terms. I noticed the other day that Accuweather is now providing a 25 day forecast for temperature and precipitation. Of course, they can create models to do this, but if the models aren't even accurate for 5 days, why would they dare to make public their 25 day predictions?

The problem with weather prediction may be that the models take into account too many variables, or perhaps the models don't weight the variables properly. If that is the case, and if people are able to improve on the computer predictions, then that method should be applied to the software.
Posted by AlanLaRue
Updated - 11th Sep
0 Votes
+ -
Not so simple
As an embedded systems programmer of over 20 years experience, you can say "This is the approach I took. Therefore, I can model a computer program to do the same." Most of the time, you don't see all of the little things you did that made the approach work. But even if you managed it, you then move on to a different but similar problem, only to discover that the tiny difference meant that you used a very different approach.

And that's where humans excel. We adapt our approaches on the fly as necessary. None of us will live long enough to see a program on a conventional digital computer that is able to do that as effectively as any one of us.
Posted by mheartwood
11th Sep
0 Votes
+ -
Fuzzy Pattern Spotting
"Computers just crunch numbers, and find patterns" - but the pattern recognition abilities of the human brain far exceed any computer program we've yet developed. A human forecaster can look at a weather map and instantly see the patterns where a computer would have to do an enormous amount of number crunching to find them. The human has no problem relating these patterns to similar patterns seen before (though they're never identical), while this is a very complex problem for a computer.

The ability to identify patterns hidden in a complex of irrelevant data was a vital survival skill to our ancestors - spotting the lion that's hidden in the bushes could save your life. Millions of years of natural selection have refined this skill in us, little wonder it's taking some time to develop computers that can rival us.
Posted by Greenknight_z
12th Sep
0 Votes
+ -
Initial Conditions
Much of my college coursework and early energy work involved computer modeling.

Weather is an energy-balance type problem with a computer making an analogy to a cube determining energy (and humidity) flows in and out, but one must really be aware of initial conditions in order to get accurate results. Computers have become more accurate over the years by increasing the number of boxes they can calculate, and with improved measurement of initial conditions.

But regardless, if the model does not closely resemble reality, there are factors which are not modeled - or not properly modeled in the equations. We need human interpretation to understand how new factors will influence the energy flows in and out of the imaginary boxes the computer is analyzing.

These factors are always changing and new factors being understood, such as the ocean waves can create ozone, global warming increases evaporation, water vapor rising higher in the atmosphere is assist the chloride based chemistries from air conditioning processes more quickly destroy ozone. Without accurate determinations of the volume and sources of each pollutant, natural element and other factors, the more complex the models, the more difficult it is to make the models match reality.

Assuming we could identify each element in its appropriate quantities, and speed up the calculations - such as with analog computing, there is no doubt models could become more accurate with little human tweaking.
Posted by Carlos Zavala
11th Sep
+2 Votes
+ -
And what about global warming models?
This article is all about how after decades we still can't model the weather accurately with computer models. And nobody sees similar problems with the computer models we use for global warming?

While weather forecasting and climate modeling aren't really the same thing, they both are about modeling energy transfer in fluids. The equations involved are massive systems of partial differential equations solved numerically on a computer. You need huge amounts of accurate data on a very small granular basis to feed these models, and you need to know the initial conditions very precisely.

With climate much of the data from before the 1980s is only surface temperatures taken on land at random sites mostly near cities. We had no idea of the temperatures of the ocean at various depths all over the world. We had no idea of the temperatures of the air at various altitudes. We had very little idea of flows in the atmosphere and especially the oceans. Yet all of these are necessary for accurate climate modeling. Instead, we are forced to use primitive proxies such as tree rings, ice cores, and ocean sediment cores to get temperatures. No weather model would ever use as data, for example, a diary entry that says "It's hot today", but that's almost what we're reduced to out of necessity in climate modeling.

The result is that climate models can't be nearly as accurate as weather models are. Scientists make guesses about data and and simplify the equations to sets that are still very complex. What happens is that they come up with a model that accurately predicts the last 150 years or so where we do have some data, and they compare models. We are supposed to get all excited when they say their models do this, but if a model doesn't it just get tweaked or the data massaged until it does. And then we are supposed to believe we can use this "foot long ruler" of the last 150 years to accurately measure the hundreds of "miles" of the earth's climate history.

This is not to say that climate scientists aren't very smart or aren't making an honest attempt to understand and model the climate. But climate science is still very young. It's like asking the physicists of the 1920s and 1930s to use their new science of quantum mechanics to come up with the Standard Model of particles when they only knew about and could predict a few particles.

Normally watching climate science mature would an exciting thing for interested laypersons. As examples, think about what we've learned about anthropology, paleontology, geology, physics, and medicine in the last 50 years. But the problem with climate science is that we are asked to globally gamble tens of trillions of dollars on this new science. Special interests are more than ready to use climate science to promote their own agenda, such as we must go back to living as our ancestors did (the same thing they were saying back in the '60s before climate became an issue). And scientists are only human. Some use it simply to advance their own careers, and many more know that their careers are dead if they question it.
Posted by zackers
Updated - 12th Sep
-1 Votes
+ -
Your money or your life.
Life is real but money is just an idea, which is more important?
Posted by shaunehunter
12th Sep
+1 Vote
+ -
Really?
So if your boss tells you he's not paying you your wages because it's just an idea, you'd be happy?
Posted by zackers
Updated - 13th Sep
-1 Votes
+ -
You fundamentally misunderstand how climate models work.
I knew someone would bring this up.

Climate models are physical models, not statistical models. They don't depend on data to make predictions. Scientists compare model output to data to test accuracy of the model but the tweaks they make are related to the modeling of physical processes and are not statistical in nature.

They don't depend on initial conditions to make predictions. You could set the initial conditions anywhere and the model would eventually converge on the same answer in the end. The closer your initial conditions are to reality the faster it converges but in the end it will.

Climate models can easily be more accurate than weather models. As an analogy consider flipping a coin. I can't tell you accurately whether the next flip will come up heads or tails however I can tell you the results of flipping a coin 1000 times with pretty good accuracy. Weather is what you get with each coin flip, climate is the results of 1000 coin flips.

Here are a couple of FAQ's on climate models for your edification:

http://www.realclimate.org/index.php/archives/2008/11/faq-on-climate-models/
http://www.realclimate.org/index.php/archives/2009/01/faq-on-climate-models-part-ii/
Posted by riverat1
12th Sep
+1 Vote
+ -
Wrong
I was not talking about statistical models, I WAS talking about physical models. These use systems of partial differential equations (PDEs) to model the physics of the climate, just as weather models use PDEs to model the physics of weather. Anybody who knows about PDEs (I have a BS in math and computer science) knows that initial conditions matter a great deal . It's also true that in solving these equations on a computer the accuracy and granularity of the data matters a great deal. It's why you need the largest supercomputers to model either the climate or the weather, because the amount of data that has to be crunched is so huge.

Climate models aren't as detailed as weather models because we just don't know nearly enough about the climate as we do the weather. They make large simplifications about wind, ocean currents, and how they interact, for example. Nobody is even sure how clouds affect the climate. In contrast, meteorologists get a new run of their models against the changing weather about every week. We only have the one run of the climate, and our accurate data only goes back 150 years or so (which really isn't long enough to qualify as a climate run).

To say that climate models "don't depend on data to make predictions" is wrong. EVERY science theory depends on data. It's not valid until the theory matches the data (it's the core of the scientific method). Why else would climate scientists spend so much time on tree rings and ice cores? Why would we be sending up satellites and establishing thousands of ground stations to record data?

As for initial conditions, climate scientists often talk about "tipping points". If the Arctic ice cap disappears, the earth may suddenly absorb too much solar energy and heat up enormously. Or the permafrost may melt and suddenly release a lot of methane. So if your initial conditions are on one side of a tipping point or another, it makes a big difference. Initial conditions do converge, but not always to the same place. Data accuracy matters a lot.
Posted by zackers
Updated - 13th Sep
0 Votes
+ -
Maybe not so wrong
You didn't read those FAQ's I cited did you? Here is a quote from one of them:

If left to run on their own, the models will oscillate around a long-term mean that is the same regardless of what the initial conditions were. Given different drivers, volcanoes or CO2 say, they will warm or cool as a function of the basic physics of aerosols or the greenhouse effect.

The basic physics that drive the climate and are incorporated into climate models is not chaotic in nature. For a given set of inputs (simulated insolation, simulated changes in greenhouse gases, simulated volcanic eruptions, etc.) they will produce a given output. Using PDEs doesn't change that.

Climate models and weather models have a lot in common since they model basically the same thing, just on different time scales and different physical scales. Weather models are detailed to catch the differences in local effects and to pinpoint local weather more accurately. But climate models incorporate things that weather models ignore because they have miniscule effects of the short time scales in which weather models operate. BTW, if weather forecasters want to make accurate forecasts I imagine they're running their models more often than once a week. Daily would make more sense to me.

When I said climate models don't depend on data I meant that is not an input to the climate model run. Of course the output of climate models is compared to real world data. That's how they get tested. So far they're better than any other method we have. Here is a comparison of model output to real world data up to the end of 2011:

http://www.realclimate.org/index.php/archives/2012/02/2011-updates-to-model-data-comparisons/

There are links to earlier comparisons in the article.

I agree with you on the tipping points issue. Things like the Arctic sea ice disappearing or permafrost melting leading to massive increases in methane may cause drastic changes but they are not included in climate models because we don't know enough about them to reliably model them yet. I note that either of those two tipping points would lead to even worse effects so climate models if anything are conservative in their projections.

BTW, on the Arctic sea ice issue, the head of the polar ocean physics group a the University of Cambridge said that the added heat from sea ice loss is equivalent to the warming from about 20 years of CO2 emissions.
Posted by riverat1
Updated - 16th Sep
0 Votes
+ -
Riverat1 - So what I stated before was not holey incorrect?
My title "Initial Conditions" is incorrect, but the body of the statement was not incorrect in that if there is a mistake in the physical conditions are not properly modeled, the convergence would not represent the physical reality as accurately as a model properly accounting for all the physical influences. And that is the value of science - to find out what influences we have not yet accounted for.

My understanding - or misunderstanding - is that initial conditions contribute to the outcome. If I remember my physical modeling classes correctly, non-dimensional analysis essentially runs the model from 0 to 1 with the amplitude plugged in later.

Your statement on "climate" vs "weather" seems to have more of a statistical flavor than the physical convergence model.
Posted by Carlos Zavala
Updated - 12th Sep
-1 Votes
+ -
I was responding to zackers.
Carlos,

My response was directed at zackers, not you.

I agree with you that in weather modeling initial conditions are very important because they model short term behavior so there's not enough time for them to clear any bias from initial conditions.

However in climate modeling the runs typically simulate at least 30 years and often much more than that. So there is "time" for the bias of initial conditions to get cleared out of the results. Still, you get a quicker resolution to what you're seeking if the initial conditions are realistic.

I don't disagree with you that if a physical model is incomplete the results may not be accurate but in any physical system there are the things that have a major effect on the system on down to the things that just tweak it around the edges. As long at you get the bigger things right the model is still likely to produce useful results.

You could certainly come up with at physics based model of a coin flip, presumably even one that would work for a biased coin. The analogy I presented just shows that despite the seemingly random results of individual events a system is not necessarily random when you look at many individual results compiled together.
Posted by riverat1
Updated - 16th Sep
+1 Vote
+ -
Initial conditions do not average out in complex systems
If you are familiar with chaos theory, you'll know that the initial conditions in complex systems such as climate or the weather matter a great deal. The so-called "butterfly effect" is an exaggeration, but the effect is very real and is part of the physical system itself. It's not something that gets introduced because the numerical computer model of the physical system is poor.

It seems intuitive that initial conditions "average out". In the simple systems most of us deal with, that's what happens. Flipping a coin is actually one such simple system. It has only two measurable results: heads or tails. Nobody cares about how many times it flipped in the air, how many times it bounced, or even if it landed on a slight angle on the carpet. So you can do statistics and predict how many heads or tails you'll get. If you also had to predict the numbers of flips and bounces and the angle at which it landed on the carpet, you'd find a statistical analysis would be much harder.

Climate or the weather are complex systems where such details matter. It's no good just to say it's getting hotter or it's getting colder. You have to say where it's getting hotter or cooler and by how much. You have to worry about feedback loops such as increased evaporation creating more clouds which might reflect sunlight back into space. Suddenly it's very complex.

Climate scientists can take a model, do a lot of simulated runs with it, and show that it converges to an average. But that's not the same thing as saying it accurately reflects nature. It doesn't justify using that model to predict the future under different conditions such as more CO2. We have only one actual climate history where we have accurate knowledge, namely the last 150 years. Coming up with a model that matches that one brief history doesn't prove that it works with the different climates of past eons. If the climate models were accurate descriptions of the physical system from first principles, you'd have a much better chance of that.
Posted by zackers
13th Sep
0 Votes
+ -
Chaos
Weather is chaotic, climate not so much. One way you could define climate would be to say climate describes the boundaries within which weather is chaotic. Climate is a function of energy balance within the physical characteristics of the Earth and the solar system. It is only chaotic to the extent those things are chaotic.
Posted by riverat1
16th Sep
0 Votes
+ -
The only person, or computer, I trust to predict the weather ...
... is Sally Solomon.
Posted by barts185
14th Sep
0 Votes
+ -
Bursa Escort
bursa escort
escort bursa
Posted by canlimacizle94
3rd Feb
Posted by canlimacizle94
3rd Feb
Join the conversation
Formatting +
BB Codes - Note: HTML is not supported in forums
  • [b] Bold [/b]
  • [i] Italic [/i]
  • [u] Underline [/u]
  • [s] Strikethrough [/s]
  • [q] "Quote" [/q]
  • [ol][*] 1. Ordered List [/ol]
  • [ul][*] · Unordered List [/ul]
  • [pre] Preformat [/pre]
  • [quote] "Blockquote" [/quote]

Join the SmartPlanet community and join the conversation! Signing up is fast and free. Don't wait -- we want to hear your opinion!