All Over Albany

Who has the best weather forecast?

Hmm... is this partly sunny or partly cloudy?

Weather forecasts are everywhere these days: TV, radio, the internet -- even those electronic billboards along I-90 now include forecasts. But how do you know that the forecast you're getting is any good?

There's only one way to find out who really gives the best forecast: put them to a head-to-head test.

The method

Each evening, I recorded the weather forecasts from five different local sources: CBS 6, News 10, NewsChannel 13, Fox 23 News, and Capital News 9. I also included The Weather Channel.

I tracked the forecast posted on each source's website (not what was broadcasted on the evening newscast). I copied down the forecasted high temperature, overnight low temperature, and conditions for as far in the future as the station would predict (six or seven days for most stations, though the Weather Channel predicts nine days ahead). I did this for the month spanning April 8 to May 8.

Using historical weather data from the Albany Airport as the benchmark, I calculated how far off each forecast was from what the actual conditions were. The smaller the difference, the better. (Here's more detail about the how the scoring worked.)

The results

So, who has the most accurate forecast in the Capital Region? Well, it depends.

Forecast conditions (sunny, cloudy, rain)

If you want the best next-day forecast, Capital News 9 is it. After two days, the Weather Channel's is best. But, really, there's not a clear winner for these short term forecasts because the all the sources were clustered together.

At three days and beyond, however, the clear winner is NewsChannel 13. Bob Kovachick and team led the other forecasters in their three- to five-day forecasts. A very honorable mention goes to Fox 23, who was a very close second (and whose six- and seven-day forecasts were stronger than WNYT's).

Surprisingly, the Weather Channel's seventh-day forecast is as good as many stations' two-day forecasts! However, given their trend it appears this just may have been dumb luck.

Forecasted daily high temperature

This is a tough horserace to call. All the stations started out at about the same place, but the Weather Channel dominated the 2-3 day range. Channel 10 barely squeaked by the Weather Channel for forecasts four days out, but both stepped back as Fox 23 took the crown for the five to seven day range.

Forecasted nightly low temperature

News 10 and the Weather Channel split the honors as the most accurate forecasters of the overnight low temperature. For the short-term, News 10 wins. Steve Caporizzo's team had the best results one, two, and four days in advance (and were only 0.08 degrees away from winning the three day forecast too). After a half week, the Weather Channel runs away with the victory. Is it fair to give the long-range crown to the Weather Channel, since it is the only station that bothers to forecast more than one week in the future? Consider that TWC's forecasts eight and nine days in advance were actually better than all of the other stations' forecasts six days in the future.

A few interesting trends

+ Almost every station was more accurate predicting the weather conditions two days in the future than they were guessing at the next day.

+ It seems there's an art to properly forecasting the overnight low. One day in advance, four stations were correct to within four degrees, while both Fox 23 and CBS 6 were more than seven degrees off.

+ Every station except for the Weather Channel reported overnight lows that were (on average) actually colder than what actually happened. This may not mean anything, as the local stations may be supplying the forecast using data from their studios and not from Albany Airport. Or, decades of experience in Albany winters may have taught them to expect the worst.

+ When it comes to predicting extreme events, no one wins. We had a hot-spell at the end of April that caught all the forecasters by surprise. On April 25, we hit an actual high temperature of 89°. Six days before that, no station predicted a temperature higher than 76°, and one even predicted a high of 57° and rain.

The bottom line

So, who is the grand champion of Albany weather forecasts? No one particular group of forecasters, unfortunately.

If conditions are what you're looking for, have a look at NewsChannel 13 in the short term (3 days) and the Weather Channel in the longer term.

For high temperatures, the place to look is the Weather Channel in the short term, and Fox 23 farther in the future.

And for overnight lows, your best bet is News 10 for the first four days, and the Weather Channel thereafter.

For more analysis of the data, please visit this post on my website at JamesCronen.com.

More details about the scoring

Each station's meteorological team was evaluated on how accurately it predicted the daily high temperature, overnight low temperature, and the weather conditions (sunny, cloudy, etc.). For temperatures, the station received a point for each Fahrenheit degree the team's forecast differed from the actual high/low temperature. For conditions, I created a six-point scale that ranked the "pleasantness" of the weather:

I found the difference between the station's predicted conditions and the actual conditions. So if, say, Fox 23 predicted partly cloudy conditions (5 on the chart) and it rained (2), they would receive three points for that particular forecast. (If you couldn't tell by now, points are bad.)

All of these scores were put into an enormous spreadsheet.