Who has the best weather forecast?

clouds

Hmm... is this partly sunny or partly cloudy?

By James Cronen

Weather forecasts are everywhere these days: TV, radio, the internet -- even those electronic billboards along I-90 now include forecasts. But how do you know that the forecast you're getting is any good?

There's only one way to find out who really gives the best forecast: put them to a head-to-head test.

The method

Each evening, I recorded the weather forecasts from five different local sources: CBS 6, News 10, NewsChannel 13, Fox 23 News, and Capital News 9. I also included The Weather Channel.

I tracked the forecast posted on each source's website (not what was broadcasted on the evening newscast). I copied down the forecasted high temperature, overnight low temperature, and conditions for as far in the future as the station would predict (six or seven days for most stations, though the Weather Channel predicts nine days ahead). I did this for the month spanning April 8 to May 8.

Using historical weather data from the Albany Airport as the benchmark, I calculated how far off each forecast was from what the actual conditions were. The smaller the difference, the better. (Here's more detail about the how the scoring worked.)

The results

So, who has the most accurate forecast in the Capital Region? Well, it depends.

Forecast conditions (sunny, cloudy, rain)

Source+1 day+2 days+3 days+4 days+5 days+6 days+7 days+8 days+9 days
Capital News 90.9261.1110.9260.8521.1481.259
Weather Channel1.0330.9001.0001.0671.3331.2330.9331.3671.300
NewsChannel 131.0971.0000.6770.7420.9031.4521.276
News 101.0320.9351.1611.1940.9681.5161.613
CBS 61.0671.0000.8330.8671.0001.4331.310
Fox 231.1720.9310.6900.8281.1031.1431.130

If you want the best next-day forecast, Capital News 9 is it. After two days, the Weather Channel's is best. But, really, there's not a clear winner for these short term forecasts because the all the sources were clustered together.

At three days and beyond, however, the clear winner is NewsChannel 13. Bob Kovachick and team led the other forecasters in their three- to five-day forecasts. A very honorable mention goes to Fox 23, who was a very close second (and whose six- and seven-day forecasts were stronger than WNYT's).

Surprisingly, the Weather Channel's seventh-day forecast is as good as many stations' two-day forecasts! However, given their trend it appears this just may have been dumb luck.

Forecasted daily high temperature

Source+1 day+2 days+3 days+4 days+5 days+6 days+7 days+8 days+9 days
Capital News 92.1852.2223.0743.8525.8896.074
Weather Channel2.3331.8672.5673.6674.9005.3006.2006.9676.600
NewsChannel 132.3232.7424.0655.3875.3556.2907.759
News 102.3233.0003.1293.6134.8066.0007.226
CBS 62.2002.6003.6334.0005.1005.1676.483
Fox 232.3103.0003.9314.5864.4835.1076.174

This is a tough horserace to call. All the stations started out at about the same place, but the Weather Channel dominated the 2-3 day range. Channel 10 barely squeaked by the Weather Channel for forecasts four days out, but both stepped back as Fox 23 took the crown for the five to seven day range.

Forecasted nightly low temperature

Source+1 day+2 days+3 days+4 days+5 days+6 days+7 days+8 days+9 days
Capital News 93.4814.1484.3704.7045.7786.444
Weather Channel3.4673.8674.3674.2004.8006.1005.5335.6005.833
NewsChannel 133.7745.1296.0975.5486.0326.793
News 103.2903.7104.4523.9354.8716.226
CBS 67.8677.4337.2336.5006.8337.036
Fox 237.2417.3107.3455.9666.0347.2868.304

News 10 and the Weather Channel split the honors as the most accurate forecasters of the overnight low temperature. For the short-term, News 10 wins. Steve Caporizzo's team had the best results one, two, and four days in advance (and were only 0.08 degrees away from winning the three day forecast too). After a half week, the Weather Channel runs away with the victory. Is it fair to give the long-range crown to the Weather Channel, since it is the only station that bothers to forecast more than one week in the future? Consider that TWC's forecasts eight and nine days in advance were actually better than all of the other stations' forecasts six days in the future.

A few interesting trends

+ Almost every station was more accurate predicting the weather conditions two days in the future than they were guessing at the next day.

+ It seems there's an art to properly forecasting the overnight low. One day in advance, four stations were correct to within four degrees, while both Fox 23 and CBS 6 were more than seven degrees off.

+ Every station except for the Weather Channel reported overnight lows that were (on average) actually colder than what actually happened. This may not mean anything, as the local stations may be supplying the forecast using data from their studios and not from Albany Airport. Or, decades of experience in Albany winters may have taught them to expect the worst.

+ When it comes to predicting extreme events, no one wins. We had a hot-spell at the end of April that caught all the forecasters by surprise. On April 25, we hit an actual high temperature of 89°. Six days before that, no station predicted a temperature higher than 76°, and one even predicted a high of 57° and rain.

The bottom line

So, who is the grand champion of Albany weather forecasts? No one particular group of forecasters, unfortunately.

If conditions are what you're looking for, have a look at NewsChannel 13 in the short term (3 days) and the Weather Channel in the longer term.

For high temperatures, the place to look is the Weather Channel in the short term, and Fox 23 farther in the future.

And for overnight lows, your best bet is News 10 for the first four days, and the Weather Channel thereafter.

For more analysis of the data, please visit this post on my website at JamesCronen.com.

More details about the scoring

Each station's meteorological team was evaluated on how accurately it predicted the daily high temperature, overnight low temperature, and the weather conditions (sunny, cloudy, etc.). For temperatures, the station received a point for each Fahrenheit degree the team's forecast differed from the actual high/low temperature. For conditions, I created a six-point scale that ranked the "pleasantness" of the weather:

ConditionsScore
Sunny6
Partly cloudy5
Mostly cloudy4
Cloudy3
Rain or snow2
Thunderstorms1

I found the difference between the station's predicted conditions and the actual conditions. So if, say, Fox 23 predicted partly cloudy conditions (5 on the chart) and it rained (2), they would receive three points for that particular forecast. (If you couldn't tell by now, points are bad.)

All of these scores were put into an enormous spreadsheet.

Comments

Wow... that post is nearly as long as those morning forecasts on WAMC!

Would be interesting to see how the National Weather Service stacked up.

Great work James! But which station forecasts the most dramatic ICY SNOWY APOCALYPSE forecasts?

@amymengel: I'm saving that for next winter. :-)

@Rob ...which I noticed was missing entirely, though I suppose since Michael Landon retired the forecast is done by Paul Caiano so it would be the same as for WNYT.

Personally, I pay little attention to the weather (always tune out Paul Caiano during the 7:50 local news break on WAMC) and just look out the window to see what it is like!

As for apocalyptic forecasts no need to wait for winter...thunderstorm season is fast approaching and local forecasters love to give us "team coverage" on them as well.

In '92 I worked as an intern for the Atmospheric Prediction Branch of the USAF and quickly discovered that I was as accurate as all of the professional meteorologists on staff...

Great report! I think I'll just roll a d6 and take my chances...

@Bob F: Hate to say this because I work in TV, but I predict (ha!) audiences will move away from long detailed forecasts in favor of thumbnail weather like they get on the web.

I'd prefer it it tv meteorologists stopped editorializing about the weather, especially their bias toward sunny days as "good weather" and rain as "bad weather." Aside from being idiotic, there's nothing "good" about many days without rain. You'd think they did the weather forecast solely for people planning picnics and days at the beach as opposed to gardeners.

In my experience, the only forecast that is accurate is Accuweather.com - and I rely on it to make decisions such as packing for three days of camping, picnics outdoors and hikes - so far, I've not ever been surprised. I'd rather look at some maps, and decide for myself what to expect. But then again, with anything as changeable as the forecast around here, its sometimes wrong there, too.

I'm tired of weather forcasts as entertainment, with a time slot that has to be filled. I don't really care about light showers in East Hell Nebraska, when there is vastly more important news that could be expanded upon in that half hour.

Last week, a forcaster said "big changes in the weather are coming, but I don't want to let the cat out of the bag just yet".

Who are you? Angela Lansbury? Just tell me damned weather!

I always found term "First Warning" to be just a tad dramatic as well.

WARNING!!!

It's partly cloudy and 72 degrees.

RUN FOR YOUR LIVES!!!

They're all charlatans. ZERO ACCOUNTABILITY WHATSOEVER. Yet, Bob Kovachick drives a Mercedes Benz S-Class. I picked the wrong profession...

I usually do the opposite of what they say and am right as many times as they are.

Just go to the source minus the weather drama - noaa.gov. Hell, get one of those NOAA radios and get into the robot voice 24/7! Just don't set the weather alarm. That damn thing goes off every time a thunder storm approaches...talk about drama, that alarm makes me think the house is burning down and dogs all over the neighborhood howl in unison!

Tommy, i think the forecaster thought there might be a light rain, and, well, we all know how cats don't like getting wet, so he probably didn't want to let the cat out of the bag.
maybe??

http://www.weather.gov - best weather ever. Nuts to this TV and Radio thing.

how did the digital billboard on 787 rate?

Say Something!

We'd really like you to take part in the conversation here at All Over Albany. But we do have a few rules here. Don't worry, they're easy. The first: be kind. The second: treat everyone else with the same respect you'd like to see in return. Cool? Great, post away. Comments are moderated so it might take a little while for your comment to show up. Thanks for being patient.

The Scoop

Ever wish you had a smart, savvy friend with the inside line on what's happening around the Capital Region? You know, the kind of stuff that makes your life just a little bit better? Yeah, we do, too. That's why we created All Over Albany. Find out more.

Recently on All Over Albany

Palace Premier Movie Series 2014-2015

The schedule for the new season of The Palace's Premier Movie Series is out. It includes 18 dates, with films that range from classic classics... (more)

Memories, of laser cats and laser dogs

Remember Schenectady High School senior Draven Rodriguez and the laser cat yearbook photo? Here's the compromise photo planned for the yearbook: Rodriguez and Mr. Bigglesworth... (more)

TOP2014: RD1: Mama Mia's v 5th and 50

Tickets are still available for the Tournament of Pizza final! Taste the final pizzas, have some beer, and vote for your favorite! And so it... (more)

Drawing: A bag of items from the Honest Weight Local Harvest Festival

This Sunday is the annual Honest Weight Local Harvest Festival in Washington Park. Dozens of local producers and food vendors will be set up in... (more)

The Sea The Sea

Something to listen to today: "Love We Are We Love" by The Sea The Sea. Singer/songwriter duo Chuck e. Costa and Mira Stanley are... (more)

Recent Comments

They've had them in Arizona for years. When I lived there, I hated the idea and hated it even more when I was caught, twice. There's something endlessly creepy about being mailed a photo-ticket of yourself driving. BUT... It stopped me from not only speeding, but from trying to slide through those yellow-ish red lights. ...

Drawing: A bag of items from the Honest Weight Local Harvest Festival

...has 119 comments, most recently from Cliff

Where to rent a passenger van?

...has 6 comments, most recently from Nina Lee

Milk bottle rustling

...has 20 comments, most recently from Nancy

The week ahead

...has 1 comment, most recently from Brooke

Memories, of laser cats and laser dogs

...has 3 comments, most recently from Susan Bardack