Who has the best weather forecast?

clouds

Hmm... is this partly sunny or partly cloudy?

By James Cronen

Weather forecasts are everywhere these days: TV, radio, the internet -- even those electronic billboards along I-90 now include forecasts. But how do you know that the forecast you're getting is any good?

There's only one way to find out who really gives the best forecast: put them to a head-to-head test.

The method

Each evening, I recorded the weather forecasts from five different local sources: CBS 6, News 10, NewsChannel 13, Fox 23 News, and Capital News 9. I also included The Weather Channel.

I tracked the forecast posted on each source's website (not what was broadcasted on the evening newscast). I copied down the forecasted high temperature, overnight low temperature, and conditions for as far in the future as the station would predict (six or seven days for most stations, though the Weather Channel predicts nine days ahead). I did this for the month spanning April 8 to May 8.

Using historical weather data from the Albany Airport as the benchmark, I calculated how far off each forecast was from what the actual conditions were. The smaller the difference, the better. (Here's more detail about the how the scoring worked.)

The results

So, who has the most accurate forecast in the Capital Region? Well, it depends.

Forecast conditions (sunny, cloudy, rain)

Source+1 day+2 days+3 days+4 days+5 days+6 days+7 days+8 days+9 days
Capital News 90.9261.1110.9260.8521.1481.259
Weather Channel1.0330.9001.0001.0671.3331.2330.9331.3671.300
NewsChannel 131.0971.0000.6770.7420.9031.4521.276
News 101.0320.9351.1611.1940.9681.5161.613
CBS 61.0671.0000.8330.8671.0001.4331.310
Fox 231.1720.9310.6900.8281.1031.1431.130

If you want the best next-day forecast, Capital News 9 is it. After two days, the Weather Channel's is best. But, really, there's not a clear winner for these short term forecasts because the all the sources were clustered together.

At three days and beyond, however, the clear winner is NewsChannel 13. Bob Kovachick and team led the other forecasters in their three- to five-day forecasts. A very honorable mention goes to Fox 23, who was a very close second (and whose six- and seven-day forecasts were stronger than WNYT's).

Surprisingly, the Weather Channel's seventh-day forecast is as good as many stations' two-day forecasts! However, given their trend it appears this just may have been dumb luck.

Forecasted daily high temperature

Source+1 day+2 days+3 days+4 days+5 days+6 days+7 days+8 days+9 days
Capital News 92.1852.2223.0743.8525.8896.074
Weather Channel2.3331.8672.5673.6674.9005.3006.2006.9676.600
NewsChannel 132.3232.7424.0655.3875.3556.2907.759
News 102.3233.0003.1293.6134.8066.0007.226
CBS 62.2002.6003.6334.0005.1005.1676.483
Fox 232.3103.0003.9314.5864.4835.1076.174

This is a tough horserace to call. All the stations started out at about the same place, but the Weather Channel dominated the 2-3 day range. Channel 10 barely squeaked by the Weather Channel for forecasts four days out, but both stepped back as Fox 23 took the crown for the five to seven day range.

Forecasted nightly low temperature

Source+1 day+2 days+3 days+4 days+5 days+6 days+7 days+8 days+9 days
Capital News 93.4814.1484.3704.7045.7786.444
Weather Channel3.4673.8674.3674.2004.8006.1005.5335.6005.833
NewsChannel 133.7745.1296.0975.5486.0326.793
News 103.2903.7104.4523.9354.8716.226
CBS 67.8677.4337.2336.5006.8337.036
Fox 237.2417.3107.3455.9666.0347.2868.304

News 10 and the Weather Channel split the honors as the most accurate forecasters of the overnight low temperature. For the short-term, News 10 wins. Steve Caporizzo's team had the best results one, two, and four days in advance (and were only 0.08 degrees away from winning the three day forecast too). After a half week, the Weather Channel runs away with the victory. Is it fair to give the long-range crown to the Weather Channel, since it is the only station that bothers to forecast more than one week in the future? Consider that TWC's forecasts eight and nine days in advance were actually better than all of the other stations' forecasts six days in the future.

A few interesting trends

+ Almost every station was more accurate predicting the weather conditions two days in the future than they were guessing at the next day.

+ It seems there's an art to properly forecasting the overnight low. One day in advance, four stations were correct to within four degrees, while both Fox 23 and CBS 6 were more than seven degrees off.

+ Every station except for the Weather Channel reported overnight lows that were (on average) actually colder than what actually happened. This may not mean anything, as the local stations may be supplying the forecast using data from their studios and not from Albany Airport. Or, decades of experience in Albany winters may have taught them to expect the worst.

+ When it comes to predicting extreme events, no one wins. We had a hot-spell at the end of April that caught all the forecasters by surprise. On April 25, we hit an actual high temperature of 89°. Six days before that, no station predicted a temperature higher than 76°, and one even predicted a high of 57° and rain.

The bottom line

So, who is the grand champion of Albany weather forecasts? No one particular group of forecasters, unfortunately.

If conditions are what you're looking for, have a look at NewsChannel 13 in the short term (3 days) and the Weather Channel in the longer term.

For high temperatures, the place to look is the Weather Channel in the short term, and Fox 23 farther in the future.

And for overnight lows, your best bet is News 10 for the first four days, and the Weather Channel thereafter.

For more analysis of the data, please visit this post on my website at JamesCronen.com.

More details about the scoring

Each station's meteorological team was evaluated on how accurately it predicted the daily high temperature, overnight low temperature, and the weather conditions (sunny, cloudy, etc.). For temperatures, the station received a point for each Fahrenheit degree the team's forecast differed from the actual high/low temperature. For conditions, I created a six-point scale that ranked the "pleasantness" of the weather:

ConditionsScore
Sunny6
Partly cloudy5
Mostly cloudy4
Cloudy3
Rain or snow2
Thunderstorms1

I found the difference between the station's predicted conditions and the actual conditions. So if, say, Fox 23 predicted partly cloudy conditions (5 on the chart) and it rained (2), they would receive three points for that particular forecast. (If you couldn't tell by now, points are bad.)

All of these scores were put into an enormous spreadsheet.

Comments

Wow... that post is nearly as long as those morning forecasts on WAMC!

Would be interesting to see how the National Weather Service stacked up.

Great work James! But which station forecasts the most dramatic ICY SNOWY APOCALYPSE forecasts?

@amymengel: I'm saving that for next winter. :-)

@Rob ...which I noticed was missing entirely, though I suppose since Michael Landon retired the forecast is done by Paul Caiano so it would be the same as for WNYT.

Personally, I pay little attention to the weather (always tune out Paul Caiano during the 7:50 local news break on WAMC) and just look out the window to see what it is like!

As for apocalyptic forecasts no need to wait for winter...thunderstorm season is fast approaching and local forecasters love to give us "team coverage" on them as well.

In '92 I worked as an intern for the Atmospheric Prediction Branch of the USAF and quickly discovered that I was as accurate as all of the professional meteorologists on staff...

Great report! I think I'll just roll a d6 and take my chances...

@Bob F: Hate to say this because I work in TV, but I predict (ha!) audiences will move away from long detailed forecasts in favor of thumbnail weather like they get on the web.

I'd prefer it it tv meteorologists stopped editorializing about the weather, especially their bias toward sunny days as "good weather" and rain as "bad weather." Aside from being idiotic, there's nothing "good" about many days without rain. You'd think they did the weather forecast solely for people planning picnics and days at the beach as opposed to gardeners.

In my experience, the only forecast that is accurate is Accuweather.com - and I rely on it to make decisions such as packing for three days of camping, picnics outdoors and hikes - so far, I've not ever been surprised. I'd rather look at some maps, and decide for myself what to expect. But then again, with anything as changeable as the forecast around here, its sometimes wrong there, too.

I'm tired of weather forcasts as entertainment, with a time slot that has to be filled. I don't really care about light showers in East Hell Nebraska, when there is vastly more important news that could be expanded upon in that half hour.

Last week, a forcaster said "big changes in the weather are coming, but I don't want to let the cat out of the bag just yet".

Who are you? Angela Lansbury? Just tell me damned weather!

I always found term "First Warning" to be just a tad dramatic as well.

WARNING!!!

It's partly cloudy and 72 degrees.

RUN FOR YOUR LIVES!!!

They're all charlatans. ZERO ACCOUNTABILITY WHATSOEVER. Yet, Bob Kovachick drives a Mercedes Benz S-Class. I picked the wrong profession...

I usually do the opposite of what they say and am right as many times as they are.

Just go to the source minus the weather drama - noaa.gov. Hell, get one of those NOAA radios and get into the robot voice 24/7! Just don't set the weather alarm. That damn thing goes off every time a thunder storm approaches...talk about drama, that alarm makes me think the house is burning down and dogs all over the neighborhood howl in unison!

Tommy, i think the forecaster thought there might be a light rain, and, well, we all know how cats don't like getting wet, so he probably didn't want to let the cat out of the bag.
maybe??

http://www.weather.gov - best weather ever. Nuts to this TV and Radio thing.

how did the digital billboard on 787 rate?

Say Something!

We'd really like you to take part in the conversation here at All Over Albany. But we do have a few rules here. Don't worry, they're easy. The first: be kind. The second: treat everyone else with the same respect you'd like to see in return. Cool? Great, post away. Comments are moderated so it might take a little while for your comment to show up. Thanks for being patient.

The Scoop

Ever wish you had a smart, savvy friend with the inside line on what's happening around the Capital Region? You know, the kind of stuff that makes your life just a little bit better? Yeah, we do, too. That's why we created All Over Albany. Find out more.

Recently on All Over Albany

10x10: Casinos, John Oliver, new spots

A while back we did a thing where wrote 10 reviews/comments in no more than 10 words each. People seemed to like it, and it... (more)

Holiday gifts: Deanna Fox

Gifts and giving are on most everyone's mind this month. So we thought we'd ask a few people to share some thoughts on presents, past... (more)

NY Thruway Guide

Might be worth a look: NY Thruway Guide, an iPhone app that's pretty much what it sounds like. It provides access to the lineup of... (more)

Stuff to do this weekend

There's lots going on the the Capital Region this weekend -- both holiday and non-holiday. So if you're looking to take a break from all... (more)

Morning Blend

The body of 5-year-old Kenneth White was found by a K-9 unit near the home where he had been staying in East Berne, according to... (more)

Recent Comments

... I tend to ask questions that make the person think about what they just said. I ask it sweetly and in a tone that notes confusion on my part. I have been called honey in the office and asked the person, " Can I ask what you mean when you call me honey? Because you don't call John honey." It calls out that he's treating you differently for being a woman. If he still doesn't get it, you can be more direct: "I appreciate that you respect my work and treat me equally, but I wouldn't want others to think otherwise based on how you address me."

Local food gifts

...has 10 comments, most recently from Carolyn

Capital Region high school graduation rates 2014

...has 2 comments, most recently from Greg

NY Thruway Guide

...has 1 comment, most recently from Rob

Good neighborhood holiday light displays?

...has 4 comments, most recently from MikeH

Where to get latkes?

...has 14 comments, most recently from Susan Anthony Brownell