15 March 2017

Properly Communicating Uncertainty Is Just as Important as Making A Good Forecast

Posted by Dan Satterfield

Deep snow near Chalfont, PA, Tuesday evening. My photo.

There has been a bit of a firestorm in the last 24 hours over an AP report that the NWS decided to stick with its snowfall forecast even after last minute model guidance showed lesser amounts were likely. There’s a story in the Washington Post and Seth Borenstein at the AP broke the story.

Here are some thoughts on this based on 37 years of forecasting, and I will say right off, I think this controversy may be a misunderstanding of the forecast process. Yes, the computer models showed less snow as the event was about to begin, and yes the NWS held onto their forecast. In this case, it was a mistake, but most of the time, it would probably be the right thing to do. What it was not, is an attempt to hide the truth from the public.

HERE IS WHY

  1. Ensemble forecast giving odds for 1,4,8, and 12 inches of snow. This is from the NOAA WRF model. Image from Penn. State. It was actually quite good, especially well inland.

    Numerical models are just guidance and each run will be different. Ensembles of several runs are usually more reliable, and I relied on the WRF models ensemble which actually did pretty well. It showed that the odds of over 8 inches of snow along the coast from Philadelphia to Boston was not that high (around 50/50), but it indicated at least 8 inches was quite likely west of I-95. Some individual runs were higher, but when I see consistency on a model, run after run, I tend to give it much more credence. The NWS forecasted up to 3-5 inches of snow for the Dover, Delaware area, and based on the ensembles, I thought this was not likely and said so. They had less than an inch.

    That said, did anyone look at the NWS snow forecasts that showed the real range?? They have these on their website, and it’s a new way of forecasting snow events. They have an “Expect at least this much” map, and then a “most likely” forecast and then, a “Could be this much if it really comes together” map.  Did anyone look at these? Did local broadcast meteorologists talk about these? I know many who did, but I am not sure this was communicated well by the NWS or broadcast meteorologists. These “low-end” maps showed almost no snow south of Wilmington.

     

  2. This is most important: There is an old rule against changing a big forecast based on one or two model runs. If you do, you end up chasing the model and confusing people, while increasing your odds of a busted forecast. The numerical models did indeed start indicating that the warmer ocean air would reach farther inland from the coast Monday night and Tuesday, and this would reduce totals, but the models tend to do rather badly with the placement of the rain/sleet/snow line. In this case, the milder air really did move well inland, but if the low had tracked 50 miles eastward, the heavy snow would have been 50 miles closer to (and right on) the coast.The problem here is not that the NWS decided to stick with their snow totals. The problem is that the forecast was communicated poorly. I posted the map below on social media, and I overestimated the snow near the coast as well, but note what I put around the NYC area. VARIABLE SNOWFALL.


    3. A busted snow forecast is where you forecast an inch and get 7 inches. In this case, people would be expecting little impact on roads/schools and there would be a major disruption. Forecasting 10-16″ and getting 5-8″ is a bad forecast, but trust me, the impacts are not much different. I rode out this storm in Doylestown, PA. where we had 6-7 inches and the winds were howling Tuesday and all of Tuesday night.
     

    There seems to be this thought out there that the NWS knew the storm was not going to produce as much snow near the coast but held onto their forecast because they thought that people wouldn’t pay as  much attention to what was going to be a bad storm. I think it’s more a case of meteorologists following the more reliable forecast rules, and if there is some blame here, it would be that NOAA and broadcast mets should have done more to remind people that the snow forecast on the coast was very tricky, but there would still be major impacts with snow and high winds. Snowfall is the most difficult forecast we make, and this was a decision based on tried and true forecast methods and not an attempt “to hide the truth”. Just another example of how the age of the conspiracy theory is impacting our daily lives.