By Charles Thibault on December 7, 2009 in BLS Nonfarm Employment.

There were two important pieces of information in this month's BLS Employment Situation Summary. First, Nonfarm Employment dropped by only 11,000 – much better than Consensus Estimate that sat at -100,000 and our forecast of -155,000. Second, employment drops from the two previous months were revised upwards: 80,000 fewer people were out of work in September than originally reported, and 79,000 fewer people were out of work in October than originally reported. Together, this month's employment contraction of only 11,000 workers and those two positive revisions of about 80,000  put the US economy much closer to employment growth than what was thought just last week.

That's great news for the economy at large, but bad news for professional forecasters. BLS revisions have a direct impact on "what our model would have said given this new information". Our forecast for November, given the revised data in this month's report, would have been a drop of 79,000 nonfarm workers instead of a drop of 155,000.

Most forecasting models will include lagged values of the dependent variable, in addition to any proprietary or public information forecasters have access to. WANTED's forecasting model, for example, includes Nonfarm Employment, Unemployment Insurance Claims, and our Hiring Demand Indicators (number of new online job ads posted on the internet).

The table below shows the information available at the time of our forecast, the information available in the most recent Employment Situation Summary, and our forecasts based on those two different sets of data:

This "relative difference" in our forecast is almost exactly the same as the BLS's own revisions (76 vs. 79/80).

Not only do revisions to the BLS's own data series creates issues with forecasting models, but it also puts forecasters on an "uneven playing field" relative to the BLS: we don't get to revise our predictions… (and if we did, we'd simply re-adjust them to the most recently published count).

The BLS will revise a monthly employment count twice before it becomes "final". This, in fact, allows us to treat the BLS's "1st preliminary" count as a forecast of its own "3rd and final" employment count. (Additionally, the BLS revisits the entire historical time-series once a year, usually in February).

So, we level the playing field by calculating "the BLS's own forecasting error relative to itself" and comparing that to the average error of professionally developed forecasts. We present this data permanently our forecast page for our readers. The Root Mean Square Error (RMSE) is a standard measure of forecast error. We can also take the RMSE as a percent of the predicted variable in order to provide a "relative magnitude". From the table below, you can see that the BLS's "error relative to itself" is 23.9%, which compares favorably to WANTED's average error of 25.4%:

You can see that, for example, the Consensus Estimate performs quite well against the "1st preliminary" number, but it does not perform so well against the final employment counts. This shows how quickly forecasts can lose accuracy when the data source you're trying to predict keeps changing its mind…

(The RMSE% for the Consensus Estimate is higher than the RMSE% for the BLS even though the absolute error is smaller because the 3rd Final number is a more volatile series than the 1st Preliminary count).

Comments are closed.