Under exceptional circumstances, WANTED Technologies will withhold its forecast of US Total Nonfarm Employment for January 2010, as reported by the Bureau of Labor Statistics.
Every year, the BLS revises the entire history of US employment in what it calls "Annual Benchmark Revisions". These revisions create a break in the data series: forecasts of month-over-month changes aren't particularly useful when there isn't a 'next month' in the series, or when the data series is only a month old. This is what happens when the BLS revises its US employment time series – the old series ended, and a new, restated one begins.
Additionally, annual revisions are released at the same time as the monthly Employment Situation Summary report which contains the data we're forecasting. In other words, not only would we be forecasting a number for a series that's no longer being used, we'd be forecasting the future values of a dataset we don't have access to.
Additionally, we are expecting substantial revisions to 2009 employment numbers because of the volatility in the US economy over the past 16 months.
In combination, these three factors, have prompted us to withhold our monthly forecast:
- The additional forecast standard error introduced by the annual Benchmarking process is 56,000 workers per month (estimated over the past 8 years). Revisions are greater during economically volatile periods: for the previous recession (2002) the benchmarking related forecast standard error was 107,000 workers per month.
- The annual benchmarking revision process creates a dataset which we don't have access to but whose future value we must predict. It would be possible to forecast employment changes if we had access to the revised series before the release of the Employment Situation report; unfortunately revisions are released concurrently with the Employment Situation Summary.
- Employment gains are "within two standard errors" of our forecast. When changes are around zero, there's a qualitatively different interpretation of changes in employment. For example, we wouldn't interpret two competing forecasts of -50,000 and +50,000 the same as we would two competing forecasts of +450,000 and +550,000 – even though both sets of forecasts are 100,000 workers apart. When exiting a recession, 'which side of zero' you are on is more important than your 'number', creating an asymmetric penalty response function to a forecast.
The BLS will release the January Employment Situation on February 5th, 2010, at 8:30am ET.
Several internal BLS data elements are revisited in detail during the Annual Benchmark Revisions:
- The "benchmark" employment level is revised, both nonfarm and at the industry-level. The benchmark employment level is changed based on unemployment insurance paperwork filed by businesses, which it takes the BLS about 10 months to sort through. For example, when reviewing its 2006 data, the BLS figured that it undercounted US employment levels by 800,000 workers (every month). The whole series gets shifted upwards by 800,000 workers in that instance.
- The infamous "Birth/Death" data is revised. In short, you can measure changes in employment by looking at how firms in a sample are adding/shedding workers. But as the BLS mentions: "There is an unavoidable lag between an establishment opening for business and its appearing in the sample frame and being available for sampling. Because new firm births generate a portion of employment growth each month, non-sampling methods must be used to estimate this growth." In other words, if there are a lot of "business births", you will have missed quite a few new workers if you only used your original sample of businesses, so you need to circle back, find those new businesses, and add those new workers into your employment counts.
- The seasonal adjustment parameters are also revised. Seasonal coefficients should be stable through time. However, seasonal coefficients are themselves estimated through models whose input data is seasonally unadjusted data. Grossly put, seasonal adjustment answers the question "what are the seasonal correction factors that will make the overall data process as smooth as possible?". If a better correction factor is discovered this year, all historical data gets re-calculated with this improved seasonal correction factor.
Let's review in more detail exactly how the annual benchmarking process causes forecasting issues.
1. Additional Forecast Standard Error:
Over the past 8 years, we can look at how the annual benchmark revisions have affected estimates of month-over-month changes in US employment. What's important to note is that the error introduced here has nothing to do with changes in the overall level of US employment (the benchmark revision). WANTED does not forecast US employment levels (like ADP). WANTED explicitly forecast changes to US employment levels. In other words, there's a difference between:
EMPt and ΔEMPt = EMPt – EMPt-1
The benchmark revisions affect EMPt, not ΔEMPt. How do benchmark revisions affect "month-over-month" changes if the overall series is going up or down some "fixed" amount? The details are in the BLS' explanation: "Employment estimates for months between the most recent benchmark and the previous year's benchmark are adjusting using a wedge-back procedure. The wedge procedure assumes that total estimation error accumulated at a steady rate since the last benchmark". In other words, the BLS was off by 800,000 workers in March 2006, so they pro-rated that difference over time to hit the correct employment number at the benchmark month. That pro-rating means that estimates of over-the-month employment changes are also affected; the series isn't "bumped by a fixed amount for all months".
Mining the last 8 years of revisions, we've found that the average monthly forecast standard error introduced by the annual benchmark revision process is 57,000 workers (we treat the BLS's "first result" as a forecast of its own final revised numbers). IF we add that to the normal forecast standard error of 78,000 (normal in the sense that that is the standard error the BLS has "with itself" between the first preliminary release and the third and final release), the forecast standard error caused by the benchmarking process is 135,000.
Remember from the basics statistics course that "two standard errors" produces a 95% confidence interval – the 95% confidence interval around the forecast is now +/- 270,000. This is quite large.
What's more, during the last recession (2002), the benchmark revision caused an additional forecast standard error of 106,000 workers per month, almost double the usual. Given that we spent 2009 in a recession, we're expecting that the BLS will revise its data more heavily now than it would if the economy had been more stable.
2. Short-Term and Long-Term Dynamic Processes:
It's hard to take issue with BLS revisions, especially when they're trying to provide the most accurate employment data possible.
We can take issue with the timing of the data release, however. The historical revisions are released at the same time as the Employment Situation Summary. We attempt to forecast the numbers in that report, but the revisions and the data we're supposed to forecast will appear at the same time.
Had the revised data been made available, say, two weeks before the release of the Employment Situation, forecasters could re-calibrate their models. Indeed, forecasting models rely on the short-term dynamics of the BLS series. Our model has an autoregressive AR(p) component – the forecasting equation uses recent over-the-month employment changes. Relying heavily on the last few months instead of year-long trends can be risky. However, econometric analysis shows that using the last three month's of data has the greatest predictive power. If using 14 months of data were better, we'd use that and the revisions wouldn't be as big of an issue. But short-term movements are critical for forecasting. The hiring cycle – about 8 to 12 weeks from post to hire – also means that this variable's estimated effect has three months impact. The fact that the new series is revealed at the same time as the number we're supposed to predict is difficult: we don't have the new series to forecast from, and our forecast would be based on a data series which is no longer applicable.
To complicate matters, our models look at the "historical interaction" of changes in UI Claims and Hiring Demand with overall employment. Changes to data can affect how our statistical models estimate the effect of changes in Hiring Demand on US employment. In other words, new data can produce new regression coefficients which in turn change the forecast.
3. Qualitative Interpretation of Employment Gains:
We're at the end of a recessionary period (hopefully), and market watchers are looking for anything that could signal the end of economic contraction.
Our estimate, with the added forecast standard error generated by the Benchmark Revision Process, has a 95% confidence interval which includes zero. If 2002 is any guide, the normal forecast error associated with this month's forecast is +/- 215,000 (plus or minus two standard errors).
When we're at the point where the economy stops shrinking and starts growing, there's a qualitatively different interpretation of the change in employment. As soon as those changes move from negative to positive, expectations on the state of the economy will shift. For example:
- 50,000 jobs = recession continues
+ 50,000 jobs = recession has ended.
To reiterate on the example above, employment gains of 450,000 or 550,000 workers both represent "robust growth". But both sets of forecasts listed here are 100,000 workers apart (+450/+550; -50/+50).
Our forecast, taking into account the large forecast error caused by the Benchmark Revision, straddles both sides of "zero".
So, given that the benchmark revision process for 2009 is likely to produce series changes that are larger than usual and that we don't have access to the series we're supposed to forecast, we think it is prudent to withhold our forecast this month.
All Entries