https://philosophicaleconomics.wordpress.com/2014/06/12/critique/

As it turns out, there is an easy way to test whether or not the chart and the model are curve-fits: just expand the time horizon. If a valuation metric can predict returns on a 10 year horizon, it should be able to predict returns on, say, a 30 year horizon. A 30 year horizon, after all, is just three 10 year horizons in series–back to back to back. Indeed, each data point on a 30 year horizon provides a *broader* sampling of history and therefore achieves a greater dilution of the outliers that drive errors. A 30 year horizon should therefore produce a *tighter* correlation than the correlation produced by a 10 year horizon.

The following chart shows the model’s predicted and actual returns using the Shiller CAPE on a 30 year prediction horizon rather than a 10 year prediction horizon.

As you can see, the chart devolves into a mess. The correlation falls from an attractive**0.813** to an abysmal **0.222**–the exact opposite of what *should* happen, given that the outliers driving the errors are being diluted to a greater extent on the 30 year horizon. Granted, the peak deviation between predicted and actual is only around 4%–but that’s 4% per year over **30** years, a truly massive deviation, worth roughly **225%** in additional total return.

The following chart plots the error terms on the 30 year horizon:

Crucially, the errors no longer offset each other. That’s why the fit breaks down.

Now, as a forecasting horizon, the choice of 30 years is just as arbitrary as the choice of 10 years. What we need to do is calculate the correlations across *all* reasonable horizons, and disclose them in full. To that end, the following table shows the correlations for 30 different time horizons, starting at 7 years and going out to 36 years. To confirm a similar breakdown, the table includes the performance of the model’s predictions using Market Cap to GDP and the Q-Ratio as valuation inputs.

At around 20 years, the correlations start to break down. By 30 years, no correlation is left. What we have, then, is clear evidence of curve-fitting. There is a coincidental pattern in the data from 1935 to 2004 that the model latches onto. At time horizons between roughly 10 years and 20 years, valuation and growth tend to overshoot their assumed means in equal and opposite directions. The associated errors cancel, and an attractive fit is generated. When different time horizons are used, such as 25 years or 30 years or 35 years, the proportionate overshooting stops occurring, the quirk of cancellation is lost, and the correlation unravels.