Index Numbers: A Black Box Full of Mischief and Mystery

The possibilities for mischief and misdirection in the construction of a business climate index are almost unlimited. There are several steps to the creation of an index, and it is possible to rely on the best available research technique at each step of the way. But the indexes critiqued on this site tend to ignore research and simply choose measures on the basis of ideological bias, convenience, or popular mythology. Due to this, the results are meaningless at best and misleading at worst.

Deciding Which Factors to Include

The first step in creating an index is to decide which factors to include. Ideally, one would include all the most relevant variables and avoid irrelevant ones. For example, one could select measures based on a review of the best research on the factors that contribute to state economic growth. Alternatively, one could simply compile a list of all the policies that appeal to your particular ideology and assert that they are conducive to growth. The latter is certainly easier and probably better serves the interest of your organization.

A particularly misleading technique is to include, for example, measures of a state’s actual economic performance, such as new business start-ups or growth in per capita income, in an index that purports to measure the climate for growth. Including performance measures in an index and then arguing that the index predicts performance is circular reasoning.

Finding the Best Measure

The second step is to find the best empirical measure of each factor. Here laziness and expediency often rule the day in business climate rankings. A particular factor — wage rates, for example — may make sense, but a good measure of wage differences across states is challenging to construct. It is much easier just to take the readily available “average hourly wage” and be done with it, ignoring the fact that average wages are determined by the skill and education level of the state’s workforce, and its industrial and occupational mix.

Combining the Measures

The third step is to combine the measures into a single index number which entails explicit or implicit weighting. An index should base the weights on research into the relative importance of various factors, but this is not done by any of the four business climate indexes reviewed on this website. One sure sign that an index isn’t serious is if its components are all weighted the same. Every factor is not of equal importance in causing economic growth, so a failure to appropriately weight factors makes an index meaningless. All of the indexes reviewed here apply quite arbitrary weights and, in so doing, produce a number that is impossible to interpret.