Rankings Are Poor Predictors of Growth
The business climate rankings all claim to be measures of public policies that produce greater economic growth and prosperity, but the researchers who have put these predictions to the test have concluded that the indexes are poor predictors of state economic performance.
Four researchers at the University of Kansas and Iowa State University published a report in 2005 titled Business Climate Indexes: Which Work, Which Don’t, and What Can They Say about the Kansas Economy?1 They attempted to measure whether state business climate indexes can explain differences in economic growth between counties on either side of a state border. The authors conclude:
None of the business climate indexes can explain a large proportion of the variation in growth across counties. The best performing business climate indexes explained at most 5 percent of the variation in relative growth at the borders, suggesting that most of the variation in economic performance is due to factors not captured by state-level business climate measures. This would seem to suggest that business climate is unimportant in driving relative growth among the states [emphasis added]. We conclude that the majority of the variation in growth is due to local business factors that affect comparative advantage, local policies, or state policies not reflected in the indexes. Undoubtedly, some of the variation in border county growth is also attributable to luck.
Of more importance are three studies that have been published in peer-reviewed academic journals. Jed Kolko, David Neumark and Marisol Mejia compared various business climate indexes to state growth, as measured in five ways: by two different total employment metrics, state gross domestic product (GDP), total payroll, and employment at new businesses.2 All told, they ran 25 statistical tests for each index. Beacon Hill’s State Competitiveness Index (SCI) did not pass any of the 25 tests for statistical significance.3 The Small Business Policy Index failed 22 of the 25 tests. The State Business Tax Climate Index (SBTCI) failed to meet the standard 5 percent criterion for statistical significance in three-fourths of the tests.
A more recent study by John Anderson tested the predictive power of the personal and corporate tax components of the SBTCI4 and concluded that, “the tax index variables have no discernible effect on state GDP growth.”
Researchers Yasuyuki Motoyama and Iris Hui approached the question differently.5 They sought to determine how a small business owner’s perception of business climate was related to supposedly objective measures of that business climate. They could find no relation between states’ scores on the SCI or the SBTCI and positive perceptions of the state’s business climate. For small business owners, the indexes produced by the Tax Foundation and Beacon Hill do not measure what is important. And if those factors do not affect business owners’ perceptions of the business climate, it is difficult to see how they would affect small business decisions.
1. George Bittlingmayer, Liesl Eathington, Arthur P. Hall, and Peter F. Orazem. “Business Climate Indexes: Which Work, Which Don’t, and What Can They Say about the Kansas Economy?” Prepared for Kansas Inc., 2005.
2. Jed Kolko, David Neumark, and Marisol Mejia. “What Do Business Climate Indexes Teach Us about State Policy and Economic Growth?” Journal of Regional Science 53 (2), 2013: 220-255.
3. Statistical significance at the 10 percent level means that there is a 10 percent probability that the relationship occurred purely by chance. The lower the level, the more likely that the relationship is causal. In academic research, scholars typically employ a standard of 5 percent or better– i.e., the probability that the relationship occurred by chance is only 1 in 20 or even better 1 in 100 – to safely reject the hypothesis that no causal relation exists.
4. John E. Anderson. “State Tax Rankings: What Do They and Don’t They Tell Us?” National Tax Journal, vol. 65, no. 4 (Dec, 2012): pp. 985-1010.
5. Yasuyuki Motoyama and Iris Hu. “How Do Business Owners Perceive the State Business Climate? Using Hierarchical Models to Examine the Business Climate Perceptions, State Rankings, and Tax Rates.” Economic Development Quarterly Vol. 29(3), 2015: 262–274.