ALEC Fails to Defend its Methodology
ALEC’s policy prescriptions embodied in Rich States, Poor States and in other of their publications, and the research methodology used by Laffer and others writing ALEC reports, have come under considerable attack in recent years, but they have failed in their attempts to defend their methodology.
In November, 2012, I published a study called Selling Snake Oil to the States: The American Legislative Exchange Council’s Flawed Prescriptions for Prosperity. The Center on Budget and Policy Priorities published a report critical of ALEC and Laffer’s policies and methods in early 2013. Several critiques of Laffer’s work were published by the Institute on Taxation and Economic Policy.
In February, 2013, ALEC attempted to strike back at its critics in a report by Eric Fruits and Randall Pozdena called Tax Myths Debunked. The arguments of Fruits and Pozdena were repeated in the 2013 edition of Rich States, Poor States, where ALEC devoted an entire chapter to attacking their critics, with considerable vituperation and sarcasm.
The first empty critique leveled in Tax Myths is directed at our analysis of the factors leading to economic growth and rising incomes among the states between 2007 and 2011. The authors of Tax Myths Debunked completely misunderstood our analysis; their criticism seems to assume that our model was predicting changes in the share of employment by sector. Instead we were simply using the states’ 2007 economic structures – measured by employment shares – to see if they predicted growth in GDP, employment, and personal income. Fruits and Pozdena’s criticisms make no sense and are completely off base; 2012 state GDP cannot be a cause of 2007 economic structure, which is the circularity they argue undermines our analysis.
A second criticism was directed at the scatter plots and associated correlations in Figures 2.1 through 2.6 in our report. Since all ALEC provided was the state rankings (without index numbers showing their relative strength or weakness), we correlated those rankings with the measures of performance that ALEC emphasizes: growth in GDP, employment, and income. Fruits and Pozdena argue a technical point here: the formula used to calculate the correlation between two continuous variables (the Pearson coefficient) is different from the formula used to calculate the correlation between two rankings (the Spearman coefficient). We had one ranked variable (ALEC’s Economic Outlook Ranking or EOR), and one continuous variable (our data on the states’ various outcomes), and used the Pearson coefficient.
To respond to this criticism, we converted our continuous variables to ranks, and then applied the Spearman coefficient. The conclusions were the same. Where there was no statistically significant relation using the Pearson formula (as was the case when we looked at the EOR as a predictor of growth in GDP or jobs), there was also no significant correlation using the Spearman. Where there was a statistically significant and negative relation (ALEC’s high ranked states have lower per capita and median family incomes) using the Pearson measure, the same result occurred with the Spearman.
In only one instance did results change, and not favorably for ALEC: Our original analysis showed a negative but not statistically significant relation between EOR and the growth in state revenues. The analysis substituting state ranks in revenue growth found a negative and statistically significant effect as measured by the Spearman coefficient.
Finally, Tax Myths presented an alternative to the analyses in Selling Snake Oil to the States, correlating the state EOR each year with the June value of the “state coincident indices” published monthly by the Federal Reserve Bank of Philadelphia for each state. The coincident indices are based on four measures of the health of the state economy: non-farm employment, average hours worked in manufacturing, the unemployment rate, and wage and salary disbursements. Fruits and Pozdena found a strong correlation between a state’s EOR and the value of the coincident index.
The state coincident indices are designed for tracking the trajectory of a state’s economy over time – whether it is sliding into recession or on a path to recovery – and are pegged to a value of 100 for every state as of 1992. They are used to compare states, but only in terms of the changes in the index over time. So the value of the index as of 2008 is in effect a measure of that state’s growth rate from 1992 to 2008, since every state started at 100. However, a high value for state X in 2008 does not mean that state X has a healthier economy than state Y with a lower value in 2008. State Y could have started out with a much higher level of prosperity in 1992 and still have higher incomes and wages than state X in 2008.
Furthermore, the correlations performed by Fruits and Pozdena are taken as evidence that ALEC policies, as represented by EOR, cause economic health, but they have done it backwards, in effect trying to demonstrate that conformance to ALEC policies in 2008 caused states to grow more rapidly from 1992 to 2008! So why didn’t they look at the policies in place as of 2008 and see if they predicted economic growth from 2008 to 2012? Probably because the correlations between the EOR in 2008 and changes in the state coincident index subsequent to that are near zero.
A more sophisticated approach to identifying the effects of a state’s EOR would entail a statistical analysis that controls for economic structure. In fact, a Philadelphia Federal Reserve Bank economist, in an article about the state coincident index, explains how state economic structure is an important determinant of the path of a state economy, as measured by changes in that index over time.1 In our regression model discussed earlier, we included the change in the state coincident index from 2007 to 2013 as one of the variables to be predicted, along with growth in GDP, jobs, income, and poverty. This is the proper use of the coincident index: using the change in the index as a measure of the change in economic health. The results were the same: When the composition of a state’s economy is controlled for, the EOR has no statistically significant relationship to the rate of improvement in the state’s economy over this period as measured by change in the state coincident index.
In sum, nothing in Tax Myths undercuts any of our analyses or conclusions. Nor does the chapter in the sixth edition of Rich States, Poor States, which can only be described as a rant. In that chapter, Laffer et al assert that “Fisher uses the wrong measures of economic health” because I used per capita income, not mentioning that I also used growth in state GDP and employment – growth rates in total GDP and jobs, not per capita GDP or jobs. (Nor did they present any case against the other standard measures of prosperity I used, median family income and median annual wages.) They assert “Fisher’s models include no ‘control variables’”; this applies only to the scatter plots and ignores the regression analyses that all include controls. They complain that I looked only at 2007 to 2011, ignoring that I did so because the policy variable, their EOR, did not exist prior to 2007, and to use 2007 policies to predict performance prior to 2007 is to fly in the face of both logic and basic statistical principles. (Note also that the analysis here, updated to 2013, comes to the same conclusion.) Finally, they repeat their usual defense: that a simple correlation between the EOR or their preferred tax policies – in fact a scatter plot just like the ones I produced and that they dismissed derisively – proves (their term) that they work. This after attacking their critics for using simple correlations without controls and pointing out (correctly) that correlation does not prove causation.
1. Theodore Crone, “What a New Set of Indexes Tells Us About State and National Business Cycles.” Federal Reserve Bank of Philadelphia, Business Review Q1 2006. http://www.philadelphiafed.org/research- and-data/publications/business-review/2006/q1/Q1_06_New Indexes.pdf