In Search of the Elusive Effects of the Regulatory State

The Trump administration is at war with the regulatory state. The fight is most intense at the Environmental Protection Agency (EPA), where Administrator Scott Pruitt is reportedly accompanied by armed guards even within the EPA building, but the Departments of Energy, Transportation, Interior and others are doing their bit. The Consumer Financial Protection Bureau is the latest agency to come under attack. Should we cheer all this on?
Certainly, the quality of the country’s regulatory regime does matter. As the Niskanen Center’s Will Wilkinson, puts it,

Whether a country’s market economy is free—open, competitive, and relatively unmolested by government — is more a question of regulation than a question of taxation and redistribution. . .

If we want to encourage freedom and prosperity, we should pay more attention to easing the grip of the regulatory state.

Still, before we take an axe, Pruitt-style, to anything that looks like it might be a regulation, it would be nice to have some actual evidence to show the degree to which regulation undermines our freedom and prosperity, and some data to help us prioritize the worst regulations for early excision. The popular indexes of economic freedom from the Heritage Foundation and the Fraser Institute, both of which have components that purport to quantify the regulatory burden, would seem like a good place to look. What follows is a summary of what we can learn from those indicators, and some thoughts about how they might be improved. 

A first look

At a glance, the data from Heritage and Fraser do suggest that regulation has a negative effect on prosperity. Consider, for example, the following scatterplot of data from the regulation component (H-REG) of the Heritage index of economic freedom against the Social Progress Index (SPI). The SPI comprises a large number of indicators that cover nutrition, health, education, personal safety, individual freedoms and other elements of human well-being. Data from both sources are available for 131 countries.

Here as everywhere in this post, it is important to note that higher scores on the Heritage and Fraser regulation indicators represent greater “regulatory freedom,” that is, a less intrusive regulatory regime. The simple correlation coefficient for H-REG and SPI is 0.69. The correlation is statistically significant at a 0.01 level of confidence, that is, given the sample size, there is less than a one percent chance that the two variables are unrelated. Squaring the correlation coefficient gives us the coefficient of determination (R²) equal to 0.46, in this case. That can be taken to mean that 46 percent of the variation in social prosperity in our sample is explained by variation in the regulation index. (Here and elsewhere in this post, the terms “statistically significant” and “significant” refer to a 0.01 level of confidence, unless otherwise specified.)

Similar patterns hold for data on regulation, freedom, and prosperity drawn from other sources, including the Fraser Economic Freedom Index, the Legatum Prosperity Index, and the Personal Freedom Index put out by the Cato Institute. 

But, not so fast

These preliminary correlations appear to be consistent with the notion that loosening the grip of the regulatory state should open the door to greater prosperity—but we shouldn’t jump to that conclusion. Although the chart shows how large a part of the variation in prosperity is “explained,” in a statistical sense, by variations in regulation, correlation is not the same as causation. We need to dig more deeply to find to find out just why life is better in countries with higher regulation scores.

The first step is to control for the effect of differences in per capita GDP among countries in the sample. It turns out that all of our measures of economic freedom, regulation, and prosperity correlate positively with per capita GDP. For example, the correlation is 0.62 for the Heritage regulation index and 0.8 for the Social Progress Index, both statistically significant. How do we know, then, How do we know, then, how much of the relationship between regulation and social prosperity is attributable simply to differences in income?

One statistical method than can help is partial correlation, which is a way of identifying the degree of association between two variables, X and Y, when both are correlated with some third variable, Z. The method proceeds in two steps. The first step is to calculate the parts of the variations in X and Y that cannot be explained by variations in Z alone, which are called the residuals of X and Y. The second step is to find the correlation between the residuals, which is called the partial correlation of X with Y, controlled for Z.

As we saw earlier, simple correlation of the Heritage regulation index with the social progress index gave an R² of 0.46. However, the partial correlation of H-REG and SPI, controlled for GDP per capita, yields an R² of just 0.15. The partial correlation between regulation and social progress is still statistically significant, but it turns out that more than two-thirds of the apparent ability of a light regulatory regime to “explain” social progress is due to the fact that both variables are highly correlated with income. Similar results hold when we use many other measures of regulation, freedom, and prosperity as the dependent variable.

But, even the modest partial correlations we get when we control for income do not necessarily reflect causal relationships. We next need to account for the interactions of regulation with other components of the economic freedom before we draw any conclusions about the effects of any one component considered in isolation.

For example, the Heritage Index of Economic Freedom has four major components in all, which cover rule of law, size of government, regulation, and openness of the economy to trade. A simple correlation of the full Heritage index with the Social Progress Index gives an R² of 0.70, suggesting that variations among countries in economic freedom jointly explain 70 percent of variations in social progress. Yet, when we run simple correlations separately for each component, we find that rule of law alone explains 10 percent of the variation in SPI, size of government explains 46 percent, regulation explains 51 percent, and openness explains 72 percent. The four variables, in total, would seem to explain not 70 percent, but 180 percent of the variation in SPI. The fact is that the simple pairwise correlations exaggerate the explanatory power of each individual variable because they fail to take into account the way the variables interact in determining the relation of economic freedom as a whole to social progress.

The standard statistical method for simultaneously determining the influence of several independent variables on one dependent variable is multiple regression. In applying this method we can, at the same time, control for income by using the residual values of the variables, as explained above, rather than their unadjusted values. The results of an income-adjusted multiple correlation are strikingly different from the those of the unadjusted pairwise correlations:

• The income-adjusted values of the four components of economic freedom jointly explain just 38 percent of the variation in the Social Progress Index.
• Rule of Law has a positive and statistically significant effect on SPI.
• Size of government has a weak negative effect on SPI. Since a larger government means a smaller score for the Heritage size-of-government variable, the negative regression coefficient means that a larger government is associated with more social progress. However, the relationship is not statistically significant at the 0.01 level of confidence. (See this earlier post for more on freedom, prosperity, and big government.)
• Economic openness has a positive and statistically significant effect on SPI, roughly as strong as that of rule of law.
• Regulation has no statistically significant effect on SPI at all.

If we run the same kind of regression with the Cato Personal Freedom Index as the dependent variable, the results are essentially the same: rule of law and openness contribute positively to freedom, a smaller size of government contributes negatively, and regulation makes no significant contribution. Repeating the same exercise using data from the Fraser index of economic freedom and the Legatum Prosperity Index produces similar patterns.

Interpretation: Good and bad indicators, good and bad regulation

Here are our findings so far:

• First, simple correlations indicate a positive cross-country association between a light regulatory regime and indicators of prosperity and personal freedom.
• Second, controlling for per capita GDP substantially weakens the apparent association.
• Third, when we account for other components of economic freedom, the explanatory power of the regulation indicator evaporates altogether.

There are two possible interpretations of these findings. One is that worries about regulation are overblown, so that improving the quality of a country’s regulatory regime would not, after all, do much to promote prosperity or personal freedom. The other possibility is that the Heritage and Fraser regulation indexes are methodologically flawed, so that they mask the true effects of regulation rather than revealing them.

After working extensively with the data, I am inclined toward the second explanation—that the regulatory state is a cause for concern, but the Fraser and Heritage indicators don’t measure the aspects of regulation that matter. 

A further remark by Wilkinson points to part of the problem:

Free markets require the presence of good regulations, which define and protect property rights and facilitate market processes through the consistent application of clear law, and an absence of bad regulation, which interferes with productive economic activity. A government can tax and spend very little — yet still stomp all over markets.

Yes, there is good regulation as well as bad regulation. One of the biggest problems with the Heritage and Fraser indexes is their failure to recognize that distinction. By indiscriminately lumping all regulation together, they come up with a hodgepodge that measures nothing.

To distinguish between “good” and “bad” regulation, we need to look both at aims and at implementation. Here are some examples.


As a first approximation, we can categorize as “good in aims” most regulations that serve as administrative substitutes for common law protections of property rights, enforcement of contracts, and redress against torts. For example, it would be fraud under common law, to sell “bread” that is made from sawdust rather than flour. The common law remedy would be a lawsuit by the buyer against the seller. The administrative remedy would be to send inspectors around to bakeries to make sure that their bread is really made from flour. Similarly, it would be negligence, under the common law, to require workers to handle dangerous chemicals without proper ventilation and clothing. The common law remedy would come through lawsuits by workers against their employers. The regulatory remedy is to send out OSHA inspectors to check whether protections are in place.

True, some advocates of minimal government favor the common law approach over regulation on both philosophical and  practical grounds. Their argument is intellectually coherent, but we don’t need to get into all that here, as long as we agree that the aims of keeping sawdust out of the bread supply and of providing safe working conditions are good. To the extent that regulations did manage to accomplish those aims, they would enhance rather than diminish freedom and prosperity compared with a situation where fraud went unchecked and negligence unpunished. 

Other regulations have bad intentions. In their new book, The Captured Economy, Brink Lindsey and Steven Teles argue that the most offensive regulations are those that restrict competition and facilitate corporate rent-seeking. Their book provides detailed case studies from financial regulation, occupational licensing, excessive protections of intellectual property, and restrictions on land use. We would expect these to undermine freedom and prosperity, as the authors confirm with convincing evidence. Tariffs and nontariff restrictions on international trade are perhaps an even more important example. 

Meanwhile, even regulations that have the best of aims can be harmful if they are badly implemented. Take the regulation of pollution, for example—an administrative substitute for common law protections against nuisance and trespass. Despite their good intentions, many pollution regulations are badly designed. Fuel economy standards for automobiles (CAFE standards) have been estimated to cost as much as 14 times more per unit of pollution as system of taxes on emissions. Ethanol blend requirements for gasoline not only fail a cost-effectiveness test as a way of reducing carbon emissions; by some estimates, they actually increase them. 

In short, when we speak of “good regulations,” we mean regulations with benign aims that are efficiently implemented. “Bad regulations” include those with bad intentions, or good intentions that are badly implemented, or—worst of all—bad intentions that are badly implemented.

Unfortunately, the Fraser and Heritage economic freedom indexes make little attempt to distinguish between the good and the bad. Consider, for example, the indicator “bureaucracy costs” that is one of 15 subcomponents of the regulation component of the Fraser economic freedom index. Statistical tests appear to show that higher bureaucracy costs are associated with higher levels of general prosperity, education, health, personal safety, and personal freedom. How can that be?

A little digging in the methodological notes to the Fraser data reveals the answer. It turns out that the indicator Fraser calls “bureaucracy costs” is based on the answer to a survey item from the World Economic Forum’s Global Competitiveness Report. The original survey question did not say anything about bureaucracy costs. Instead, it asked business executives around the world to respond to the statement, “Standards on product/service quality, energy and other regulations (outside environmental regulations) in your country are: (1 = Lax or non-existent, 7 = among the world’s most stringent).” Fraser converts the responses to a 0-to-10 scale, and reverses the hierarchy of responses, so that “lax or non-existent” becomes a 10 and “among the world’s most stringent” becomes a 0. 

Countries such as Sweden, Japan, and Germany get very low scores for this indicator. Evidently, when executives from these countries responded to the survey, they proudly assigned a seven to their own countries’ insistence on high-quality goods and reliable energy supplies, and gave disdainful zeros to other countries where product quality and energy regulations are “lax or nonexistent.” In short, it is likely that respondents are treating this item as if it asked, “How good are product service/quality and energy regulations in your country?” Several other items in both the Heritage and Fraser regulation measures have similar problems. 

The failure to distinguish between good and bad regulation is not the only weakness of the Heritage and Fraser regulation indicators. Another problem lies in what their authors include under the heading of “regulation,” and what leave out.

The most conspicuous omission is the relegation of trade regulation to a separate category. Barriers to trade are the most damaging policies for freedom and prosperity. Environmental regulations are another omission. Regulations on the sale of real property are included in the legal system and property rights component of F-REG, rather than in the regulatory component. Other regulations, such as restrictions on ownership of foreign currency, are also removed from the regulatory component of the economic freedom indexes.

At the same time, some items that seem completely inappropriate are tucked in under the heading of regulation. For example, Heritage lumps a subcomponent called “monetary freedom” into its regulation category. This turns out to be a measure of the rate of inflation, with an adjustment for price controls. To be sure, price stability is important, but I think that is something quite different from what most people have in mind when they speak of easing the grip of the regulatory state. 

In short, both the Heritage and Fraser measures of regulation are deeply flawed. Once they are turned inside out and examined, it is hardly surprising that they have little statistical power to explain anything. 

Conclusions

When all is said and done, our search among the economic freedom data from Heritage and Fraser for evidence of the effects of the regulatory state has been frustrating. We are left with the following conclusions:

1. Simple correlations do find positive and statistically significant relationships between measures of regulation and commonly used measures of prosperity and personal freedom.
2. Half or more of the relationships implied by simple correlations turn out to come from the strong correlations of regulation, prosperity, and personal freedom indicators with GDP per capita. Controlling for income, wealthy countries with light regulation have only slightly better freedom and prosperity outcomes than wealthy countries with average regulation.
3. In multiple regressions that account for the interaction of regulation with other components of economic freedom, the statistical power of the Fraser and Heritage regulation indicators to explain cross-country variations in prosperity and personal freedom evaporates altogether.
4. Close examination reveals serious methodological problems in the way both the Fraser and Heritage regulation components are constructed. Neither makes adequate efforts to distinguish between helpful and harmful aspects of regulation. Both include some indicators that fit poorly with common notions of what the regulatory state really is and does, and both exclude important aspects of regulation (especially of international trade).

In my view, none of this should be taken as proof that regulation is always harmless. Although there can be good regulation as well as bad, the bad can become truly ugly. The failure to show that is, in my opinion, one of the main flaws of the Fraser and Heritage regulation indexes.

The bottom line, going forward, is that the search for the economic impact of the regulatory state needs to take a more nuanced approach. It needs to focus more on the details of specific regulations than on broad indicators. It needs to pay more attention to trade regulation, environmental regulation, and financial regulation. It remains possible that further work will lead to useful aggregate measures of regulation, but that will require a substantially different approach than the one taken by Fraser and Heritage.

Reposted from NiskanenCenter.com