Mismeasuring Progress

by Will Wilkinson on August 24, 2006

It is shocking to discover just how much of the debate over politics and policy rests on semi-arbitrary government standards for measuring things. For example, if you believe the Consumer Price Index speaks with absolute authority, then you will believe obviously absurd things, like the idea that real wages have stagnated. Virginia Postrel has a nice short essay in Forbes [free reg. req.] on this aspect of the mismeasurement of economic progress. If Bureau of Labor Statistics true-believers are right, then

… you have to wonder who’s buying all those flat-screen TVs, serving precooked rotisserie chicken for dinner or organizing their closets with Elfa systems. “Anybody who thinks things are getting worse should go to Best Buy and notice the type of people who go to Best Buy,” says economist Robert J. Gordon of Northwestern University.

Gordon is the author of a much-cited study showing that from 1966 to 2001 real income kept up with productivity gains for only the top 10% of earners. What the pessimists who tout his study don’t say is that, while Gordon does find that inequality is increasing, he’s convinced that the picture of middle-class stagnation is false.

“The median person has had steadily improving standards of living,” he says. But real incomes have been understated. The problem lies in how the U.S. Bureau of Labor Statistics calculates the cost of living.

Similarly, the American Enterprise Institute’s Nicolas Eberstadt has a terrific essay on the bizarre and inaccurate method by which the government calculates the poverty rate in the new Policy Review. Eberstadt shows that the official poverty statistics often get things backwards, indicating that poverty is getting worse when it is in fact getting better according to a number of other noncontroversial measures of economic well-being:

The official poverty rate is incapable of representing what it was devised to portray: namely, a constant level of absolute need in American society. The biases and flaws in the poverty rate are so severe that it has depicted a great period of general improvements in living standards — three decades from 1973 onward — as a time of increasing prevalence of absolute poverty. We would discard a statistical measure that claimed life expectancy was falling during a time of ever-increasing longevity, or one that asserted our national finances were balanced in a period of rising budget deficits.

Journalists unfortunately tend to take government numbers as gospel, and therefore end up communicating to the public a badly distorted picture of the state of our economy and society. And far too often intellectually savvy commentators who ought to know better repair to government statistics as if they are pure data, untainted by systematic methodological bias. However, far from a neutral picture of empirical economic reality, we get a funhouse mirror. I don’t think there is any intentional bias in these measurement methods. But there sure is ideological resistance to replacing them with more empirically adequate measures. Things really are getting better all the time, but “reality-based” economic measures might get in the way of some people’s pet policies. And we can’t have that! I think we’ll eventually get better official methods for measuring real income and poverty, but not without a fight.

[cross-posted from Cato@Liberty]

  • Curtis
    I think "reflexive" is your word for "principled." A typical pragmatist perspective. Of course, there are other words for "pragmatist."

    You have me completely beaten on the beret. Smooth, very smooth.

    And I apologize: I should have said "reason" rather than "logic." In fact, that's a good way to express the failure of 20th-century philosophy: it was the triumph of (formal) logic over reason.

    In other words, like so many other fields (such as mine, computer science; and of course economics), philosophy was overrun by a Raspailesque horde of second-rate mathematicians. The result was arguably formal, and formal anything beats informal anything else in the kingdom of the bureaucrat. It was also of no use to anyone. In fact, it had negative value, because it displaced actual thinking.

    As for my evilometer, it needs no calibration at all.

    The US today is the greatest success of civilization, period. But this proves nothing. If in 1989 the US and its European satrapies had never existed, the DDR would have been the greatest success of civilization. But it still would have been just as idiotic and corrupt.

    In other words, the US's tragedy is that it has no America to compare itself to. If any of the liberal Western states of the 19th century had survived into the 1990s with its structures of law, trade, learning and government intact, the analogy DDR:US as US:X would seem obvious. Since none did, the analogy seems - to a pragmatist - ridiculous. But how can anyone's ethical verdict depend on such an accident of history? Or are your ethics pragmatic, too?

    Country X could be anywhere. Even the France of Napoleon III would probably do, in a pinch. But I'm assuming that, since you are a libertarian, you agree with me that central North America would be a very different and considerably improved place if the Great Society, the New Deal, the Progressive Era, the Civil War, and hopefully the Federalist period, had never happened.

    If this assumption is wrong, perhaps you should consider a different word for your intellectual affiliation. I suggest "nationalist," "federalist," or, of course, "progressive." Isn't the New America Foundation just across the street?

    Obviously, empirical observation - unless we play the little game of defining hypothetical abstraction as "empirical" - can tell us nothing about what an Articles of Confederation America would look like in 2006. I'm afraid actual thought may be required.

    All these mistakes proceed from one fallacy: the conflation of economic progress and technical progress. This keeps us on topic, because "economic growth" is a mixture of exactly these two orthogonal qualities (plus a third, population growth).

    Economic progress is liberalization, pure and simple. It is also called globalization. Its reverse is centralization, parochialization, etc.

    Technical progress is the discovery of new methods of production. Unlike economic progress, technical progress is difficult to reverse, though it can happen (for example, in the fall of Rome).

    Economic progress stimulates technical progress. But technical progress has its own internal structure. It is a serious fallacy to measure economic progress by technical indicators, especially if we use absolute rather than relative comparisons.

    For example, that we have iPods and HDTVs in 2006, and we didn't in 1706, does not tell us that the legal and economic structures of North America in 2006 are vastly superior to those of 1706. (In fact, I'd say the truth is the opposite. But then, I'm a libertarian.)

    The hallmark of the 20th century is the combination of enormous technical progress with a mixture of economic progress and reversal. As in the DDR, the combination of technical progress and economic reversal can sometimes appear as overall progress.

    Productive methods in the Eastern bloc definitely advanced during the Communist period. It was much slower than in the West. But compared to any time before the Industrial Revolution, it remained impressive. And had the West not provided a yardstick of the possible, the DDR's mandarins could easily have passed its sluggishness off as natural "maturity."

    What failures are our mandarins passing off as maturity? Is Microsoft our Robotron? If we restrict our brains to the approved positivist procedures, we're not even allowed to guess.

    In particular, it's worth noting the late Communist predilection for supporting their consumer economies with massive loans from more economically advanced countries. Doesn't sound familiar at all, does it?

    As for table-pounding intellectual stagnation, you repeat the familiar myth that all reason is equivalent to theological hairsplitting. I suppose that somewhere in your expensive education, you were probably taught that Socrates is not a cat.

    I'd take another look at that phrase "wholly convinced." From someone perceptive enough to note his own susceptibility to absolutist thinking, it may be a warning sign.

    Sorry about the scribbled-on, scanned-in PDF. I still recommend it. K-L was not exactly a libertarian, but I suspect he had more things in his philosophy than either you or I.
  • Will Wilkinson
    Curtis, I think your reflexive anti-statism causes a serious failure of perspective. The GDR and USSR analogues are just atrociously inept. Despite it's problems, the USA circa now is one of the greatest successes of human civilization. Our democracy is indeed part of that, but only a small part. You really do need to recalibrate your evilometer.

    I really don't think Quine was Derrida without the beret. In fact, Quine wore a beret.
    http://www.lawrence.edu/fast/ryckmant/Quine.gif

    Funny how the century that "rejected logic" saw more advances in logic than all previous centuries combined.

    In any case, it is curious to accuse one of the last centuries greatest deductive logicians with a failure to appreciate deductive logic. Though, true, Quine's semantic indeterminacy ideas are rather "postmodern". But that's what's wrong about Quine. Get rid of his behaviorism and his description theory of reference, and we're A-OK. Positivism is dead. Long live positivism!

    I submit that fixation on deductive reasoning from allegedly apodictic truths is a way of avoiding the much more difficult work of developing reliable methods of inductive inference, and developing the corresponding virtues of mind. Axiom-and-grind is a recipe for table-pounding intellectual stagnation.

    Sorry about the .doc!
  • JohnDewey
    Steve Sailer,

    Do you mean that the poor must live in high crime areas, surrounded by those who have abandoned all hope of achieving a comfortable lifestyle through legitimate means? If so, then I agree somewhat.

    It wasn't like this 50 years ago, when I was a child. I started out life in poor neighborhoods. We weren't burdened by crime or by neighbors who resented my father's climb into the middle class.
  • Curtis
    Will,

    Linking to a Word document! Tsk, tsk.

    Failing to make a distinction between deductive and inductive thinking is simply a matter of wordplay. It's this sort of intellectual pathology that has convinced most intelligent people in this century that philosophy doesn't matter and they don't need to learn it. It's Derrida without the beret.

    Perhaps the whole universe is running on a quantum computer. One can certainly define it as such. But when you kiss a girl, you kiss a girl, you don't collapse her wave function.

    Surely you'll agree that it's a curious coincidence that the century which rejected logic was also the century of the State. How convenient a trope positivism has proved! How pleasant for the status quo that any syllogism which could demonstrate its corruption can be rejected as an unprovable hypothesis, which is impractical to test and hence will never be tested!

    I would be remiss if I didn't comment on a phrase you used earlier, "armchair philosopher." Perhaps you could clarify as to what seating arrangements you consider appropriate for the practice of philosophy. An Aeron? A Louis Quinze? Or are you of the peripatetic school? Or, as I suspect, is the converse of "armchair" simply "licensed?"

    There is a very straightforward way to liquidate the United States. Treat it as an ordinary corporation, not a divine institution, and proceed according to the common law. Its liabilities are its bonds and its entitlements. Its assets are its physical plant and its rights of taxation, whose closest medieval analogue are the rights you own when you own a serf. And its shares can be divided in some proportion between its voters and its current managers, hopefully corresponding to their actual influence in practice.

    I love phrases like "space of feasible alternatives." It trips lightly off the tongue of the DC libertarian. Who is otherwise distinguished by his utter inability to grasp the lesson of Boromir.

    If you look at the history of the Eastern bloc, you notice a curious fact. Right up until the end of, say, East Germany, the reformers kept appearing. The new Dubceks and Nagys, men like Egon Krenz. They had their dreams of socialism with a human face. With them in charge, of course. And there were plenty of intellectuals who were ready to hop on the wagon and follow them.

    But instead, the whole thing just disappeared. Pfft.

    Here is the big secret about the (post-Stalin) Soviet system: everyone was a reformer. Everyone wanted to work within the system in order to improve it. Many talented and energetic people were attracted by this task. Kieslowski's film Blind Chance (now available on DVD) is a wonderful chronicle of this process.

    These people are mostly all still alive. And I'm sure most of them are smart enough to regret the fact that they worked as shills for a giant criminal organization.

    Of course, democracy is different. We have elections, they didn't, so we are good, whereas they were evil. There are no Brezhnevs in DC. I must be looking for a completely different couple of droids.

    One of the many errors that people who criticize Washington make is to assume that all the problems of the system are the result of it having the wrong leaders. Or the wrong employees, or whatever. Since I grew up as a Federal dependent, I am not subject to this comforting delusion - I'm aware that almost all the people who work in the machine, both formal employees and contractors such as yourself, and even the actual politicians, are intelligent and cultured people with only the best of intentions. I hope this explains my belief that it's the organization itself that must be liquidated.

    Perhaps I'm wrong. Perhaps "liberal democracy" really is the end of history. Perhaps the US Constitution really was divinely inspired. Who can know these things? But before you take this one to the bank, you might want to read this.
  • The real problem with being poor in America today is not that you don't own enough stuff but that you have to live around other poor people.
  • Will Wilkinson
    Curtis, Good stuff. I was waiting for the "scientism" charge! Our first real, deep difference is over a priori knowledge. I don't think even the laws of logic are knowable a priori. They're the broadest, most secure, generalizations. I was wholly persuaded of this by my teacher Michael Devitt, and his teacher, Quine. I wish I could be more original, but my view is almost exactly the same as Devitt's here. To my ears, then, the claim that "a system of replicators, heritable variation and selective competition result in descent with modification," is not an empirical conjecture sounds bizarre.

    I agree that the state should not be managing the minds of its citizens. I am in fact writing something about why the state should not manage the minds of its citizens. I think we probably think about the same thing, though probably for different reasons.

    We do disagree about incrementalism. I do not see the constructive alternative. I am not an anarchist, nor a jacobin; my goal is to find the best path starting from here through the space of feasible alternatives to a better system.

    I've actually got a very high tolerance for kookiness. I grew up in a minority sect of Mormonism, and spent much of my twenties an Ayn Rand devotee. And I love Nietzsche. And spinning out logically valid implications from empirically false first principles can be an illuminating and worthwhile excercise. But it is not the height of intellectual achievement.
  • Curtis
    [I'll drop this in again: it doesn't seem to have cleared the spam filter.]

    JohnDewey: I apologize for the implication! I'm sure you must get that a lot. I hope I haven't added to the burden.

    Will, from most incendiary to least:

    I do not attribute Hayek's differences from Mises to a betrayal of principle. Has there ever been anyone of remotely that intellectual stature - libertarian or fascist, Catholic, Protestant or Buddhist, you name it - that didn't believe the things he was saying? Even Hitler was remarkably sincere. Intellectuals do it for fun. Hypocrisy is no fun.

    Rather, I attribute Hayek's success to his differences from Mises. As, it seems, do you. I don't think there is any disagreement here. All we disagree on is who was right and who was wrong.

    It was, however, unfair of me to impute that the Cato Institute is a den of soulless bureaucrats whose veins are white from massive grant injections, whose favorite novel is How to Liquidate Trotskyists and Other Double Dealers, and whose only pastime is staring into their green Fresnel lenses and checking each others' bookshelves for unapproved literature. I withdraw the remark.

    Empiricism, or pragmatism as some call it, or as Hayek called it scientism, is precisely your problem. You think of it as progress, as part of the 20th-century victory of science. You are still in the thrall of the Brandeis brief.

    In fact, empiricism was a core part of the revolt against reason.

    Empiricism is simply a systematic immunity to deductive logic. It is a sort of controlled stupidity. Its answer to every new idea, to every device of classical rhetoric is: well, that certainly sounds quite clever, but what is it but a bunch of words?

    The attitude is certainly a useful mental tool, in moderation. It is especially useful as a way to inoculate large populations against mass philosophical delusion. Most people are simply not smart enough to be trusted with Marx, or even with Rosenberg. Refuting ethical and political syllogisms is not a task for the median voter.

    However, it should not be mistaken for actual thinking.

    You've probably heard the strange, penguinlike cries of socialists who accuse you of "free-market fundamentalism." What does one say to such a charge? Are libertarians mere cultists, groaning under the sway of our sacred texts, mindlessly parroting our guru of choice, our Friedman or Hayek, Rothbard or Hoppe?

    As I hope you agree, it's precisely the opposite. It's everyone else who's in the cult. The cult of the mainstream has many names, but Science must be its favorite. "Scientists say..."

    In reality, reason is more important than science. It's not that reason works because reason is scientific. It's that science works because science is reasonable.

    A good example is Darwinian evolution. Darwinian evolution is apodictically true. It is neither a theory nor a fact. It is not a mere conjecture, and it does not need to be confirmed by any kind of evidence. In a system of replicators, heritable variation and selective competition result in descent with modification. It happens that Earth's biosphere is a system of replicators which exhibits heritable variation and selective competition. But if it was not (for example, if DNA repair was perfect), Darwin's logic would still be correct.

    Misesian (the usual contemporary usage) economic logic is precisely like Darwinian evolutionary logic. It is generally more complex than anything in Darwin, although contemporary evolutionary biologists have certainly closed that gap and then some. But it plays by exactly the same rules.

    For example, the statement that free markets clear is exactly in the same logical class as the statement of evolution. And yet socialist economists release a steady stream of empirical studies which purport to prove that labor markets do not obey it.

    For more in this vein, point your Fresnel lens at the essays collected as "Method" in Money, Method and the Market Process. It might be a fun thing to review on your blog if you have any interesting disagreements with it.

    As for the problems with central planning, of course there are many which have a reasonable claim to being "fundamental." I cannot disagree with Caplan on this, although his essay on the Austrians definitely deserves a new orifice or two.

    But if you're running with that whole Marginal Revolution crowd, I can certainly see how you miss the point about punk. It's amazing how a libertarian moves to DC and in a few years he's writing books about how great it is that we have Official Art.

    The point of "empirical punk" is that a democratic government cannot measure punk, because punk is entirely a phenomenon of taste, and a democratic government cannot have or measure taste. The very nature of democracy is to be impersonal. When you try to legislate taste, you get what John Dolan calls the Beigeocracy.

    The wine comparison is illustrative. Taste in wine is actually, as you say, not that complicated. Witness Enologix. Wine tasting, unlike rock and roll, is not an actual art. There is no Punkologix.

    To see what happens when government gets into the taste business, we need look no farther than architecture. Architecture never lies. Healthy societies make attractive buildings. Unhealthy societies make ugly ones. Brutalism and empiricism are good buddies.

    As for liquidating the government, I suppose you think that if one day the good Lord came down and raptured up all of his peoples as he could find between Glen Burnie and Alexandria, North America would be suffering from some kind of shortage of official authority. Who needs jumbo shrimp when we have nationalist libertarians?

    The Misesian line is not that good psychology leads to statism. It is that the state should not be in the business of psychology. It should not be managing the minds of its citizens. Security is certainly an essential service, and someone has to provide it. But whoever those people are, they should not be in the public information business.

    And BTW, no one has indoctrinated me (though I hope I do make it to Alabama one of these days). If you are looking for Misesians to mock, you certainly don't have to look much farther than lewrockwell.com.

    But an exaggerated fear of kooks is one of the primary ways that our contemporary orthodoxy succeeds in imposing itself. In fact, it is not at all uncommon for people to have some ideas which are kooky and others which are extremely good. Witness, for example, Nietszche. I would hope that an actual intellectual, such as yourself, would be able to evade this obvious trap.
  • Dain
    Once one realizes what the origins of the state are, and that it is (somehow) the sole legitimate purveyor of violence, how can one defend it without simultaneously throwing out any kind of universal morality? If the state is allowed to steal, for instance, why aren't I?
  • Will Wilkinson
    Curtis, You're genuinely funny! But you're mostly wrong. What is the problem with aggregating subjective judgments? Suppose 94 was the average subjective wine-rating of 1000 randomly selected people (or sommoliers). Then there is a high probability that a person picked at random will like that wine better than one that rated an 84 average.

    Better yet, the science of the future may be able to tell us what it is chemically about a wine, and what it is about the brain of an expert judge, that leads them to prefer that wine over another.

    "The fundamental problem with central planning is that its performance cannot be measured."

    You mean the fundamental practical problem? I agree with Bryan Caplan that the the fundamental problem is incentive-compatibility, not calculation.

    You could bother to defend Mises. His a priorism is simply gibberish. I think Mises was an incandescently brilliant economic thinker, and I have his books spine out. But his philosophy and psychology was just second-rate. Hayek on the other hand was also a great psychological theorist, and an empiricist with a thoroughgoing scientific sensibility, which prevented him from endorsing some of Mises's armchair philosophy-cum economics. Their differences were substantively and honestly intellectual, and it happens that Hayek was generally correct. It's bogus ad hominem, and speaks ill of your indoctrinators, to attribute Hayek's differences from Mises to a betrayal of principle.

    If you think Cato or anyone could have liquidated the government simply by putting more Misean dogma in the air (and we've done plenty of that, too, I'm afraid) , then you're in the grip a utopian fantasy. In any case, I think liquidating the government is an awful idea, and the real problem is in fact making it better.

    You do see, I'm sure, that the conviction that one could in principle empirically measure punk is totally logically independent of anything whatsoever having to do with government. Don't you?

    So, is the neo-Misean line that good psychology leads to statism? Sort of like evangelicals who think that good biology leads to immorality? If so, that's extremely silly.
  • JohnDewey
    Curtis,

    Like hundreds of other John Dewey's in America, my name has no connection to the collectivist, "progressive" philosopher.

    "Dewey" has been my father's family surname for at least 250 years. Members of our family have fought in nearly every major American war. My first and middle names were given to me in honor of two uncles, both pilots, both killed in World War II combat.

    I am proud of my family, and of the uncles I never met. I will always respect the heritage contained in the name I was given at birth.
  • Curtis
    Will,

    If Consumer Reports wants to give an HDTV four stars, even if the Wine Spectator wants to give a cabernet a 94, I have no objection at all (although some wine snobs would certainly disagree).

    This is because these numbers are not, in any way, disguised as quantitative observations. Because they are simply illustrations of qualitative personal judgments. Because I know, in a general way, who is making these judgments, and I know that they are not under any kind of political pressure. Because if the critics who make them go around the bend, they will lose their jobs and be replaced.

    And, most of all, because neither Consumer Reports nor the Wine Spectator is actively engaged in an attempt to manage the entire friggin' planet.

    Accepting these meaningless aggregates is the first step toward believing in central planning. The fundamental problem with central planning is that its performance cannot be measured. Once you deny this fact, which is not a consequence of any empirical observation but a matter of pure logical deduction, you are simply quibbling over policy, and your name might as well be Ira Magaziner.

    If Mises scares you, this same observation is easy to find in the works of his student Hayek. Hayek made significant compromises in order to make his work acceptable to nationalists, and he was not nearly as good a writer as Mises. But you can probably put his books on your shelf with the spines facing out.

    JohnDewey: you have quite a handle for someone commenting on a libertarian blog! You might enjoy this little piece.

    Measuring inflation by price indexes is like measuring who won a baseball game by graphing the crowd noise. If you actually wanted to measure inflation, you would measure the growth of money and government-guaranteed credit. Unfortunately the latter is very hard to quantify, but various attempts can be made. The result is certainly an actual number with no fudge factors mixed in.

    It is true that various contractual instruments exist which are marked to the CPI. TIPS are another example. These contracts (including entitlements) cannot simply be voided. Private actors have never shown any interest at all in price indexing, but if you want to liquidate the government, the first step is to treat it as a private actor. All this means is that shutting down the BLS is not the same kind of no-brainer as, say, NASA. Such indexed instruments should not be issued in future, but the existing ones have to be managed.

    Of course, if Catonians put one tenth of the brainpower they put into improving the Federal government into liquidating it, Washington would already be a museum. When will libertarians learn that Fabian strategies work for growing the state, but not for shrinking it?

    But no. Instead, we get centrally planned punk. A Department of Punk - upgraded to Cabinet level to attact the important punk vote. Punk studies programs at every major university - taught by leading punks - training America's best young minds in how to achieve true punkness - rated in milliramones by US News and World Report.
  • JohnDewey
    Curtis,

    I disagree with the assertion that measuring inflation is ridiculous. But even if it is, we still have to live with inflation estimates. Billions and billions of $ are riding on the measures of inflation - for indexing of income tax brackets, increases in social security benefits, and more. I would prefer that economists at least help improve the measure rather than just ignore it.
  • Will Wilkinson
    Curtis, I don't think the idea of measuring progress on any given dimension of value is absurd. Misean arguments on this score strike me as willfully obscurantist and ideologically anti-empirical. What exactly is "ridiculous" about putting a number to how much better in terms of picture quality this year's HDTV is than last years? It is fairly easy to think of a study in which you could do exactly that. And then you can say how much extra you have to pay for how much extra picture quality.

    I'm a pluralist, so I don't think there is a single dimension of progress. But it is pretty easy to determine what it is that people do in fact value in certain kinds of products. And it surely in principle possible to measure that, and say how much it costs per unit.

    In any case, the government IS going to continue trying to measure inflation. It is better to get them to improve their methods than to snark off about how it can't be done.

    I'm still totally psyched by the idea of a milliramone. It think that could definitely be put into practice!
  • Will Wilkinson
    Milliramones!
  • Curtis
    The very idea of officially measuring progress is absurd.

    It's like having a number for how punk you are. If we lived in a punkocracy where punk was considered the highest possible social value, perhaps each of us would get a rating in milliramones, using a set of criteria computed by the BLS, which could run them all through its supercomputer and tell us that sure enough, America is 3% punker this year.

    The problem is simply that measuring "inflation" in the current sense of the word requires putting a number to how much better this year's HDTV is than last years. Which is, again, ridiculous. But which the nice people at the BLS do, in fact, spend their lives doing.

    Since this number then goes on to contaminate all other economic statistics, they are all, by definition, just as useless. In the software game we have a name for this - it rhymes with vitiligo.

    More technically, the delusion of a general welfare function is the central error of the discipline of official economics, to which you folks at Cato, for some reason, have devoted yourself. Perhaps it is the excellent shad fishing to be found on the central Atlantic seaboard.

    Fortunately, you do not actually have to move to Alabama to get the Irving Fisher out of your system - your browser can make the trip instead.
blog comments powered by Disqus

Previous post:

Next post: