Knowledge And Decisions

PREFACE TO THE 1996 EDITION

As of the time when it was first published in 1980, Knowledge and Decisions was by far my most important and most comprehensive book. None of my subsequent writings has surpassed it, though it has been joined by work of comparable scope in A Conflict of Visions in 1987 and Race and Culture in 1994. The re-publication of Knowledge and Decisions now, sixteen years after its initial debut, reflects the continuing importance of the issues raised in it, and offers an opportunity to update some conclusions while re-affirming others.

The social vision underlying the analysis in Knowledge and Decisions is the opposite of what I have elsewhere called The Vision of the Anointed.¹ There the focus was on a set of people and their perception of their role in a world whose problems they saw as caused by the inadequacies of others. Here the focus is not on particular people or their beliefs, but on social processes and the constraints within which those processes operate. The analysis begins with one of the most severe constraints facing human beings in all societies and throughout history—inadequate knowledge for making all the decisions that each individual and every organization nevertheless has to make, in order to perform the tasks that go with living and achieve the goals that go with being human.

How a variety of social institutions and processes coordinate innumerable scattered fragments of knowledge, enabling a complex society to function, is the central theme of the first half of Knowledge and Decisions. This approach focuses on the advantages and disadvantages of particular institutions and processes in mobilizing the knowledge needed for making particular kinds of decisions. Families, markets, armies, churches, corporations, and sports teams are just some of the wide spectrum of formal and informal social mechanisms for mobilizing and coordinating the knowledge and experience of many to guide or influence individual decisions by others.

Perhaps the most important feature of the first half of Knowledge and Decisions is simply its analysis of decision-making processes and institutions in terms of the characteristics and consequences of those processes themselves—irrespective of their goals. As noted in Chapter 6, this approach rejects the common practice of “characterizing processes by their hoped-for results rather than their actual mechanics.” “Profit-making” businesses, “public interest” law firms, and “drug prevention” programs are just some of the many things commonly defined by their hoped-for results, rather than by the characteristics of the decision-making processes involved and the incentives created by those processes. So-called “profit-making” businesses, for example, often fail to make a profit and most of them become extinct within a decade after being founded. In Knowledge and Decisions the owners of such businesses are defined not as profit-makers but as residual claimants to the firm’s income—that is, to what is left over after employees, suppliers, and others have been paid. Put this way, it is clear from the outset that what is left over may be positive, negative, or zero. There is no more reason to expect “drug prevention” programs to prevent drug usage or “public interest” law firms to serve the public interest than to expect that most “profit-making” enterprises will in fact make profits. Whether any of these organizations do or do not live up to their expectations or claims is a question of empirical evidence. Pending the presentation of such evidence, such organizations can be analyzed in terms of what they actually do, not what they hope or claim to achieve. In Germany a 1933 “Law for Removing the Distress of People and Reich” gave the chancellor dictatorial powers,² which in turn allowed Adolf Hitler to start wars that brought unprecedented distress—indeed, devastation—to the German people and nation.

The point here is not simply that laws, policies, and programs can have counterproductive results. The point is that, when social processes are described in terms of their hoped-for results, this obscures the more fundamental question as to just what they actually do—and circumvents questions as to whether doing such things is likely to lead to the results expected or proclaimed. More specifically, we need to know what incentives and constraints are created by these social processes. Therefore socialism, for example, is defined in this book not in terms of such goals as equality, security, economic planning, or “social justice,” but as a system in which property rights in agriculture, commerce, and industry may be assigned and re-assigned only by political authorities, rather than through transactions in the marketplace.

To the socialist, of course, government ownership of the means of production is but a means to the various social ends being sought, but such results that are hoped for tell us nothing about the institutional processes set in motion or the incentives inherent in those processes, much less their actual consequences. Indeed, lofty goals have long distracted attention from actual consequences, most notably in many Western intellectuals’ determined resistance to acknowledging the devastating consequences of communism in the Soviet Union, which the Communists themselves eventually acknowledged during the era of glasnost under Gorbachev. The lofty goals of communism—always receding before them like the horizon—kept many Soviet sympathizers in the West mesmerized for decades, while more millions were slaughtered under Stalin than in Hitler’s death camps.

Insurgent movements in general—whether religious, political, or academic—look very different when viewed in terms of their respective goals than they do when viewed in terms of their incentives and constraints. Whether the goal of an insurgency has been to establish the Christian religion in the days of the Roman Empire, to create an Interstate Commerce Commission in nineteenth-century America or to promote civil rights for minorities in the twentieth century, what a successful insurgency does in institutional or process terms is to change the incentives and constraints facing others, as well as the incentives and constraints facing themselves and their successors. Against this background, it is not surprising that there should be certain patterns common to insurgent movements, whether those movements have been promoting religion, political ideology, minority interests, or innumerable other causes.

One of these patterns in the history of many insurgent movements has been a disappointment in the direction that the movement has taken after victory, including claims that the revolution has been “betrayed” and that the later leaders and followers have failed to live up to the high standards set by their predecessors in the insurgency. None of this is surprising when such movements are examined in terms of their processes, including the incentives and constraints at work.

First of all, the kinds of people attracted to the original insurgency, under the initial set of incentives and constraints, tend to be very different from the kinds of people who gravitate to it after it has become successful and achieved a major part of its goals. By definition, an insurgent movement forms under a set of incentives and constraints very different from those which it seeks to create. Often the members face a certain amount of hostility, or even persecution, from those around them or from an elite currently benefitting from the status quo. These original insurgents may even face dangers to their careers or to their lives. These are not conditions which tend to attract timid careerists or mere opportunists, unless the opportunists foresee a high probability that the insurgency will succeed within a period of time that is relevant to their personal ambitions.

After the success of the insurgency, however, radically different incentives and constraints are created by that very success. Many Christians in the Roman Empire, for example, went from being a poor and persecuted minority to being among the powerful agents of a state religion, able to enrich themselves and to persecute others. A similar pattern marked the history of the Communists in Russia many centuries later. On a smaller scale, the life cycle of regulatory agencies in the United States has often been seen to follow a pattern leading to control by very different kinds of people and policies from those behind the movement which led to creation of the agency in the first place.³ Cries of the betrayal of the original ideals of the American civil rights movements have also been widely heard in recent years, along with lamentations about the caliber of the later individual and organizational leadership of that movement.

None of this should be surprising. A successful insurgency not only presents different incentives and constraints to its successors, that success itself can winnow out its original supporters in a non-random way. Opportunists within the insurgency are among those most likely to remain after the spoils of success become available, while those most activated by the original ideals that have now been largely achieved may be among those more likely to drift away, either to private life or to other crusades on other issues that remain unresolved in the society. Outside opportunists and careerists are also likely to be attracted in growing numbers over the years. Degeneracy and betrayal are hardly surprising under such circumstances.

The point here is not to single out insurgent movements for criticisms that might be equally (or more) applicable to supporters of the status quo. Indeed, much of the current status quo is the result of prior insurgencies. The more general and more important point is to distinguish between (1) examining issues and institutions in terms of their process characteristics versus (2) examining them in terms of their proclaimed goals or ideals.

Among the ways in which various decision-making processes differ is in the extent to which they are institutionally capable of making incremental trade-offs, rather than attempting categorical “solutions.” Consumers continually make incremental trade-offs when deciding what to buy in supermarkets or in automobile dealerships, but appellate courts may have only a stark choice to make between declaring a statute constitutional or unconstitutional. This is one of the central points explored in the first half of Knowledge and Decisions and remains as relevant today as when the book was written. The importance of incremental trade-offs—whether in economic, social, or personal decisions—is emphasized here, in part because it is so often lost in the shuffle of more emotionally appealing categorical priorities that are its antithesis. The distinction between incremental trade-offs and categorical “solutions” has been highlighted by recent trends in laws, policies, and judicial decisions creating such categorical goals as protecting endangered species, eliminating the last “vestiges” of segregation, creating innumerable “rights,” and promoting “safety” of many sorts—all this not explicitly at all costs, but often in practice treating costs as somehow unworthy considerations to be almost always over-ridden.

Diminishing returns alone can make categorical decision-making counterproductive in its impact, when the point is reached where trivial amounts of one thing are being gained at the cost of devastating losses of another. But, even aside from diminishing returns, categorical decision-making means that the very benefit being sought in one form may be sacrificed in another form. One particular kind of safety, for example, may be achieved by creating vastly greater dangers of another kind, as when pesticides are banned to eliminate their residual dangers in the environment, at the cost of a thousand-fold increase in the incidence of deadly, insect-borne diseases, such as malaria.

The second half of Knowledge and Decisions looks at how different kinds of decision-making processes have been evolving and how particular decisions once made in one kind of place are increasingly being made in other kinds of places—in the schools, rather than in the home, in the courts rather than in the marketplace, and so on. More important, the implications of such changes in the locations of decisions are examined, not only in terms of the accuracy and scope of the knowledge conveyed to different decision makers, but also in terms of what it all means for human freedom.

The analysis in the first half of this book stands as I wrote it more than fifteen years ago. More examples could be added now, but are not needed. However, the trends discussed in the second half have, of course, continued to evolve and so will have to be updated here. The central theme of the second half of Knowledge and Decisions is contained in two sentences in Chapter 7:

In many areas, these trends have clearly continued, but in some—especially in the economic area during the 1980s—there was some movement toward reversing such trends. From a global perspective, the collapse of communism in Eastern Europe and of apartheid in South Africa in the 1990s both represent trends towards greater individual freedom. What history will make of these mixed trends of the past fifteen years is of course a question which only the future can answer. What can be done today is to examine these developments in more detail in social, economic, and political terms.

SOCIAL TRENDS

In one sense, it might seem that there has been a growing trend in the United States, and in Western civilization generally, toward greater individual freedom from both government control and social control. Words that could not have been used in public just a generation or so ago are now broadcast through the mass media. Both still and motion pictures that would have been banned then are today not only widely available but are even being shown to children in the public schools. Flouting convention haws become almost a convention itself and, for critics of existing society, indignation has become a way of life. As with so many other seemingly revolutionary developments in history, however, the question must be raised whether there has been a net diminution of taboos or a substitution of new taboos and repressions for old. From the standpoint of a study of the social coordination of knowledge and the role of decision-making processes, further questions must be raised as to whether these new social trends represent better or worse ways of coping with the inherent inadequacies of human knowledge when making decisions.

One of the central themes of Knowledge and Decisions is that feedback mechanisms are crucial in a world where no given individual or manageably-sized group is likely to have sufficient knowledge to be consistently right the first time in their decisions. These feedback mechanisms must convey not only information but also incentives to act on that information, whether these incentives are provided by prices, love, fear, moral codes, or other factors which cause people to act in the interest of other people. Moreover, not all information is new information. History is a vast storehouse of experience from generations and centuries past. So are traditions which distill the experiences of millions of other human beings over millennia of time. How are all these things affected by the new social trends?

Whether in the media, the arts, or educational institutions from the kindergarten to the university, old taboos have been replaced by new ones. Well-known entertainer Anita Bryant vanished from the media and became a nonperson almost immediately after voicing criticisms of homosexuality. Criticism of any aspect of the values or way of life of any racial or ethnic minority would threaten a similar social extinction to the career of any television commentator, university professor, or newspaper editorial writer so bold as to challenge the new taboos. Merely a failure to use the ever-growing list of “politically correct” terms for all sorts of things can have serious repercussions. Moreover, more than a passive imposition of taboos is involved. Very often, these taboos are accompanied by militant promotions of new social visions throughout educational, religious, and other institutions. The instruments of intimidation include vaguely-worded speech codes under which students may be punished or expelled from many colleges, insulting harangues by “diversity consultants” employed by corporations, colleges, and other institutions, and threats to the careers of military officers, civilian officials, and corporate executives who do not march in step with the new orthodoxies.

The specifics of these visions can be left to be explored elsewhere, as they have been.⁴ What is important in the present context is the question as to how they affect the coordination of knowledge and the functioning of feedback mechanisms that govern decision-making processes.

Many representatives of the new orthodoxies question the very existence of the knowledge which is crucial to decision making. To them, tested knowledge is nothing more than “socially constructed” beliefs—which can be readily replaced by other beliefs which they will construct. The many social verification processes which weed out failing notions and preserve validated knowledge thus disappear from the discourse, as if by sleight of hand, when ideas and practices are seen as merely “constructed” and thus capable of being “deconstructed,” whether in literature, law, or other fields.

The apparent sophistication of this approach can be scrutinized with a physical example, in order to avoid the distractions of ideological presuppositions. Eyesight is, in some sense “constructed,” because it is not merely a matter of light entering the eye and travelling to the optic nerve. From these light patterns the brain must construct a world and project it outward as something that we see. For example, it is not these light patterns themselves but our presuppositions about perspective which enable us to decide that the chair next to us, which looms much larger in our field of vision than the automobile across the street, is nevertheless not as big as the automobile. We know that dogs do not see the same world we do because they are color-blind and that other creatures with different kinds of eyes, and creatures with sonarlike perception systems, such as bats, must construct their picture of the world from different raw materials of the senses. But does any of this mean that what we see is merely a set of conventions, no more valid than an abstract painting or a vision to be conjured up by the words of articulate writers or orators, or by psychedelic drugs?

Would anyone walk into a lion’s cage because both the lion and the cage, as we see them, are ultimately things constructed in our brains? More important, why not? Only because the verification processes so deftly made to disappear in theory could become very quickly, very brutally, and very agonizingly apparent. That is also the very reason why dogs do not run into a roaring flame and why bats swerve to avoid colliding with a stone wall. All these differently constructed worlds are subjected to verification processes. All these creatures’ worlds, like our own, are indeed “perceptions” but they are not just perceptions. The position of the observer is indeed an integral part of the data, but it is not the only part of the data.

The whole approach of Knowledge and Decisions is the antithesis of that of deconstructionism, for here the prevailing theme is that there is an independent reality which each individual perceives only imperfectly, but which can be understood more fully with feedback that can validate or invalidate what was initially believed. This is applied not only to physical reality but also to social realities, whose many ramifications may not all be understood by any given individual, but whose feedback nevertheless forces the decision maker to change course in spite of whatever predilections that decision maker may have. To take a trivial and non-controversial example, the initial decision of the Coca-Cola company to change the flavor of their drink to what they thought the public would prefer was rescinded with embarrassing haste when the market response belied the company’s expectations. Stock markets are likewise an ongoing economic referendum on what goods and services people do and don’t want, often disappointing—and punishing—those who guessed wrong, even with the best professional advice available.

Given the crucial importance of feedback in using knowledge to make decisions, the transfer of decisions from one kind of institution to another raises serious and even grave questions as to which institution is inherently more open to feedback and which more thoroughly insulated from it. The nature of the feedback process is also important: Is it mere articulation, in which some may have great talents without a corresponding depth of understanding, and in which others may choose to listen to or ignore, or is it inarticulate but powerful mechanisms ranging from money to love?

Plain and commonsensical as this approach may seem, it goes directly counter to the way many of the issues of the day are discussed. Much of the literature on racial or sexual prejudices and their discriminatory economic effects, for example, proceeds in utter disregard of knowledge-validation processes, such as competition in the marketplace. It has often been asserted that women receive only about two thirds of what men receive for doing the same work. While this assertion is open to very serious challenge on empirical grounds,⁵ the more relevant analytical point here is that it treats employers’ perceptions as if they were independent of the validation processes of economic competition. For women to be paid only two thirds of what men are paid for doing the same work with the same productivity would mean that an employer’s labor costs would be 50 percent higher than necessary with an all-male labor force. If all that was involved was blind prejudice, that might seem to be a viable situation. But even a cursory consideration of the economic implications of trying to compete and survive in the marketplace with labor costs 50 percent higher than they need be must at the very least raise serious questions. Similarly, the owner of a professional basketball team might read Mein Kampf and become a convinced racist but, if he were then to refuse to hire black basketball players, would there be no economic repercussions—or would he be more likely to disappear as a basketball club owner via the bankruptcy courts?

Note that what is involved here is not enlightened self-interest on the part of individual economic decision makers but the systemic effects of competitive processes which winnow out those whose decisions diverge most from reality. Under special circumstances, such as those of a government-regulated monopoly or cartel, the costs of arbitrary discrimination can be reduced or eliminated, thereby allowing discrimination to continue indefinitely because of insulation from market feedback. The point here is not that discrimination is impossible, nor even an attempt to assess how much discrimination there is. The point is precisely that we must look at the actual characteristics of decision-making processes—their incentives and constraints—if we want to gauge the likely outcomes of particular decisions in particular circumstances. Put differently, sweeping assertions about the consequences of perceptions alone—even racist or sexist perceptions—ignore inherent circumstantial constraints, such as those affecting dogs, bats, and people. Much, if not most, of what is said about many of the great issues of the day pays little or no attention to these kinds of concerns, which predominate in the first half of Knowledge and Decisions.

The growing prevalence of words like “perceptions,” “stereotypes,” and “socially constructed” serves ultimately to mute or eradicate the distinction between ideas and realities. Yet it is precisely the role of feedback through decision-making processes to sharpen that distinction. The disparagement of facts in history, or of original meanings of words and phrases in the Constitution, is part of the more general tendency to treat reality as plastic and the fashions of the times as equal to, or better than, the evolved understandings produced by experience and validated by the assent of successive generations. When works of literature which have gained the respect of generation after generation of readers are called “privileged” writings, not only is a validation process made to disappear into thin air but the very concept of achievement ex post is equated with a privilege ex ante.

Economic achievement, for example, is often seen as mere “privilege” and failure as “disadvantage,” again obliterating the distinction between the ex ante and the ex post, to the detriment of any empirical study of the foundations of achievement and failure, since the very distinction itself vanishes by verbal magic.

ECONOMIC TRENDS

Knowledge and Decisions was published some months before Ronald Reagan was elected president of the United States. His eight years in office were marked by both by changing economic conditions in the United States and by changes in economic decision-making institutions and processes—the “Reagan revolution”—that were imitated by many other countries around the world. Whether that revolution marked an enduring change in economic and social institutions, or just another passing blip on the great screen of history, is the large, unanswered question of our time.

The most striking aspect of the decade of the 1980s in the United States was that it marked the longest peacetime economic expansion in history.⁶ Despite various attempts to rewrite this history,⁷ standards of living rose in the country at large and government tax receipts rose by hundreds of billions of dollars, even as tax rates were reduced. The enduring significance of this economic boom was that it inspired other governments, of both the political left and right, to make similar institutional changes, such as privatizing government-run enterprises and reducing the degree of government regulation in general. As far away as New Zealand, “Reaganomics” inspired “Rogemomics,” named for finance minister Roger Douglas of the Labour Party, who instituted similar policies there. Many third-world countries which emerged into independence during the 1960s and took socialism as axiomatically both the most efficient and the most humane form of economy were nevertheless later forced, either by their own bitter experience or by the demands of outside lending agencies, to privatize and deregulate their economies.

All this represented a break with the trends discussed in Chapter 8 of this book, toward a movement of economic decisions into governmental institutions. One symptom of the change in the United States was that the steady growth in the size of the Federal Register