SOCIAL FORECASTING

Forecasting has been important in sociological thought. Early European sociologists argued that societies progress through inevitable historical stages; those theories helped sociologists predict all societies' futures. Early American sociologists adopted the pragmatists' rule that a science proves it "works" by predicting future events (Schuessler 1971). Sociologists, however, have only recently adopted methods appropriate for those early goals. The review in this article of the delayed development of social forecasting includes (1) three sociologists' conceptual uses of forecasting and some reasons their suggestions were not followed, (2) qualitative and quantitative methods of forecasting, and (3) recent indications of increased interest in forecasting.

FORECASTING TRADITIONS

Sociologists have contributed several social forecasting concepts that were historically significant enough to become traditional orientations in the analysis of the future. William F. Ogburn "held that in the modern world technological inventions commonly come first and social effects later. By reason of this lag, it is possible, he argued, to anticipate the future and plan for its eventualities" (Schuessler 1971, p. 309). For example, new possibilities came into conflict with family values when the invention of effective birth control gave women new choices. Ogburn's contribution was to suggest that cultural lags are inevitable but that the period of disruption they cause can be shortened (Reiss 1986).

Merton (1949) challenged Ogburn's idea that the effects of inventions can be easily anticipated. Each invention has an apparent goal, or manifest function, that it is hoped it will perform in society. Each change, however, also contains the possibility of performing a number of latent functions. These are unanticipated side effects that often are not desired and sometimes are dangerous. The institutions of society are closely intertwined, and an invention in one area can cause shocks throughout the system. The automobile is an example. Its manifest function of changing transportation has been fulfilled, but at the cost of serious ecological and sociological changes.

Merton's (1949) second warning was that social forecasting is unique because it tries to predict the behavior of humans, who change their minds. The self-fulfilling prophecy is a forecast that makes people aware of real or imagined new opportunities or dangers to be avoided. Merton demonstrated that false forecasts can have powerful effects if they gain public acceptance. For example, a sound bank can be destroyed by a run on its funds caused by a prediction of failure. Henshel's more inclusive concept—the self-altering prediction—shows that forecasts can be self-defeating as well as self-fulfilling. W. I. Thomas's theorem, "If men define situations as real, they are real in their consequences," applies particularly to the definitions societies make of the future (Henshel 1978, p. 100).

Moore challenged sociologists to go beyond safe prophecy based on orderly trends and attack the difficult problem of "how to handle sharp changes in the magnitude of change, and sharp (or at least clear) changes in direction" (Moore 1964, p. 332). There are four types of discontinuous societal change: (1) Some societies are changed drastically by an exogenous variable, an idea or value from another society. Modern Japan is an example. (2) A society's rate of development can increase spontaneously, creating an abundance of new ideas. This is an exponential acceleration, a change in the rate of change. (3) Moore attributes changes in the direction of change to the existence of a dialectic of values in each society's apparent trend. For example, a society may appear to be profit-oriented and ecologically exploitative, but there also exists a counterset of values that stress harmony with each other and with nature. If a shift in such basic value emphases could be predicted, many other associated forecasts could be made. (4) Finally, Moore recognizes that there are pure emergents, inventions such as money and writing, that cannot be thought of as parts of trends.

Moore drew a methodological moral from these complexities: "One must somehow move from discrete necessary conditions to cumulative and sufficient ones" (Moore 1964, p. 334). That is, the search for the one trend or causal variable that drives societal change should be abandoned. The summation and particularly the interaction of many component developments create events.

In 1966 Moore asked sociologists to put aside value-free scientific rules and attempt to construct preferable futures that might help "mankind survive for the next twenty years" (Moore 1966, p. 270). Moore was confronting what he felt to be the main reason why forecasting was done so infrequently. It is professionally permissible for sociologists to examine social change both currently and retrospectively, but making a forecast leaves one liable to being labeled a utopian (Winthrop 1968, p. 136). Utopian thinking is in disrepute because past advocates allowed their values to cloud their constructions. However, images of the future provide goals and determine how people plan and therefore how they behave in the present. Moore sought utopias that would perform a necessary social planning function by constructing alternative directions for human purpose.

WHY SOCIAL FORECASTING HAS DEVELOPED SLOWLY

Sociologists' basic methodological orientations preclude an interest in forecasting. Sociologists analyze society's static interconnections and concentrate on the social structures that persist. They have not developed skill in isolating the sequences of dynamic social behavior (Moore 1966). They are better at categorizing and typing people than at predicting how individuals might change from one type to another.

Many sociologists feel that not enough is known to predict future events. They point to economists and demographers and ask, If they are failing with their more quantifiable data, how can complex social changes be anticipated? One school of thought sees sociology as a qualitative art form that will never be a statistically modeled science. Critical sociologists object on moral grounds. They feel that society requires essential restructuring before positive change can be effected. Since most forecasting is based on models of the current structure, they feel that it sanctions unjust social arrangements (Henshel 1982).

JUDGMENTAL AND QUALITATIVE FORECASTING METHODS

The futurists (Bell 1997; Kurian and Molitor 1996) see "the challenge being not just to forecast what the future will be, but to make it what it ought to be" (Enzer 1984, p. 202). The actual future is too complex to be predefined, but possible futures can be constructed that can be instructive. In addition, secondary forecasts can be made that estimate the effects of policy actions on the original course of development (Colquhoun 1996). The pace of change is considered too rapid to be captured by traditional methods reliance on a careful quantitative reconstruction of the past. This justifies the use of experts' opinions, and futurists' methods are ways of systematizing those judgments (Allen 1978, p. 79).

A discontinuous social change usually is preceded by a "substantial restructuring of basic tenets and beliefs" (Holroyd 1978, p. 37). Such paradigm shifts are revolutionary, such as the rejection of the earth as the center of the universe. They appear in fields of knowledge in which one system of thought seems to be in control but is unable to solve important problems. Holroyd, for example, predicts a paradigm shift in economics because its current theories are unable to deal with essential problems such as scarcity of natural resources. Futurists anticipate shifts by compiling lists of crucial issues in the institutions of society. When the gap between current and desired conditions is large, that area is monitored closely for discontinuous change (Holroyd 1978, p. 38).

Cross-impact matrices are constructed by listing all possible future events in the problem area under study (Allen 1978, pp. 132–145). Each event is recorded as a row and a column in a square matrix. This allows the explicit examination of every intersection of events when one asks: What is the probability that the first will occur if it has been preceded by the other? The probabilities of occurrence can be derived from available data but are often judgments. Cross-impact analysis is a systematic way of heeding Merton's warning about not overlooking possibly damaging latent consequences. It is a tool for spotting crucial turning points or originating novel viewpoints by examining the intersections of change at which experts' judgments conflict.

Delphi surveys constitute an ingenious method for allowing the interaction of expert judgments while avoiding the contamination of social status or damage to reputations because of radical or mistaken pronouncements (Henshel 1982). In a series of survey rounds, everyone sees the distribution of others' responses without knowing the proponents' identities. A composite forecast emerges as anonymous modifications are made at each round.

After a review of forecasting methods, Ascher (1978) chose scenarios as one of only two methods he could recommend. A scenario is "a hypothetical sequence of events constructed for the purpose of focusing attention on causal processes and decision points" (Herman Kahn, quoted in Wilson 1978, p. 225). It is a story, but a complex one based on all available data and usually constructed after a cross-impact analysis has isolated possible turning points. Usually, two or three related scenarios are constructed to illustrate alternative futures that could be determined by particular decisions.

It is not surprising that an expert's decision process can be made explicit. What is surprising is that in many studies the systematic model of an expert often forecasts better than the person does (Armstrong 1978). In bootstrapping, the forecaster's individualized decision procedures become the "bootstraps" by which a systematized procedure is "lifted" into an orderly routine. Such a model can be made deductively through interviews that isolate and formalize the decision rules or inductively by starting with a series of past forecasts and attempting to infer the rules that accounted for the differences between them.

Metaforecasting (Makridakis 1988) represents an essential summary of these considerations and a bridge to more quantitative methods. It combines judgmental and statistical estimates. It attempts to include historical and social information to overcome the tendency to ignore or overreact to changes in established patterns or relationships.

SOCIAL DEMOGRAPHY

Demography is the most established form of social forecasting, and its methods and record can be found elsewhere (Henshel 1982). This article will discuss only two elements from its continuing development: a method that has had wide influence and what can be learned from its frequent failures to predict future population sizes.

A cohort is an aggregate of individuals of similar age who therefore experience events during the same time period (Reiss 1986, p. 47). Cohort analysis was first used by Norman Ryder to study the changing fertility behaviors of women born during the same five-year periods. Since that time, cohorts have been used in the study of many areas of social change to differentiate the changes that are result from individuals maturing through the stages of life from those caused by powerful societal events or value shifts.

Demographers failed to anticipate the postwar baby boom and the onset of its decline. These errors were due to assumption drag, "the continued use of assumptions long after their validity has been contradicted by the data" (Ascher 1978, p. 53). Henshel (1982) says that demographers probably ignored these turning points because they simply talked to each other too much. They reassured each other that their assumptions and their extrapolations from past trends would soon reassert themselves in the data. Recognition of this error of developing an isolated club of forecasters has helped economists and will help sociologists avoid a similar regimentation of estimates.

The mix of assumptions and actual data varies widely in simulation models. The most useful models test a set of explicit assumptions so that no interactions between variables are overlooked. Models have contributed the idea of the feedback loop as an important caution against unidirectional thinking. This common system characteristic occurs when an effect reaches a sensitive level and begins a reaction that modifies its own cause (Simmons 1973, p. 195). Often, however, the mix of assumptions and facts in simulations leans too heavily toward judgments. So-called black-box modeling (McLean 1978), in which equations are hidden, can produce output that is plausible and provocative but also unrealistic. The creator of the Limits to Growth study admitted that "in World Dynamics . . . there is no attempt to incorporate formal data. . . . All relationships are intuitive" (Simmons 1973, p. 208). That study extrapolated what have come to be seen as extreme assumptions of geometric growth unchecked by social adaptation. Its dramatic predictions of imminent shortages had a wide but unwarranted impact (Cole et al. 1973). A comment on those failed predictions and their popularity at the time of their publication sets the context in which all "modeled" forecasts should be received: "The apparent detached neutrality of a computer model is as illusory as it is persuasive. Any model of a social system necessarily involves assumptions about the workings of that system, and these assumptions are necessarily colored by the attitudes and values of the individuals or groups concerned. . . . [C]omputer models should be regarded as an integral part of political debate. . . . The model is the message" (Freeman 1973, p. 7).

PRAGMATIC STATISTICAL ANALYSIS OF TIME SERIES

Attention has shifted to techniques that are less concerned with demonstrating the effects of assumed patterns. Time series are records of observations through time. Traditional time series analysis projects "future values of a variable based entirely on the past and present observations of that variable" (Levine et al. 1999). It involves isolating the trend inside the many "noisy" or seasonal factors that may obscure it. The techniques have been well developed, are taught in undergraduate management statistics courses, and have been adapted for spreadsheet software available on most computers. The problem, however, is how much faith one can put in the idea that "people do what they usually do." Time series projections are essential first steps in discovering patterns of behavior of aggregates of people over time. Such patterns often persist, but some shock (invention, immigration, social redefinition such "the sixties," or adjustment of tradition such as decreasing sexism) may cause disruption. In recognition of these sociological disruptions, time series are being explored from the viewpoint that any variable may be uniquely complex and subject to sudden change.

Time series regressions uncover structural relationships involved in the history of two or more variables. Before the relationship can be assessed, sources of error must be isolated and controlled. The most important of these errors are (1) the overall trend of change that would obscure any specific interrelationship and (2) the autocorrelation effect of internal dependence of an observation on previous observations. If a relationship seems to explain the data series' movements, it is tested with ex-post forecasts that can be verified within the range of available data. If these succeed, "ex-ante-forecasts can be used to provide educated guesses about the path of the variables into the blind future" (Ostrom 1990, p. 77).

Autoregressive moving average (ARIMA) models predict a variable's current status by using a combination of its previous observations and mathematically approximated random shocks. The goal is to find a pattern that fits the immediate data, not to understand relationships. ARIMA models are useful in interrupted time series analysis, in which the impact of a policy or another intervention can be examined by seeing how different the variable's patterns are before and after the intervention (McDowall et al. 1980). Autoregressive models have a limitation important for social forecasting, in which historical data are relatively scarce. "Because ARIMA models must be identified from the data to be modeled, relatively long time series are required" (McCleary and Hay 1980, p. 20). Fifty observations are recommended.

Exponential smoothing is widely used and is as reliable as more complicated methods (Gardner 1985). In its simplest form, the next period's forecast is based on the current forecast plus a portion of the error it made. That is, the difference between the current time period's forecast and the actual value is weighted and used to adjust the next period's expected value. The higher the value of the weight used is, the more the error adjustment contributes and the more quickly the model will respond to changes. Exponential smoothing is used in early detection of curvilinear changes, when the rate of change speeds or slows (Gardner 1987).

FUTURE TRENDS

Forecasting is being done. It is central in business and government planning. Even though many of these forecasts' essential variables are social or are found in social contexts (such as family decisions to move, build, and purchase or the development of social problems), economists have become society's designated forecasters (Henshel 1982; Stimson and Stimson 1976). Sociologists will not change this imbalance easily, but there are some indications that forecasting may finally become part of everyday sociological work.

Assumptions that a particular cycle or curve is the natural or underlying process of all change have been abandoned, and pragmatic methods are now widespread. It is also accepted that a forecast is developed only to be monitored for possible discontinuities. Trend extrapolations rarely are done without accompanying methods for describing the expected deviations.

Two forecasting methods are particularly promising because they allow sociologists to build on traditional skills. Componential or segmentation forecasting (Armstrong 1978) recognizes that an aggregate forecast can be improved by combining forecasts made on the population's component social groups. Sociologists are best able to distinguish the groups that should be treated separately. Pooled time series analysis (Sayrs 1989) combines cross-sectional descriptions such as one-time surveys. Sociologists are expert at describing interconnections in the structures of organizations or societies, and now they have the opportunity to study these social arrangements over time.

Society has recognized the wisdom of the early concern about anticipating the latent effects of social and technological inventions. Progress no longer seems inevitable. The popular question now is, Can someone assure us that a new element will not be as destructive as past changes?

Sociologists seem to be uniquely suited to help forecasting become more plausible because their working assumptions counter the weaknesses of current methods. The idea that technological innovation or economic cycles drive social change has produced today's mechanistic, ultrarational, anti-individualistic models that assume that the population is homogeneous (Dublin 1992). All these weaknesses are naturally contradicted when sociologists expand their vision of a population to include the cultural diversity of the social contexts that produce, accept or reject, and always modify the effects of technological and economic circumstances.

The future acceptance of forecasting also depends on sociologists' ability to improve the preparation and presentation of forecasts by using their traditional strengths. Forecasts will be accepted by policymakers and the public only when the quasi-theories they hold about the future are specifically addressed and proved false. Sociologists know this better than other social scientists do; they often are called on to dispel labels and popular theories that are so entrenched that they make any new attempt at explanation seem a "fool's experiment" to the forecaster's audience They also are used to the idea of various and multiple causes acting in a situation and therefore are skilled at isolating "unanticipated consequences."

Forecasts will improve and become more plausible when they place less importance on traditional scientific formulations. A forecast is not a hypothesis. Hypotheses must be made in advance of the behavior they are meant to predict to assure a full and objective test of the theories that produced them. Forecasts demand monitoring of predictions and adaptation of forecasts to circumstances. A forecast is as good as its ability to anticipate and allow the inclusion of changing social forces. That is, its main function is not to make an accurate prediction of future events but to isolate and interrelate the many factors in the current situation that may be causally powerful. Understanding the current social situation's complexity is the most important factor.

references

Allen, T. Harrell 1978 New Methods in Social ScienceResearch: Policy Sciences and Futures Research. New York: Praeger.

Armstrong, J. Scott 1978 Long-Range Forecasting: FromCrystal Ball To Computer. New York: Wiley.

Ascher, William 1978 Forecasting: An Appraisal forPolicymakers and Planners. Baltimore: Johns Hopkins University Press.

Bell, Wendell 1997 Foundations of Futures Studies, vols. I and II. New Brunswick, N.J.: Transaction.

Cole, H. S. D., Christopher Freeman, Marie Jahoda, and K. L. R. Pavitt, eds. 1973 Models of Doom: A Critique ofthe Limits to Growth. New York: Universe.

Colquhoun, Robert 1996 "The Art of Social Conjecture: Remembering Bertrand de Jouvenel." History of theHuman Sciences 9(1): 27–42

Dublin, Max 1992 FutureHype: The Tyranny of Prophecy. New York: Plume.

Enzer, Selwyn 1984 "Anticipating the Unpredictable." Technological Forecasting and Social Change 26:201–204.

Freeman, Christopher 1973 "Malthus with a Computer" In H. S. D. Cole, Christopher Freeman, Marie Jahoda, and K. L. R. Pavitt eds., Models of Doom: A Critique ofthe Limits to Growth. New York: Universe.

Gardner, Everette S., Jr. 1985 "Exponential Smoothing: The State of the Art." Journal of Forecasting 4:1–28.

—— 1987 "Short-Range Forecasting." LOTUS 3:54–58.

Henshel, Richard L. 1978 "Self-Altering Predictions." In Jib Fowles, ed., Handbook of Futures Research. Westport, Conn.: Greenwood Press.

—— 1982 "Sociology and Social Forecasting." In Ralph H. Turner and James F. Short, eds., AnnualReview of Sociology, vol. 8. Palo Alto, Calif.: Annual Reviews.

Holroyd, P. 1978 "Change and Discontinuity: Forecasting for the 1980s." Futures 10:31–43.

Kurian, George T., and Graham Molitor 1996 Encyclopedia of the Future, vols. I and II. New York: Macmillan.

Levine, David M., Mark L. Berenson, and David Stephan 1999 Statistics for Managers Using Microsoft Excel. Upper Saddle River N.J.: Prentice-Hall

Makridakis, Spyros 1988 "Metaforecasting: Ways of Improving Accuracy and Usefulness." InternationalJournal of Forecasting 4(3):467–91.

McCleary, Richard, and Richard A. Hay, Jr. 1980 AppliedTime Series Analysis for the Social Sciences. Beverly Hills, Calif.: Sage.

McDowall, David, Richard McCleary, Errol E. Meidinger, and Richard A. Hay, Jr. 1980 Interrupted Time. Newbury Park, Calif.: Sage.

McLean, J. Michael 1978 "Simulation Modeling." In Jib Fowles, ed., Handbook of Futures Research. Westport, Conn.: Greenwood Press.

Merton, Robert K. 1949 Social Theory and Social Structure, rev. ed. Glencoe, Ill.: Free Press.

Moore, Wilbert E. 1964 "Predicting Discontinuities in Social Change." American Sociological Review 29:331–338.

—— 1966 "The Utility of Utopias." American Sociological Review 31:756–772.

Ostrom, Charles W., Jr. 1990 Time Series Analysis. Regression Techniques, 2nd ed. Newbury Park, Calif.: Sage.

Reiss, Albert J., Jr. 1986 "Measuring Social Change." In Neil J. Smelser and Dean R. Gerstein, eds., Behavioraland Social Science: Fifty Years of Discovery. Washington, D.C.: National Academy Press.

Sayrs, Lois W. 1989 Pooled Time Series Analysis. Beverly Hills, Calif.: Sage.

Schuessler, Karl 1971 "Continuities in Social Prediction. "In H. L. Costner, ed., Sociological Methodology,1971. San Francisco: Jossey-Bass.

Simmons, Harvey 1973 "System Dynamics and Technocracy." In H. S. D. Cole, Christopher Freeman, Marie Jahoda, and K. L. R. Pavitt eds., Models of Doom:A Critique of the Limits to Growth. New York: Universe.

Stimson, John, and Ardyth Stimson 1976 "Sociologists Should Be Put to Work as Forecasters." AmericanSociologist 11:49–56.

Wilson, Ian H. 1978 "Scenarios." In B. Fowles, ed. Handbook of Futures Research. Westport, Conn.: Greenwood Press.

Winthrop, Henry 1968 "The Sociologist and the Study of the Future." American Sociologist 3:136–145.

John Stimson