SAMPLE SELECTION BIAS

In a linear regression model, sample selection bias occurs when data on the dependent variable are missing nonrandomly, conditional on the independent variables. For example, if a researcher uses ordinary least squares (OLS) to estimate a regression model in which large values of the dependent variable are underrepresented in a sample, estimates of slope coefficients typically will be biased.

Hausman and Wise (1977) studied the problem of estimating the effect of education on income in a sample of persons with incomes below $15,000. This is known as a truncated sample and is an example of explicit selection on the dependent variable. This is shown in Figure 1, where individuals are sampled at three education levels: low (L), middle (M), and high (H). In the figure, sample truncation leads to an estimate of the effect of schooling that is biased downward from the true regression line as a result of the $15,000 ceiling on the dependent variable. In a variety of special conditions (Winship and Mare 1992), selection biases coefficients downward. In general, however, selection may bias estimated effects in either direction.

A sample that is restricted on the dependent variable is effectively selected on the error of the regression equation; at any value of X, observations with sufficiently large positive errors are eliminated from the sample. As is shown in Figure 1, as the independent variable increases, the expected value of the error becomes increasingly negative, making these two elements negatively correlated. Because this contradicts the standard assumption of OLS that the error and the independent variables are not correlated, OLS estimates become biased.

A different type of explicit selection occurs when the sample includes persons with incomes of $15,000 or more but all that is known about those persons is their educational attainment and that their incomes are $15,000 or more. When the dependent variable is outside a known bound but the exact value of the variable is unknown, the sample is censored. If these persons' incomes are coded as $15,000, OLS estimates are biased and inconsistent for the same reasons that obtain in the truncated sample.

A third type of selection that leads to bias occurs when censoring or truncation is a stochastic function of the dependent variable. This is termed implicit selection. In the income example, individuals with high incomes may be less likely to provide information on their incomes than are individuals with low incomes. As is shown below, OLS estimates also are biased when there is implicit selection.

Yet another type of selection occurs when there is selection on the measured independent variable(s). For example, the sample may be selected on educational attainment alone. If persons with high levels of schooling are omitted from the model, an OLS estimate of the effect for persons with lower levels of education on income is unbiased if schooling has a constant linear effect throughout its range. Because the conditional expectation of the dependent variable (or, equivalently, the error term) at each level of the independent variable is not affected by a sample restriction on the independent variable, when a model is specified properly OLS estimates are unbiased and consistent (DuMouchel and Duncan 1983).

EXAMPLES

Sample selection can occur because of the way data have been collected or, as the examples below illustrate, may be a fundamental aspect of particular social processes. In econometrics, where most of the basic research on selection bias has been done, many of the applications have been to labor economics. Many studies by sociologists that deal with selection problems have been done in the cognate area of social stratification. Problems of selection bias, however, pervade sociology, and attempts to grapple with them appear in the sociology of education, family sociology, criminology, the sociology of law, social networks, and other areas.

Trends in Employment of Out-of-School Youths. Mare and Winship (1984) investigate employment trends from the 1960s to the 1980s for young black and white men who are out of school. Many factors affect these trends, but a key problem in interpreting the trends is that they are influenced by the selectivity of the out-of-school population. Over time, that selectivity changes because the proportion of the population that is out of school decreases, especially among blacks. Because persons who stay in school longer have better average employment prospects than do persons who drop out, the employment rates of nonstudents are lower than they would be if employment and school enrollment were independent. Observed employment patterns are biased because the probabilities of employment and leaving school are dependent. Other things being equal, as enrollment increases, employment rates for out-of-school young persons decrease as a result of the compositional change in this pool of individuals. To understand the employment trends of out-of-school persons, therefore, one must analyze jointly the trends in employment and school enrollment. The increasing propensity of young blacks to remain in school explains some of the growing gap in the employment rates between blacks and whites.

Selection Bias and the Disposition of Criminal Cases. A central focus in the analysis of crime and punishment involves the determinants of differences in the treatment of persons in contact with the criminal justice system, for example, the differential severity of punishment of blacks and whites (Peterson and Hagan 1984). There is a high degree of selectivity in regard to persons who are convicted of crimes. Among those who commit crimes, only a portion are arrested; of those arrested, only a portion are prosecuted; of those prosecuted, only a portion are convicted; and among those convicted, only a portion are sent to prison. Common unobserved factors may affect the continuation from one stage of this process to the next. Indeed, the stages may be jointly determined inasmuch as legal officials may be mindful of the likely outcomes later in the process when they dispose cases. The chances that a person will be punished if arrested, for example, may affect the eagerness of police to arrest suspects. Analyses of the severity of sentencing that focus on persons already convicted of crimes may be subject to selection bias and should take account of the process through which persons are convicted (Hagan and Parker 1985; Peterson and Hagan 1984; Zatz and Hagan 1985).

Scholastic Aptitude Tests and Success in College. Manski and Wise (1983) investigate the determinants of graduation from college, including the capacity of the Scholastic Aptitude Test (SAT) to predict individuals' probabilities of graduation. Studies based on samples of students in colleges find that the SAT has little predictive power, yet those studies may be biased because of the selective stages between taking the SAT and attending college. Some students who take the SAT do not apply to college, some apply but are not admitted, some are admitted but do not attend, and those who attend are sorted among the colleges to which they have been admitted. Each stage of selection is nonrandom and is affected by characteristics of students and schools that are unknown to the analyst. When one jointly considers the stages of selection in the college attendance decision, along with the probability that a student will graduate from college, one finds that the SAT is a strong predictor of college graduation.

Women's Socioeconomic Achievement. Analyses of the earnings and other socioeconomic achievements of women are potentially affected by nonrandom selection of women into the labor market. The rewards that women expect from working affect their propensity to enter the labor force. Outcomes such as earnings and occupational status therefore are jointly determined with labor force participation, and analyses that ignore the process of labor force participation are potentially subject to selection bias. Many studies in economics (Gronau 1974; Heckman 1974) and sociology (Fligstein and Wolf 1978; Hagan 1990; England et al. 1988) use models that simultaneously represent women's labor force participation and the market rewards that women receive.

Analysis of Occupational Mobility from Nineteenth-Century Censuses. Nineteenth-century decennial census data for cities provide a means of comparing nineteenth- and twentieth-century regimes of occupational mobility in the United States. Although one can analyze mobility by linking the records of successive censuses, such linkage is possible only for persons who remain in the same city and keep the same name over the decade. Persons who die, emigrate, or change their names are excluded. Because mortality and migration covary with socioeconomic success, the process of mobility and the way in which observations are selected for the analysis are jointly determined. Analyses that model mobility and sample selection jointly offer the possibility of avoiding selection bias (Hardy 1989).

MODELS OF SELECTION

Berk (1983) provides an introduction to selection models; Winship and Mare (1992) provide a review of the literature before 1992. We start by discussing the censored regression, or tobit, model. We forgo discussion of the very closely related truncated regression model (Hausman and Wise 1977).

Tobit Model. The censored regression, or tobit, model is appropriate when the dependent variable is censored at an upper or lower bound as an artifact of how the data are collected (Tobin 1958; Maddala 1983). For censoring at a lower bound, the model is
where for the ith observation, Y1i* is an unobserved continuous latent variable, Y1i is the observed variable, Xi is a vector of values on the independent variables, εi is the error, and ß is a vector of coefficients. We assume that εi is not correlated with Xi and is independently and identically distributed. The model can be generalized by replacing the threshold zero in equations (2) and (3) with a known nonzero constant. The censoring point also may vary across observations, leading to a model that is formally equivalent to models for survival analysis (Kalbfleisch and Prentice 1980).

Standard Sample Selection Model. A generalization of the tobit model involves specifying that a second variable Y2i* affects whether Y1i is observed. That is, retain the basic model in equation (1) but replace equations (2) and (3) with Variants of this model depend on how Y2i is specified. Commonly, Y2i* is determined by a binary regression model: where Y2i* is a latent continuous variable. The classic example is a model for the wages and employment of women where Y1i is the observed wage, Y2i is a dummy variable indicating whether a woman works, and Y2i* indexes a woman's propensity to work (Gronau 1974). In a variant of this model, Y2i is hours of work and equations (6) through (8) are a tobit model (Heckman 1974). In both variants, Y1i* is observed only for women with positive hours of work. One can modify the model by assuming, for example, that Y1i is dichotomous. If εi and υi follow a bivariate normal distribution, this leads to a bivariate probit selection model.

Estimation of equation (1) using OLS will lead to biased estimates. When Y2i* > 0, The OLS regression of Y1i on Xi is biased and inconsistent if εi is correlated with υi − Ziα, which occurs if εi is correlated with υi or Zi or both. If the variables in Zi are included in Xi, εi and Zi are not correlated by assumption. If, however, Zi contains additional variables, εi and Zi may be correlated. When σευ = 0, selection depends only on the observed variables in Zi, not those in Xi. In this case, selection can be dealt with either by conditioning on the additional Z's or by using propensity score methods (Rosenbaum and Rubin 1983).

Equation (9) shows how selectivity bias may be interpreted as an omitted variable bias (Heckman 1979). The term E[εi| Y2i* > 0] can be thought of as an omitted variable that is correlated with Xi and affects Y1. Its omission leads to biased and inconsistent OLS estimates of ß.

Nonrandom Treatment Assignment. A model intimately related to the standard selection model that is not formally presented here is used when individuals are assigned nonrandomly to some treatment in an experiment. In this case, there are essentially two selection problems. For individuals not receiving the treatment, information on what their outcomes would have been if they had received treatment is "missing." Similarly, for individuals receiving the treatment, we do not know what their outcomes would have been if they had not received the treatment. Heckman (1978) explicitly analyzes the relationship between the nonrandom assignment problem and selection. Winship and Morgan (1999) review the vast literature that has appeared on this question in the last two decades.

ESTIMATORS

A large number of estimators have been proposed for selection models. Until recently, all these estimators made strong assumptions about the distribution of errors. Two general classes of methods—maximum likelihood and nonlinear least squares—typically assume bivariate normality of εi and υi. The most popular method is that of Heckman (1979), known as the lambda method, which assumes only that υi in equation (6) is normally distributed and E[εi|υi] is linear.

For a number of years, there has been concern about the sensitivity of the Heckman estimator to these normality and linearity assumptions. Because maximum likelihood and nonlinear least squares make even stronger assumptions, they are typically more efficient but even less robust to violations of distributional assumptions. The main concern of the literature since the early 1980s has been the search for alternatives to the Heckman estimator that do not depend on normality and linearity assumptions.

Heckman's Estimator. The Heckman estimator involves (1) estimating the selection model (equations [6] through [8]), (2) calculating the expected error, υi = E[υi|υi > -Ziα], for each observation using the estimated α, and (3) using the estimated error as a regressor in equation (1). We can rewrite equation (9) as If εi and υi are bivariate normal and Var (υi) = 1, then E(εi|υi) = σευυi and
where ϕ and Φ are the standardized normal density and distribution functions, respectively. The ratio λ(-Ziα) is the inverse Mills's ratio. Substituting equation (11) into equation (10), we get


where ηi is not correlated with both Xi and λ(-Ziα). Equation (12) can be estimated by OLS but is preferably estimated by weighted least squares since its error term is heteroskedastic (Heckman 1979).

The precision of the estimates in equation (12) is sensitive to the variance of λ and collinearity between X and λ. The variance of λ is determined by how effectively the probit equation at the first stage predicts who is selected into the sample. The better the equation predicts, the greater the variance of λ is and the more precise the estimates will be. Collinearity will be determined in part by the overlap in variables between X and Z. If X and Z are identical, the model is identified only because λ is nonlinear. Since it is seldom possible to justify the form of λ on substantive grounds, successful use of the method usually requires that at least one variable in Z not be included in X. Even in this case, X and λ(-Ziα) may be highly collinear, leading to imprecise estimates.

Robustness of Heckman's Estimator. Because of the sensitivity of Heckman's estimator to model specification, researchers have focused on the robustness of the estimator to violations of its several assumptions. Estimation of equations (6) through (8) as a probit model assumes that the errors υi are homoskedastic. When this assumption is violated, the Heckman procedure yields inconsistent estimates, though procedures are available to correct for heteroskedasticity (Hurd 1979). The assumed bivariate normality of υi and εi in the selection model is needed in two places. First, normality of υi is needed for consistent estimation of α in the probit model. Second, the normality assumption implies a particular nonlinear relationship for the effect of Ziα on Y2i through λ. If the expectation of εi conditional on υi is not linear and/or υi is not normal, λ misspecifies the relationship between Ziαand Y2i and the model may yield biased results.

Several studies have analytically investigated the bias in the single-equation (tobit) model when the error is not normally distributed. In a model with only an intercept—that is, a model for the mean of a censored distribution—when errors are not normally distributed, the normality assumption leads to substantial bias. This result holds even when the true distribution is close to normal (for example, the logistic) (Goldberger 1983). When the normality assumption is wrong, moreover, maximum likelihood estimates may be worse than estimates that simply use the observed sample mean. For samples that are 75 percent complete, bias from the normality assumption is minimal; in samples that are 50 percent complete, bias is substantial in the truncated case but not in the censored case; and in samples that are less than 50 percent complete, bias is substantial in almost all cases (Arabmazar and Schmidt 1982).

The fact that estimation of the mean is sensitive to distributional misspecification suggests that the Heckman estimator may not be robust and raises the question of how often such problems arise in practice. In addition, even when normality holds, the Heckman estimator may not improve the mean square error of OLS estimates of slope coefficients in small samples (50 or less) (Stolzenberg and Relles 1990). This appears to parallel the standard result that when the effect of a variable is measured imprecisely, inclusion of the variable may enlarge the mean square error of the other parameters in the model (Leamer 1983).

No empirical work that the authors know of directly examines the sensitivity of Heckman's method for a standard selection model. Work by LaLonde (1986) using the nonrandom assignment treatment model suggests that in specific circumstances the Heckman method can inadequately adjust for unobserved differences between the treatment and control groups.

Extensions of the Heckman Estimator. There are two main issues in estimating equation (12). The first is correctly estimating the probability for each individual that he or she will be selected. As it has been formulated above, this means first correctly specifying both the linear function Zα and second specifying the correct, typically nonlinear relationship between the probability of selection and Zα. The second issue is the problem of what nonlinear function should be chosen for λ. When bivariate normality of errors holds, λ is the inverse Mills's ratio. When this assumption does not hold, inconsistent estimates may result. Moreover, since Xi and Zi are often highly collinear, estimates of ß in equation 12 may quite be sensitive to misspecification of λ.

The first problem is handled rather easily. In the situation where one has a very large sample and there are multiple individuals with the same Z, the simplest approach is to estimate the probability of selection nonparameterically by directly estimating the probability of being selected for individuals with each vector of Z's from the observed frequencies. With smaller samples, kernel estimation methods are available. These methods also consist of estimating probabilities directly by grouping individuals with "similar" Z's and directly calculating the probability of selection from weighted frequencies. Variants of this approach involve different definitions of similarity and/or weight (Hardle 1990). In both methods, the problem of estimating how the probability of selection depends on Z is bypassed.

Semiparametric methods are also available. These methods are useful if their underlying assumptions are correct, since they generally produce more efficient estimates than does a fully nonparametric approach. These methods include Manski's maximum score method (1975), nonparametric maximum likelihood estimation (Cosslett 1983), weighted average derivatives (Stoker 1986; Powell et al. 1989), spline methods, and series approximations (Hardle 1990).

The problem in the second stage is to deal with the fact that one generally has no a priori knowledge of the correct functional form for λ. Since λ is simply a monotonic function of the probability of being selected, this is equivalent to asking what nonlinear transformation of the selection probability should be entered into equation (12). A variety of approaches are available here. One approach is to approximate λ through a series expansion (Newey 1990; Lee 1982) or by means of step functions (Cosslett 1991). An alternative is to control for λ by using differencing or fixed effect methods (Heckman et al. 1998). The essential idea is to control for the probability of selection by implicitly including a series of dummy variables in equation (1), with each dummy variable being used to indicate a set of individuals with the same probability of selection or, equivalently, the same λ. Generally, this will produce significantly larger standard errors of the slope estimates. This is appropriate, however, since selection increases uncertainty. With small samples, these methods can be generalized through kernel estimation (Powell 1987; Ahn and Powell 1990). Newey et al. (1990) apply a variety of methods to an empirical problem.

CONCLUSION

Selection problems bedevil much social science research. First and foremost, it is important for investigators to recognize that there is a selection problem and that it is likely to affect their estimates. Unfortunately, there is no panacea for selection bias. Various estimators have been proposed, and it is important for researchers to investigate the range of estimates produced by different methods. In most cases this range will be considerably broader than the confidence intervals for the OLS estimates. Selection bias introduces greater uncertainty into estimates. New methods for correcting for selection bias have been proposed that may provide a more powerful means for adjusting for selection. This needs to be determined by future research.

references

Ahn, H., and J. J. Powell 1990 Semiparametric Estimationof Censored Selection Models with a Nonparametric Selection Mechanism. Madison: Department of Economics, University of Wisconsin.

Arabmazar, A., and P. Schmidt 1982 "An Investigation of the Robustness of the Tobit Estimator to Non-Normality." Econometrica 50:1055–1063.

Berk, R. A. 1983 "An Introduction to Sample Selection Bias in Sociological Data." American Sociological Review 48:386–398.

Cosslett, S. R. 1983 "Distribution-Free Maximum Likelihood Estimator of the Binary Choice Model." Econometrica 51:765–781.

—— 1991 "Semiparametric Estimation of a Regression Model with Sampling Selectivity." In W. A. Barnett, J. Powell, and G. Tauchen, eds., Nonparametricand Semiparametric Methods in Econometrics and Statistics. Cambridge: Cambridge University Press.

DuMouchel, W. H., and G. J. Duncan 1983. "Using Sample Survey Weights in Multiple Regression Analyses of Stratified Samples." Journal of the American Statistical Association 78:535–543.

England, P., G. Farkas, B. Kilbourne, and T. Dou 1988 "Explaining Occupational Sex Segregation and Wages: Findings from a Model with Fixed Effects." AmericanSociological Review 53:544–558.

Fligstein, N. D., and W. Wolf 1978 "Sex Similarities in Occupational Status Attainment: Are the Results Due to the Restriction of the Sample to Employed Women?" Social Science Research 7:197–212.

Goldberger, A. S. 1983 "Abnormal Selection Bias." In S. Karlin, T. Amemiya, and L. A. Goodman, eds., Studies in Econometrics, Time Series, and Multivariate Statistics. New York: Academic Press.

Gronau, R. 1974 "Wage Comparisons—Selectivity Bias." Journal of Political Economy 82:1119–1143.

Hagan, J. 1990 "The Gender Stratification of Income Inequality among Lawyers." Social Forces 68:835–855.

——, and P. Parker, 1985. "White-Collar Crime and Punishment." American Sociological Review 50:302–316.

Hardle, W. 1990 Applied Nonparametric Regression. New York: Cambridge University Press.

Hardy, M. A. 1989 "Estimating Selection Effects in Occupational Mobility in a 19th-Century City." American Sociological Review 54:834–843.

Hausman, J. A., and D. A. Wise, 1977 "Social Experimentation, Truncated Distributions, and Efficient Estimation." Econometrica 45:919–938.

Heckman, J. J. 1974 "Shadow Prices, Market Wages and Labor Supply." Econometrica 42:679–694.

—— 1978 "Dummy Endogenous Variables in a Simultaneous Equation System." Econometrica 46:931–959.

—— 1979. "Sample Selection Bias as a Specification Error." Econometrica 47:153–161.

——, H. Ichimura, S. Smith, and P. Todd 1998 "Characterizing Selection Bias Using Experimental Data." Econometrica 6:1017–1099.

Hurd, M. 1979 "Estimation in Truncated Samples." Journal of Econometrics 11:247–58.

Kalbfleisch, J. D., and R. L. Prentice 1980 The StatisticalAnalysis of Failure Time Data. New York: Wiley.

Lalonde, R. J. 1986 "Evaluating the Econometric Evaluations of Training Programs with Experimental Data." American Economic Review 76:604–620.

Leamer, E. E. 1983 "Model Choice and Specification Analysis." In Z. Griliches and M. D. Intriligator, eds., Handbook of Econometrics, vol. 1. Amsterdam: North-Holland.

Lee, L. F. 1982 "Some Approaches to the Correction of Selectivity Bias." Review of Economic Studies 49:355–372.

Maddala, G. S. 1983 Limited-Dependent and QualitativeVariables in Econometrics. Cambridge: Cambridge University Press.

Manski, C. F., and D. A. Wise 1983 College Choice inAmerica. Cambridge, Mass.: Harvard University Press.

Mare, R. D., and C. Winship 1984 "The Paradox of Lessening Racial Inequality and Joblessness among Black Youth: Enrollment, Enlistment, and Employment, 1964–1981." American Sociological Review 49:39–55.

Newey, W. K. 1990 "Two-Step Series Estimation of Sample Selection Models." Paper presented at the 1988 European meeting of the Econometric Society.

——, J. L. Powell, and J. R. Walker 1990 "Semiparametric Estimation of Selection Models: Some Empirical Results." American Economic Review 80:324–328.

Peterson, R. and J. Hagan 1984 "Changing Conceptions of Race: Towards an Account of Anomalous Findings of Sentencing Research." American SociologicalReview 49:56–70.

Powell, J. L. 1987 Semiparametric Estimation of BivariateLatent Variable Models. Working Paper No. 8704. Madison: Social Systems Research Institute, University of Wisconsin.

——, J. H. Stock, and T. M. Stoker 1989 "Semiparametric Estimation of Index Coefficients." Econometrica 57:1403–1430.

Rosenbaum, P., and D. B. Rubin 1983 "The Central Role of the Propensity Score in Observational Studies for Causal Effects." Biometrika 70:41–55.

Stoker, T. M. 1986 "Consistent Estimates of Scaled Coefficients." Econometrica 54:1461–1481.

Stolzenberg, R. M., and D. A. Relles 1990 "Theory Testing in a World of Constrained Research Design: The Significance of Heckman's Censored Sampling Bias Correction for Nonexperimental Research." Sociological Methods Research 18:395–415.

Tobin, J. 1958 "Estimation of Relationships for Limited Dependent Variables." Econometrica 26:24–36.

Winship, C., and R. D. Mare 1992 "Models for Sample Selection Bias." Annual Review of Sociology 18:327–350.

——, and S. Morgan 1999 "The Estimation of Causal Effects from Observational Data." Annual Review ofSociology. 25:657–704.

Zatz, M. S., and J. Hagan 1985 "Crime, Time, and Punishment: An Exploration of Selection Bias in Sentencing Research." Journal of Quantitative Criminology 1:103–126.

Christopher Winship
Robert D. Mare