Polls And Polling

Opinion polling has a long history in American politics. On July 24, 1824, a Harrisburg, Pennsylvania newspaper reported on a "straw" poll taken "without discrimination of parties" which indicated that Andrew Jackson was the popular choice for U.S. President over John Quincy Adams. As it turned out, Jackson did receive the most electoral votes. However, Jackson did not have at least 50 percent of the electoral votes, so the election was decided by the U.S. House of Representatives. The methods used by the Harrisburg newspaper to take the poll are not known. Modern statistical theories on opinion polling had not yet been invented.

Scientific methods for measuring public opinion had to wait for advances in the mathematical topics of probability theory and statistics , and the technique of sampling . These advances came in the late 1800s and early 1900s.

Random Sampling

In 1935, George Gallup founded the American Institute of Public Opinion to apply the latest techniques in mathematical statistics and sampling theory to take public opinion polls on various vital issues. The first major test of Gallup's scientific sampling techniques was the presidential election of 1936 between Franklin D. Roosevelt and Alfred E. Landon. This election provided a great contrast between the old and new methods of opinion polling. The Literary Digest magazine, one of the most respected publications of the time, conducted a massive survey of more than two million Americans, which predicted that Landon would win the presidency. Roosevelt

won the election, as predicted by the new Gallup poll, which was based upon a sample less than one one-thousandth the size of the Digest's.

Analysis of the sampling technique used by the Literary Digest's pollsters revealed that they had chosen the names of people to be surveyed from telephone directories and automobile registration lists, a common practice at that time. By 1936, however, Roosevelt's New Deal to end the Great Depression had helped millions of poorer families get public works jobs and welfare assistance. Vast numbers of these lower income people migrated from the Republican Party to the Democratic Party to show appreciation for Roosevelt's programs. But wealthier Americans who did not benefit as much from the Democratic policies were more likely to remain Republicans or switch to the Republican Party. Thus, using telephone and automobile ownership to select the sample to be polled biased the sample in the direction of the people who were wealthy enough to own cars and phones and who were, therefore, more likely to vote for the Republican (Landon) than for the Democrat (Roosevelt).

Gallup's organization, on the other hand, relied less on the size of the sample and more on the randomness of the selection process. This technique, known as random sampling, makes it far more likely that the sample will have proportional representation of various subgroups according to their proportions in the entire population.

Random sampling is based upon the mathematical theory of probability. There are two key principles involved. The first is called "equal probability of selection" and says that if every member of a population has an equal probability of being selected to the sample, then the sample will be representative of the entire population. The second principle is in the form of a mathematical equation that allows the polling organization to determine the size of a random sample necessary to give a specified degree of accuracy to the results of the poll.

From the initial poll in 1936 to the mid-1980s, Gallup's interviews were conducted in person at randomly selected residential addresses throughout the country. By 1985 the number of households with telephones was approaching 95 percent, which meant that it was no longer necessary to avoid randomly selected telephone numbers as a method for conducting interviews. But one remaining problem is that about 30 percent of households have unlisted telephone numbers.

To get around the problem of unlisted numbers, Gallup uses a technique called random digit dialing. A computer is given a list of all telephone exchanges in the country and then randomly generates lists of numbers from those exchanges. In essence, the computer creates a list of all possible phone numbers in the United States and then randomly generates a subset of that list for the Gallup interviewers to call.

Sample Size

Once a random sample is assured, the next question becomes: How many people should be called? The answer depends on how much possible error one is willing to tolerate in the results of the poll. Of course, ideally, we want no error at all. Unfortunately, this can only be accomplished by interviewing every voting-age American, which, of course, is an impractical task. So we are back to the question of how much error is acceptable given the amount of money and time available to the organization commissioning the poll. Once this acceptable level of error is established, the answer is purely mathematical.

Although the mathematics is advanced, it is straightforward and unambiguous. It says that if you are willing to settle for poll results that will be within plus or minus 5 percent of the actual election results, then interview 385 people. If you would rather be within ±3 percent of the actual results, then survey 1,068 people. If ± 2 percent sounds better, call 2,401 people. For an error of ±1 percent, you will need to make over 9,600 phone calls.

In practice, most organizations are willing to settle for poll results that are within ±3 percent of actual population results due to the costs involved in the polling process. Notice that to go from 3 percent to 2 percent requires more than doubling the sample size, which will also more than double the cost of the poll. To get results that are within ±1 percent of the true election results would increase the cost of the polling by nine-fold over the ±3 percent level.

It should also be stated that the mathematics does not really guarantee that when 1,068 people are interviewed, you will always get results within ±3 percent of the true population values. It only guarantees that it will happen 95 percent of the time. That is, if one hundred different polls of 1,068 people each were carried out, in ninety-five of those polls we would get our ±3 percent accuracy. In the other five polls, we would not. Statisticians call this the 95-percent confidence level for the results of the poll. You can increase the confidence level to 98 percent or 99 percent or even 99.9 percent, but, again, it will come at huge increases in the sample sizes and, therefore, the cost of the poll. It is standard practice to use a 95-percent confidence level for public opinion polling.

The Exit Poll

Another type of poll that has come under great scrutiny in recent years is the "exit" poll commissioned by the television networks and other news media, and carried out at polling sites in certain "key" precincts across the nation on election day. The idea is that by doing random sampling of voters just after they have voted, the news media will be able to "project" the winner as soon as the polls close. This process has long been criticized, particularly in presidential elections, because it has been contended that projecting the results of the election in the eastern time zone before the polls have closed in the rest of the country may cause voters in the west to stay away from the polls.

Whether or not this has actually ever occurred, no one really knows, but in the presidential election of 2000, events happened that brought calls for congressional investigations of the entire process of exit polling. Prior to Election Day, nearly every major poll indicated that the race between Democrat Al Gore and Republican George W. Bush was going to be one of the closest in history. On the evening of the election, the Associated Press and all the major television networks reported that results of exit polling in key precincts in the state of Florida showed that Gore would win the state's electoral votes.

Later that same evening, the media began to backtrack on its earlier call, reporting that something had gone wrong in the exit polling and that they were putting Florida's electoral votes back into the undecided column. Within the next 2 hours, the networks determined that they once again had sufficient information to declare the winner in Florida, only this time it was George W. Bush. This pushed Bush's total above the 270 needed for election and the networks declared George W. Bush the 46th President of the United States.

This, however, was not the last word. The media would have to do yet another flip-flop as late returns from Florida showed Gore dramatically closing the gap. Florida was once again returned to the undecided column and remained there until a controversial series of Florida recounts, court cases, and an ultimate U.S. Supreme Court decision would declare Bush the winner on December 12, 2000.

What went so terribly wrong in the exit polling? Most of the scrutiny has been focussed on Voter News Service (VNS), the polling organization that all the major networks used in their projections, and on the media themselves for rushing to judgement based on too little data in an election that was ultimately decided by fewer than 500 votes out of six-million cast in the state of Florida. If the VNS "key" precincts truly showed either candidate winning by more than the margin of error in the exit polls, then something was wrong in the selection of those precincts.

In an election this close, those polls should have been showing the race too close to call, especially early in the evening when the media projected Gore the winner in Florida. The television networks' reliance on a single source (VNS) for their exit polling and their desire to be the first to call the race are certainly areas for investigation.

see also Census; Data Collection and Interpretation; Predictions.

Stephen Robinson