Sampling

(source)
Modern public opinion polling depends critically on sampling for its viability, since it rests on the assumption that we can learn about what the public as a whole thinks by asking just a fraction of its members. To get a better feel for how this works, consider these questions:

First, what is sampling? As the video clip above suggests, sampling refers simply to the selection of a subset of individuals (i.e. a "sample") from a larger population. ABC News's "Guide to Polls and Public Opinion" points out that sampling isn't unique to public opinion polling; instead, it's analogous to a blood test, in which only a little bit of blood is studied in order to draw conclusions about the health of an entire organism. Another good analogy is to a chef who tastes just a spoonful of sauce to determine whether it's ready to serve. In both examples, it would be ludicrous to suggest that the entire bloodstream or pot of sauce would need to be tested; so too in public opinion polling, only a small sample of the population is needed to make determinations about the views and attitudes of the whole.

Second, what are the characteristics of a good sample? In a nutshell, good sampling accounts for two factors: size (how big the sample is) and representativeness (the extent to which the sample faithfully reflects key characteristics of the whole population). Of the two, representativeness matters more than size. Consider the contest between the Literary Digest and Gallup polls to predict the 1936 election (see here to refresh your memory). The Literary Digest poll had a significantly larger sample of millions of respondents compared to Gallup's couple of thousand. However, Gallup's quota sampling technique provided a sample that was more representative of the public as a whole than the Literary Digest's reliance on telephone and automobile registry listings, which tended to oversample voters from the wealthier end of the socioeconomic spectrum. That said, size does matter, especially when it comes to determining a poll's margin of error, as suggested by another one of our required texts for this unit: Public Agenda's "Best Estimates: A Guide to Sample Size and Margin of Error."

Finally, how should samples be selected? The final required text for this unit, "Types of Sampling" from the Columbia Center for New Media Teaching and Learning, lays out several types of sampling strategies that are commonly used in public opinion polling and the circumstances that are best suited to the different techniques. ...Read more

Measuring Public Opinion

Our first full unit is devoted to the question of how we know what the American public thinks about politics -- in other words, how we measure public opinion. The short answer is that we ask the people what their views and attitudes are, which seems kind of obvious -- except that it's just not practical to question everyone every time we want to know what the country is thinking.

To get around this problem, the predominant means for measuring public opinion is the scientific public opinion poll or survey, terms we'll use interchangeably to refer to questionnaires that are administered to a small but representative sample of the population in order to get a sense of what the population as a whole thinks. The first required text for this unit, "Guide to Polls & Public Opinion," by Gary Langer, the director of polling for ABC News, provides a pretty clear overview of what public opinion polls are, how they work, and the extent of their usefulness. Later on in this unit, we'll get more into the nitty-gritty of "how they work" (i.e. the logic and best practices of sampling); then, in Topic II, we'll turn to "the extent of their usefulness" and steps pollsters can take to improve the usefulness of their poll and survey results.

While the belief that public opinion should have a hand in determining political and policy outcomes dates back to the founding of the United States (see my last post), the emergence of scientific public opinion polling and survey research is a relatively recent development. A few of this unit's required texts drive this home:

• The segment on "George Gallup and the Scientific Public Opinion Poll" from a PBS documentary on the twentieth century traces back to the 1930s, when a few enterprising marketing researchers, including George Gallup, were developing new survey research methods. Prior to that time, public opinion was measured primarily through "straw polls," unscientific polls that were used to estimate likely election outcomes on a local basis (e.g. in a single town) as early as 1820s.


• "The Black & White Beans" is a Time magazine article that was written in order to explain the logic of public opinion polling to the American public at a time when it was all still fairly new -- just a little over a decade after Gallup bested the Literary Digest poll with his prediction of the 1936 election. It's probably worth noting that this article was published in May 1948 -- several months before all the major pollsters were fundamentally off base in their forecasts of the 1948 presidential election.


• On that note, in "US Election 1948: The First Great Controversy about Polls, Media, and Social Science," a Swedish social scientist named Hans Zetterberg provides an account of the pollsters' 1948 mishap and changes in polling practices that have been made since then to prevent a similar situation in the future.

...Read more