Polling: "Margin of Error"

This is not a particularly recent poll, although the assertion is still true. But that's not the point.

The New York Times > Washington > New Poll Finds Bush Priorities Are Out of Step With Americans

The poll was conducted by telephone with 1,111 adults from Thursday through Monday. It has a margin of sampling error of plus or minus three percentage points.

Have you ever wondered where that "margin of error of + or – 3 % points" comes from, or why the weird number of people polled (which most people react to by thinking it's much too small)?
The simple answer is simple. Follow along:

  1. The sample size is 1,111
  2. The square root of 1,111 is 33.33
  3. 33.33 / 1111 = o.03, or 3%

The answer is that simple. In random sampling from a uniform population, the best estimate of how good the average result is will be

+/- [sqrt(N) / N] = [1/sqrt(N)],

where N is the number of [statistically independent] samples.
The inverse works too. If you are told that the error is E%, then

N = 1 / (E/100)2

is the original sample size.
There is no mystery about this relationship between error and sample size in polls, and it is not what determines careful or "scientific" polling. It is simply an unvarying, mathematical result giving the best guess you can make about the error in an average calculated from random (i.e., statistically independent) samples taken from the larger population that one is trying to characterize.
The trick, of course, is in that bit about taking "random samples". That's the part that polling organizations work very hard at: to convince their customers that they (and they alone among their competitors) know how to take very good, very nearly "random samples" from any given population — all Americans, all likely Republican voters, all women under 18 who watch MTV, all men over 50 who eat chocolate ice cream at least twice a week, whatever group the poll's sponsor is interested in.
All the work, or artistry (some would like to say "science") goes into selecting the samples so that they will be randomly drawn from the population of interest; none of it goes into calculating the margin of error.
So now, when you hear a margin of error quoted, you can amaze all your friends by revealing the exact number of people who were asked the question, and sound amazingly clever.

Posted on June 24, 2005 at 15.02 by jns · Permalink
In: All, It's Only Rocket Science, The Art of Conversation

2 Responses

Subscribe to comments via RSS

  1. Written by S.W. Anderson
    on Friday, 24 June 2005 at 21.37
    Permalink

    Thanks, that's very interesting and answers a question I hadn't realized I was curious about. Great post.

  2. Written by Bearcastle Blog
    on Wednesday, 29 March 2006 at 14.03
    Permalink

    Old Fartdom

    So, I read about this poll about "profanity":

    …the AP-Ipsos poll on attitudes about profanity. The results are taken from a poll of 1,001 adults…*

    Not surprisingly,

    Two-thirds said they think people use profanity more now than they did …

Subscribe to comments via RSS

Leave a Reply

To thwart spam, comments by new people are held for moderation; give me a bit of time and your comment will show up.

I welcome comments -- even dissent -- but I will delete without notice irrelevant, rude, psychotic, or incomprehensible comments, particularly those that I deem homophobic, unless they are amusing. The same goes for commercial comments and trackbacks. Sorry, but it's my blog and my decisions are final.