# Margin of Error

**Posted:**November 19, 2008

**Filed under:**nerdism 2 Comments

Few weeks ago, I was at a friendly dinner party, discussing the upcoming elections, and specifically the results of recent voter surveys. One of the participants in the discussion said “I never pay attention to the error margins, since they apply to both candidates”.

I think he meant that if a specific poll said that 52% of the sampled voters preferred Obama and 46% preferred McCain, and the poll has a margin of error of 3%, then perhaps the “real” numbers are 55% for Obama and 49% for McCain, or maybe 49% Obama and 43% McCain, but it doesn’t really matter since the difference between them is constant.

This is of course, very false. For three important reasons:

- The margin of error is 3%, which means that the result of 49% for Obama and 49% for McCain cannot be ruled out. It is possible that Obama has no lead at all. It is important to understand that the 0% difference between the candidates is just at likely as the 6% difference the poll result actually show. There is no statistical way to differentiate detween these scenarios and both are just as real.
- 3% margin of error actually means that there are 95% chance that the “real” result is within 3% of the reported result. (Where “real” means what theresults would be if the entire adult population had been polled with complete accuracy). Remember that around election times, many polls are published. 5% of them have a bigger error than they report. How big? We have no idea.
- The reported margin of error is correct assuming that the sampling was perfect. Which means that no one refused to answer questions, no one lied, the questions were not worded or ordered in a way that caused bias, the selection of the sample was not biased, etc, etc. All these factors are likely to cause errors much larger than the theoretic sampling error, and what’s worse – we have no idea how big they can be and in which direction.

If you are really interested in the subject and not afraid of some mathematical notation, Terence Tao has a much deeper analysis of the subject.

Well, it is survey marketing after all… if the result would have end with 60% Mc Kain and 40% O’Bama, it would simply mean the people taking part of the survey did not provide honnest answer, and nobody is forced to reveal his voting intention. Actually, this would just prove the survey is worthless, as in France 2002 elections …

http://french.peopledaily.com.cn/french/200203/27/fra20020327_53353.html

they gave Jospin at 52% and he did not even pass the first tour, mostly due to electors who did not reveal there voting intention…

Whale no election on the moon

I hear similar nonsense from friends and family as you heard at this dinner party. I can’t express how frustrating this kind of thing can be. Polls are intentionally misleading, but that doesn’t stop supporters from whoever is leading in said polls to use these “facts” as an instant indication of victory.