7.4: Errors in Sampling

7.4.1: Expected Value and Standard Error

Expected Value

In probability theory, the expected value (or expectation, mathematical expectation, EV, mean, or first moment) of a random variable is the weighted average of all possible values that this random variable can take on. The weights used in computing this average are probabilities in the case of a discrete random variable, or values of a probability density function in the case of a continuous random variable.

The expected value may be intuitively understood by the law of large numbers: the expected value, when it exists, is almost surely the limit of the sample mean as sample size grows to infinity. More informally, it can be interpreted as the long-run average of the results of many independent repetitions of an experiment (e.g. a dice roll). The value may not be expected in the ordinary sense—the “expected value” itself may be unlikely or even impossible (such as having 2.5 children), as is also the case with the sample mean.

The expected value of a random variable can be calculated by summing together all the possible values with their weights (probabilities):

where $\text{x}$represents a possible value and $\text{p}$ represents the probability of that possible value.

Standard Error

The standard error is the standard deviation of the sampling distribution of a statistic. For example, the sample mean s the usual estimator of a population mean. However, different samples drawn from that same population would in general have different values of the sample mean. The standard error of the mean (i.e., of using the sample mean as a method of estimating the population mean) is the standard deviation of those sample means over all possible samples of a given size drawn from the population.

Expected Value and Standard Error of a Sum

Suppose there are five numbers in a box: 1, 1, 2, 3, and 4. If we were to selected one number from the box, the expected value would be:

Now, let’s say we draw a number from the box 25 times (with replacement). The new expected value of the sum of the numbers can be calculated by the number of draws multiplied by the expected value of the box: $25\cdot 2.2=55$. The standard error of the sum can be calculated by the square root of number of draws multiplied by the standard deviation of the box: $\sqrt{}\text{of box}=5\cdot 1.17=5.8$. This means that if this experiment were to be repeated many times, we could expect the sum of 25 numbers chosen to be within 5.8 of the expected value of 55, either higher or lower.

7.5.2: Using the Normal Curve

Z-Value

The functional form for a normal distribution is a bit complicated. It can also be difficult to compare two variables if their mean and or standard deviations are different. For example, heights in centimeters and weights in kilograms, even if both variables can be described by a normal distribution. To get around both of these conflicts, we can define a new variable:

This variable gives a measure of how far the variable is from the mean ($\text{x}-\mu$), then “normalizes” it by dividing by the standard deviation ($\sigma$). This new variable gives us a way of comparing different variables. The $\text{z}$-value tells us how many standard deviations, or “how many sigmas”, the variable is from its respective mean.

Areas Under the Curve

To calculate the probability that a variable is within a range, we have to find the area under the curve. Normally, this would mean we’d need to use calculus. However, statisticians have figured out an easier method, using tables, that can typically be found in your textbook or even on your calculator.

These tables can be a bit intimidating, but you simply need to know how to read them. The leftmost column tells you how many sigmas above the the mean to one decimal place (the tenths place).The top row gives the second decimal place (the hundredths).The intersection of a row and column gives the probability.

For example, if we want to know the probability that a variable is no more than 0.51 sigmas above the mean, $\text{P}(\text{z}<0.51)$, we look at the 6^th row down (corresponding to 0.5) and the 2^nd column (corresponding to 0.01). The intersection of the 6^th row and 2^nd column is 0.6950, which tells us that there is a 69.50% percent chance that a variable is less than 0.51 sigmas (or standard deviations) above the mean.

A common mistake is to look up a $\text{z}$-value in the table and simply report the corresponding entry, regardless of whether the problem asks for the area to the left or to the right of the $\text{z}$-value. The table only gives the probabilities to the left of the $\text{z}$-value. Since the total area under the curve is 1, all we need to do is subtract the value found in the table from 1. For example, if we wanted to find out the probability that a variable is more than 0.51 sigmas above the mean, $\text{P}(\text{z}>0.51)$, we just need to calculate $1-\text{P}(\text{z}<0.51)=1-0.6950=0.3050$, or 30.5%.

There is another note of caution to take into consideration when using the table: The table provided only gives values for positive $\text{z}$-values, which correspond to values above the mean. What if we wished instead to find out the probability that a value falls below a $\text{z}$-value of $-0.51$, or 0.51 standard deviations below the mean? We must remember that the standard normal curve is symmetrical, meaning that $\text{P}(\text{z}<-0.51)=\text{P}(\text{z}>0.51)$, which we calculated above to be 30.5%.

We may even wish to find the probability that a variable is between two z-values, such as between 0.50 and 1.50, or P(0.50).

68-95-99.7 Rule

Although we can always use the $\text{z}$-score table to find probabilities, the 68-95-99.7 rule helps for quick calculations. In a normal distribution, approximately 68% of values fall within one standard deviation of the mean, approximately 95% of values fall with two standard deviations of the mean, and approximately 99.7% of values fall within three standard deviations of the mean.

7.4.3: The Correction Factor

In probability theory, the expected value refers, intuitively, to the value of a random variable one would “expect” to find if one could repeat the random variable process an infinite number of times and take the average of the values obtained. More formally, the expected value is a weighted average of all possible values. In other words, each possible value the random variable can assume is multiplied by its assigned weight, and the resulting products are then added together to find the expected value.

The weights used in computing this average are the probabilities in the case of a discrete random variable (that is, a random variable that can only take on a finite number of values, such as a roll of a pair of dice), or the values of a probability density function in the case of a continuous random variable (that is, a random variable that can assume a theoretically infinite number of values, such as the height of a person).

From a rigorous theoretical standpoint, the expected value of a continuous variable is the integral of the random variable with respect to its probability measure. Since probability can never be negative (although it can be zero), one can intuitively understand this as the area under the curve of the graph of the values of a random variable multiplied by the probability of that value. Thus, for a continuous random variable the expected value is the limit of the weighted sum, i.e. the integral.

Simple Example

Suppose we have a random variable X, which represents the number of girls in a family of three children. Without too much effort, you can compute the following probabilities:

$\begin{array}{c}\text{P}[\text{X}=0]=0.125\end{array}$

$\begin{array}{c}\text{P}[\text{X}=1]=0.375\end{array}$

$\begin{array}{c}\text{P}[\text{X}=2]=0.375\end{array}$

$\begin{array}{c}\text{P}[\text{X}=3]=0.125\end{array}$

The expected value of X, E[X], is computed as:

$=0\cdot 0.125+1\cdot 0.375+2\cdot 0.375+3\cdot 0.125$

$=1.5$

This calculation can be easily generalized to more complicated situations. Suppose that a rich uncle plans to give you $2,000 for each child in your family, with a bonus of $500 for each girl. The formula for the bonus is:

$\text{Y}=1,000+500\text{X}$

What is your expected bonus?

$=1000\cdot 0.125+1500\cdot 0.375+2000\cdot 0.375+2500\cdot 0.125$

$=1750$

We could have calculated the same value by taking the expected number of children and plugging it into the equation:

$\text{E}[1,000+500\text{X}]=1,000+500\text{E}\left[\text{X}\right]$

Expected Value and the Law of Large Numbers

The intuitive explanation of the expected value above is a consequence of the law of large numbers: the expected value, when it exists, is almost surely the limit of the sample mean as the sample size grows to infinity. More informally, it can be interpreted as the long-run average of the results of many independent repetitions of an experiment (e.g. a dice roll). The value may not be expected in the ordinary sense—the “expected value” itself may be unlikely or even impossible (such as having 2.5 children), as is also the case with the sample mean.

Uses and Applications

To empirically estimate the expected value of a random variable, one repeatedly measures observations of the variable and computes the arithmetic mean of the results. If the expected value exists, this procedure estimates the true expected value in an unbiased manner and has the property of minimizing the sum of the squares of the residuals (the sum of the squared differences between the observations and the estimate). The law of large numbers demonstrates (under fairly mild conditions) that, as the size of the sample gets larger, the variance of this estimate gets smaller.

This property is often exploited in a wide variety of applications, including general problems of statistical estimation and machine learning, to estimate (probabilistic) quantities of interest via Monte Carlo methods.

The expected value plays important roles in a variety of contexts. In regression analysis, one desires a formula in terms of observed data that will give a “good” estimate of the parameter giving the effect of some explanatory variable upon a dependent variable. The formula will give different estimates using different samples of data, so the estimate it gives is itself a random variable. A formula is typically considered good in this context if it is an unbiased estimator—that is, if the expected value of the estimate (the average value it would give over an arbitrarily large number of separate samples) can be shown to equal the true value of the desired parameter.

In decision theory, and in particular in choice under uncertainty, an agent is described as making an optimal choice in the context of incomplete information. For risk neutral agents, the choice involves using the expected values of uncertain quantities, while for risk averse agents it involves maximizing the expected value of some objective function such as a von Neumann-Morgenstern utility function.

7.4.4: A Closer Look at the Gallup Poll

Overview of the Gallup Poll

The Gallup Poll is the division of Gallup, Inc. that regularly conducts public opinion polls in more than 140 countries around the world. Historically, the Gallup Poll has measured and tracked the public’s attitudes concerning virtually every political, social, and economic issue of the day, including highly sensitive or controversial subjects. It is very well known when it comes to presidential election polls and is often referenced in the mass media as a reliable and objective audience measurement of public opinion. Its results, analyses, and videos are published daily on Gallup.com in the form of data-driven news. The poll has been around since 1935.

How Does Gallup Choose its Samples?

The Gallup Poll is an opinion poll that uses probability sampling. In a probability sample, each individual has an equal opportunity of being selected. This helps generate a sample that can represent the attitudes, opinions, and behaviors of the entire population.

In the United States, from 1935 to the mid-1980s, Gallup typically selected its sample by selecting residences from all geographic locations. Interviewers would go to the selected houses and ask whatever questions were included in that poll, such as who the interviewee was planning to vote for in an upcoming election .

There were a number of problems associated with this method. First of all, it was expensive and inefficient. Over time, Gallup realized that it needed to come up with a more effective way to collect data rapidly. In addition, there was the problem of non-response. Certain people did not wish to answer the door to a stranger, or simply declined to answer the questions the interviewer asked.

In 1986, Gallup shifted most of its polling to the telephone. This provided a much quicker way to poll many people. In addition, it was less expensive because interviewers no longer had to travel all over the nation to go to someone’s house. They simply had to make phone calls. To make sure that every person had an equal opportunity of being selected, Gallup used a technique called random digit dialing. A computer would randomly generate phone numbers found from telephone exchanges for the sample. This method prevented problems such as under-coverage, which could occur if Gallup had chosen to select numbers from a phone book (since not all numbers are listed). When a house was called, the person over eighteen with the most recent birthday would be the one to respond to the questions.

A major problem with this method arose in the mid-late 2000s, when the use of cell phones spiked. More and more people in the United States were switching to using only their cell phones over landline telephones. Now, Gallup polls people using a mix of landlines and cell phones. Some people claim that the ratio they use is incorrect, which could result in a higher percentage of error.

Sample Size and Error

A lot of people incorrectly assume that in order for a poll to be accurate, the sample size must be huge. In actuality, small sample sizes that are chosen well can accurately represent the entire population, with, of course, a margin of error. Gallup typically uses a sample size of 1,000 people for its polls. This results in a margin of error of about 4%. To make sure that the sample is representative of the whole population, each respondent is assigned a weight so that demographic characteristics of the weighted sample match those of the entire population (based on information from the US Census Bureau). Gallup weighs for gender, race, age, education, and region.

Potential for Inaccuracy

Despite all the work done to make sure a poll is accurate, there is room for error. Gallup still has to deal with the effects of nonresponse bias, because people may not answer their cell phones. Because of this selection bias, the characteristics of those who agree to be interviewed may be markedly different from those who decline. Response bias may also be a problem, which occurs when the answers given by respondents do not reflect their true beliefs. In addition, it is well established that the wording of the questions, the order in which they are asked, and the number and form of alternative answers offered can influence results of polls. Finally, there is still the problem of coverage bias. Although most people in the United States either own a home phone or a cell phone, some people do not (such as the homeless). These people can still vote, but their opinions would not be taken into account in the polls.

Attributions

• Expected Value and Standard Error
  • “Boundless.”
    http://www.boundless.com/.
    Boundless Learning
    CC BY-SA 3.0.
  • “Expected value.”
    http://en.wikipedia.org/wiki/Expected_value.
    Wikipedia
    CC BY-SA 3.0.
  • “Standard error.”
    http://en.wikipedia.org/wiki/Standard_error.
    Wikipedia
    CC BY-SA 3.0.
  • “standard deviation.”
    http://en.wiktionary.org/wiki/standard_deviation.
    Wiktionary
    CC BY-SA 3.0.
  • “Standard deviation diagram.”
    http://commons.wikimedia.org/wiki/File:Standard_deviation_diagram.svg.
    Wikimedia
    CC BY-SA.
• Using the Normal Curve
  • “Boundless.”
    http://www.boundless.com/.
    Boundless Learning
    CC BY-SA 3.0.
  • “Using Normal Distributions – IB Math Stuff.”
    http://ibmathstuff.wikidot.com/usingnormaldistributions.
    Wikidot
    CC BY-SA.
  • “Chapter 7: Normal distribution – Statistics.”
    http://statistics.wikidot.com/ch7.
    Wikidot
    CC BY-SA.
  • “standard deviation.”
    http://en.wiktionary.org/wiki/standard_deviation.
    Wiktionary
    CC BY-SA 3.0.
  • “Standard deviation diagram.”
    https://en.wikipedia.org/wiki/File:Standard_deviation_diagram.svg.
    Wikipedia
    CC BY-SA.
  • “Using Normal Distributions – IB Math Stuff.”
    http://ibmathstuff.wikidot.com/usingnormaldistributions.
    Wikidot
    CC BY-SA.
  • “Chapter 7: Normal distribution – Statistics.”
    http://statistics.wikidot.com/ch7.
    Wikidot
    CC BY-SA.
• The Correction Factor
  • “Boundless.”
    http://www.boundless.com/.
    Boundless Learning
    CC BY-SA 3.0.
  • “Expected value.”
    http://en.wikipedia.org/wiki/Expected_value.
    Wikipedia
    CC BY-SA 3.0.
  • “weighted average.”
    http://en.wiktionary.org/wiki/weighted_average.
    Wiktionary
    CC BY-SA 3.0.
  • “random variable.”
    http://en.wiktionary.org/wiki/random_variable.
    Wiktionary
    CC BY-SA 3.0.
  • “integral.”
    http://en.wiktionary.org/wiki/integral.
    Wiktionary
    CC BY-SA 3.0.
  • “Stats: Expected value and moments (July 29, 2005).”
    http://www.pmean.com/05/Moments.asp.
    P.Mean Website
    CC BY.
• A Closer Look at the Gallup Poll
  • “Boundless.”
    http://www.boundless.com/.
    Boundless Learning
    CC BY-SA 3.0.
  • “Opinion poll.”
    http://en.wikipedia.org/wiki/Opinion_poll.
    Wikipedia
    CC BY-SA 3.0.
  • “Gallup (company).”
    http://en.wikipedia.org/wiki/Gallup_(company).
    Wikipedia
    CC BY-SA 3.0.
  • “Nonprobability sampling.”
    http://en.wikipedia.org/wiki/Nonprobability_sampling.
    Wikipedia
    CC BY-SA 3.0.
  • “nonresponse.”
    http://en.wiktionary.org/wiki/nonresponse.
    Wiktionary
    CC BY-SA 3.0.
  • “Voter poll.”
    http://en.wikipedia.org/wiki/File:Voter_poll.jpg.
    Wikipedia
    CC BY-SA.