18 Chi-square

We come at last to our final statistic: chi-square (χ²). This test is a special form of analysis called a nonparametric test, so the structure of it will look a little bit different from what we have done so far. However, the logic of hypothesis testing remains unchanged. The purpose of chi-square is to understand the frequency distribution of a single categorical variable or find a relationship between two categorical variables, which is a frequently very useful way to look at our data.

Categories and Frequency Tables

Our data for the χ² test are categorical, specifically nominal, variables. Recall from Unit 1 that nominal variables have no specified order and can only be described by their names and the frequencies with which they occur in the dataset. Thus, unlike our other variables that we have tested, we cannot describe our data for the χ² test using means and standard deviations. Instead, we will use frequencies tables.

	Cat	Dog	Other	Total
Observed	14	17	5	36
Expected	12	12	12	36

Table 1. Pet Preferences

Table 1 gives an example of a frequency table used for a χ² test. The columns represent the different categories within our single variable, which in this example is pet preference. The χ² test can assess as few as two categories, and there is no technical upper limit on how many categories can be included in our variable, although, as with ANOVA, having too many categories makes our computations long and our interpretation difficult. The final column in the table is the total number of observations, or N. The χ² test assumes that each observation comes from only one person and that each person will provide only one observation, so our total observations will always equal our sample size.

There are two rows in this table. The first row gives the observed frequencies of each category from our dataset; in this example, 14 people reported preferring cats as pets, 17 people reported preferring dogs, and 5 people reported preferring a different animal. The second row gives expected values; expected values are what would be found if each category had equal representation. Note: Chi-square tests should not be used when an expected value for any cell will be less than 5.

The expected values correspond with the null hypothesis for χ² tests: equal representation of categories. Our first of two χ² tests, the Goodness-of-Fit test, will assess how well our data lines up with, or deviates from, this assumption.

Goodness-of-Fit Test

The first of our two χ² tests assesses one categorical variable against a null hypothesis of equally sized frequencies. Equal frequency distributions are what we would expect to get if placement into a particular category was completely random. We could, in theory, also test against a specific distribution of category sizes if we have a good reason to (e.g., we have a solid foundation of how the regular population is distributed), but this is less common, so we will not deal with it in this text. For example, if you know that for every 2 cats available for adoption there are 5 dogs available, you might expect more dogs than cats in your expected values row.

Step 1 – State the Hypotheses

All χ² tests, including the goodness-of-fit test, are nonparametric. This means that there is no population parameter we are estimating or testing against; we are working only with our sample data. Because of this, there are no mathematical statements for χ² hypotheses. This should make sense because the mathematical hypothesis statements were always about population parameters (e.g., μ), so if our test is nonparametric, we have no parameters and therefore no statistical notation hypothesis statements.

We do, however, still state our hypotheses in written form. For goodness-of-fit χ² tests, our null hypothesis is that there is an equal number of observations in each category. That is, there is no difference between the categories in how prevalent they are. Our alternative hypothesis says that the categories do differ in their frequency. We do not have specific directions or one-tailed tests for χ², matching our lack of mathematical statement. That is:

H₀: There is not difference in number of observations between categories

H_A: There is a difference in number of observations between categories

Step 2 – Find the Critical Value

Our degrees of freedom for the χ² test are based on the number of categories we have in our variable, not on the number of people or observations like it was for our other tests. Luckily, they are still as simple to calculate.

Figure 1. First 10 rows of a χ² table

Step 3 – Calculate the Test Statistic

The calculations for our test statistic in χ² tests combine our information from our observed frequencies (O) and our expected frequencies (E) for each level of our categorical variable. For each cell (category) we find the difference between the observed and expected values, square them, and divide by the expected values. We then sum this value across cells for our test statistic.

Step 4 – Make a Decision and Interpret the Results

Now we can compare the test statistic of 6.49 to the critical value of 5.99. Because 6.49 is greater than 5.99, we can reject the null hypothesis and state that pet preference is different from random chance. So, let’s interpret this in APA style.

Finding a Relationship between Two Categorical Variables

 

The goodness-of-fit test is a useful tool for assessing a single categorical variable. However, what is more common is wanting to know if two categorical variables are related to one another. This type of analysis is similar to a correlation, the only difference being that we are working with nominal data, which violates the assumptions of traditional correlation coefficients. This is where the χ² test for independence comes in handy.

As noted above, our only description for nominal data is frequency, so we will again present our observations in a frequency table. When we have two categorical variables, our frequency table is crossed. That is, each combination of levels from each categorical variable are presented. This type of frequency table is called a contingency table because it shows the frequency of each category in one variable contingent upon the specific level of the other variable.

An example contingency table is shown in Table 3 which displays whether or not 168 college students watched college sports growing up (Yes/No) and whether the students’ final choice of which college to attend was influenced by the college’s sports teams (Yes – Primary, Yes – Somewhat, No).

College Sports		Affected Decision
		Primary	Somewhat	No	Total
Watched	Yes	47	26	14	87
	No	21	23	37	81
	Total	68	49	51	168

Table 3. Contingency table of college sports and decision making

In contrast to the frequency table for our goodness-of-fit test, our contingency table does not contain expected values, only observed data. Within our table, wherever our rows and columns cross, we have a “cell”. A cell contains the frequency of observing its corresponding specific levels of each variable at the same time. The top left cell in table 3 shows us that 47 people in our study watched college sports as a child AND had college sports as their primary deciding factor in which college to attend.

Cells are numbered based on which row they are in (rows are numbered top to bottom) and which column they are in (columns are numbered left to right). We always name the cell using (R, C), with the row first and the column second. Based on this convention, the top left cell containing our 47 participants who watched college sports as a child and had sports as a primary criteria is cell (1, 1). Next to it, which has 26 people who watched college sports as a child but had sports only somewhat affect their decision, is cell (1, 2), and so on. We only number the cells where our categories cross. We do not number our total cells, which have their own special name: marginal values. Marginal values are the total values for a single category of one variable, added up across levels of the other variable. In table 3, these marginal values have been italicized for ease of explanation, though this is not normally the case. We can see that, in total, 87 of our participants (47+26+14) watched college sports growing up and 81 (21+23+37) did not. The total of these two marginal values is 168, the total number of people in our study. Likewise, 68 people used sports as a primary criteria for deciding which college to attend, 50 considered it somewhat, and 50 did not use it as criteria at all. The total of these marginal values is also 168, our total number of people. The marginal values for rows and columns will always both add up to the total number of participants, N, in the study. If they do not, then a calculation error was made and you must go back and check your work.

Expected Values of Contingency Tables

Our expected values for contingency tables are based on the same logic as they were for frequency tables, but now we must incorporate information about how frequently each row and column was observed (the marginal values) and how many people were in the sample overall (N) to find what random chance would have made the frequencies out to be.

Notice that the marginal values still add up to the same totals as before. This is because the expected frequencies are just row and column averages simultaneously. Our total N will also add up to the same value.

χ² Test for Independence

The χ² test performed on contingency tables is known as the test for independence. In this analysis, we are looking to see if the values of each categorical variable (that is, the frequency of their levels) is related to or independent of the values of the other categorical variable. Let’s do the four-step test for this example.

Step 1 – State the Hypotheses

Because we are still doing a χ² test which is nonparametric, we still do not have statistical versions of our hypotheses. The actual interpretations of the hypotheses are quite simple: the null hypothesis says that the variables are independent or not related, and alternative says that they are not independent or that they are related.

H₀: Watching college sports as a child is not related to choosing college.

OR

H₀: Watching college sports as a child is independent of college choice.

H_A: Watching college sports as a child is related to choosing college.

OR

H_A: Watching college sports as a child is not independent of college choice.

Step 2 – Find the Critical Value

For step 2, the only change is degrees of formula. Our critical value will come from the same table that we used for the goodness-of-fit test, but our degrees of freedom will change. Because we now have rows and columns (instead of just columns) our new degrees of freedom is found by multiplying two numbers together.

Step 3 – Calculate the Test Statistic

As you can see below, we calculate our test statistic for the test for independence the same way we did for the goodness of fit test. Our equation just gets a bit longer because we now have to do it for every cell in the contingency table. So, for our 2×3 contingency table, we will have 6 components in our equation.

Reject H₀. Based on our data from 168 people, we can say that there is a statistically significant relationship between whether or not someone watches college sports growing up and how much a college’s sports team factor in to that person’s decision on which college to attend, χ²(2) = 20.31, p < 0.05.

Effect Size for χ²

Like all other significance tests, χ² tests – both goodness-of-fit and tests for independence – have effect sizes that can and should be calculated for statistically significant results. There are many options for which effect size to use, and the ultimate decision is based on the type of data, the structure of your frequency or contingency table, and the types of conclusions you would like to draw. For the purpose of our introductory course, we will focus only on a single effect size that is simple and flexible: Cramer’s V. This is appropriate to use when the χ² test involves a matrix larger that 2 X 2. Cramer’s Vis a type of correlation coefficient that can be computed on categorical data.