Quantitative Methods in Social Science 360-300 • CEGEP, Montreal
Revision sheet: cross-tabulations, chi-square and association (360-300)
This sheet deals with the most used test of the whole course and the one most poorly written up: the chi-square of a crosstab. It does not restate the formula, it gives the three questions to ask in order, is there a relationship, where is it, how strong is it, and the eight places where a paper answers one of them believing it answers all three.
It is written for Social Science students at CEGEP taking the compulsory Quantitative Methods course, 360-300. The corrected exercise set on the same chapter goes through every step, from computing the expected frequencies to reading a table published in a journal.
Mark this sheet as read or add it to your favourites: a free account, no password, keeps your read sheets and favourites from one visit to the next, tells you which chapter to tackle next and lets you ask for the method or exercise you are missing. Create your space, an email is enough.
The thread of the chapter
A crosstab only becomes an answer once its percentages are computed IN THE RIGHT DIRECTION. The chi-square then says whether there is a relationship, never where it is nor whether it is strong: three distinct questions, and the exam asks all three.
The table, its margins, and the three possible percentages
•A cell can be related to the total of its ROW, to the total of its COLUMN or to the GRAND total: three different numbers, three different questions.
•One rule: compute percentages IN THE DIRECTION of the independent variable, then compare those percentages from one category of that variable to the next.
•The MARGINS are the row and column totals. They are identical in the observed table and in the expected table.
•A crosstab describes an ASSOCIATION. It says nothing about the direction of the relationship or about its cause.
The margins 200, 200, 180, 220 and the total 400 are the only numbers that the expected table keeps unchanged.
On an exam, writing next to the table “I compute row percentages, because the CEGEP is the independent variable” earns the method mark.
The test, in four numbers
•Expected frequency of a cell: Eij=nRi×Cj, that is, what the cell would contain if the two variables were independent.
•χ2=∑E(O−E)2: the differences are squared because their sum is always zero.
•df=(r−1)(c−1). Critical values at 5%: 3.841 for 1 degree of freedom, 5.991 for 2, 7.815 for 3, 9.488 for 4.
•V=n×min(r−1,c−1)χ2, between 0 and 1: below 0.10 negligible, 0.10 to 0.20 weak, 0.20 to 0.40 moderate.
The chi-square distribution with 1 degree of freedom: beyond 3.841 (red), only 5% of the possible values remain under independence. The 36.36 of the example is far to the right.
The chi-square answers “is there a relationship”, the cell-by-cell CONTRIBUTIONS answer “where”, and V answers “how strong”. Three questions, three tools.
The conditions, and what to do when they fail
•INDEPENDENT observations: each person counts in one cell only. This condition is never negotiable.
•EXPECTED frequencies above 5, with at most 20% of the cells between 1 and 5. These are the expected ones, never the observed ones.
•Calculation on COUNTS, never on percentages: the chi-square depends on the sample size, so starting from percentages amounts to declaring n=100.
•If the conditions fail: merge neighbouring categories, or switch to Fisher's exact test on a 2 by 2 table.
Yates' correction, on a 2 by 2 table, subtracts 0.5 from each absolute difference before squaring it. It ALWAYS lowers the chi-square.
The mistakes that cost marks
These are the errors I correct most often in session. Each one costs marks on a paper, even when the reasoning behind it is right.
1.Computing percentages in the wrong direction
2 marks, and the whole interpretation of the table
What not to write
“Among those who answered yes, 66.7% come from CEGEP A: CEGEP A therefore says yes more.”
What to write
“At CEGEP A, 200120=60% say yes, against 30% at CEGEP B: I compute row percentages, because the CEGEP is the independent variable.”
The two pairs of bars come from the same table: on the left the share of “yes” in each CEGEP, on the right the share of each CEGEP among the “yes”. Only the left one answers the question asked.
Why: The column percentage describes the composition of the respondents, which depends on the sizes of the two CEGEPs. Only the percentage in the direction of the independent variable compares groups.
2.Computing the chi-square on percentages
the whole question, since the result bears no relation to the true one
What not to write
“The table is already in percentages: I apply the formula directly.”
What to write
“I rebuild the COUNTS from the percentages and the margins, then compute. The chi-square depends on n, and a table of percentages would fix n=100.”
Why: Multiplying every count by k multiplies the chi-square by k. Starting again from percentages therefore throws away the information about the sample size, which is precisely what the test measures.
3.Mistaking significance for strength
2 marks, and it is the most frequent mistake of the whole course
What not to write
“χ2=36.36, far above 3.841: the relationship between the two variables is therefore very strong.”
What to write
“The relationship is highly significant, which means it is probably not due to chance. Its STRENGTH is read on Cramér's V, here 0.302, a moderate association.”
Illustration: 51% against 49% among 10,000 people is significant but negligible (V = 0.02); 60% against 30% among 400 is a moderate association (V = 0.30).
Why: The chi-square grows with the sample size; V does not. With enough people, the smallest difference becomes significant without becoming important.
4.Concluding independence because you fail to reject
1.5 marks, systematically deducted
What not to write
“χ2=2.1, below 3.841: the two variables are therefore independent.”
What to write
“We fail to reject H0: the data do not allow us to conclude that there is a relationship. This is not the same as demonstrating independence, especially with a small sample.”
A chi-square of 2.1 falls before the critical value 3.841: we fail to reject H₀, which does not demonstrate independence, especially on a small sample.
Why: A test without power detects nothing, even when something is there. Absence of evidence is not evidence of absence, and that is exactly what the marker wants to hear.
5.Checking the conditions on the observed frequencies
1 mark, and the conclusion of the test becomes questionable
What not to write
“All the observed cells exceed 5: the conditions are met.”
What to write
“It is the EXPECTED frequencies that must exceed 5. I compute all of them before concluding on the conditions.”
Illustration table: every observed cell is at least 6, yet the cell A-yes expects only 14 × 24 / 132 = 2.5. The condition applies to the expected frequencies.
Why: An observed cell of 8 can have an expected frequency of 2.5 if its margins are small: it is precisely this imbalance that makes the chi-square unstable.
6.Counting the degrees of freedom as rows times columns
1 mark, and often a reversed decision
What not to write
“The table is 3×4, so df=12.”
What to write
“df=(3−1)(4−1)=6, and the 5% critical value is 12.592.”
3 × 4 table with known margins: six cells (blue) are enough, the rest follows from the totals. Hence (3 − 1)(4 − 1) = 6. R: row totals, C: column totals, n: total.
Why: With the margins fixed, you only need to know (r−1)(c−1) cells to rebuild the whole table: that is the number of cells that are really free.
7.Believing that Cramér's V moves when the sample size changes
1.5 marks, and the question on the difference between the two indicators
What not to write
“With four hundred people instead of forty, the association is ten times stronger.”
What to write
“Multiplying every count by 10 multiplies the chi-square by 10 and leaves V UNCHANGED at 0.302: the proportions have not moved, so neither has the association.”
Three tables with identical proportions: the chi-square is multiplied by ten from one row to the next, and the last column does not move by a thousandth.
Why: V divides the chi-square by n: that is exactly what makes it comparable from one survey to another. The chi-square measures the EVIDENCE, V measures the RELATIONSHIP.
8.Concluding causation from a crosstab
the whole conclusion question
What not to write
“Students who work more than twenty hours succeed less: paid work makes them fail.”
What to write
“The table establishes an ASSOCIATION. To speak of a cause, we would also need temporal precedence and the elimination of confounding variables, here the socio-economic background.”
The table shows the association between work and success; the socio-economic background (orange) can act on both and produce the same table.
Why: A crosstab satisfies only one of the three conditions of causation. Elaboration, that is, redoing the table within each category of a third variable, is what allows you to go further.
Which method to choose
Which direction should the percentages go?
Identify the INDEPENDENT variable, then look at where it sits in the table.
Independent variable (IV) in rows: each row makes 100% and you compare the columns; in columns: each column makes 100%.
If the independent variable is in ROWS → compute ROW percentages, each row totalling 100%, then compare the columns with each other
Example: CEGEPs in rows, answer in columns
If the independent variable is in COLUMNS → compute COLUMN percentages, each column totalling 100%, then compare the rows
Example: gender in columns, choice of program in rows
If neither variable is clearly independent → compute percentages on the GRAND total, and limit yourself to describing the distribution
Example: two opinions crossed with no assumed direction
If the question asks for the composition of a specific group → compute percentages in the other direction, and SAY so: it is then a profile, not a comparison
Example: “who are the respondents in favour?”
Immediate check: the percentages you are going to COMPARE must belong to different groups, never to the same total.
What to do after the chi-square?
Look at the decision of the test, then at what the question still asks.
The four contributions of the example: 10 and 8.18 for each CEGEP. The overall chi-square says there is a relationship, the contributions say where.
If you reject H0 and the question asks WHERE the relationship is → read the CONTRIBUTIONS cell by cell and name the two or three largest
Example: a cell that brings 10 of the 36.4 total
If you reject H0 and the question asks the STRENGTH of the relationship → compute Cramér's V and place it on the usual landmarks
Example: V=0.302, moderate association
If you fail to reject H0 → conclude to a lack of evidence, never to independence, and point out the sample size
Example: χ2=2.1 with n=40
If the question supplies a third variable → ELABORATION: redo the table within each category of that variable and compare the results
Example: the relationship disappears in each subgroup, so it was spurious
Elaboration has three outcomes: REPLICATION when the relationship holds everywhere, EXPLANATION when it disappears, SPECIFICATION when it holds only in some subgroups.
How the answer is expected to be written
A marker ticks steps. Here they are in order, with the concluding sentence expected word for word.
Writing up a complete chi-square test
When to use it: Whenever the question asks whether there is a relationship between two categorical variables.
1State the hypotheses in words: H0, the two variables are independent; H1, they are not.
2Build the table of EXPECTED frequencies and check that all of them exceed 5, saying so.
3Compute the chi-square cell by cell, keeping the contributions visible.
4Give the degrees of freedom (r−1)(c−1) and the critical value read at the stated risk.
5Decide, then conclude IN THE CONTEXT of the question, with the level of risk.
Concluding sentence
“At the 5% level, χ2=36.36 exceeds the critical value 3.841 at 1 degree of freedom: we reject the hypothesis of independence. The choice of CEGEP and the answer are related, and Cramér's V of 0.302 places this association at the moderate level.”
The trap: Concluding “the chi-square is significant” without naming the two variables: a conclusion out of context earns no mark.
Marking: 1 mark for the hypotheses, 1 mark for the expected frequencies and the conditions, 2 marks for the calculation, 1 mark for the decision, 1 mark for the conclusion in context.
Interpreting the contributions cell by cell
When to use it: When the question asks where the relationship lies, or to comment on the table after a rejection.
1Write the four or six contributions E(O−E)2 and their sum.
2Pick out the ones that weigh most in the total, as a percentage of the chi-square.
3Look at the SIGN of O−E for those cells: above or below independence.
4Express each large cell in one sentence about content, not in a sentence about calculation.
Concluding sentence
“The two cells of CEGEP A account for 18.2 of the 36.4 of the chi-square: 120 favourable answers are observed against 90 expected, a net excess relative to independence.”
The trap: Commenting on a cell without looking at the sign of the difference: a large contribution can come from a SHORTFALL as well as from an excess.
Marking: 1 mark for the contributions computed, 1 mark for picking out the large ones, 1 mark for the sentence about content.
Check before you hand in
Five minutes of checking recover more marks than one more problem started in a hurry.
The margins of the expected table are those of the observed one
Add up the rows and the columns of the expected table: you must find exactly the same totals.
90+110=200 and 90+90=180: the margins are kept, so the expected frequencies are right.
The sum of the differences is zero
Add up the O−E of all the cells: the total is always zero. This is the quickest check of an expected table.
+30−30−30+30=0: the four differences of a 2 by 2 table are equal in absolute value.
Cramér's V stays between 0 and 1
A V above 1 comes from forgetting the min(r−1,c−1) in the denominator, or from a chi-square computed on percentages.
A reported V of 1.9 always signals a denominator error, never an exceptional association.
The order of magnitude of the critical value
The 5% critical values are 3.8, 6.0, 7.8 and 9.5 for 1 to 4 degrees of freedom: remembering them saves a trip to the table and catches an error in the degrees of freedom.
A critical value read as 18.3 on a 2 by 2 table signals that you counted 10 degrees of freedom instead of 1.
The typical problem, taken apart
A 2 by 2 table, from the chi-square to Cramér's V
Four hundred students from two CEGEPs answer a question with yes or no. At CEGEP A, 120 yes and 80 no; at CEGEP B, 60 yes and 140 no.
The question asks whether there is a relationship between the CEGEP and the answer, where it lies, and how strong it is.
The counts of the question: 120 yes and 80 no at CEGEP A, 60 yes and 140 no at CEGEP B.
Step 1
Row percentages: 60% yes at CEGEP A, 30% at CEGEP B.
Why
The CEGEP is the independent variable, so we compute row percentages. This step is not decorative: it already gives the answer, and the test will only confirm it.
Step 2
Expected frequencies: E11=400200×180=90, E12=110, E21=90, E22=110. All of them are well above 5.
Why
The expected table is that of perfect independence. Its margins are the same as those of the observed table, which serves as an immediate check.
The four differences of a 2 by 2 table are always equal in absolute value: that is why the sum of the O−E is zero and why they are squared.
Step 4
df=(2−1)(2−1)=1, critical value at 5%: 3.841. We reject H0.
Why
A single degree of freedom: as soon as one cell is known, the margins give the other three. Write the critical value you read, not only the decision.
Step 5
Contributions: the two cells of CEGEP A weigh 18.18 out of 36.36, exactly half of the chi-square.
Why
The overall chi-square does not say where the relationship is. Here, the excess of yes at CEGEP A, 120 observed against 90 expected, carries half of it.
Step 6
V=400×136.36≈0.302. With n=40 and the same proportions, χ2=3.64 and V=0.302 still.
Why
The same set of proportions would be NOT significant at forty people and highly significant at four hundred, even though the association is identical. That is the whole difference between evidence and relationship.
The conclusion, written out
“At the 5% level, χ2=36.36 with 1 degree of freedom exceeds 3.841: the CEGEP attended and the answer are related. The gap comes mostly from the excess of favourable answers at CEGEP A, 120 against 90 expected, and Cramér's V of 0.302 places this association at the moderate level.”
The classic mistake on this problem: Stopping at “the chi-square is significant”: the question asked three things, and two thirds of the marks are in the contributions and in V.
Learn by heart
•Compute percentages IN THE DIRECTION of the independent variable, then compare from one category to the next.
•Eij=nRi×Cj, and the expected table has exactly the same margins as the observed one.
•df=(r−1)(c−1), never r×c. Critical values at 5%: 3.841, 5.991, 7.815, 9.488.
•The chi-square is computed on COUNTS. Multiplying every count by k multiplies the chi-square by k.
•The chi-square measures the EVIDENCE, Cramér's V measures the RELATIONSHIP. V does not depend on n.
•Failing to reject never demonstrates independence: it is a lack of evidence, often a lack of power.
•ELABORATION: replication, explanation or specification, depending on what happens to the relationship in each subgroup.
Frequently asked questions
In which direction should the percentages of a crosstab be computed?
In the direction of the independent variable, the one whose action you assume. If the groups being compared are in rows, compute row percentages, each row totalling one hundred, then compare the columns with each other. The percentage in the other direction describes the composition of the respondents, which depends mostly on the sizes of the groups.
How do you compute the expected frequencies of the chi-square?
For each cell, multiply the total of its row by the total of its column, then divide by the overall total. A cell whose row makes 200 and whose column makes 180 in a table of 400 people has an expected frequency of 90. Immediate check: the totals of the expected table must be identical to those of the observed table.
What is the difference between the chi-square and Cramér's V?
The chi-square says whether a relationship exists, Cramér's V says how strong it is. The first grows with the number of people surveyed, so that with a very large sample the smallest difference becomes significant. The second divides by the sample size and stays between zero and one, which makes it comparable from one survey to another.
How many degrees of freedom does a crosstab have?
The number of rows minus one, multiplied by the number of columns minus one. A 2 by 2 table therefore has a single degree of freedom, and a 3 by 4 table has six. The idea is simple: since the totals are fixed, you only need to know that many cells to rebuild the whole table.
Does a non-significant chi-square prove independence?
No. Failing to reject the hypothesis of independence means that the data are not enough to conclude, which happens very often with small samples. The same set of proportions can be non-significant at forty people and highly significant at four hundred. You therefore conclude to a lack of evidence, never to an absence of relationship.
Practise it
Corrected exercises: Cross-tabulations, chi-square and association, 360-300
A method is proved on a paper, not on a sheet. The set for the same chapter takes each of these traps into a problem, with the solution written out step by step.
Something missing from this sheet?A method, a trap, an exercise to practise it, a mistake you spotted: tell me, I read every request.
Sending goes through your free space: that is how I can answer you and tell you when the exercise is added. An email is enough, no password. Create your space.
Rather not create an account? Write to me directly.
Get in touch for a first session. Quantitative Methods is the only compulsory mathematics course in Social Science, and it is also the one that most often decides the R score of a semester.
