Corrected exercises: cross-tabulations, chi-square and association (360-300)

Percentage of the GRAND TOTAL: $\dfrac{21}{300}=7.0\%$, which answers “what share of all respondents both work a lot AND dropped a course”. ROW percentage: $\dfrac{21}{60}=35.0\%$, which answers “among those who work 21 hours or more, what share dropped a course”. COLUMN percentage: $\dfrac{21}{63}\approx 33.3\%$, which answers “among those who dropped a course, what share worked a lot”. Three numbers, three questions, and only one of the three is the question of the survey.

Row percentages of dropping a course: $\dfrac{18}{120}=15.0\%$, $\dfrac{24}{120}=20.0\%$, $\dfrac{21}{60}=35.0\%$. They show that the share of students who dropped a course rises steadily with the hours worked, and that it more than doubles between the first and the last profile. That is exactly the reading that a 100% stacked bar chart makes visible at a glance.

You must compute the percentages IN THE DIRECTION of the independent variable, here the hours of work, so by row. The general rule is simple and worth remembering: compute percentages within the categories of the INDEPENDENT variable, and compare those percentages from one category to the next. Each group is then brought to a base of 100, which neutralizes the fact that the three profiles do not have the same size, 120, 120 and 60.

Column percentages: $\dfrac{18}{63}\approx 28.6\%$, $\dfrac{24}{63}\approx 38.1\%$, $\dfrac{21}{63}\approx 33.3\%$. They describe the COMPOSITION of the group of students who dropped a course, and they depend as much on the size of each profile in the sample as on the relationship under study. The “21 h or more” profile looks unremarkable there, 33.3%, when it is by far the most at risk, simply because it has only 60 people. Reading a table in the wrong direction is the most common error of the semester, and it regularly reverses the conclusion.

Reply that the table establishes an ASSOCIATION, not a causal relationship. Three objections stand on these data alone. The causation may run the other way: a struggling student lightens the course load and takes a job, so dropping the course comes before the work. A confounding variable may explain both: the financial situation of the household pushes students to work and makes studies more fragile. Finally, the sample is not experimental: nobody was randomly assigned to 21 hours of work. What the table does allow you to say is that students who work a lot drop courses more often, and that this deserves an explanation.

Under independence, the share who dropped a course would be the SAME in all three profiles, and it would equal the overall share: $\dfrac{63}{300}=21.0\%$. That is the very definition of independence between two variables: knowing the number of hours worked would give no information about the probability of dropping a course.

Expected frequency: $E_{ij}=\dfrac{\text{row } i \text{ total}\times \text{column } j \text{ total}}{n}$. Row “0 to 10 h”: $\dfrac{120\times 63}{300}=25.2$ and $\dfrac{120\times 237}{300}=94.8$. Row “11 to 20 h”: the same, 25.2 and 94.8, since the row total is identical. Row “21 h or more”: $\dfrac{60\times 63}{300}=12.6$ and $\dfrac{60\times 237}{300}=47.4$. The expected frequencies are decimals, and they must never be rounded: they are expected values, not counts.

Row totals: $25.2+94.8=120$, $25.2+94.8=120$, $12.6+47.4=60$. Column totals: $25.2+25.2+12.6=63$ and $94.8+94.8+47.4=237$. Grand total: 300. This check is the best error control of the chapter, and it is immediate: by construction, the expected table has exactly the same marginal totals as the observed table, and only the internal split changes.

Differences $O-E$: $18-25.2=-7.2$; $102-94.8=+7.2$; $24-25.2=-1.2$; $96-94.8=+1.2$; $21-12.6=+8.4$; $39-47.4=-8.4$. Their sum is zero, and so is the sum within each row. This is unavoidable since the two tables have the same margins: what is missing in one cell is necessarily found in another cell of the same row. This property is the reason why the differences are SQUARED before being added, since otherwise the sum would always be zero and would measure nothing.

In ABSOLUTE terms, it is the “21 h or more” profile with a difference of 8.4, against 7.2 for the first profile. In RELATIVE terms, the gap is even clearer: $\dfrac{8.4}{12.6}=66.7\%$ more students who dropped a course than expected, against $\dfrac{7.2}{25.2}=28.6\%$ fewer than expected for the first profile. This is exactly what the chi-square formalizes by dividing the square of the difference by the expected frequency: a difference of 8.4 in a cell that expected only 12.6 weighs far more than the same difference in a cell that expected 95.

NULL hypothesis: hours of paid work and dropping a course are INDEPENDENT in the population, meaning that the share who drop a course is the same whatever the profile. ALTERNATIVE hypothesis: the two variables are not independent. Two remarks matter for what follows: the null hypothesis is the one we try to refute, and the alternative says nothing about the DIRECTION of the relationship, which is a limitation of the chi-square.

Contributions $\dfrac{\left(O-E\right)^{2}}{E}$: $\dfrac{\left(-7.2\right)^{2}}{25.2}=\dfrac{51.84}{25.2}\approx 2.057$; $\dfrac{51.84}{94.8}\approx 0.547$; $\dfrac{1.44}{25.2}\approx 0.057$; $\dfrac{1.44}{94.8}\approx 0.015$; $\dfrac{70.56}{12.6}=5.600$; $\dfrac{70.56}{47.4}\approx 1.489$. Total: $\chi^{2}\approx 9.765$.

$df=\left(3-1\right)\left(2-1\right)=2$. The critical value at 5% is 5.991. Since $9.765>5.991$, we REJECT the hypothesis of independence: the association observed between hours of work and dropping a course is statistically significant at the 5% level. We can be more precise: the value also exceeds the 1% critical value, 9.210, but not the 0.1% one, 13.816. The p-value therefore lies between 0.001 and 0.01.

The cell “21 h or more, dropped a course” contributes 5.600, that is $\dfrac{5.600}{9.765}\approx 57.3\%$ of the total chi-square. Adding its neighbour in the same row, 1.489, the heavy-workers profile alone accounts for more than 72% of the departure from independence. This is an essential and often neglected reading: the overall chi-square says there is a relationship, the table of contributions says WHERE it is, and here it is entirely concentrated on the third profile.

No. Rejection says that the observed gap is too large to be attributed to sampling chance alone; it says nothing about the STRENGTH of the relationship. The proof is that, for an identical structure, the chi-square grows in proportion to the sample size: any gap, however tiny, becomes significant with enough respondents. You must therefore also compute a normalized measure of ASSOCIATION, Cramér's V or the contingency coefficient, which does not depend on $n$.

First condition, the INDEPENDENCE of the observations: each individual appears in only one cell, which rules out multiple-response questions and repeated measurements on the same people. Second condition, all the expected frequencies must be ABOVE 5, or more flexibly, at most 20% of the cells may have an expected frequency between 1 and 5, and none may fall below 1. Third condition, the test is applied to COUNTS, never to percentages: computing a chi-square on a table of percentages amounts to pretending that the sample has exactly 100 people.

Total for “Arts and Letters”: $8+11+4+2=25$. Column totals: $38+22+8=68$; $52+31+11=94$; $14+9+4=27$; $6+3+2=11$. Grand total: 200. Expected frequencies of the row: $\dfrac{25\times 68}{200}=8.5$; $\dfrac{25\times 94}{200}=11.75$; $\dfrac{25\times 27}{200}=3.375$; $\dfrac{25\times 11}{200}=1.375$. Two cells out of four fall below 5, and among the twelve cells of the table, the three cells below 5 make up 25% of the total, which exceeds the 20% tolerance. The test is therefore not applicable as it stands.

Merge the two lowest levels of satisfaction into one, “not very satisfied or dissatisfied”. The row “Arts and Letters” then has observed frequencies 8, 11 and 6, and expected frequencies 8.5, 11.75 and $\dfrac{25\times 38}{200}=4.75$. Only one cell in nine remains below 5, that is 11%, which is acceptable. The cost is twofold: you lose the distinction between “not very satisfied” and “dissatisfied”, so information is lost, and above all you chose the merging AFTER seeing the data, a decision that must be honestly declared in the report. A merging decided in advance, for reasons of content, is always preferable.

Table 12, 8 / 6, 14: the row totals are 20 and 20, the column totals 18 and 22, and $n=40$. Expected frequencies: $\dfrac{20\times 18}{40}=9$ and $\dfrac{20\times 22}{40}=11$, in both rows. Without correction: $\chi^{2}=\dfrac{9}{9}+\dfrac{9}{11}+\dfrac{9}{9}+\dfrac{9}{11}\approx 3.636$, below the critical value 3.841: we fail to reject. With Yates' correction, each absolute difference of 3 becomes 2.5: $\chi^{2}=\dfrac{6.25}{9}\times 2+\dfrac{6.25}{11}\times 2\approx 2.525$, even further from the critical value. The correction is therefore CONSERVATIVE: it always lowers the chi-square and makes rejection harder, which compensates for the fact that the continuous chi-square distribution approximates a discrete reality.

Three routes. Merge categories, which remains the simplest solution when the merging makes sense in terms of content. Use Fisher's EXACT test, which directly computes the probability of every possible table with fixed margins and requires no condition on the frequencies; it is instantaneous with software today, and it is the right answer for a small 2 by 2 table. Finally, enlarge the sample, which is the only solution if you want to keep all the categories. What you must never do is apply the test anyway and hope that nobody checks: p-values obtained on nearly empty cells are plainly wrong, not just approximate.

$\min\left(3-1,\ 2-1\right)=1$, so $V=\sqrt{\dfrac{9.765}{300\times 1}}=\sqrt{0.03255}\approx 0.180$.

A value of 0.18 corresponds to a WEAK association, only just above the threshold of negligibility. In other words, the relationship between hours of work and dropping a course is real in the statistical sense, it is much larger than what chance would produce, and it remains modest: knowing the work profile of a student improves the prediction of dropping a course only marginally. This is the nuance that the headline of a press release always loses.

If the two variables are independent, every observed frequency equals its expected frequency, all the differences are zero, so $\chi^{2}=0$ and $V=0$. At the other extreme, on a 2 by 2 table where all the observations lie on one diagonal, for example 50 and 0 in the first row, 0 and 50 in the second, the expected frequencies are all equal to 25 and $\chi^{2}=4\times \dfrac{625}{25}=100=n$. Since $\min\left(r-1,\ c-1\right)=1$, we get $V=\sqrt{\dfrac{100}{100}}=1$. So $V$ is indeed normalized between 0 and 1, and it is precisely the division by $n$ that makes it insensitive to the sample size.

$C=\sqrt{\dfrac{9.765}{9.765+300}}=\sqrt{\dfrac{9.765}{309.765}}\approx 0.178$. The value is very close to $V$, which is normal for a weak association. $V$ is preferred because the contingency coefficient NEVER reaches 1: its maximum depends on the dimensions of the table, about 0.707 for a 2 by 2 table and 0.816 for a 3 by 3. Two contingency coefficients from tables of different formats are therefore not comparable, whereas two values of $V$ are.

“Among 300 students, the proportion who dropped a course rises from 15.0% among those who work at most 10 hours a week to 35.0% among those who work 21 hours or more. The association is statistically significant, $\chi^{2}=9.77$ with 2 degrees of freedom, p-value below 0.01, but its strength is weak, $V=0.18$. More than half of the departure from independence comes from the 21 hours or more profile alone. Since these data are cross-sectional and non-experimental, they do not establish a cause-and-effect relationship.” Four elements never to omit: the percentages that show the DIRECTION, the test that shows the SIGNIFICANCE, the measure that shows the STRENGTH, and the limit of the design.

The chi-square is MULTIPLIED BY TWO, going from 9.765 to 19.53. Algebraically: if all the observed frequencies are multiplied by $k$, the row and column totals are as well, so each expected frequency $\dfrac{R\times C}{n}$ is multiplied by $\dfrac{k\times k}{k}=k$. Each term then becomes $\dfrac{\left(kO-kE\right)^{2}}{kE}=\dfrac{k^{2}\left(O-E\right)^{2}}{kE}=k\dfrac{\left(O-E\right)^{2}}{E}$. The chi-square is therefore exactly proportional to $n$, while the critical value does not move: it depends only on the degrees of freedom.

Cramér's V is UNCHANGED: $V=\sqrt{\dfrac{k\chi^{2}}{kn\times \min\left(r-1,c-1\right)}}$, and the factor $k$ cancels. With $n=900$ and $\chi^{2}=29.29$, we get back $V=\sqrt{\dfrac{29.29}{900}}\approx 0.180$. This is precisely the property that makes $V$ a measure of STRENGTH: it describes the structure of the table, not the amount of data collected.

On a 2 by 2 table with equal groups and proportions 0.51 and 0.49, the chi-square is approximately $n\times \dfrac{\left(p_{1}-p_{2}\right)^{2}}{4\overline{p}\left(1-\overline{p}\right)}$ with $\overline{p}=0.50$, that is $n\times \dfrac{0.0004}{1}=0.0004n$. We need $0.0004n\ge 3.841$, so $n\ge 9{,}603$. With nearly ten thousand respondents, a gap of two points therefore becomes significant, while Cramér's V is $\sqrt{\dfrac{3.841}{9{,}603}}=0.02$, a perfectly negligible association.

That it must stop being read as “important”. “Significant” means nothing more than “too large to be reasonably attributed to sampling chance, given the sample size”. A serious report therefore always publishes three things together: the raw measure of the gap, in percentage points, the result of the test, and a normalized measure of strength. On very large administrative files, where $n$ exceeds a million, the test loses all practical interest since absolutely everything is significant.

The symmetric problem is a lack of POWER: with a sample that is too small, a real and even important relationship may fail to reach significance, and you wrongly conclude that there is no relationship. Failing to reject the null hypothesis has never meant that it is true. You guard against it by computing the power BEFORE the survey, which amounts to choosing the sample size according to the smallest gap you want to be able to detect, and by publishing confidence intervals rather than the rejection decision alone.

Overall, 72.5% of the participants pass all their courses against 57.5% of the non-participants, a gap of 15.0 POINTS in favour of the participants. A hasty report would conclude that extracurricular activities promote success, and the institution would fund the program.

Among those who work little: 80% against 80%, ZERO gap. Among those who work a lot: 50% against 50%, also ZERO gap. With hours of work held constant, participation has strictly no relationship with success. The overall association, though perfectly real, is found in NEITHER of the two subgroups.

Participants: $150+50=200$, of whom $120+25=145$ passed, that is 72.5%. Non-participants: $50+150=200$, of whom $40+75=115$ passed, that is 57.5%. The total is indeed 400 students and 260 who passed. This check is indispensable: it establishes that the reversal is not a calculation error but a property of the data.

The mechanism is that of a common ANTECEDENT variable. Hours of paid work determine both participation, since nobody joins the debating club while working 25 hours a week, and success, which drops from 80% to 50% among heavy workers. Participants are therefore massively recruited among those who work little, 150 out of 200, and non-participants among those who work a lot, 150 out of 200. The overall comparison actually compares two populations with very different availability, and the gap of 15 points measures that difference in composition, not an effect of extracurricular activities.

Three possible outcomes. REPLICATION: the association remains identical in each stratum, the third variable changes nothing, and the initial relationship is strengthened. EXPLANATION, or spurious relationship: the association disappears in all the strata because the third variable is ANTECEDENT to the other two. SPECIFICATION, or interaction: the association remains in some strata and not in others, which specifies the conditions of the relationship. The present case is an EXPLANATION: hours of work logically and chronologically precede participation as well as success, and the overall association was spurious. If the third variable came after the first, we would speak of interpretation, and the variable would be intervening rather than confounding.

FALSE. The value of the chi-square depends on three things: the strength of the relationship, the sample size and the dimensions of the table. A chi-square of 42.7 on 5,000 respondents corresponds to a Cramér's V of 0.09, a negligible association. Correct statement: the chi-square settles SIGNIFICANCE; strength is read on Cramér's V, which divides by $n$ precisely for that reason.

FALSE, it is a column reading presented as an explanation. The 33% describe the COMPOSITION of the group who dropped a course, which depends mostly on the size of each profile in the sample. The right reading is by row: 35% of the heavy workers drop a course against 15% of the light workers. Correct statement: the share who drop a course is more than twice as high among those who work 21 hours or more, and nothing in this table attributes a share of the dropped courses to a cause.

FALSE, and the error is serious. The chi-square is computed on COUNTS. Converting to percentages amounts to imposing $n=100$ in each row, which manufactures a fictitious sample size and makes the p-value entirely wrong, in one direction or the other depending on the true size. Correct statement: percentages are presented to the reader, and the test is computed on the raw counts.

FALSE. Failing to reject the null hypothesis means that the data are COMPATIBLE with independence, not that they demonstrate it. With a small sample, an important association may well fail to reach the threshold: that is a lack of power, not proof of absence. Correct statement: at the 5% level, the data do not allow us to conclude that there is an association; to go further, we would need a larger sample or a confidence interval on the difference.

FALSE, or rather backwards. Elaboration does not invalidate the initial analysis, it COMPLETES it: the overall relationship did exist in the data, and the control reveals that it ran through a third variable. Correct statement: the observed association was spurious, that is, attributable to a common antecedent variable; this is a RESULT of the analysis, often the most interesting one, and it must be published as such rather than hidden.

Success rates: $\dfrac{96}{120}=80.0\%$ with a dedicated space, $\dfrac{56}{90}\approx 62.2\%$ with a shared space, $\dfrac{18}{40}=45.0\%$ with no place to study at all. The trend is monotone and strong: 35 points separate the two extremes, and the rate drops at every step. This is the correct direction of calculation, since the place to study is the independent variable here.

Margins: rows 120, 90 and 40; columns 170 and 80; $n=250$. Expected frequencies: $\dfrac{120\times 170}{250}=81.6$ and $\dfrac{120\times 80}{250}=38.4$; $\dfrac{90\times 170}{250}=61.2$ and $\dfrac{90\times 80}{250}=28.8$; $\dfrac{40\times 170}{250}=27.2$ and $\dfrac{40\times 80}{250}=12.8$. All exceed 5, so the condition is satisfied.

Contributions: $\dfrac{\left(96-81.6\right)^{2}}{81.6}=\dfrac{207.36}{81.6}\approx 2.541$; $\dfrac{207.36}{38.4}=5.400$; $\dfrac{\left(56-61.2\right)^{2}}{61.2}=\dfrac{27.04}{61.2}\approx 0.442$; $\dfrac{27.04}{28.8}\approx 0.939$; $\dfrac{\left(18-27.2\right)^{2}}{27.2}=\dfrac{84.64}{27.2}\approx 3.112$; $\dfrac{84.64}{12.8}=6.613$. Total: $\chi^{2}\approx 19.05$, with $df=\left(3-1\right)\left(2-1\right)=2$. Since $19.05>5.991$ and even $>13.816$, we reject independence with a p-value below 0.001.

$V=\sqrt{\dfrac{19.05}{250\times 1}}=\sqrt{0.07619}\approx 0.276$. The association is MODERATE, clearly stronger than the one observed with hours of work, which was 0.18. This is a result the office can actually use, unlike an association of 0.05 that would be significant without being actionable.

“The rate of passing every course falls from 80.0% among students with a dedicated study space to 62.2% among those whose space is shared, then to 45.0% among those who have none. The association is significant, $\chi^{2}=19.05$ with 2 degrees of freedom, p-value below 0.001, and of moderate strength, $V=0.28$. Since these data are cross-sectional, they do not allow us to attribute the difference to the place to study alone: having a dedicated space is strongly linked to household income and to the size of the home, variables that would have to be controlled before guiding an intervention.” It contains the three mandatory elements, direction, significance and strength, plus the limit of the design and the control variable that should be measured.

$df=\left(3-1\right)\left(2-1\right)=2$: the reported value is correct. Volunteering rates: $\dfrac{38}{170}\approx 22.4\%$, $\dfrac{52}{200}=26.0\%$, $\dfrac{44}{140}\approx 31.4\%$. The trend rises with income, by nine percentage points between the extremes, which is not nothing even if the test does not conclude.

The author can say that, at the usual 5% level, the data do NOT allow rejecting the hypothesis of independence: the observed gap could be produced by sampling chance about one time in eight. The author can NOT say that volunteering is independent of income: with $N=510$ spread over three brackets, the power of the test is modest, and a real difference of nine points may perfectly well fall below the threshold. What should be published here is the confidence interval on the gap between the extreme brackets, which would show the whole range of differences compatible with these data.

$V=\sqrt{\dfrac{4.12}{510\times \min\left(2,1\right)}}=\sqrt{\dfrac{4.12}{510}}=\sqrt{0.008078}\approx 0.0899$, rounded to 0.09. The reported figure is therefore consistent. This check takes ten seconds and it regularly catches transcription errors in published tables: it is worth doing systematically.

Nonresponse to the income question is not random: it is notoriously more frequent at both ends of the distribution, among the poorest out of embarrassment and among the richest out of caution. The 94 excluded persons represent 15.6% of the initial sample, a considerable share, and removing them may have flattened the relationship by emptying precisely the extreme brackets, the ones that carried the gap. A serious analysis would compare respondents and nonrespondents on the other available variables, age, schooling, region, and say so in the text.

“In this sample, the volunteering rate rises from 22.4% among households under 40,000 dollars to 31.4% among those at 80,000 dollars or more, but the gap does not reach significance at the 5% level, $\chi^{2}=4.12$; $df=2$; $p=0.127$; $V=0.09$. These data therefore do not allow us to conclude that there is a relationship, nor to rule one out, especially since 15.6% of the respondents were excluded for not reporting their income.” The difference from the author's sentence is decisive: it replaces a claim of absence with a finding of indecision, which is the only thing the test allows.