Corrected exercises: concepts, variables and measurement scales (360-300)

The CONCEPT is the general idea, here poverty: it has no numerical existence, you can neither observe it nor count it. The DIMENSIONS are the distinct faces of the concept, here income, living conditions and exclusion: they cut the concept into parts, each of which can be approached separately. The INDICATORS are the measures actually collected, here after-tax income per equivalence unit, the share of income spent on housing, and self-reported food insecurity. Only the last level produces numbers. The whole chain is read downward when you build the instrument, and upward when you interpret the results: it is on the way up that people cheat, by attributing to the concept what only the indicator showed.

The equivalence unit is $\sqrt{4}=2$, so the indicator is $\dfrac{68{,}000}{2}=34{,}000$ dollars per equivalence unit. What the division corrects is the household's ECONOMIES OF SCALE: two people living together do not need two rents or two refrigerators. Dividing by 4, the number of people, would assume there are no economies of scale at all; not dividing at all would assume they are total. The square root is a compromise, and it is exactly the one Statistics Canada uses for its Low Income Measure. Note that the choice of the exponent is not a technical detail: it moves thousands of households from one side of the threshold to the other.

The four-person household is at 34,000 dollars per equivalence unit, so it is ABOVE the 26,500 threshold. The one-person household has an equivalence unit of $\sqrt{1}=1$, so its indicator is 25,000 dollars: it is BELOW the threshold, even though its total income is almost three times smaller than the other household's. That is precisely the intended effect: comparing households of different sizes requires bringing income to a common scale, otherwise every large family would look rich.

The politician is not wrong, and that is a reason to publish the definition, not to give up on measuring. The number of people below the threshold depends on the choice of threshold, on the exponent of the equivalence unit and on whether you reason before or after tax: three decisions, three different numbers for the same reality. What stays solid, on the other hand, is the COMPARISON under a constant definition, over time or between neighbourhoods. A serious organization therefore fixes its definition, publishes it, and does not change it: it is that stability, not a supposed neutrality, that makes the measure useful.

A possible indicator: the proportion of students who say they have at least one friend in their class, or the participation rate in extracurricular activities. What the indicator misses: a student can take part in everything without feeling accepted, and a student who is alone by choice is not poorly integrated. No indicator covers its concept entirely; the work consists in choosing several, whose blind spots do not overlap, and saying which ones you kept.

Mother tongue: NOMINAL, categories with no order. Rank in a ranking: ORDINAL, there is an order but the gap between first and second has no reason to equal the gap between second and third. Temperature in degrees Celsius: INTERVAL, differences are comparable but the zero is conventional. Number of children: RATIO, with an absolute zero and a discrete variable. Year of birth: INTERVAL, year zero is a calendar convention. Annual income: RATIO, zero income is a real absence of income.

No. The ratio $\dfrac{20}{10}=2$ makes no sense on an interval scale, because zero degrees Celsius does not represent an absence of heat, it is the melting point of water. In kelvins, a ratio scale whose zero is absolute, the two days are $283.15$ and $293.15$, and the ratio is $\dfrac{293.15}{283.15}\approx 1.035$, that is 3.5% more. The ratio goes from 2 to 1.035 depending on the unit chosen: a ratio that changes when you change the unit is not a property of reality, it is an artifact of the scale. The DIFFERENCE, on the other hand, is 10 degrees in both systems, and that is why it stays legitimate. The short rule to remember: no absolute zero, no ratio.

2.1 is worth nothing as a mean education level, because the codes 1, 2, 3 are ordered labels and not quantities: nothing says the gap between high school and CEGEP equals the gap between CEGEP and university. The number 2.1 can be computed, it cannot be interpreted. What you can legitimately report is the MEDIAN, here the CEGEP level, or the percentage breakdown. The trap is common because the software accepts the calculation without complaint: it is the researcher, not the software, who knows the scale.

Only on a RATIO scale. The coefficient of variation is the quotient of the standard deviation by the mean, so a ratio, and a ratio only makes sense if the zero is absolute. On the Celsius temperatures of a Montreal March, the mean can approach zero and the coefficient can blow up toward infinity without anything remarkable having happened; in kelvins, the same series gives a tiny coefficient. Two irreconcilable numbers for the same series are enough to disqualify the indicator on that scale.

Yes, by changing the indicator rather than the concept. “Education level” coded from 1 to 3 is ordinal; “number of years of study completed” is ratio, with an absolute zero and interpretable differences. You gain the right to compute a mean and a coefficient of variation, and you lose a little of the richness of the diploma, which cannot be reduced to a duration. The lesson holds for the whole course: the scale is not imposed by the concept, it is chosen at the moment you decide how to measure.

A negative item codes belonging backwards: a 5 there means the opposite of a 5 elsewhere. Summing without reversing would amount to adding degrees of warmth to degrees of cold. Reversal on a scale from 1 to 5 is written $x'=6-x$, general form $x'=(\min+\max)-x$. Respondent A goes from 2 to $6-2=4$, B stays at $6-3=3$, C goes from 5 to $6-5=1$. You can check that the transformation preserves the range, sends 1 to 5 and 5 to 1, and leaves the neutral point fixed: those three properties are exactly what we ask of it.

Respondent A: $5+4+4+5=18$. Respondent B: $3+3+3+3=12$. Respondent C: $2+1+1+2=6$. With four items scored from 1 to 5, the total score ranges from $4\times 1=4$ to $4\times 5=20$. A is therefore high on the scale, C low, and B exactly in the middle of the range, which is consistent since B answered at the neutral point everywhere.

Each item taken alone is ORDINAL: nothing guarantees that the gap between “disagree” and “neutral” equals the gap between “neutral” and “agree.” The total score is treated in practice as an INTERVAL variable, and that is an openly assumed convention, not a proof: summing several items smooths out the irregularities of each one and makes the distribution continuous enough for the mean and standard deviation to become useful summaries. You gain two things by summing: RELIABILITY, because misreadings of one item partly cancel out, and COVERAGE, because four wordings touch more facets of the concept than a single one. You lose the right to say that the gap between 12 and 14 is the same as between 18 and 20; the prudent course is to interpret relative positions rather than distances.

Removing the neutral point forces each respondent to take a side, which increases the variance and removes the shelter of the middle, where the indifferent and the cautious alike take refuge. The cost is twofold: a respondent who truly has no opinion is forced into a false answer, and the scores are no longer comparable with those collected on the five-level scale, including those of the previous wave of the same survey. Changing the number of levels in the middle of a longitudinal study is one of the surest ways to make a time series useless.

It is a problem of RELIABILITY, that is, of the stability of the instrument. A six-point gap on a range of sixteen points, when the sense of belonging has no reason to have moved in three weeks, indicates that the instrument is largely measuring noise: mood of the day, interpretation of the wording, order of the items. Validity is a different question: it would ask whether the score really measures belonging and not, for example, general satisfaction. An instrument can be perfectly reliable and completely invalid, and that is the most dangerous case, because the consistency of the results gives an illusion of rigour.

INDEPENDENT variable: the number of study hours per week, the one whose action we assume. DEPENDENT variable: the final grade, the one we assume varies in response. The vocabulary is not trivial: it fixes the direction of the arrow, therefore what the study claims to establish. Swapping the two would give a different and equally plausible hypothesis, the one where good results encourage students to study more.

A CONFOUNDING variable acts on the independent variable AND on the dependent variable, without lying on the causal path between them. A highly motivated student studies more, and also does better for reasons that do not go through study hours: attention in class, questions asked, assignments handed in. The observed correlation between study and grade is then partly a reflection of motivation, and not the effect of study. A MEDIATING variable, for its part, lies ON the path: study hours increase mastery of the exercises, which improves the grade. The distinction is decisive because you must control for a confounding variable, whereas controlling for a mediating variable would make the very effect you are trying to measure disappear.

The drop from 0.42 to 0.11 means that at constant motivation, the relationship between study hours and grade almost vanishes. Motivation therefore accounted for most of the initial correlation: we speak of a SPURIOUS relationship, or at least a largely confounded one. Beware of a second possible reading, though: by keeping only the motivated students, you reduced the variance of the independent variable, which mechanically lowers a correlation coefficient. A careful researcher therefore checks the standard deviations in both groups before concluding, and attributes the drop to confounding only if the spread of study hours stayed comparable.

The strongest design is the RANDOMIZED EXPERIMENT: you draw lots to decide who receives tutoring and who does not. Random assignment spreads all variables evenly on average, including the ones you did not think to measure, which no statistical control can do. It requires three things: a sample large enough for chance to actually balance the groups, an ethical and practical possibility of withholding tutoring from some students, and follow-up that keeps the groups from contaminating each other. When random assignment is impossible, which is the ordinary case in the social sciences, you fall back on quasi-experimental designs, whose conclusions are weaker and must be announced as such.

Two reasonable controls: high school average, which captures prior level, and the number of paid work hours per week, which captures real availability. You measure them at the start, then bring them into the analysis, either by comparing subgroups of the same prior level or by adding them as explanatory variables in a multiple regression. In both cases the goal is the same: compare what is comparable, so that the remaining difference can reasonably be attributed to tutoring.

VALIDITY is the agreement between what the instrument measures and what it claims to measure: the shots fall around the centre. RELIABILITY is the consistency of the instrument from one measurement to the next: the shots are grouped, wherever they land. Target 1, grouped at the centre: valid and reliable. Target 2, grouped at the upper right: reliable but not valid, the error is systematic. Target 3, scattered around the centre: valid on average but not reliable. Target 4, scattered and off-centre: neither. The two qualities are logically independent, which is why having only one of them is never enough.

This scale is case 2: very RELIABLE, with a repeatability of 0.1 kg, and INVALID for measuring an absolute weight, because of the constant 2.0 kg bias. For tracking weight loss, however, it is perfectly usable, because the quantity of interest is a DIFFERENCE: $\left(p_{2}+2\right)-\left(p_{1}+2\right)=p_{2}-p_{1}$, the constant bias cancels out. That is the practical lesson of the exercise: the quality required depends on the question asked, and a biased but stable instrument remains good for comparing, bad for locating.

The test-retest correlation of 0.88 measures RELIABILITY, specifically temporal stability. It says nothing about validity: a test that actually measured pessimism, and not anxiety, would give results that are just as stable. It also says nothing about the internal consistency of the items, which is estimated differently, nor about whether two weeks might be short enough for respondents to remember their answers, which artificially inflates the correlation. High reliability is a necessary condition, never a guarantee.

First way, CRITERION validity: you compare the test with a recognized external reference, for example the diagnosis made independently by clinicians, and check whether the test ranks the same people. Second way, CONTENT and CONSTRUCT validity: you show that the items cover the recognized facets of anxiety, physiological, cognitive and behavioural, and that the score behaves as theory predicts, that is, it rises before an exam and correlates with insomnia without being confused with depression. Neither proves validity in the strict sense; they argue for it, and a serious instrument accumulates both kinds of arguments.

Yes, it is target 3: unbiased but widely scattered. An INDIVIDUAL measurement becomes unusable there, since a difference observed in one person may be nothing but noise. The mean of a large group, on the other hand, stays correct, because random errors cancel out: the standard error of the mean decreases as $\dfrac{s}{\sqrt{n}}$. Hence a reading rule that serves the rest of the course: an instrument with low reliability can be used to compare large groups and never to decide the case of one person.

Health: $z_{1}=\dfrac{72-60}{75-60}=\dfrac{12}{15}=0.80$. Education: $z_{2}=\dfrac{11-6}{16-6}=\dfrac{5}{10}=0.50$. Income: $z_{3}=\dfrac{28{,}000-20{,}000}{60{,}000-20{,}000}=\dfrac{8{,}000}{40{,}000}=0.20$. Min-max normalization brings quantities that cannot be compared, years and dollars, onto the same scale from 0 to 1: that is the condition for the addition that follows not to be a sum of apples and oranges.

Arithmetic mean: $\dfrac{0.80+0.50+0.20}{3}=\dfrac{1.50}{3}=0.50$. The neighbourhood therefore sits exactly in the middle of the reference scale. Note that this 0.50 hides a strong heterogeneity, from 0.20 to 0.80: that is the congenital flaw of any composite index, and it is why the components are always published next to the index.

Weighted index: $0.5\times 0.80+0.3\times 0.50+0.2\times 0.20=0.40+0.15+0.04=0.59$. The neighbourhood gains nine hundredths, not because its situation has changed, but because the weighting rewards the dimension where it does best. The weights express a value judgment about the relative importance of the dimensions: they cannot be deduced from any data. Publishing them, and showing how the index moves when they are changed, is the only honest way to proceed; this is called a sensitivity analysis.

Geometric mean: $\sqrt[3]{0.80\times 0.50\times 0.20}=\sqrt[3]{0.08}\approx 0.43$. It is lower than the arithmetic mean, and it always will be as soon as the components differ. What it penalizes is IMBALANCE: it refuses to let one strength fully make up for a deficiency, and it tends to zero as soon as one component collapses. That is exactly why the United Nations Development Programme switched to the geometric mean in computing its Human Development Index.

The new $z_{3}=\dfrac{36{,}000-20{,}000}{40{,}000}=0.40$, so the index becomes $\dfrac{0.80+0.50+0.40}{3}=\dfrac{1.70}{3}\approx 0.567$: an increase of about 0.067, nearly seven points out of a hundred. On sensitivity to the bounds: if the upper bound for income were set at 80,000 dollars rather than 60,000, the same 8,000-dollar increase would add only $\dfrac{8{,}000}{60{,}000}\approx 0.133$ to the component, so about 0.044 to the index. The progress measured therefore depends on an administrative choice of bounds, which rules out comparing two indices built on different references, and requires keeping the same bounds from one year to the next.

Across the five regions, the relationship is strongly POSITIVE and, as part c) will confirm, perfectly linear: the higher the share of immigrants, the higher the measured rate, with a correlation coefficient of 1. Across the twenty individuals, the points on the contrary line up along a NEGATIVE slope within each region: at the level of persons, the relationship reverses. Both readings are correct, they simply do not concern the same object: the first describes regions, the second describes persons.

It is the ECOLOGICAL FALLACY, described by Robinson in 1950 on US census data: the states where the proportion of immigrants was highest were also the most literate, while individually the immigrants were less so. The rule broken is simple to state and easy to forget: a relationship established at one level of aggregation does not carry over to another level. The mechanism is always the same: immigrants settled in the most developed states, where the whole population, immigrant or not, was more literate. The region variable carries most of the information, and aggregation erases the internal variation, which is precisely the variation that answers the individual-level question.

The five pairs are $(5, 1)$, $(20, 3)$, $(35, 5)$, $(50, 7)$, $(65, 9)$. The gaps along the horizontal axis are 15 throughout, and the gaps along the vertical axis are 2 throughout: the ratio is constant, so the five points lie exactly on a line and the correlation is 1 in absolute value. The slope is $\dfrac{2}{15}\approx 0.133$ rate points per percentage point. A perfect aggregate correlation is not stronger evidence: on the contrary, aggregation removes individual noise and mechanically inflates the coefficients, which explains why ecological correlations are almost always higher than individual correlations.

The symmetric error is the ATOMISTIC, or individualistic, FALLACY: concluding from the individual level to the collective level. Example: within a school, the students who read the most get the best grades; you infer that a school where a lot of reading goes on gets better results overall, which does not follow, because social composition, selection at entry and peer effects operate at the level of the school and not at the level of the student. Both errors have the same root: believing that a relationship is a transportable property, when it belongs to the level at which it was measured.

The question is a trap because class size is a property of the CLASS, whereas success is a property of the STUDENT: the two variables do not live at the same level. The correct treatment is a multilevel analysis, which keeps students as units and introduces the class as a higher level, with its own variables. Lacking such a tool, you choose the student as the unit and add the size of their class as a context variable, knowing that students in the same class are not independent observations, which distorts ordinary statistical tests toward an excess of significance.

FALSE. A postal code can be sorted, but sorting is not ordering in the sense of measurement scales: the order must correspond to an increasing DEGREE of the property under study. H1A does not have “more” of anything than H2B, it designates a different place. Correct statement: the postal code is a NOMINAL qualitative variable; the characters that make it up serve as a label and not as a quantity, and the decisive test is that its mean means nothing.

FALSE, and this is the costliest mistake in the chapter. A reliable instrument gives consistent results, including when it is consistently wrong: a scale that reads two kilos too high is reliable and invalid. Correct statement: reliability is NECESSARY but not sufficient for validity. Without reliability, no individual validity is possible; with reliability alone, you can measure the wrong thing very precisely.

FALSE for the second half only. Saying that the mean exceeds the neutral point is an acceptable summary, provided you accept the convention that treats the total as an interval variable. But the claim about equal gaps explicitly asserts an interval scale that the levels of a Likert scale do not guarantee. Correct statement: on an ordinal scale, you interpret POSITIONS and not DISTANCES; the gap between 3.4 and 4.4 is not comparable to the gap between 1.0 and 2.0, and the median remains the summary that needs no extra assumption.

FALSE twice over. First, a correlation computed on regions describes regions and does not carry over to individuals: that is the ecological fallacy. Second, a correlation coefficient is neither a probability nor a percentage chance: 0.86 means the points fit a line well, and its square, about 0.74, gives the share of shared variance. Correct statement: across the regions of Quebec, average income and graduation rate vary strongly together, and individual data would be needed to say anything about an individual.

FALSE. Decimals are a matter of precision of WRITING, rigour is a matter of the validity of the instrument and of the data collection plan. Writing a median income as 28,431.7382 dollars when it comes from a sample of 300 households gives a false impression of exactness: the margin of error here is counted in hundreds of dollars, and every decimal displayed beyond that is noise. Correct statement: the number of digits published must reflect the real uncertainty of the measurement; adding decimals adds only misplaced confidence.

First dimension, course success: the percentage of courses passed on first enrolment, a RATIO variable, with an absolute zero. Second dimension, persistence: still being enrolled in the third semester, a dichotomous NOMINAL variable, or the number of semesters completed, a ratio variable. Third dimension, certification: obtaining the diploma within the expected time, a dichotomous nominal variable, or the delay in semesters to the diploma, a ratio variable. The three dimensions do not overlap: a student can pass all their courses and drop out, persist without obtaining the diploma, or graduate late after a bumpy path.

No, and for a reason that has nothing to do with the students' talent: the averages come from different courses, graded by different teachers, in groups whose requirements are not aligned. Comparing 76.4 and 73.1 amounts to comparing two uncalibrated thermometers. A less misleading measure is the course pass rate, less sensitive to the grading scale, or better, a comparison at equal entry level, taking into account the high school average of the two cohorts. This is exactly the logic that gave birth to the R score: making comparable grades that are not.

You must publish BOTH, explicitly naming the time window of each. The on-time rate, 41%, measures the smoothness of student paths as much as success: it counts as failures the students who change programs, lighten their semester to work or take one more semester, when many of them will earn their diploma. Publishing that figure alone overstates failure by nearly twenty-two percentage points and pushes the institution to discourage atypical paths rather than support them. Publishing only the long-term rate, conversely, would hide delays. The rule holds beyond this case: a rate does not exist without its observation window.

Weighted index: $0.4\times 0.72+0.25\times 0.58+0.35\times 0.41=0.288+0.145+0.1435=0.5765$, so about 0.58. With the geometric mean $\sqrt[3]{0.72\times 0.58\times 0.41}\approx 0.555$, you get slightly less, because the geometric mean refuses to let a strong component fully make up for a weak one, here graduation. The choice between the two is not technical: it says whether the institution accepts that a good course pass rate buys back a low graduation rate.

First argument: the R score is a RELATIVE measure, built to rank a student within their group. Its mean over an entire cohort is close to a constant by construction, which makes it almost insensitive to the institution's real progress. Second argument: it says nothing about persistence or graduation, since it exists only for courses taken; a student who drops out disappears from the statistic and improves it. An indicator you can improve by losing students is exactly what a student success office must not make its target.

TARGET population: all Quebec residents aged 18 and over, about 6.8 million people. Sampling frame actually used: the members enrolled in an online volunteer panel, a few tens of thousands of people who put themselves forward. The gap between the two is the COVERAGE defect: the frame excludes people without regular internet access, those who refuse to join a panel, and those who never answer solicitations. These exclusions are not spread at random: they concentrate older, isolated or less educated people, precisely groups whose opinions often differ from the average.

Weighting gives each respondent a weight such that the structure of the sample, by age, sex, region and language, reproduces that of the population known from the census. If 18-to-24-year-olds make up 10% of the population and 5% of respondents, each of them counts double. What weighting does NOT correct: the fact that the panel's 18-to-24-year-olds do not resemble Quebec's 18-to-24-year-olds. It corrects composition, never selection within a category. In other words, it treats an observable imbalance and leaves intact the self-selection bias, which is the real problem of a volunteer panel.

The margin of error is a quantity defined for a PROBABILITY sample, where every member of the population has a known, nonzero probability of being chosen. In a volunteer panel, that probability does not exist: nobody can say what chance a given Quebecer had of being interviewed. Publishing a margin anyway amounts to borrowing the precision of a method that was not used. Serious polling firms therefore write, as their own professional standards recommend, that a non-probability sample does not allow a margin of error to be computed, and provide at best a credibility interval accompanied by explicit assumptions.

$E=1.96\sqrt{\dfrac{0.25}{1{,}012}}=1.96\sqrt{0.000247}=1.96\times 0.01572\approx 0.0308$, that is 3.08 points, which does round to 3.1 points. The calculation therefore confirms the arithmetic of the sheet, and that is precisely what makes the figure misleading: it is exact and it does not apply. Note in passing how slowly $\dfrac{1}{\sqrt{n}}$ decreases: to bring the margin down to 1.5 points, you would have to quadruple the sample, to about 4,270 respondents.

First problem, the direction: 54% are IN FAVOUR and not against; the headline reverses the result, which is a plain misreading. Second problem, the gap with respect to uncertainty: the headline turns a sample result into a statement about “Quebecers,” whereas 54% with a margin of 3.1 points means a range of 50.9% to 57.1%, and that reasoning assumes a probability sample that does not exist here. A third criticism is possible: the 5% no opinion disappeared from the headline, which suggests a split between only two camps. The correct release would say that a majority of respondents report being in favour, giving the range and the nature of the panel.