Corrected exercises: descriptive statistics (201-337)

$\overline{x}=\frac{14+9+21+17+12}{5}=\frac{73}{5}=14.6$.

Sorted data: 18, 22, 27, 31, 35, 39, 44. With $n=7$, the median is the 4th value: $31$. Pitfall: taking the middle value of the UNSORTED list, $27$.

Sorted data: 3, 6, 8, 9, 11, 14. With $n=6$ even, the median is the mean of the 3rd and 4th values: $\frac{8+9}{2}=8.5$. Pitfall: taking $\frac{3+9}{2}=6$, the two central values of the unsorted list.

The value 7 appears three times, the value 4 twice, the others once: the mode is $7$. The mode is a VALUE, not its frequency $3$.

$n=5+8+4+2+1=20$ and $\sum n_{i}x_{i}=0\times 5+1\times 8+2\times 4+3\times 2+4\times 1=0+8+8+6+4=26$, so $\overline{x}=\frac{26}{20}=1.3$. Pitfall: dividing by $5$, the number of distinct values, which gives $5.2$.

Sorted data: 3, 5, 6, 8, 9, 11, 12, 15, 17, 24. Range $=24-3=21$. Lower half 3, 5, 6, 8, 9, so $Q_{1}=6$; upper half 11, 12, 15, 17, 24, so $Q_{3}=15$. $IQR=15-6=9$.

$\overline{x}=\frac{40}{5}=8$. Deviations $-4$, $-1$, $0$, $2$, $3$, with squares 16, 1, 0, 4, 9, whose sum is $30$. A sample, so $s^{2}=\frac{30}{5-1}=7.5$. Pitfall: dividing by $n=5$, which gives $6$, the population variance.

$\overline{x}=\frac{144}{6}=24$. Deviations $-3$, $1$, $-6$, $6$, $2$, $0$, with squares 9, 1, 36, 36, 4, 0, whose sum is $86$. So $s^{2}=\frac{86}{5}=17.2$ and $s=\sqrt{17.2}\approx 4.15$. Pitfall: stopping at the variance $17.2$.

Same sum of squares, divided this time by $n=6$: $\sigma^{2}=\frac{86}{6}\approx 14.33$ and $\sigma\approx 3.79$. The population standard deviation is always the smaller of the two, since we divide by a larger number.

$CV=\frac{s}{\overline{x}}\times 100=\frac{7.3}{48.5}\times 100\approx 15.05$%. Pitfall: dividing the mean by the standard deviation, $\frac{48.5}{7.3}$.

Eye colour: qualitative NOMINAL, the categories have no order. Level of satisfaction from 1 to 5: qualitative ORDINAL, the categories are ordered but the gaps are not measurable; nothing says that going from 1 to 2 is the same as going from 4 to 5. Number of siblings: quantitative DISCRETE, you count, and in-between values do not exist. Commuting time: quantitative CONTINUOUS, any value in an interval is possible, only the precision of the instrument limits it. Postal code: qualitative NOMINAL, despite its digits: computing the mean of postal codes would make no sense, and that is the decisive test.

Population: the 4,200 students of the CEGEP. Statistical unit: one student. Variable: the commuting time from home to the college, in minutes. Sample: the 300 students surveyed. A PARAMETER is a characteristic of the POPULATION, here the mean time $\mu$ of the 4,200, a single value, fixed and unknown. A STATISTIC is the same characteristic computed on the SAMPLE, here $\overline{x}$, which changes from one sample to the next. We are looking for the parameter, and all we have is the statistic: the rest of the course is about measuring the gap between the two.

SIMPLE random sampling: number the 4,200 students, then draw 300 numbers at random, each student having the same probability and every group of 300 being equally possible. SYSTEMATIC: the sampling interval is $k=\frac{4200}{300}=14$; draw one student at random among the first 14, then take every 14th student on the list. Proportional STRATIFIED: the science stratum is 60% of the population, so it must be 60% of the sample too, that is $(0.60)(300)=180$ science students and 120 students outside science.

Online poll open to whoever wants to answer: SELF-SELECTION bias, the most motivated answer, typically those with a long commute or a grievance. Questionnaire at the library exit one Tuesday at 4 p.m.: COVERAGE bias, the sampling frame is not the population; it reaches neither evening students nor those who go home early because they live far away, precisely the ones the study is about. Email with 300 replies out of 4,200: NONRESPONSE bias, with a response rate of 7% there is no reason to believe that the respondents resemble the 3,900 who stayed silent. In all three cases, increasing the size corrects nothing: the flaw is not in the number, it is in the way people are chosen.

Because size NEVER makes up for a selection bias. The magazine had drawn its addresses from telephone directories and car registration lists: in the middle of the Great Depression, owning a telephone or a car was the privilege of a well-off minority, rather hostile to Roosevelt. Every new reply therefore added to a sample that was systematically off target, without ever correcting the shift. The institute, on the other hand, had built a sample representative of the social structure of the country. The lesson is fundamental: the size of the sample only reduces RANDOM error, the one that comes from the luck of the draw. SYSTEMATIC error depends only on the method, and a million badly recruited replies are still a million wrong replies. That is also why a serious poll publishes its method before its results.

The sum is $32+35+38+38+41+45+52+219=500$, so the mean is $\overline{x}=\dfrac{500}{8}=62.5$ thousand dollars. For the median, the data are already sorted and $n=8$ is even: take the mean of the two central values, the fourth and the fifth, that is $\dfrac{38+41}{2}=39.5$. The mode is 38, the only value that appears twice.

SEVEN employees out of eight earn less than the mean of 62.5. Only the owner is above it. A mean with 87.5% of the data below it obviously does not describe a typical individual: it is PULLED upward by one extreme value, because computing the mean gives 219 the same weight as 32 even though it lies very far from it.

Without the 219, the sum of the seven remaining salaries is 281, so the mean becomes $\dfrac{281}{7}=40.1$, and the median, the fourth value of a data set of seven, becomes 38. The mean dropped from 62.5 to 40.1, a change of 22.4; the median went from 39.5 to 38, a change of 1.5. The MEAN therefore moved fifteen times more: it is sensitive to extreme values, while the median is ROBUST.

The management will quote the MEAN, 62,500 dollars, which gives the most flattering picture. The union will quote the MEDIAN, 39,500 dollars, closer to everyday reality. It is the median that best describes an ordinary employee, since by construction half earn less and half earn more. Neither is false: they answer different questions, and that is precisely why statistical agencies publish the MEDIAN income and not the mean income.

The mean, 62.5, is clearly GREATER than the median, 39.5. This means that the distribution is SKEWED TO THE RIGHT, with a tail of a few high values that pull the mean. It is the typical shape of every distribution of incomes, of wealth and of company sizes. The opposite case, a mean smaller than the median, would correspond to a skew to the left; equality of the two is what a symmetric distribution shows.

$\overline{x}=\dfrac{12+15+18+20+25}{5}=\dfrac{90}{5}=18$. The deviations $x_{i}-\overline{x}$ are $-6$, $-3$, $0$, $+2$, $+7$, and their squares 36, 9, 0, 4, 49. The sum of the squares is 98.

Because the sum of the RAW deviations is always ZERO: here $-6-3+0+2+7=0$, and it is no accident, it is a property of the mean. The positive deviations exactly cancel the negative ones, so their sum measures no spread at all. Squaring makes every term positive, which prevents the cancelling, and penalizes large deviations more than small ones. You could also take absolute values, which gives the mean absolute deviation, but the square is much easier to handle algebraically, which explains why it is used everywhere.

As a SAMPLE: $s^{2}=\dfrac{98}{5-1}=24.5$ and $s=\sqrt{24.5}\approx 4.95$. As a POPULATION: $\sigma^{2}=\dfrac{98}{5}=19.6$ and $\sigma=\sqrt{19.6}\approx 4.43$. The sample version is therefore always the LARGER of the two.

Because the mean $\overline{x}$ used in the calculation was itself ESTIMATED from the same data. The five deviations are therefore not independent: since their sum must be zero, knowing four of them determines the fifth. There are really only $n-1=4$ free deviations, which are called the degrees of freedom. Dividing by 5 would systematically underestimate the spread of the population the sample is drawn from; dividing by 4 corrects this bias. When you have the WHOLE population, the mean $\mu$ is not estimated but known, and the correction is no longer needed.

$CV=\dfrac{s}{\overline{x}}\times 100=\dfrac{4.95}{18}\times 100=27.5$%. It is used to compare the spread of data sets that have neither the same unit nor the same order of magnitude. A standard deviation of 5 is huge for marks out of 20 and negligible for salaries in dollars: relative to the mean, the coefficient of variation becomes a percentage with no unit, directly comparable. It is commonly used to compare the consistency of two manufacturing processes or the volatility of two investments.

$z_{Anne}=\dfrac{78-70}{8}=\dfrac{8}{8}=1.00$ and $z_{Beno\hat{i}t}=\dfrac{85-80}{10}=\dfrac{5}{10}=0.50$.

ANNE, with a z-score of 1.00 against 0.50. The result is surprising at first, since her raw mark, 78, is LOWER than Benoît's 85. But Anne stands a full standard deviation above her group, while Benoît is only half a standard deviation above his. Measured against the difficulty of the test and the spread of the group, Anne's performance is therefore better. This is exactly the reasoning behind the R score used for university admission in Quebec.

A NEGATIVE z-score means that the value is BELOW the mean of the group; its absolute value says by how many standard deviations. A z-score of ZERO means that the value is exactly equal to the mean. The z-score has no unit: it is the ratio of two quantities with the same unit, which is precisely what makes comparison possible between different distributions.

Anne, at $z=1.00$, sits exactly at the edge of the interval containing the central 68%: about 16% of the students in her group do better than she does. Benoît, at $z=0.50$, is well inside this central interval; about 31% of his group do better. Neither reaches $z=2$, the threshold beyond which you enter the top 2.5% and speak of an exceptional performance.

From $z=\dfrac{x-\overline{x}}{s}$ we get $x=\overline{x}+z\,s=55{,}000+1.5\times 6000=55{,}000+9000=64{,}000$ dollars. The z-score therefore reads both ways: it turns a value into a relative position, and a relative position into a value. Note in passing that the announcement is cleverly worded: it describes a salary of 64,000 dollars, better than that of about 93% of the industry, which is good but less spectacular than the statistical vocabulary suggests.

The data set has $n=10$ values, already sorted. The median is the mean of the fifth and sixth: $Q_{2}=\dfrac{22+25}{2}=23.5$. The lower half is 12, 15, 18, 20, 22, whose middle value is $Q_{1}=18$. The upper half is 25, 28, 30, 32, 60, whose middle value is $Q_{3}=30$.

$IQR=Q_{3}-Q_{1}=30-18=12$. The five-number summary is: minimum 12, $Q_{1}=18$, median 23.5, $Q_{3}=30$, maximum 60.

The fences are $Q_{1}-1.5\,IQR=18-18=0$ and $Q_{3}+1.5\,IQR=30+18=48$. Any value outside $\left[0, 48\right]$ is an outlier. The value 60 exceeds 48: it is the ONLY outlier in the data. All the other values are inside the interval.

The box runs from 18 to 30, cut by a line at 23.5. The left whisker goes down to the smallest non-outlying value, 12; the right whisker goes up to 32, the last non-outlying value, and the 60 is marked separately by an isolated point. The median 23.5 is not in the centre of the box, it is shifted slightly to the left since $23.5-18=5.5$ against $30-23.5=6.5$: the upper half is a little more spread out. Combined with the high outlier, this indicates a distribution skewed to the RIGHT.

Because the range rests only on the TWO extreme values, that is, precisely on the ones most likely to be outliers. Here the range is $60-12=48$, a number entirely dictated by one suspicious data value, while the interquartile range is 12 and describes the central 50% of the data, without being affected by what happens at the ends. The interquartile range is therefore ROBUST, just like the median, and that is why the two naturally go together: the mean and the standard deviation on one side, the median and the interquartile range on the other.

Final mark: $(0.15)(72)+(0.25)(65)+(0.20)(88)+(0.40)(79)=10.80+16.25+17.60+31.60=76.25$. Simple mean: $\frac{72+65+88+79}{4}=\frac{304}{4}=76.00$. They differ because the weighted mean gives each mark an importance proportional to its weight: here, the 79 on the heaviest assessment counts for more than the 88 on the assessment worth 20%. The two would be equal if and only if all the weights were EQUAL, that is 25% each: the simple mean is just a special case of the weighted mean.

The first three assessments bring in $(0.15)(72)+(0.25)(65)+(0.20)(88)=44.65$ points out of the 60% already assessed. She needs 80 overall, so $80-44.65=35.35$ points must come from an assessment worth 40%: her mark must be $\frac{35.35}{0.40}\approx 88.4$. It is demanding but reachable, and the calculation shows exactly what weighting is for: an assessment worth 40% still leaves everything to play for.

The true mean is computed over the 42 students combined: $\frac{(30)(68)+(12)(80)}{42}=\frac{2040+960}{42}=\frac{3000}{42}\approx 71.4$. The teacher computed $\frac{68+80}{2}=74$, that is, the mean of the two means, which amounts to treating the two groups as if they had the same size. Yet the weaker group is two and a half times larger: it must weigh that much more. The gap, 2.6 points, is not trivial when cohorts are being compared.

The mean of means is correct only if all the subgroups have the SAME size. In general, each mean must be weighted by the size of its group: $\overline{x}=\frac{\sum n_{i}\overline{x_{i}}}{\sum n_{i}}$. The reason is that the overall mean is the sum of ALL the values divided by their number, and that the sum of the values of one group is $n_{i}\overline{x_{i}}$: the group size is already there, it cannot disappear. It is one of the most widespread errors in reports, because it is invisible: the wrong calculation looks perfectly reasonable.

$\frac{(4)(80)+(2)(65)+(3)(90)}{4+2+3}=\frac{320+130+270}{9}=\frac{720}{9}=80.0$. We weight by credits because they measure the workload: a 4-credit course represents twice as much learning as a 2-credit course, and it would be absurd for a minor course to count as much as a major one in the results of a semester. The principle is general: a weighted mean always expresses a judgement about the relative IMPORTANCE of the items, and that judgement is a decision, not a calculation. Changing the weights changes the mark without any result having changed.

The midpoint of each class is the mean of its two boundaries: 5, 15, 25, 35 and 45 minutes. The total frequency is $6+14+18+9+3=50$, which matches the statement.

$\overline{x}=\dfrac{\sum n_{i}c_{i}}{n}$ where $c_{i}$ is the class midpoint. The numerator is $6\times 5+14\times 15+18\times 25+9\times 35+3\times 45=30+210+450+315+135=1140$. So $\overline{x}=\dfrac{1140}{50}=22.8$ minutes.

Compute $\sum n_{i}\left(c_{i}-\overline{x}\right)^{2}$. The deviations are $-17.8$, $-7.8$, $2.2$, $12.2$ and $22.2$, whose squares are $316.84$, $60.84$, $4.84$, $148.84$ and $492.84$. Weighted by the frequencies: $1901.04+851.76+87.12+1339.56+1478.52=5658$. Then $\sigma^{2}=\dfrac{5658}{50}=113.16$ and $\sigma=\sqrt{113.16}\approx 10.6$ minutes.

The MODAL class is the one with the largest frequency: $\left[20, 30\right)$ with 18 students. For the median class, look for where the 25th and 26th observations lie. The cumulative frequencies are 6, then 20, then 38: the 25th and 26th therefore fall in the third class. The MEDIAN class is also $\left[20, 30\right)$. The fact that they coincide, together with a mean of 22.8 close to the midpoint of that class, suggests a fairly symmetric distribution, with a slight skew to the right due to the long commutes.

Because each of the 50 actual values was replaced by the MIDPOINT of its class, which amounts to assuming that the data are spread uniformly within each class, or at least symmetrically around the midpoint. If, for example, the 18 students in the class $\left[20, 30\right)$ were in fact all around 21 minutes, the true mean would be lower than 22.8. The result would be EXACT if, in each class, the actual mean of the data coincided with the midpoint of the class. In practice the errors largely cancel from one class to the next, and the approximation is excellent as soon as the classes are narrow enough and the frequencies large enough.

Cumulative frequencies: 6 at 10 min, $6+14=20$ at 20 min, $20+18=38$ at 30 min, $38+9=47$ at 40 min, $47+3=50$ at 50 min. As percentages: 12%, 40%, 76%, 94%, 100%. The last value must equal the total frequency exactly; it is the check to make every time.

Because the cumulative frequency answers the question “how many students take LESS THAN so many minutes”. We only know for certain that 20 students are under 20 minutes once the class $[10, 20)$ has been entirely counted, that is, at its upper boundary. Plotting the running total at the midpoint of the class would amount to claiming that half the class is reached in the middle, which is precisely what we do not know. The ogive is therefore an increasing curve that starts from zero at the lower boundary of the first class and reaches the total frequency at the upper boundary of the last one.

The median corresponds to the 25th student, that is 50% of 50. The cumulative frequency is 20 at 20 minutes and 38 at 30 minutes: the median therefore falls in the class $[20, 30)$. We must move $25-20=5$ students into it out of the 18 in the class, a fraction $\frac{5}{18}$ of its width of 10 minutes: $M=20+\frac{5}{18}(10)\approx 22.8$ minutes. On a histogram, we could only name the median class, $[20, 30)$, and go no further: that is the whole advantage of the ogive, which gives a value and not an interval.

Ninth decile: 90% of 50 is 45 students. The cumulative frequency is 38 at 30 minutes and 47 at 40 minutes, so $D_{9}$ is in $[30, 40)$: $D_{9}=30+\frac{45-38}{9}(10)\approx 37.8$ minutes. First decile: 10% of 50 is 5 students, in the first class: $D_{1}=0+\frac{5}{6}(10)\approx 8.3$ minutes. Concretely, one student in ten arrives in less than 8 minutes, and one in ten takes more than 38 minutes. For a transit service, it is this last figure that matters: planning on the mean would leave 10% of the riders in a situation clearly worse than what one believes has been provided for.

At 35 minutes, we are in the middle of the class $[30, 40)$: the cumulative frequency is $38+\frac{5}{10}(9)=42.5$ students, so $50-42.5=7.5$ students take more than 35 minutes, that is 15%. The assumption made when interpolating is that the data are spread UNIFORMLY within each class, which allows the ogive to be treated as a series of straight-line segments. It is reasonable when the classes are narrow and the frequencies large, but it becomes wrong as soon as the spread inside a class is very uneven: if, in the class $[30, 40)$, all the students actually took 31 minutes, only the 3 students of $[40, 50)$ would take more than 35 minutes, and the exact answer would be 6%, not 15%. The remedy is the same as always: narrower classes, as long as the frequencies allow it.

Mean: $\overline{F}=1.8(18.0)+32=32.4+32=64.4$ °F. Standard deviation: $s_{F}=1.8(4.0)=7.2$ °F. The 32 plays no part because adding a constant SHIFTS the whole data set as a block, without bringing the values closer together or moving them apart: the gap between any two readings stays the same, and so does the spread. Only the multiplication by 1.8 stretches the gaps, and it stretches all of them by the same factor.

For $y=ax+b$: $\overline{y}=a\overline{x}+b$, the variance becomes $s_{y}^{2}=a^{2}s_{x}^{2}$ and the standard deviation $s_{y}=|a|\,s_{x}$. The absolute value matters: multiplying by $-2$ flips the data but doubles its spread, and a standard deviation can never be negative. Note also that the median and the quartiles follow the same rule as the mean, and that the range and the interquartile range follow the rule of the standard deviation.

In Celsius: $CV=\frac{4.0}{18.0}\times 100\approx 22.2$%. In Fahrenheit: $CV=\frac{7.2}{64.4}\times 100\approx 11.2$%. The coefficient of variation was CUT IN HALF although no data value changed, only the unit. The conclusion is clear: the coefficient of variation only makes sense for a variable measured on a RATIO scale, that is, one with an absolute zero, such as a mass, a duration or an income. The Celsius and Fahrenheit temperatures have purely conventional zeros, so the ratio $\frac{s}{\overline{x}}$ means nothing there. In kelvins, the question would become legitimate again.

In Celsius: $z=\frac{22-18.0}{4.0}=1.00$. In Fahrenheit: $z=\frac{71.6-64.4}{7.2}=\frac{7.2}{7.2}=1.00$. Identical. Proof: if $y=ax+b$ with $a>0$, then $z_{y}=\frac{y-\overline{y}}{s_{y}}=\frac{(ax+b)-(a\overline{x}+b)}{a s_{x}}=\frac{a(x-\overline{x})}{a s_{x}}=\frac{x-\overline{x}}{s_{x}}=z_{x}$. The $b$ disappears by subtraction, the $a$ by cancellation. This is exactly why the z-score makes it possible to compare measurements made in different units: it is the only indicator of the chapter that depends neither on the origin nor on the scale.

Adding 5 points is $y=x+5$: the mean goes up by 5, the standard deviation does not move, and the ranking is strictly unchanged since everyone gains the same amount. Multiplying by 1.1 is $y=1.1x$: the mean and the standard deviation are both multiplied by 1.1, and the ranking also stays unchanged, since the function is increasing. No linear transformation with a positive slope can change an order, which explains why none changes the z-scores either. Finally, applying $y=\frac{x-\overline{x}}{s}$ means taking $a=\frac{1}{s}$ and $b=-\frac{\overline{x}}{s}$: the new data set has mean $\frac{\overline{x}-\overline{x}}{s}=0$ and standard deviation $\frac{s}{s}=1$. It is called the STANDARDIZED data set, and it is what makes two assessments marked on different scales comparable. The whole logic of the z-score, and of the standardized scores used for university admission, fits in these two lines.

The comparison with the MEDIAN is missing. The gap between 71,000 and 48,000 is considerable: the mean exceeds the median by nearly 50%, which reveals a distribution strongly skewed to the right. Concretely, a minority of very well-off households pulls the mean upward, while the typical household earns 48,000 dollars. Calling the neighbourhood well-off on the basis of the mean alone is describing a minority as if it were the norm.

$CV=\dfrac{55{,}000}{71{,}000}\times 100=77.5$%. Such a high coefficient of variation signals extreme heterogeneity: the spread is of the same order of magnitude as the average level itself. This neighbourhood has no “typical” income in the usual sense; it puts side by side very different situations. Any single measure of central tendency, whichever it is, will necessarily mislead there.

YES, it is correct, and it is even the definition of the median: half of the households earn more than 48,000 dollars and the other half less. The claim is beyond reproach on substance. It is still slanted by what it leaves out: one could just as truthfully write that half of the households earn LESS than 48,000 dollars. Choosing to present the upper half rather than the lower half is a rhetorical choice, not a statistical one.

The two neighbourhoods have the same typical household, but the first is highly UNEQUAL while the second is HOMOGENEOUS: in the second, if incomes are spread roughly in a bell shape around the median, about 95% of households lie within two standard deviations of it, between 24,000 and 72,000 dollars, whereas in the first, incomes spread over a range several times wider. Two neighbourhoods with the same median can therefore offer entirely different social realities, which no measure of central tendency can reveal on its own.

First, the MEDIAN, 48,000 dollars, as the measure of the typical household, because it resists the extreme values that are clearly present here. Second, the INTERQUARTILE RANGE, or failing that the quartiles themselves, because it describes the spread of the central 50% without being distorted by the tail of the distribution, unlike the standard deviation of 55,000, which is itself inflated by the high incomes. Third, the share of households below a reference THRESHOLD, for example the low-income cut-off, because a position indicator answers the real social question where no numerical summary does. One could add a histogram, which remains the most honest way to present a distribution: it summarizes nothing and therefore hides nothing.