Descriptive statistics 201-SN1-RE: revision sheet

Q: When should the median be used rather than the mean in descriptive statistics?

When you want to describe a typical individual and the distribution is skewed or contains an extreme value: flows with a flood, concentrations with a pollution day, incomes. The median depends only on ranks and resists extremes. If the question is about a total, such as a volume or total rainfall, the mean is what you need, because it alone preserves the sum.

Q: Why do we divide by n minus 1 to compute the standard deviation of a sample?

Because the deviations are measured around the sample mean, which is the point closest to the sample's own data. The sum of squared deviations is therefore in general too small compared with the one we would get using the true population mean. Dividing by n minus 1 instead of n corrects exactly this bias for the variance, on average over all possible samples.

Q: How do you compute the median of data grouped into classes?

Find the median class, the one where the cumulative frequency passes half of the total frequency. Then interpolate: the lower boundary of the class, plus the amount still missing to reach half, divided by the frequency of the class, multiplied by its width. This calculation assumes the data are spread evenly within the class, so it is an estimate.

Q: What does a z-score of 2 mean?

The value exceeds the mean of its group by two standard deviations. It is not twice the mean. A negative z-score indicates a value below the mean. For a roughly bell-shaped distribution, a z-score beyond 2 in absolute value is unusual; the z-score is mainly used to compare individuals from different groups or measured in different units.

Q: Should an outlier be removed from a data set?

No, not without proof of an error. An outlier is only a value that lies far away according to a rule, for example more than one and a half times the interquartile range above the third quartile. If it is real, like a smoke day or a contaminated well, it is often the most important information: flag it, draw it as an isolated point, and publish the median, which resists it.

The thread of the chapter

A summary does not replace the distribution: its SHAPE (skewness, extreme value, classes) chooses the measure to publish, and the standard deviation gives the scale for judging a deviation.

Topics covered

Population and sample
Parameter and statistic
Discrete and continuous variables
Data grouped into classes
Mean, median and mode
Skewness of a distribution
Variance and standard deviation
Standard score (z-score)
Percentile rank
Quartiles and box plot
Histogram and frequency density

The essentials

Three centres, three different questions

• The MEAN shares the total equally: $\overline{x}=\dfrac{\sum x_{i}n_{i}}{n}$ . It is the only one that preserves a total ( $n\,\overline{x}$ ), and it follows the extreme values.
• The MEDIAN splits the SORTED data into two halves. It depends only on ranks: changing the largest value does not move it.
• The MODE is the most frequent value or class; it is the only centre of a categorical nominal variable.
• Right skew (long tail toward large values): in general mode $<$ median $<$ mean. Left skew: the reverse order. This ordering is a tendency, not a theorem.

A distribution stretched to the right: the long tail pulls the mean (

\approx 3.6

) beyond the median (

\approx 2.7

), itself beyond the mode (

\approx 1.5

Before computing, look at the shape: the shape, together with the question asked, chooses the measure to publish.

Dispersion: two divisors, one good reason

• Sample: $s^{2}=\dfrac{\sum(x_{i}-\overline{x})^{2}}{n-1}$ ; whole population: $\sigma^{2}=\dfrac{\sum(x_{i}-\mu)^{2}}{N}$ .
• Computing formula: $\sum(x_{i}-\overline{x})^{2}=\sum x_{i}^{2}-n\,\overline{x}^{2}$ , with $\overline{x}$ EXACT.
• Why $n-1$ : the deviations are taken around $\overline{x}$ , the point that minimizes $\sum(x_{i}-c)^{2}$ ; they are therefore too small on average, and $s^{2}$ corrects exactly this bias.
• $s$ is in the unit of the data, $s^{2}$ in the unit squared. Coefficient of variation: $CV=\dfrac{s}{\overline{x}}\times 100\%$ , with no unit.

Position: where a value stands

• z-score: $z=\dfrac{x-\overline{x}}{s}$ , the deviation from the mean counted in standard deviations; conversely $x=\overline{x}+z\,s$ .
• Percentile rank (a common convention): $PR(x)=\dfrac{N_{<}+\frac{1}{2}N_{=}}{n}\times 100$ , the share of the group lying BELOW $x$ .
• Quartiles (a common convention): medians of the lower and upper halves of the sorted data. Outlier: outside $[Q_{1}-1.5\,IQR, Q_{3}+1.5\,IQR]$ , with $IQR=Q_{3}-Q_{1}$ .
• Classes: $P_{k}=L+\dfrac{\frac{k}{100}n-F}{n_{c}}\times w$ ; the median is $P_{50}$ .

Quartile and percentile rank conventions vary from one college and one software package to another: write down the one you follow.

The rules in table form

Each row reads from left to right: the assumptions, then the result. A red cell is not an answer, it is the finding that the form settles nothing and the instruction to rewrite it. Every case is followed by a worked example.

Changing units: what happens to the mean and the standard deviation

Each value is transformed by $y=ax+b$ (unit conversion, shift of origin). The last column is the one people get wrong.

Transformation	Mean	Standard deviation
$y=x+b$	$\overline{x}+b$	$s_{x}$
Example: Lake temperatures in kelvins: $21.75\ ^{\circ}\mathrm{C}$ becomes $294.90$ K, the standard deviation stays $0.81$ .
$y=ax$	$a\,\overline{x}$	$\|a\|\,s_{x}$
Example: Nitrate-nitrogen to nitrate ion, $a=4.43$ : $\overline{x}=5.61$ becomes $24.85$ mg/L and $s=3.99$ becomes $17.69$ mg/L.
$y=ax+b$	$a\,\overline{x}+b$	$\|a\|\,s_{x}$
Example: Data $1, 2, 3$ ( $\overline{x}=2$ , $s=1$ ) and $y=-2x+5$ : $3, 1, -1$ , with mean $1$ and standard deviation $2$ , not $-2$ .
$y=x+b$	$\overline{x}+b$	$s_{x}+b$ rule that does not exist
Example: Data $1, 2, 3$ plus $10$ : $11, 12, 13$ still has $s=1$ , not $11$ . What to do: A shift changes no deviation from the mean: the standard deviation does not move, only the mean shifts.

The z-score does not change for $a>0$ : this is why it is used to compare measurements taken in different units.

The mistakes that cost marks

These are the errors I correct most often in session. Each one costs marks on a paper, even when the reasoning behind it is right.

1. Reading the median without sorting the data

the whole question, and the following ones that use it

What not to write

“The flows are $24, 22, 23, \dots, 27$ ; there are $15$ days, so the median is the flow on day 8, $88\ \mathrm{m^{3}/s}$ .”

What to write

“Sorted data: the 8th value is $28$ , so $\mathrm{Med}=28\ \mathrm{m^{3}/s}$ .”

Why: The median is a position in the SORTED data, never in the order of time or of recording.

2. Publishing the mean to describe a typical individual of a skewed distribution

1 to 2 marks of interpretation

What not to write

“On a usual day, the river flows at $35.2\ \mathrm{m^{3}/s}$ .”

What to write

“On half of the days, the flow is at most $28\ \mathrm{m^{3}/s}$ (median); the mean, $35.2$ , is pulled up by three flood days and is used for the total volume.”

Why: A few large values pull the mean far from the bulk of the data; the median resists them. But for a total, the mean is what you need.

3. Dividing by n for a sample

1 mark, and a wrong value that propagates

What not to write

“ $s=\sqrt{\dfrac{4.62}{8}}\approx 0.76\ ^{\circ}\mathrm{C}$ for the $8$ points of the lake.”

What to write

“The $8$ points are a sample of the lake: $s=\sqrt{\dfrac{4.62}{7}}\approx 0.81\ ^{\circ}\mathrm{C}$ .”

Why: Deviations from $\overline{x}$ are too small on average; the divisor $n-1$ compensates. Read the calculator key: $s_{x}$ or $\sigma_{n-1}$ , not $\sigma_{x}$ .

4. Rounding the mean before the computing formula

the whole question: a negative sum of squares is impossible

What not to write

“ $\sum x_{i}^{2}-n\overline{x}^{2}=3789.12-8\times 21.8^{2}=-12.8$ .”

What to write

“With the exact $\overline{x}=21.75$ : $3789.12-3784.5=4.62$ .”

Why: The formula subtracts two large, nearly equal numbers; rounding the mean by 0.05 destroys the result. Keep the mean exact or in memory.

5. Comparing raw values from two populations

2 marks, the conclusion is reversed

What not to write

“The $31.0$ cm trout from lake B is relatively larger than the $27.3$ cm trout from lake A.”

What to write

“ $z_{B}=\dfrac{31.0-33.4}{3.0}=-0.80$ and $z_{A}\approx 0.62$ : relative to its lake, the $27.3$ cm trout is the larger.”

Why: Two groups with different means and standard deviations can only be compared after standardization.

6. Reading a z-score as a ratio

1 mark of interpretation

What not to write

“ $z=2$ : the value is twice the mean.”

What to write

“ $z=2$ : the value exceeds the mean by two standard deviations, $x=\overline{x}+2s$ .”

Why: The z-score measures a distance from the mean in standard deviations; it never compares the value with the mean by a quotient.

7. Using the frequencies as heights with unequal classes

the whole graph, and the interpretation that reads it

What not to write

“I drew each bar at the height of its frequency: $20$ , $15$ , $15$ .”

What to write

“Height $=$ density $\dfrac{n_{i}}{w_{i}}$ : $2$ , $1.5$ and $0.5$ per unit; the area of each bar is its frequency.”

Same data: on the left, the wide class shows an area of

450

against

200

; on the right, the densities give areas of

20

15

and

15

, the true frequencies.

Why: The eye reads areas: with the frequencies as heights, the wide class looks more than twice the first one when it holds fewer data.

8. Removing an outlier, or drawing the whisker all the way to it

1 to 2 marks, and sometimes the only information that matters

What not to write

“The day at $27\ \mu\mathrm{g/m^{3}}$ is an outlier, I remove it from the report.”

What to write

“ $27>Q_{3}+1.5\,IQR=16.75$ : an outlier, drawn as an isolated point and flagged (smoke day), not removed.”

The whisker stops at

13

, the largest value below the fence

16.75

; the smoke day, at

27

, stays visible as an isolated point.

Why: “Outlier” is a criterion of position, not proof of an error. A value is removed only if a measurement or data-entry error is proved.

9. Judging a standard on the mean

the conclusion of the study

What not to write

“The mean of the wells, $5.61$ mg/L, is below the $10$ mg/L guideline: the water complies.”

What to write

“The median is $4.6$ mg/L, but $3$ wells out of $20$ exceed $10$ mg/L: these wells must be reported.”

Why: A standard applies to each individual; the mean describes the centre and says nothing about the tail of the distribution.

Which method to choose

Which measure to publish?

Look at the QUESTION asked, then at the shape of the distribution.

If the question is about a total (volume, total rainfall, water balance) → the mean, or the sum directly

Example: $15$ days at $\overline{x}=35.2\ \mathrm{m^{3}/s}$ : $V=528\times 86\,400\approx 4.56\times 10^{7}\ \mathrm{m^{3}}$
If you describe a typical individual and the distribution is skewed or has an extreme value → the median, with the interquartile range

Example: PM2.5 with a smoke day: $\mathrm{Med}=9$ and $IQR=3.5$ , rather than $\overline{x}\approx 10.67$ and $s\approx 5.53$
If the distribution is roughly symmetric, with no extreme value → the mean, with the standard deviation

Example: Lake temperatures: $\overline{x}=21.75\ ^{\circ}\mathrm{C}$ , $s\approx 0.81\ ^{\circ}\mathrm{C}$
If the question is about a standard or safety → the relevant extreme, or the proportion beyond the threshold

Example: Ice: a single hole at $9$ cm, below a $15$ cm threshold, closes the trail, whatever the mean
If you compare two individuals from two different groups → the z-scores

Example: $z_{A}\approx 0.62$ against $z_{B}=-0.80$
If the variable is categorical nominal → the mode and the frequencies

a mean of categories coded 1, 2, 3 makes no sense

No measure is the best in absolute terms. In a report, both centres are often published: the gap between them tells you about the shape.

Which formula for the form of the data?

Look at how the data are given.

If raw data → $\overline{x}=\dfrac{\sum x_{i}}{n}$ ; median from the sorted data

Example: $6, 7, 8, \dots, 27$ : $\overline{x}=\dfrac{128}{12}\approx 10.67$
If table of values and frequencies (tabulated data) → weight each value by its frequency; median from the cumulative frequencies

Example: values $1$ to $4$ , frequencies $3, 7, 6, 4$ : $\overline{x}=\dfrac{51}{20}=2.55$ , $\mathrm{Med}=2.5$
If classes → class midpoints for the mean and the standard deviation; interpolation for the median and the percentiles

Example: $[0, 4)$ , $[4, 8)$ , $[8, 12)$ with frequencies $5, 12, 3$ : $\overline{x}=5.6$ , $\mathrm{Med}\approx 5.67$
If classes of unequal widths to be graphed → histogram of the densities $\dfrac{n_{i}}{w_{i}}$

Example: $15$ values in $[20, 50)$ : height $0.5$ per unit

How the answer is expected to be written

A marker ticks steps. Here they are in order, with the concluding sentence expected word for word.

Computing and interpreting a standard deviation

When to use it: The problem asks for the dispersion of a data set, or to “characterize” measurements.

1 Say whether the data are a sample or a population, and deduce the divisor, n minus 1 or n.
2 Compute the exact $\overline{x}$ , then $\sum(x_{i}-\overline{x})^{2}$ with the table of deviations or with $\sum x_{i}^{2}-n\overline{x}^{2}$ .
3 Give the variance with its squared unit, then the standard deviation in the unit of the data.
4 Interpret in context, as an order of magnitude, and compare with the mean if you are comparing data sets (coefficient of variation).

Concluding sentence

“The $8$ points form a sample of the lake: $s=\sqrt{\dfrac{4.62}{7}}\approx 0.81\ ^{\circ}\mathrm{C}$ ; the temperatures along the transect deviate from the mean of $21.75\ ^{\circ}\mathrm{C}$ by about $0.8\ ^{\circ}\mathrm{C}$ .”

The trap: Forgetting to say WHY you divide by n minus 1: the grader wants to read “sample”.

Marking: Typically: divisor justified, calculation, unit, interpretation.

Concluding a descriptive study

When to use it: The last question of a study asks you to “conclude”, to “write a summary” or to “answer the initial question”.

1 Restate the population, the sample and the variable, with the unit.
2 Give the centre suited to the shape (median if skewed or with an extreme value) and its dispersion.
3 Give what the question is really after: a total, a proportion beyond a threshold, an extreme.
4 Flag the outliers and their known cause, without removing them.
5 Say what the study does not allow you to conclude (non-random sample, extension to the population).

Concluding sentence

“Over these $12$ days, the typical PM2.5 concentration is $9\ \mu\mathrm{g/m^{3}}$ (median, interquartile range $3.5$ ); a smoke day at $27\ \mu\mathrm{g/m^{3}}$ is a real outlier, flagged separately.”

The trap: Concluding with a single number, the mean, and leaving out the extreme that answers the question.

Check before you hand in

Five minutes of checking recover more marks than one more problem started in a hurry.

The sum of the deviations is zero

Add up the deviations $x_{i}-\overline{x}$ in the table: the total must be $0$ , up to rounding.

Lake temperatures: $2.40-2.40=0$ . A total of $0.8$ signals a wrong mean or a missing deviation.
The centres lie between the minimum and the maximum

The mean and the median necessarily fall within the range of the data, and the median leaves as many values on each side.

A mean of $35.2$ for flows from $22$ to $88$ is plausible; a median of $88$ , the largest value, is not.
The order of magnitude of the standard deviation

For data with no extreme value, the standard deviation usually falls between one sixth and one half of the range.

Lake: range $2.5$ , $s\approx 0.81$ , between $0.42$ and $1.25$ . An $s$ of $4.6$ would betray a variance taken for a standard deviation or the reverse.
The frequencies and the areas give back n

Add up the frequencies in the table, or the areas of the density histogram: you must find the total frequency.

$50+16+12+9+5=92$ days; densities $25$ , $5.33$ , $2.4$ , $0.9$ , $0.25$ multiplied by their widths do give back $92$ .

The typical problem, taken apart

A fine-particle station and its smoke day

A station measures the daily mean concentration of fine particles $\mathrm{PM_{2.5}}$ , in $\mu\mathrm{g/m^{3}}$ , over $12$ summer days. Fictitious data: $6$ ; $8$ ; $7$ ; $9$ ; $12$ ; $10$ ; $8$ ; $11$ ; $27$ ; $9$ ; $13$ ; $8$ . On day 9, smoke from a distant forest fire reached the city.

Summarize these data for the station's bulletin: centre, dispersion, position of the smoke day.

Step 1

Sorted data: $6, 7, 8, 8, 8, 9, 9, 10, 11, 12, 13, 27$ ; $n=12$ , $\sum x_{i}=128$ .

Why

Sorting first serves the median and the quartiles; counting n avoids dividing by the wrong number.

Step 2

$\overline{x}=\dfrac{128}{12}\approx 10.67$ and $\mathrm{Med}=\dfrac{9+9}{2}=9\ \mu\mathrm{g/m^{3}}$ .

Why

Mean above the median: a first clue that one value is pulling upward.

Step 3

$\sum x_{i}^{2}=1702$ , so $s^{2}=\dfrac{1702-12\times 10.\overline{6}^{2}}{11}\approx 30.61$ and $s\approx 5.53$ .

Why

A sample of days: divisor n minus 1, mean kept exact in the computing formula.

Step 4

$z_{27}=\dfrac{27-10.67}{5.53}\approx 2.95$ .

Why

The z-score says the smoke day is nearly three standard deviations away, even though it inflates that standard deviation itself.

Step 5

$Q_{1}=8$ , $Q_{3}=11.5$ , $IQR=3.5$ ; upper fence $11.5+1.5\times 3.5=16.75<27$ : outlier.

Why

The interquartile range rule does not depend on the extreme value, unlike the z-score: it gives a clearer verdict.

Step 6

Without the smoke day: $\overline{x}\approx 9.18$ and $s\approx 2.14$ ; the median stays $9$ .

Why

A single day multiplies the standard deviation by more than two and leaves the median still: the median is what describes the typical day.

The conclusion, written out

“Over these $12$ days, the typical concentration is $9\ \mu\mathrm{g/m^{3}}$ (median, interquartile range $3.5$ ). The smoke day, at $27\ \mu\mathrm{g/m^{3}}$ , is a real outlier: it is flagged separately, not removed.”

The classic mistake on this problem: Publishing “ $10.67\pm 5.53$ ”: the mean describes a usual day poorly, and the standard deviation mostly measures a single day.

Learn by heart

• The median is read on the SORTED data; the mean preserves the total.
• Sample: divisor $n-1$ ; whole population: divisor $n$ .
• $\sum(x_{i}-\overline{x})=0$ and $\sum(x_{i}-\overline{x})^{2}=\sum x_{i}^{2}-n\overline{x}^{2}$ , with the EXACT mean.
• $z=\dfrac{x-\overline{x}}{s}$ : a distance in standard deviations, never a ratio.
• $y=ax+b$ : $\overline{y}=a\overline{x}+b$ and $s_{y}=|a|\,s_{x}$ ; a shift does not change the standard deviation.
• Unequal classes: height $=\dfrac{n_{i}}{w_{i}}$ , the AREA is the frequency.
• Outlier does not mean erroneous: flag it, do not remove it without proof.

Frequently asked questions

When should the median be used rather than the mean in descriptive statistics?

When you want to describe a typical individual and the distribution is skewed or contains an extreme value: flows with a flood, concentrations with a pollution day, incomes. The median depends only on ranks and resists extremes. If the question is about a total, such as a volume or total rainfall, the mean is what you need, because it alone preserves the sum.

Why do we divide by n minus 1 to compute the standard deviation of a sample?

Because the deviations are measured around the sample mean, which is the point closest to the sample's own data. The sum of squared deviations is therefore in general too small compared with the one we would get using the true population mean. Dividing by n minus 1 instead of n corrects exactly this bias for the variance, on average over all possible samples.

How do you compute the median of data grouped into classes?

Find the median class, the one where the cumulative frequency passes half of the total frequency. Then interpolate: the lower boundary of the class, plus the amount still missing to reach half, divided by the frequency of the class, multiplied by its width. This calculation assumes the data are spread evenly within the class, so it is an estimate.

What does a z-score of 2 mean?

The value exceeds the mean of its group by two standard deviations. It is not twice the mean. A negative z-score indicates a value below the mean. For a roughly bell-shaped distribution, a z-score beyond 2 in absolute value is unusual; the z-score is mainly used to compare individuals from different groups or measured in different units.

Should an outlier be removed from a data set?

No, not without proof of an error. An outlier is only a value that lies far away according to a rule, for example more than one and a half times the interquartile range above the third quartile. If it is real, like a smoke day or a contaminated well, it is often the most important information: flag it, draw it as an isolated point, and publish the median, which resists it.

Mark this sheet as read or add it to your favourites: a free account, no password, keeps your read sheets and favourites from one visit to the next, tells you which chapter to tackle next and lets you ask for the method or exercise you are missing. Create your space, an email is enough.

Practise it

Corrected exercises: Descriptive statistics, 201-SN1-RE

A method is proved on a paper, not on a sheet. The set for the same chapter takes each of these traps into a problem, with the solution written out step by step.

11 corrected exercises
110 points
150 minutes

Do the exercises

Something missing from this sheet? A method, a trap, an exercise to practise it, a mistake you spotted: tell me, I read every request.

Sending goes through your free space: that is how I can answer you and tell you when the exercise is added. An email is enough, no password. Create your space. Rather not create an account? Write to me directly.

Next sheet Counting

This chapter comes back in

The chapters that build on it, later in the year or in the years that follow.

Confidence intervals: mean, Student's t and proportion

© Ahmed Squalli Houssaini. Revision sheet published at www.letuteurscientifique.ca/en/fiches/cegep-sn1-descriptive-statistics. Free for personal and classroom use; republishing it elsewhere requires written permission (legal notice).

Revision sheet: descriptive statistics (201-SN1-RE)

The essentials

Three centres, three different questions

Dispersion: two divisors, one good reason

Position: where a value stands

The rules in table form

Changing units: what happens to the mean and the standard deviation

The mistakes that cost marks

1. Reading the median without sorting the data

2. Publishing the mean to describe a typical individual of a skewed distribution

3. Dividing by n for a sample

4. Rounding the mean before the computing formula

5. Comparing raw values from two populations

6. Reading a z-score as a ratio

7. Using the frequencies as heights with unequal classes

8. Removing an outlier, or drawing the whisker all the way to it

9. Judging a standard on the mean

Which method to choose

Which measure to publish?

Which formula for the form of the data?

How the answer is expected to be written

Computing and interpreting a standard deviation

Concluding a descriptive study

Check before you hand in

The typical problem, taken apart

A fine-particle station and its smoke day

Learn by heart

Frequently asked questions

When should the median be used rather than the mean in descriptive statistics?

Why do we divide by n minus 1 to compute the standard deviation of a sample?

How do you compute the median of data grouped into classes?

What does a z-score of 2 mean?

Should an outlier be removed from a data set?

Corrected exercises: Descriptive statistics, 201-SN1-RE

This chapter comes back in

See also

Looking for a 201-SN1-RE tutor in Montreal?