My account Contact me

Quantitative Methods in Social Science 360-300 • CEGEP, Montreal

Revision sheet: concepts, variables and measurement scales (360-300)

This sheet does not go over the course again: you already have it in your notes. It deals with what decides your mark on the first Quantitative Methods exam, that is, the gap between what the software agrees to compute and what the measurement scale really allows. A spreadsheet will happily average a column of postal codes: it is up to you to know that this number means nothing.

It is written for Social Science students at the CEGEP, in Montreal and elsewhere, who take the mandatory course Quantitative Methods, 360-300. Once the sheet is read, the corrected exercise set on the same chapter puts every reflex to the test, from operationalization to reading the technical sheet of a survey.

Mark this sheet as read or add it to your favourites: a free account, no password, keeps your read sheets and favourites from one visit to the next, tells you which chapter to tackle next and lets you ask for the method or exercise you are missing. Create your space, an email is enough.

The thread of the chapter

A measurement scale is not a property of the numbers written in the column, it is a property of what they represent. Almost every lost mark in this chapter comes from computing on codes what you would only be entitled to compute on quantities.

This chapter is part of 360-300, Quantitative Methods in Social Science

The essentials

Three levels, and only one produces numbers

  • • The CONCEPT is what you are talking about: success, poverty, trust in institutions. It is never measured directly.
  • • The DIMENSIONS cut it into distinct aspects: success breaks down into results, persistence and engagement.
  • • The INDICATORS are the questions or records that produce numbers: the R score, the rate of courses passed, the hours of involvement per week.
  • • Every critique of a study starts at this level: an indicator can be excellent while the operationalization has forgotten an entire dimension.
concept: successresultsR scorepersistencecoursepass rateengagementinvolvementhours
The concept is cut into dimensions, each dimension goes down to an indicator; only the indicators, in blue, produce numbers.

In a critique question, naming the level where the problem lies earns the mark: “the problem is not the measurement, it is the missing dimension.”

The four scales, and what each one allows

  • • NOMINAL: categories with no order. Mode, counts, percentages, chi-square.
  • • ORDINAL: an order, but unknown distances. You add the median, quartiles, percentile rank and Spearman's coefficient.
  • • INTERVAL: constant distances, a conventional zero. You add the mean, the standard deviation and the Z score. Ratios stay forbidden.
  • • RATIO: a zero that means the absence of the thing measured. You add ratios, the coefficient of variation and the geometric mean.
  • • Each scale inherits everything allowed by the previous one: it is a staircase, not four separate boxes.
SCALEWHAT IT ALLOWS IN ADDITIONNominalmode, counts, chi-squareOrdinalmedian, quartiles, percentile rankIntervalmean, standard deviation, Z scoreRatioratios, coefficient of variation
Each row adds tools to the previous row and removes none: the last column reads “in addition to everything above.”

The scale is a property of the VARIABLE, not of the software. Excel computes the mean of a column of program codes without complaint.

Four roles, and two of them are handled in opposite ways

  • • INDEPENDENT: the one whose action we assume. DEPENDENT: the one that varies in response.
  • • CONFOUNDING: it acts on the other two without being on the path from one to the other. You must CONTROL it.
  • • MEDIATING: it is on that path, the effect goes through it. You must NEVER control it.
  • • Controlling for a variable means redoing the analysis inside each of its categories.
independentdependentconfoundingmediatingto controlto leave
The confounder (orange) acts on both variables off the path: you control it. The mediator (green) is on the path: you leave it.

The exam question never asks “what is the role of this variable” without expecting what you do about it: control it or leave it alone.

The mistakes that cost marks

These are the errors I correct most often in session. Each one costs marks on a paper, even when the reasoning behind it is right.

1. The mean of a nominal variable coded in digits

2 points, plus everything the question built on it

What not to write

“The five programs are coded 1 to 5 and the mean is 2.82.8: the average program is therefore the third one.”

What to write

“The variable is nominal: the digits are labels. The only measure of central tendency available is the MODE, here program 4, chosen by 31% of respondents.”

24%A1519%B2418%C3331%D428%E51coding 1coding 2“mean” 2.8“mean” 3.2mode: D
Illustrative breakdown with a mean of 2.8: renumbering the five programs from 5 to 1 gives 3.2 without changing a single answer. The mode, D, does not move.

Why: A code is not a quantity. Renumbering the programs in another order would change the mean without changing anything in the data: an indicator that depends on the order of data entry is not an indicator.

2. The ratio “twice as much” on a scale with no absolute zero

1 point, and the marker spots the mistake in a second

What not to write

“Group A scores 44 and group B scores 22: group A is twice as satisfied.”

What to write

“Group A is 22 steps above group B. Talking about a ratio would require an absolute zero, which the scale does not have.”

The same two answers, two codingscoding 1 to 5coding −2 to +212345−2−10+1+2BABA
Points A and B sit at the same places on both rulers: only the numbering changes. The gap of two steps survives (4 − 2 = 2 and 1 − (−1) = 2), the ratio does not (4/2 = 2, but 1/(−1) = −1).

Why: On an interval scale, the zero is a choice. Recoding the same scale from −2-2 to +2+2 moves no answer, and yet the ratio becomes −1-1: a number that changes with the coding does not describe the data.

3. The negative item reversed with a minus sign

0.5 point per item, and the index built afterward is entirely wrong

What not to write

“The item is worded backwards, so I reverse it: the answer 44 becomes −4-4.”

What to write

“On a scale from 11 to 55, the reversed item is x′=(1+5)−x=6−xx'=(1+5)-x=6-x: the answer 44 becomes 22, the answer 11 becomes 55.”

−4−20123456 − 4 = 2correct: 6 − x−4−2012345−4: outside the scalewrong: −x
Reversal 6 − x sends 4 to 2 and stays inside the scale from 1 to 5 (green); the minus sign sends 4 to −4, outside the scale (red).

Why: The reversed value must stay inside the scale. The minus sign takes it out, and an index that mixes values from 11 to 55 with values from −5-5 to −1-1 no longer measures anything.

4. Mistaking reliability for validity

2 points, since the question usually covers both qualities

What not to write

“The two administrations give exactly the same results: the instrument is therefore valid.”

What to write

“The two administrations agree, so the instrument is RELIABLE. Validity is another question: does it really measure the stated concept?”

neither reliable nor validreliable, but not validvalid, but not very reliablereliable and valid
Reliability is read from how TIGHT the group of shots is, validity from whether it is CENTRED on the target: at the upper right, a very tight group completely off target.

Why: A scale that reads two kilos too high always gives the same figure: perfectly reliable, completely invalid. Reliability is necessary for validity, never sufficient.

5. Comparing two normalized indices built on different bounds

the whole conclusion of the question

What not to write

“Region A's index is 0.720.72 and region B's is 0.650.65: region A does better.”

What to write

“The two indices were normalized on different bounds. I recompute both on the same minimum and maximum before comparing them.”

40900.72A on [40, 90]301100.65B on [30, 110]30110A: 0.575same boundsB: 0.65
Worked example: A is 76 on [40, 90], B is 82 on [30, 110]. On common bounds, A drops to 0.575 and falls behind B.

Why: Min-max normalization places each value on a ruler defined by the two bounds chosen. Two different rulers give two numbers that cannot be compared any more than centimetres and inches.

6. Concluding about individuals from regional data

2 points, and it is the mistake markers look for first in the critique question

What not to write

“Neighbourhoods where average income is high vote more for this party: so wealthy people vote for this party.”

What to write

“At the level of NEIGHBOURHOODS, average income and vote share are related. Nothing in these data says what individuals do: individual data would be needed.”

The three clouds rise, the dashed line joining their means falls: the slope is not the same depending on the level at which you measure it.

Why: This is the ecological fallacy. The relationship between group means can have the opposite sign to the relationship between the individuals in each group, with no contradiction at all.

7. Controlling for the variable that carries the effect

the whole interpretation, and the conclusion is turned upside down

What not to write

“I control for the number of study hours, and the effect of motivation on the grade disappears: motivation is therefore useless.”

What to write

“Study hours are the MEDIATING variable through which motivation acts. That the effect disappears when I control for it shows the MECHANISM, it does not cancel it.”

motivationstudy hoursgradecontrolled: the path is cut
Controlling for study hours cuts both arrows through which the effect of motivation passes: the effect disappears because you removed it.

Why: You control for a confounder to rule out a competing explanation. Controlling for a mediator amounts to asking for the effect of motivation among students who study the same number of hours: by construction, nothing is left.

8. Announcing a margin of error on a panel of volunteers

1 method point, and the credibility of the whole report

What not to write

“We collected 1,2001{,}200 online answers: the margin of error is 2.82.8 points, 19 times out of 20.”

What to write

“The sample is not a probability sample: no margin of error can be computed. The results describe the respondents, not the population.”

random drawvolunteers
Random draw (green) is spread across the whole population; volunteers (red) come from a single corner. The margin formula assumes the first case.

Why: The formula runs on any number, including on a forum poll. What it assumes is a known, nonzero inclusion probability for each unit, which a panel of volunteers does not have.

Which method to choose

Which scale, therefore which statistic?

Look at the data column and ask three questions in this order: can the values be ranked, are the gaps constant, does zero mean “nothing.”

nominalordinalintervalratioSSArtsNS1234−10 °C0 °C10 °C20 °C0 h10 h20 h30 hzero = nothingchosen zerounknown gapsno order
The four steps: names with no order, an order with unknown gaps, equal gaps around a chosen zero, equal gaps from a true zero.

  • If the values are names that no natural order ranks → NOMINAL scale: mode, counts, percentages, chi-square

    Example: program of study, region, postal code

    the fact that the values are entered as digits changes nothing

  • If you can rank the values, but the gap between two neighbours has no known value → ORDINAL scale: you add median, quartiles, percentile rank and Spearman

    Example: Likert scale, highest diploma obtained

    treating a Likert scale as interval is common, but it must be DECLARED

  • If the gaps are constant and interpretable, but the zero is conventional → INTERVAL scale: you add mean, standard deviation and Z score; ratios stay forbidden

    Example: temperature in degrees Celsius, Z score, calendar year

  • If the zero means the complete absence of the thing measured → RATIO scale: everything is allowed, ratios and coefficient of variation included

    Example: income, age, number of study hours per week

Decisive test between interval and ratio: does the phrase “twice as much” make sense? 2020 °C is not twice as hot as 1010 °C, whereas 2020 dollars is indeed twice 1010 dollars.

What role should a third variable get?

Draw the assumed arrow, from the independent variable to the dependent one, then ask where the third variable plugs in.

XYCconfoundingcontrol itXYMmediatingleave itXYWmoderatingsplit the groups
Where the third variable plugs in: onto X and Y (confounding), onto the path from X to Y (mediating), onto the arrow itself (moderating).

  • If it acts on the independent variable AND on the dependent one, without being on the path between them → CONFOUNDING: control for it, otherwise the observed relationship may be entirely spurious

    Example: age, in the link between years of experience and salary

  • If the independent variable acts on it, and it then acts on the dependent variable → MEDIATING: do not control for it, you would erase the very effect you are looking for

    Example: study hours, between motivation and final grade

  • If the strength of the relationship changes depending on the category you are in → MODERATING: redo the analysis separately within each category and compare the results

    Example: sex, in the link between sports participation and stress

If no branch applies, the variable is simply off topic: do not bring it into the analysis just because it appears in the file.

How the answer is expected to be written

A marker ticks steps. Here they are in order, with the concluding sentence expected word for word.

Justifying the measurement scale of a variable

When to use it: Whenever the question asks “what type of variable is this” or “which measure of central tendency is appropriate.”

  1. 1 Name the variable and quote two of its values as they appear in the table.
  2. 2 Say whether the values can be ranked, and by what: an order exists or it does not, it is not a matter of opinion.
  3. 3 Say whether the gap between two neighbouring values is constant and interpretable.
  4. 4 Say whether the zero means the absence of the thing measured or is conventional.
  5. 5 Conclude on the scale, THEN list the statistics it allows.

Concluding sentence

“The variable ‘satisfaction level’ takes the values 11 to 55, which rank from weakest to strongest but whose gaps are not guaranteed to be constant: it is an ORDINAL scale. You can therefore report the mode, the median and the quartiles, but neither the mean nor the standard deviation without declaring the assumption of equal gaps.”

The trap: Concluding on the scale without saying what it allows: half the marks of the question are in the second sentence.

Marking: 1 point for the scale, 1 point for the justification by order and zero, 1 point for the statistics allowed.

Building a composite index in four moves

When to use it: When the question asks you to sum up several indicators in a single comparable number.

  1. 1 Declare the indicators chosen and the scale of each.
  2. 2 Set and WRITE DOWN the normalization bounds, saying where they come from: observed minimum and maximum, theoretical bounds or official thresholds.
  3. 3 Normalize each indicator by z=x−min⁡max⁡−min⁡z=\dfrac{x-\min}{\max-\min}, then check that each zz falls between 00 and 11.
  4. 4 Reverse the indicators whose high value is bad, by 1−z1-z, before any aggregation.
  5. 5 Aggregate, saying whether the components have the same weight and why.

Concluding sentence

“The index is 0.580.58 on a scale from 00 to 11, built on the bounds declared above; it can only be compared with an index built on exactly the same bounds.”

The trap: Forgetting to reverse a negative indicator: the failure rate then enters the index as if it were good news.

Marking: 1 point for the declared bounds, 2 points for the normalizations, 1 point for the justified aggregation.

Check before you hand in

Five minutes of checking recover more marks than one more problem started in a hurry.

The typical problem, taken apart

Operationalizing success at CEGEP, then building the index

You want to sum up the success of a Social Science student in a single index between 00 and 11.

Three indicators are kept: the R score on the record, 2828; the proportion of courses passed in the semester, 0.900.90; the number of hours of extracurricular involvement per week, 44.

The bounds chosen are: R score from 2020 to 3535, proportion from 0.500.50 to 11, involvement from 00 to 1010 hours.

203528R score0.510.90courses passed0104involvement (h)
The student's three indicators, each placed between the bounds declared in the statement.

Step 1

Concept: success at CEGEP. Dimensions: results, persistence, engagement. Indicators: R score, proportion of courses passed, hours of involvement.

Why

Write the three levels before any calculation: it is the only place where you can still see that a dimension is missing, and it is a marking point in itself.

Step 2

Scales: the R score is an INTERVAL scale, the proportion and the hours are RATIO scales. All three allow the mean.

Why

Min-max normalization only makes sense from the interval level up: on an ordinal variable, it would invent distances the data do not contain.

Step 3

z1=28−2035−20=815≈0.533z_{1}=\dfrac{28-20}{35-20}=\dfrac{8}{15}\approx 0.533; z2=0.90−0.501−0.50=0.800z_{2}=\dfrac{0.90-0.50}{1-0.50}=0.800; z3=4−010−0=0.400z_{3}=\dfrac{4-0}{10-0}=0.400.

2035R scorez = 0.5330.51courses passedz = 0.8010involvement (h)z = 0.401common scale

Why

The three indicators arrive in incomparable units. Without normalization, the R score weighs thirty times more than the proportion in any sum.

Step 4

Arithmetic mean: 0.533+0.800+0.4003≈0.578\dfrac{0.533+0.800+0.400}{3}\approx 0.578.

Why

Equal weights, so you have to say so: “the three dimensions are assumed to be equally important, for lack of an argument to rank them.” A choice that is not declared is a hidden choice.

Step 5

Geometric mean: 0.533×0.800×0.4003≈0.555\sqrt[3]{0.533\times 0.800\times 0.400}\approx 0.555, that is 0.0230.023 less.

Why

The geometric mean penalizes imbalance among components. The gap of two hundredths measures exactly how uneven this student's profile is.

Step 6

Check: the three zz values are indeed in [0,1][0, 1], and 0.400≤0.578≤0.8000.400 \le 0.578 \le 0.800.

Why

A mean that left the range of its components would reveal a data entry or normalization error, without your having to redo a single calculation.

The conclusion, written out

“This student's success index is 0.580.58 on a scale from 00 to 11, built on the declared bounds: R score from 2020 to 3535, proportion of courses passed from 0.500.50 to 11, involvement from 00 to 1010 hours. It can only be compared with an index built on exactly the same bounds.”

The classic mistake on this problem: Adding the three raw indicators, 28+0.90+428+0.90+4: the R score crushes the other two and the index measures nothing but it.

Learn by heart

  • • NOMINAL: mode. ORDINAL: plus the median and quantiles. INTERVAL: plus the mean and standard deviation. RATIO: plus ratios.
  • • A ratio (“twice as much”) requires an ABSOLUTE ZERO. Without it, you only compare DIFFERENCES.
  • • Item reversed on a scale from min⁡\min to max⁡\max: x′=(min⁡+max⁡)−xx'=(\min+\max)-x. From 11 to 55, that gives x′=6−xx'=6-x.
  • • RELIABILITY is necessary for validity and never sufficient: an instrument can be wrong in the same way every time.
  • • You control for a CONFOUNDING variable. You NEVER control for a mediating variable.
  • • A relationship belongs to the LEVEL at which it was measured: regions say nothing about individuals.
  • • A margin of error assumes a PROBABILITY sample. On a panel of volunteers, the formula gives a number that means nothing.

Frequently asked questions

How do I know whether a variable is ordinal or interval?

Ask two questions in order. Can the values be ranked from smallest to largest? If not, the variable is nominal. If so, is the gap between two neighbouring values the same everywhere and interpretable? If so, it is an interval scale and the mean becomes legitimate. Otherwise the variable stays ordinal, and you stick to the median and quartiles.

Can you compute a mean on a Likert scale?

Strictly speaking, no: a Likert scale is ordinal, and nothing guarantees that going from 2 to 3 is the same gap as going from 4 to 5. In practice, a lot of research does it when the scale has at least five levels and the distribution is not too skewed. The exam rule: do it only by declaring the assumption, and report the median as well.

What is the difference between validity and reliability?

Reliability is consistency: the same instrument, applied twice under the same conditions, gives the same result. Validity is accuracy: the instrument really measures the stated concept. A scale that reads two kilos too high is perfectly reliable and yet invalid. Reliability is necessary for validity, never sufficient.

What is the ecological fallacy in the social sciences?

It is concluding about individuals from data measured on groups. If neighbourhoods with high average income vote more for a party, that does not say that wealthy people vote for that party: the relationship can even be reversed inside each neighbourhood. A relationship belongs to the level at which it was measured.

How do you reverse a negative item in a questionnaire?

On a scale running from the minimum to the maximum, the reversed value is the sum of the two bounds minus the answer. On a scale from 1 to 5, you compute 6 minus the answer: a 4 becomes 2, a 1 becomes 5. You never use the minus sign, which would leave the scale and make any index built afterward negative.

Practise it

Corrected exercises: Concepts, variables and measurement scales, 360-300

A method is proved on a paper, not on a sheet. The set for the same chapter takes each of these traps into a problem, with the solution written out step by step.

  • 10 corrected exercises
  • 100 points
  • 150 minutes
Do the exercises
Something missing from this sheet? A method, a trap, an exercise to practise it, a mistake you spotted: tell me, I read every request.

Sending goes through your free space: that is how I can answer you and tell you when the exercise is added. An email is enough, no password. Create your space. Rather not create an account? Write to me directly.

What is it about?
I read every request myself, nobody else sees it. No need to write your name or your school.
When your request is dealt with, you hear about it by email and in your space.

By sending, you agree that I read your request to answer it. It is kept with your account, erased with it, and never published: privacy policy.

Next sheet Sampling and questionnaires

© Ahmed Squalli Houssaini. Revision sheet published at www.letuteurscientifique.ca/en/fiches/cegep-qm-concepts-variables-scales. Free for personal and classroom use; republishing it elsewhere requires written permission (legal notice).

See also

All revision sheets All exercise sets Fiche: concepts, variables et échelles de mesure (in French)

Looking for a 360-300 tutor in Montreal?

Get in touch for a first session. We go back over the moves that cost marks on the Quantitative Methods exam, from naming a measurement scale to criticizing a poll, then test them on exercises at the level of real CEGEP tests.

Book a session WhatsApp
Site by Studio Squalli