My account Contact me

COMP 202 Foundations of Programming • McGill University, Montreal

Revision sheet: Strings and text processing (COMP 202)

This sheet is not a summary of the string chapter of COMP 202: you already have the slides. It answers one question, what loses marks on strings, on the paper midterm and final and in the assignments marked by the autograder, and which precise gesture avoids each loss.

The angle of the chapter: a string cannot be changed, so every method call is a value that must be kept, and every search answers with a position that must be tested. Almost every trap below is one of those two facts in disguise, and the autograder catches the ones the eye does not.

Mark this sheet as read or add it to your favourites: a free account, no password, keeps your read sheets and favourites from one visit to the next and tells you which chapter to tackle next. Create your space, an email is enough.

The thread of the chapter

A string is a read-only sequence. Everything you do TO it builds a new string that must be given a name, and everything you look for IN it comes back as a position. The marks of this chapter go to whoever keeps the result of every call and tests the position before using it.

This chapter is part of COMP 202, Foundations of Programming (McGill)

The essentials

Nothing changes a string: every call is a value to keep

  • • s.upper(), s.strip(), s.replace('a', 'o'), s.lower(): each one BUILDS a new string and returns it. The string named s is exactly what it was before the call.
  • • So a method call on its own line does nothing visible. The line to write is s = s.strip(), or t = s.upper() when the original is still needed.
  • • s[0] = 'J' raises TypeError, str object does not support item assignment. To change one character, build a new string: s = 'J' + s[1:].
  • • The same holds inside a loop: for c in s: c = c.upper() changes the copy named c and leaves s untouched. A transformed string is ACCUMULATED, out = out + c.upper(), or collected in a list and joined.
  • • Consequence for a function: the transformed string has to be RETURNED. A function that prints it returns None, and None is what the autograder compares.
s.upper() on its own linesHelloHELLOno namebuilt, then thrown awayt = s.upper()sHellotHELLOs unchanged, t holds the result
Left: s.upper() builds 'HELLO' and nobody names it, so s still reads 'Hello'. Right: the same call with t = in front, and the result survives under the name t.

On a paper trace, the marker looks for the assignment. A line that reads name.strip() with nothing on the left is marked as if it were absent, because for the program it is.

Two rulers on one string, and three questions with three kinds of answer

  • • Indices sit ON the characters, 0 to len(s) - 1, and -1 is the last. Slice cuts fall BETWEEN the characters, so s[i:j] takes the characters from i included to j excluded and has j−ij - i of them.
  • • A slice of length n starting at i is s[i:i+n], never s[i:n]. The second form has a fixed right end and stops sliding.
  • • Is it there? sub in s gives a bool. Where is it? s.find(sub) gives an int, -1 when absent, while s.index(sub) raises ValueError. How many? s.count(sub) gives an int and counts non-overlapping occurrences.
  • • A string compares to a string character by character, by character code: '10' < '9' is True because '1' comes before '9', and 'Zebra' < 'apple' is True because every capital comes before every lower case letter.
  • • Character arithmetic: ord('a') is 97, ord('A') is 65, chr goes back. The letter k places further along the alphabet is chr((ord(c) - ord('a') + k) % 26 + ord('a')), where the % 26 does the wrap around.

The exam gives one string and asks five slices of it. Write the index ruler above the string once, then read every slice off the ruler: the marks go to the reading, not to the memory.

The rules in table form

Each row reads from left to right: the assumptions, then the result. A red cell is not an answer, it is the finding that the form settles nothing and the instruction to rewrite it. Every case is followed by a worked example.

One word, seven questions: what comes back on s = 'banana'

The same string, and the answers that seven lines give. The red lines are lines students write in the belief that they mean something else: they run without a crash, or crash, but never do what was meant.

you writegives backon 'banana'
'an' in s a bool True

Example: 'an' in 'banana' is True; 'nab' in 'banana' is False, since 'nab' occurs nowhere as 3 consecutive characters.

s.find('an') an int 1

Example: 'banana'.find('an') is 1: 'an' occurs at positions 1 and 3, and find reports the first.

s.find('x') an int -1

Example: 'banana'.find('x') is -1, not an error; s[-1] is then 'a', the LAST character, if nobody tests the -1 first.

s.count('an') an int 2

Example: 'banana'.count('an') is 2, and 'banana'.count('ana') is 1, since the second 'ana' overlaps the first at index 3.

s.count('aeiou') an int 0 not the vowel count

Example: 'banana'.count('aeiou') is 0: the 5-character substring 'aeiou' occurs nowhere. The 3 vowels are found with a loop: total = 0, then for c in s: if c in 'aeiou': total = total + 1.

What to do: count takes a SUBSTRING. To count characters from a set, loop over s and test c in 'aeiou'.

s[0] = 'B' TypeError no result does not exist

Example: 'banana'[0] = 'B' raises TypeError; 'B' + 'banana'[1:] gives 'Banana', 6 characters, the first one replaced.

What to do: Build the new string from slices: s = 'B' + s[1:], or s.capitalize() when it is the first letter.

s.strip('ba') a new string 'nan' not what strip removes

Example: 'banana'.strip('ba') is 'nan': strip removes every 'b' and every 'a' from BOTH ends until it meets an 'n', so 3 characters go on the left and 0 on the right, then the trailing 'a' goes too, leaving 3.

What to do: The argument of strip is a SET of characters to remove from the ends. To remove a prefix, test s.startswith(p) and take s[len(p):]; to remove every occurrence, use replace.

Every line of this table leaves 'banana' exactly as it was. Not one of the seven changes the string it is called on.

The mistakes that cost marks

These are the errors I correct most often in session. Each one costs marks on a paper, even when the reasoning behind it is right.

1. Calling a method and throwing its result away

every test of the assignment, since the function returns the raw input; on paper, the whole trace

What not to write

“name.strip() then name.lower(), and now name is clean.”

What to write

“name = name.strip().lower(): each call returns a new string, and the assignment is what keeps it.”

Why: A string cannot be changed, so a method can only hand back a new one. With nothing on the left of the call, the new string has no name and is lost at once. The autograder sees the difference on its first test: the expected 'ana' against the returned ' Ana '.

2. Assigning a character by its index

the whole question: TypeError on the first call, so zero tests pass

What not to write

“To capitalise the word, word[0] = word[0].upper().”

What to write

“word = word[0].upper() + word[1:], since a string does not support item assignment.”

Why: Index assignment exists for lists, not for strings, and the two look alike on paper. The gesture is always the same: cut the string into the pieces that stay, insert the new piece, and name the result.

3. Writing a window as s[i:n] instead of s[i:i+n]

2 to 3 marks on a trace, all the tests past the first on an assignment

What not to write

“Every group of 3 letters is s[i:3] for i in range(len(s)).”

What to write

“The window of length 3 at position i is s[i:i+3], for i in range(len(s) - 3 + 1): on 'BANANA' that is i from 0 to 3 and 4 windows.”

B0A1N2A3N4A5s[1:1+3]gives ANAB0A1N2A3N4A5s[3:3+3]gives ANAB0A1N2A3N4A5s[3:3]gives empty
Both ends move with i in s[1:1+3] and s[3:3+3], and the window of 3 slides. In s[3:3] the right end stayed at 3, the cuts coincide and the slice is empty.

Why: The right end of a slice is an INDEX, not a length. s[i:3] has a fixed end, so it shrinks as i grows and is empty from i = 3 on. The figure shows the window sliding when both ends move together, and dying when only the left one does.

4. Comparing strings as if they were numbers

the trace question, 2 marks, and a sort of marks read from a file that comes out in the wrong order

What not to write

“'10' < '9' is False, since ten is bigger than nine.”

What to write

“'10' < '9' is True: strings compare character by character, and '1' comes before '9'. To compare the numbers, convert first: int('10') < int('9') is False.”

Why: The comparison stops at the first character that differs and decides on it, by character code. It also explains 'Zebra' < 'apple', True, because every capital letter has a smaller code than every lower case letter: lower() both sides before comparing words, int() both sides before comparing numbers.

5. Treating 'aeiou' as a set of letters

the whole question, and the answer 0 looks like a program that ran

What not to write

“vowels = s.count('aeiou'), or in the loop, if c == 'aeiou': total = total + 1.”

What to write

“for c in s: if c in 'aeiou': total = total + 1. The in operator asks whether the ONE character c is among the five.”

Why: count looks for the five characters in a row, and c == 'aeiou' compares a one-character string with a five-character one, which is never equal. Neither crashes, so the wrong count of 0 travels into the average or the report. The same confusion in the other direction, if 'a' or 'e' in s, is always True because the string 'a' is truthy on its own.

6. Giving strip a substring to remove

1 to 2 marks on a trace, a silent wrong answer on data whose ends contain the same letters

What not to write

“'www.mcgill.ca'.strip('www.') removes the www. and gives 'mcgill.ca'.”

What to write

“strip('www.') removes every w and every dot from both ends: 'mcgill.ca' here by luck, but 'www.wow.ca'.strip('www.') is 'ow.ca', the first letter of the name gone. To drop a prefix, test startswith and slice: s[4:].”

Why: The argument of strip is a set of characters, not a word, and the removal continues character by character until it meets one outside the set. It works by accident on most inputs, which is what makes it survive testing: on 'www.wow.ca' it eats 5 characters instead of 4.

7. Printing the string the function was asked to return

every test of the function: assert reverse('abc') == 'cba' compares 'cba' with None

What not to write

“def reverse(s): print(s[::-1]), and it displays 'cba', so it works.”

What to write

“def reverse(s): return s[::-1]. The test reads the value that comes back, and print sends it to the screen instead, leaving None.”

Why: A displayed string and a returned string look identical in the shell and are two different things to the program. The autograder never reads the screen. The same slip inside a loop, print(out) at every pass, is marked on paper as an output of n lines where one was asked.

8. Reassigning the loop variable to transform the string

the whole question: the function returns its input untouched

What not to write

“for c in s: c = c.upper(), and after the loop s is in capitals.”

What to write

“out = '' before the loop, then for c in s: out = out + c.upper(), and s is unchanged while out holds 'ABC'.”

sa0b1c2cba copy of s[1]Bafter c = c.upper()s still 'abc'
c points to a copy of s[1]; c = c.upper() moves c to a new 'B' and the three cells of s never move. What was wanted is a fourth object, out, built beside s.

Why: The loop variable holds a COPY of each character in turn. Assigning to c rebinds that name and touches neither s nor the next character. A transformed string is built beside the original, with an accumulator or a list to join.

Which method to choose

Which tool, by the question the statement asks

Read the verb of the statement, not the name of the chapter

  • If is a piece of text PRESENT in the string → sub in s, a bool; never s.find(sub) as a truth test, since position 0 is false

    Example: 'an' in 'banana' is True

  • If WHERE a piece of text is → s.find(sub), then if pos == -1 before any use of pos as an index

    Example: 'banana'.find('an') is 1, 'banana'.find('x') is -1

    s.index(sub) raises instead of returning -1: choose it only when absence is a bug

  • If HOW MANY times a piece of text occurs → s.count(sub) for a substring; a loop with c in 'aeiou' for a set of characters

    Example: 'banana'.count('an') is 2, vowels of 'banana' are 3

  • If TAKE a part of the string: first n, last n, a window at i → a slice, s[:n], s[-n:], s[i:i+n]; an index only for one character

    Example: 'BANANA'[1:1+3] is 'ANA', 'BANANA'[-2:] is 'NA'

  • If CHANGE a part of the string → build a new one: 'J' + s[1:], s.replace(old, new), or the loop with an accumulator; then name the result

    Example: 'B' + 'banana'[1:] is 'Banana'

  • If BREAK a line into fields, or GLUE fields into a line → s.split(sep) keeps empty fields, s.split() collapses whitespace; sep.join(list_of_strings) for the way back

    Example: 'a,,b'.split(',') has 3 pieces, ' a b '.split() has 2

  • If COMPARE two strings, or CLEAN one before comparing → strip() and lower() both sides, then ==; int() both sides when the strings hold numbers

    Example: ' Ana '.strip().lower() == 'ana'; int('10') < int('9') is False

  • If SHIFT or NUMBER a letter → ord(c) - ord('a') for the position 0 to 25, arithmetic, % 26, then chr back

    Example: shifting 'x' by 3 gives 'a', since (23 + 3) % 26 is 0

Every branch ends with a value. If the value has no name and is not returned, the branch was taken for nothing: that is the fil of the chapter, and it is where the marks go.

How the answer is expected to be written

A marker ticks steps. Here they are in order, with the concluding sentence expected word for word.

Tracing a string expression on paper

When to use it: The midterm gives s = 'Montreal' and asks for the value of s[2:5], s[-3:], s[::2], s.find('t') or len(s.split('e')).

  1. 1 Write the string once with its index ruler above it, 0 to len(s) - 1, and the negative ruler below when a negative index appears.
  2. 2 For a slice, mark the two cuts BETWEEN the characters and copy what lies between them; count the characters and check that the count is stop minus start.
  3. 3 For a method, name what it gives back before computing it: a string, a list, an int or a bool. The type is worth a mark on its own and prevents the answer 1 for a find that should read -1.
  4. 4 Evaluate a chain left to right, one intermediate value per line: s.strip() first, then .lower(), then .replace.
  5. 5 Write the answer as a Python value: with its quotes for a string, with its brackets for a list, and add its length in brackets when the question is about a slice.

Concluding sentence

“s = 'Montreal', indices 0 to 7. s[2:5] takes indices 2, 3, 4, so s[2:5] = 'ntr' (3 characters).”

The trap: Reading the right end of a slice as included: s[2:5] has 3 characters, not 4, and the marker checks the count.

Marking: Typically 1 mark per value, half of it for the correct type and the quotes: 'ntr' and not ntr, ['Montr', 'al'] and not Montr al.

Writing a function that returns a transformed string

When to use it: An assignment asks for def clean(s), def initials(name), def encode(text, k): a string in, a string out, marked by an autograder.

  1. 1 Start the accumulator before the loop: out = '' for a string, or pieces = [] when the result is joined at the end.
  2. 2 Loop over the characters, for c in s, or over the words, for w in s.split(), and decide with if, elif, else what each one contributes; the else branch says what happens to the characters that are not letters.
  3. 3 Add the contribution to the accumulator, out = out + piece, never to the loop variable.
  4. 4 Return the accumulator ONCE, after the loop and outside it, with return and not print.
  5. 5 Run the function in your head on '' and on a one-character string before submitting: the empty string must give '' without an IndexError.

Concluding sentence

“def initials(name): out = '' / for w in name.split(): out = out + w[0].upper() + '.' / return out”

The trap: A return indented inside the loop, which ends the function after the first character and passes exactly one test, the one on a single letter.

Marking: Usually a test on the ordinary case, one on the empty string, one on punctuation or spacing, one on upper and lower case: four tests, four chances to lose a quarter each.

Check before you hand in

Five minutes of checking recover more marks than one more problem started in a hurry.

The typical problem, taken apart

Initials of a full name, marked by an autograder

Write a function initials(name) that returns the initials of a full name, in upper case, each followed by a dot: initials('jean marie tremblay') returns 'J.M.T.'. The name may carry extra spaces at either end or between the words.

Give the function, then trace it on ' ana ' and on the empty string.

ww[0]w[0].upper() + '.'out so far'jean''j''J.''J.''marie''m''M.''J.M.''tremblay''t''T.''J.M.T.'
The loop runs 3 times, one word per pass; each pass adds its own dot, so the accumulator ends on a dot without any special case for the last word.
Python
def initials(name):
    out = ''
    for w in name.split():
        out = out + w[0].upper() + '.'
    return out

Step 1

Cut the name into words with name.split(), no argument: 'jean marie tremblay' gives the list of 'jean', 'marie', 'tremblay', 3 words, and ' ana ' gives the list holding 'ana' alone.

Why

split() with no separator collapses every run of whitespace and drops the outer spaces, so no word is ever empty and w[0] can never raise. split(' ') would have produced empty words on the double spaces, and w[0] on one of them is an IndexError.

Step 2

Start the accumulator, out = '', before the loop, and for each word add w[0].upper() + '.': after the three passes out reads 'J.', then 'J.M.', then 'J.M.T.'.

Why

The dot travels with each initial, so the last one arrives on its own and nothing has to be removed at the end. Building with a separator between the pieces would have needed a slice out[:-1] or a join, and one more place to be off by one.

Step 3

Return out after the loop, at the indentation of the for, not inside it.

Why

The autograder calls initials and compares the value that comes back with 'J.M.T.'. A print shows the right text and returns None, which fails every test; a return inside the loop stops after 'J.', which passes only the test on a single word.

Step 4

Trace on ' ana ': split gives one word, the loop runs once, out is 'A.' and that is what is returned.

Why

The extra spaces are absorbed by split(), so nothing else in the function has to know about them. This is the test on spacing, worth a quarter of the marks.

Step 5

Trace on '': split gives an empty list, the loop body never runs, and out is still '', which is returned. Check: no index, no slice, no error, and the result has length 0.

Why

The empty string is the test students fail most, usually with an IndexError on name[0] taken before the loop. Here the accumulator starts empty and the loop simply does not run, which is the whole point of initialising out before the for.

The conclusion, written out

“initials builds out from '' by adding w[0].upper() + '.' for each word of name.split() and returns out: 'J.M.T.' on 'jean marie tremblay', 'A.' on ' ana ', '' on ''.”

The classic mistake on this problem: Taking the first letter with name[0] before splitting: on ' ana ' that is a space, and the returned string starts with ' .' instead of 'A.'.

Learn by heart

  • • s[i] is ONE character, i from 0 to len(s) - 1, and s[-1] is the last. s[i:j] runs from i included to j excluded and has j−ij - i characters; a window of length n at i is s[i:i+n].
  • • No method changes a string. Write s = s.strip(), t = s.upper(); s[0] = 'x' is a TypeError, and the replacement is 'x' + s[1:].
  • • sub in s gives a bool. s.find(sub) gives an int, -1 when absent, and -1 is a valid index. s.index(sub) raises ValueError. s.count(sub) counts NON-OVERLAPPING occurrences of a substring, never characters from a set.
  • • s.split(sep) keeps empty fields and gives k + 1 pieces for k separators; s.split() collapses whitespace. sep.join(parts) needs a list of strings.
  • • Strings compare character by character by code: '10' < '9' and 'Z' < 'a' are both True. lower() both sides for words, int() both sides for numbers.
  • • strip(chars) removes a SET of characters from both ends, not a substring; startswith and a slice remove a prefix.
  • • ord('a') is 97, ord('A') is 65; chr((ord(c) - ord('a') + k) % 26 + ord('a')) shifts a lower case letter by k with wrap around.
  • • A function that transforms a string RETURNS it once, after the loop; the loop variable is a copy, so the result is accumulated in out.

Frequently asked questions

Why does s.upper() not change my string in Python?

Because a string cannot be changed at all. Every method builds a new string and returns it, so s.upper() on its own line computes a value and drops it. Keep the result with an assignment: s = s.upper(), or t = s.upper() if you still need the original. The same holds for strip, lower, replace and every other string method.

How do I change one character of a string in Python?

You cannot assign to an index: s[0] = 'J' raises a TypeError. Build a new string from the pieces that stay and the piece that changes, s = 'J' + s[1:], and give it a name. For a first letter, s.capitalize() does it; for every occurrence of a character, s.replace(old, new) does it; for a rule applied character by character, loop over s and accumulate the result in a new string.

What is the difference between find and index for strings in Python?

Both return the position of the first occurrence of a substring. They differ only when the substring is absent: find returns minus one and index raises a ValueError. Never use the result of find as an index without testing for minus one first, because minus one is a valid position, the last character. When you only need to know whether the substring is there, use the in operator, which returns True or False.

Why is '10' < '9' True in Python?

Because strings compare character by character, by character code, and stop at the first difference. The first characters are '1' and '9', and '1' comes first, so the comparison is decided there. To compare the numbers, convert both sides with int first. The same rule makes every capital letter come before every lower case letter, so lower both words before comparing them.

How do I count the vowels in a string in Python?

Loop over the characters and test each one against the set of vowels: total = 0, then for c in s, if c in 'aeiou', add one to total. Lower the string first if capitals count. Do not use s.count('aeiou'): count looks for those five letters in a row and returns zero. And do not write c == 'aeiou', which compares one character with a five character string and is never true.

Practise it

Corrected exercises: Strings and text processing, COMP 202

A method is proved on a paper, not on a sheet. The set for the same chapter takes each of these traps into a problem, with the solution written out step by step.

  • 10 corrected exercises
  • 100 points
  • 150 minutes
Do the exercises
Previous sheet Loops and the patterns they carry Next sheet Functions, scope and program design

© Ahmed Squalli Houssaini. Revision sheet published at www.letuteurscientifique.ca/en/fiches/comp202-strings-text-processing. Free for personal and classroom use; republishing it elsewhere requires written permission (legal notice).

See also

All revision sheets All exercise sets Computer science tutoring

Stuck in COMP 202?

I tutor COMP 202 at McGill in English or in French, in person in Montreal or online. Get in touch for a first session.

Book a session WhatsApp
Site by Studio Squalli