Corrected exercises: strings and text processing, COMP 202 at McGill

a) word[0] is 'P', the first character. word[-1] is 'N', the last, and it is the form to prefer over word[len(word) - 1]. word[2:5] is 'THO': it starts at index 2 and stops BEFORE index 5, so three characters. word[-3:] is 'HON', the last three, since a missing end means to the end. word[:0] is the empty string, because it stops before the first character, and an empty result is a legitimate value rather than an error.

b) word[::-1] is 'NOHTYP', the string reversed, and word[::2] is 'PTO', every second character starting at the first. The two empty positions are the start and the end of the slice, and leaving them empty means from the beginning and to the end. With a negative step the defaults flip and the walk runs from the end backwards, which is why word[::-1] does not need any index at all.

c) word[10] raises IndexError, string index out of range, because an INDEX must name an existing character. word[2:99] returns 'THON' without complaint, because a SLICE clamps its bounds to the string. The one sentence: indexing asks for one character and fails when it is not there, slicing asks for a region and quietly keeps the part that exists. That difference is the reason a slice never protects you from a wrong bound, and why an empty result is worth testing for.

d) Because -0 and 0 are the same integer, so word[-0] is word[0], the FIRST character. Negative indexing starts at -1 for the last character precisely because zero is already taken. The smallest correct form for the last character is word[-1], and it works whatever the length, whereas word[len(word) - 1] needs the length to be computed and read correctly.

e) There are $j - i$ characters, provided both indices lie inside the string, because the slice includes the character at $i$ and excludes the one at $j$. So word[2:5] has $5 - 2 = 3$ characters. The same subtraction gives the length of range(i, j), and that is not a coincidence: both are half open intervals, and half open intervals are the only ones whose length is a subtraction with no plus one.

a) strip builds a NEW string with the spaces removed and returns it; it cannot change name, because a string cannot be changed at all. The line therefore computes a value and throws it away, and name still holds what it held. The repair is to keep the result: name = name.strip(). This is the single most common wasted line in a first assignment, and the same reasoning covers lower, upper, replace and every other string method.

b) A new string: upper, replace, join. A list: split. A number: count and index. A boolean: startswith and isalpha. Two of them deserve a note. join is called on the SEPARATOR, not on the list, which is why it reads ', '.join(parts) rather than parts.join(', '). And index returns a number but raises when the substring is absent, which is what separates it from find.

c) Each call is applied to the result of the one before, left to right: the line is stripped of its outer spaces, then lowered, then every remaining space becomes an underscore. On ' Ana Marie ' the result is 'ana_marie'. If replace came first, the OUTER spaces would become underscores too and the strip would then find nothing to remove, giving '__ana_marie__'. Chaining is convenient and the order is part of the meaning.

d) '12'.isdigit() is True. '-12'.isdigit() is False, because the minus sign is not a digit. '3.5'.isdigit() is False, for the same reason with the point. ' 12'.isdigit() is False, since the space is not a digit either. To test for a number that may be negative or decimal, the readable answer in COMP 202 is to try the conversion inside a try and catch the ValueError, which accepts exactly what float itself accepts, and nothing else.

e) The rule is left to right: the leftmost call runs first and each following call is applied to what the previous one returned. It matters wherever one method changes what the next one can see, and the clearest case is exactly c): stripping then replacing is not the same as replacing then stripping. It also matters for lower before a comparison, since 'Ana'.lower() == 'ana' while 'Ana' == 'ana'.lower() reads the same but tests the untouched left side.

a) It gives the list holding 'Ana', '78' and '91', three STRINGS. The marks are strings too, because cutting a string can only produce strings, and '78' + '91' is '7891' rather than 169. Nothing is a number until int or float is applied, and that conversion is the step forgotten in half the file exercises of the term.

b) 'a,,b'.split(',') gives 'a', the EMPTY string, and 'b': three pieces, because two commas in a row delimit an empty field, and an empty field is data. ' a b '.split() with no argument gives 'a' and 'b' only: called without a separator, split treats any run of whitespace as one cut and drops the leading and trailing ones. ' a b '.split(' ') gives five pieces, an empty string, 'a', an empty string, 'b' and an empty string, because with an explicit separator every single space is a cut. The rule: no argument for human spacing, an explicit separator for a data format where empty fields matter.

c) join builds a string by putting the separator between the pieces, so every piece must already BE a string; given integers it raises TypeError, sequence item 0: expected str instance, int found. The version that works converts first: ','.join([str(v) for v in [1, 2, 3]]), which gives '1,2,3'. Doing the conversion inside the comprehension rather than earlier keeps the original list numeric, which is what the rest of the program needs.

d) line.split(':', 1), which cuts at the FIRST colon only and returns two pieces, 'note' and ' remember to split, then join'. The second argument, maxsplit, is what stops the cut from continuing through the commas of the text. Without it the label would still be right and the rest would arrive in pieces, which is a bug that only shows up on the day a value contains the separator.

e) The first is always equal, since split with an explicit separator keeps every empty field and join puts exactly the same separator back between them: the round trip is exact. The second is not: split with no argument drops the leading and trailing whitespace and collapses every run of spaces into one cut, so ' a b ' comes back as 'a b'. That normalisation is often what is wanted, but it must be chosen and not discovered.

a) 'Hello'.find('l') is 2, the FIRST occurrence, scanning from the left. 'Hello'.rfind('l') is 3, the last one. 'Hello'.find('z') is -1, meaning not found. 'Hello'.index('z') raises ValueError, substring not found, instead of returning anything. So find and index differ on exactly one point, what they do when the answer is nothing, and that is the point on which the choice is made.

b) It prints the LAST character of s, because -1 is the index of the last character. Nothing crashes, nothing warns, and a program that extracts a field this way keeps running with a wrong value that looks like data. It is worse than a crash because a crash names the line and stops; this walks the wrong value into the rest of the program, where it will be blamed on something else. The guard is one line: if pos == -1, handle the absence, and only then index.

c) 'aaaa'.count('aa') is 2 and 'aaa'.count('aa') is 1. Both follow the same rule: count finds NON OVERLAPPING occurrences, scanning from the left and resuming after the match it just took. In 'aaaa' it takes positions 0 and 2; in 'aaa' it takes position 0, then resumes at index 2 where only one character is left. Counting overlapping occurrences needs a loop with find and a start that advances by one, not by the length of the pattern.

d) pos = text.find('ss'), then while pos != -1: print(pos), pos = text.find('ss', pos + 1). On 'mississippi' it prints 2 and 5. The second argument of find is the index to start looking from, and advancing by one rather than by two is what makes the search count overlapping occurrences as well. Advancing by len(sub) instead would reproduce the behaviour of count. Forgetting to advance at all gives an infinite loop that prints 2 for ever, which is the same missing update as in the loops chapter.

e) The in operator: if sub in s. It says exactly what is meant, it returns a boolean rather than a number to be compared with -1, and there is no sentinel to mishandle. Writing if s.find(sub) != -1 works and reads as a puzzle; writing if s.find(sub) as a truth test is simply wrong, since a substring found at position 0 makes the test false.

a) out = '' before the loop, then for name in names: out = out + name + ', ', which gives 'Ana, Ben, Chen, ' with a trailing separator that no name follows. It is removed afterwards with a slice, out[:-2], or avoided by adding the separator BEFORE every piece except the first, which needs a test on the index. Both work; both are the reason join exists.

b) ', '.join(names) gives the same string with no special case at either end. The loop version is quadratic because a string cannot be extended in place: each pass BUILDS a new string by copying everything accumulated so far, so the copies cost $1 + 2 + \cdots + n$ characters, which is $n(n+1)/2$. join walks the list once to compute the total length, allocates the result once and copies each piece once, which is linear. On three names nobody notices; on a file of fifty thousand lines the difference is minutes against milliseconds.

c) f'{mark / 10:.1%} for {name:<12}' with mark = 8.5 gives '85.0% for Ana '. The percent format multiplies by a hundred and adds the sign itself, so the value passed must be the ratio 0.85 and not 85. The colon inside the braces separates the expression from its format specification, and everything after the colon is about presentation only.

d) The first fails with TypeError, can only concatenate str to str, because the plus sign between a string and an int has no meaning, as in the types chapter. The second works because print accepts any number of arguments, converts each one with str and puts a space between them, which is also why print('Total:', 12) shows a space that nobody typed. The f-string version is f'Total: {12}', and it is the one to prefer: it converts without being asked and puts exactly the characters written.

e) f'{name:<12}{average:>8.2f}' gives a row where the name is left aligned in a field of twelve characters and the average right aligned in a field of eight with two decimals. The aligning is done by the two format specifications, not by spaces typed between the fields: a name of four letters and a name of eleven both end at the same column, which is what makes the numbers underneath line up. Right alignment is the correct choice for numbers, since it puts the units digits under each other.

a) It gives four strings: ' Tremblay, Ana ', ' 78 ', ' 91 ' and ' 65 \n'. Every one of them keeps the spaces that surrounded it, because split cuts at the separator and hands back everything else untouched, and the last one keeps the newline as well since nothing removed it. The list has four items, not three: the name is a field like the others.

b) int(parts[3]) WORKS and returns 65, because int tolerates whitespace on either side of the number, newline included. The comparison fails, parts[3] == '65' is False since the two strings differ by four characters. The test fails too, ' 65 \n'.isdigit() is False because a space is not a digit. The one that works is the dangerous one: a field that is only ever converted looks perfectly clean, so the newline survives unnoticed until the day a field is compared, used as a label or printed, and then the bug appears far from the line that caused it.

c) line = line.strip(), then parts = line.split(';'), then marks = [int(p) for p in parts[1:]], with name = parts[0].strip() alongside. The order matters: stripping the whole line first removes the newline once and for all, and the per field strip that remains is for the spaces around each value. Doing the split first forces a strip on every field, which is three chances to forget one.

d) With a comma as the separator, 'Tremblay, Ana' would have been cut in two, the line would have produced five fields instead of four, and parts[1] would have been ' Ana ' rather than a mark. The program would then raise a ValueError on the conversion, or worse, shift every field by one and report the marks of the wrong column. The lesson: a separator must be a character the data cannot contain, which is why real formats use a semicolon or a tab, or quote their fields.

e) For the blank line, if line == '': continue, placed immediately after the strip and before the split, since a blank line splits into a list of one empty string and the indexing that follows would raise IndexError. For the empty field, a test before the conversion, since int('') raises ValueError: either skip the record, or record it as missing rather than as zero, because a missing mark and a mark of zero are different facts and averaging them is not the same thing.

a) The test is word == word[::-1]. It is exact on a single word in one case, and it answers no on anything a human would call a palindrome but that carries capitals, spaces or punctuation: 'Racecar' fails on its capital R, and 'A man, a plan' fails on its spaces and commas. The test is not wrong, the input is not normalised, and separating those two statements is what question b) is about.

b) clean = ''.join([c.lower() for c in s if c.isalpha()]), which keeps the letters, lowers them and glues them back into one string. On the sentence given it produces 'amanaplanacanalpanama', twenty one characters, and clean == clean[::-1] is then True. Note the shape: a comprehension that FILTERS with its if and TRANSFORMS with its expression, joined once at the end rather than accumulated in a loop, exactly as in exercise 5.

c) i = 0 and j = len(word) - 1, then while i < j: if word[i] != word[j]: the answer is no and the loop stops; otherwise i = i + 1 and j = j - 1. The condition is i < j and not i != j: with an even length the two indices cross without ever being equal, so a test on equality would run past each other and compare the string with itself in reverse, doing twice the work at best and never stopping at worst.

d) No special case is needed. On 'racecar', seven characters, the pairs compared are the ones at indices 0 and 6, then 1 and 5, then 2 and 4, and the loop stops with i and j both at 3, the middle character, which never needs comparing since it is its own mirror. On 'abba', four characters, the pairs are 0 and 3, then 1 and 2, and the loop stops with i at 2 and j at 1, having crossed. Three comparisons in the first case, two in the second, and the same condition covers both.

e) The slice version builds a reversed copy of the whole million characters, then compares character by character until the second one, so it does a million copies for two comparisons. The loop version compares the first pair, then the second, finds the difference and stops: two comparisons and no copy at all. Both are correct and the difference is entirely in the wasted work, which is invisible on a word of seven letters and is the whole answer on a large input. On a string that IS a palindrome, the two do comparable work.

a) False. '-12'.isdigit() is False, because the minus sign is not a digit, and isdigit asks about EVERY character. The same goes for '3.5' and for ' 12'. Correct version: isdigit accepts a string made only of digits, so it recognises unsigned integers and nothing else; for anything else, attempt the conversion and catch the ValueError.

b) False. It returns a list of ONE element, the whole string. The counterexample: 'hello'.split(',') gives the list holding 'hello', whose length is 1, so a program that tests the length against 3 to validate a record correctly rejects it, while a program that assumes at least one comma and reads parts[1] raises IndexError. Correct version: split always returns at least one piece, and cutting at $k$ separators gives $k+1$ pieces.

c) False, it is 1. count takes non overlapping occurrences: it matches at position 0, then resumes the search AFTER that match, at index 2, where only one character remains. The counterexample is in the statement itself, and 'aaaa'.count('aa') being 2 rather than 3 makes the same point. Correct version: count counts non overlapping occurrences; counting overlapping ones needs a loop with find and a start advanced by one.

d) False. It raises IndexError, since the valid indices run from 0 to len(word) - 1. The counterexample: with word = 'abc', len(word) is 3 and word[3] is out of range while word[2] is 'c'. The confusion comes from the length being one more than the last index, which is exactly what makes word[-1] the form worth using. Correct version: the last character is word[-1], or word[len(word) - 1].

e) False. A string cannot be extended in place, so out = out + piece builds a whole new string on every pass and copies everything accumulated so far. Over $n$ pieces that is about $n^2/2$ character copies, against $n$ for join, which computes the total length once and copies each piece once. The counterexample is measurable: building a string of fifty thousand pieces takes seconds by accumulation and milliseconds by join. Correct version: collect the pieces in a list and join once.

a) chr((ord(c) - ord('a') + k) % 26 + ord('a')). The first subtraction moves the letter from its ASCII code to a position between 0 and 25, where arithmetic makes sense; the modulo brings the shifted position back into that range, which is the wrap around from z to a; the final addition puts the result back into the code range so that chr gives a letter. Doing the modulo on the raw code would be meaningless, since the codes do not start at zero, and that is the mistake to look for in a wrong answer.

b) For 'a' with $k = 3$: $97 - 97 = 0$, then $0 + 3 = 3$, then $3 \bmod 26 = 3$, then $3 + 97 = 100$, which is 'd'. For 'x': $120 - 97 = 23$, $23 + 3 = 26$, $26 \bmod 26 = 0$, $0 + 97 = 97$, which is 'a'. For 'z': $122 - 97 = 25$, $25 + 3 = 28$, $28 \bmod 26 = 2$, $2 + 97 = 99$, which is 'c'. The two last cases are the ones a test suite must contain, since they are the only ones that exercise the wrap.

c) Walk the message character by character and decide for each one: if it is a lower case letter, shift it with base ord('a'); if it is upper case, shift it with base ord('A') so that the capital stays a capital; otherwise append it unchanged. The pieces go into a list and are joined at the end, for the reason of exercise 5. The test on the case is what makes the difference between a cipher and a program that turns a comma into a letter, and the whole structure is a cascade of elif inside a for loop.

d) Either shift by $-k$, since the modulo of a negative number in Python is already non negative and $(p - k) \bmod 26$ lands in range by itself, or shift by $26 - k$, which is the same value. A marker prefers a single function with the shift as a parameter, called with $k$ to encode and with $-k$ to decode, because writing a second nearly identical function is the duplication the functions chapter exists to remove.

e) Encoding twice with 13 returns the original message, since $13 + 13 = 26$ is a full turn of the alphabet. It is the only non zero shift with that property because it is the only one that is exactly half of 26, and any other $k$ would need $2k \equiv 0 \pmod{26}$ with $k$ strictly between 0 and 26, which has no other solution. That is the cipher known as ROT13, and its whole point is that one function both encodes and decodes.

a) Nine words, given by len(text.split()). Splitting with no argument is the right call here, since the words are separated by single spaces and any accidental double space would be handled correctly anyway. Note that this counts what lies between the spaces, so a sentence ending with a full stop would give a last word carrying its punctuation, which is why a real word count strips the punctuation first.

b) The total is 35 letters, sum([len(w) for w in text.split()]), and the average is $35/9 = 3{,}889$ to three decimals, printed with f'{total / count:.3f}'. Note that the total does not count the spaces: the sentence itself is 43 characters long, and the difference of 8 is exactly the number of spaces, one fewer than the number of words.

c) The longest words are quick, brown and jumps, all of length 5. The two step method is the one to remember: first compute the maximum, best = max([len(w) for w in words]), then select everything that reaches it, [w for w in words if len(w) == best]. A single loop that keeps the longest word seen so far returns only the FIRST of the three, and that is the classic wrong answer to a question that says which words rather than which word.

d) The letter o appears 4 times, text.count('o'), in brown, fox, over and dog. The word the appears twice, but text.count('the') also returns 2 only by luck here: count works on characters, not on words, so it would count the inside another, together or theory. The honest answer counts on the LIST of words, sum([1 for w in text.split() if w == 'the']), which is the question actually asked. Counting a word with a substring search is a classic silent error.

e) for w in text.split(): print(f'{w:<8}{len(w):>3}'), which lines the numbers up in a column whatever the word. For the frequency of every word, the method of this chapter needs a pass over the whole list for each distinct word, which is the quadratic double loop of the loops chapter; a dictionary replaces it with a single pass that counts as it goes, because looking a key up costs the same whatever the number of keys. That is what a dictionary buys, and it is the subject of the set on lists and dictionaries.