Corrected exercises: files, exceptions and real data, COMP 202 at McGill

a) Nothing at all. Mode 'w' truncates the file to zero length at the moment it is opened, whether or not anything is ever written, so the check finds an empty file and the original content is gone. The mode to open a file for looking at it is 'r', and the mode to add without destroying is 'a'. This is the single most expensive typo of a first course, and it is worth the habit of never writing 'w' next to a filename that holds real data.

b) infile.read() returns the whole content as ONE string, newlines included. infile.readlines() returns a LIST of strings, one per line, each still carrying its newline. Iterating with for line in infile: hands the lines over one at a time, which is the form to prefer because it never holds more than one line in memory and reads naturally. read is right for a small file that has to be searched as a whole; readlines is rarely the best of the three.

c) with open('marks.txt', 'r') as infile: then the reading inside the block. It closes the file automatically at the end of the block, INCLUDING when an exception leaves the block early, which a plain open followed by close does not do unless a finally is written by hand. In COMP 202 a file left open costs style marks; in a long running program it costs file handles, and on the writing side it costs data, since what has not been flushed is lost.

d) FileNotFoundError, naming the path it tried. What to do depends on whose fault it is: if the name came from the user, catch it and ask again, since a typo is an expected event and not a bug; if the name is fixed in the program, let it propagate, because the message names the file and the line, which is exactly what is needed to fix the real problem. Catching an exception only to print a vaguer message than the one Python already gave is worse than not catching it.

e) Both truncate the file on opening and both write from position zero, so the result depends on the order in which the two handles happen to flush, and it is usually a mixture of the two outputs or the shorter one padded with the tail of the longer. Nothing raises. The rule is one writer at a time, and inside a single program that means one function owns the writing. It is also why a program that reads a file and rewrites it in place should write to a temporary file and rename it at the end.

a) The strings 'Ana,8\n' and 'Ben,5\n', six characters each, the newline included and counted. That is what makes int(parts[1]) work and parts[1] == '8' fail, as the strings set showed: int tolerates surrounding whitespace, an equality test does not. Everything downstream is cleaner if the newline is removed once, at the top of the loop, with line = line.strip().

b) Zero. A file handle carries a POSITION, and reading to the end leaves it there; a second loop starts from where the first stopped, which is the end, and finds nothing. Nothing is raised, so the symptom is an empty result rather than an error. The two repairs: read the file into a list once, and walk that list as often as needed, which is the usual answer; or move the position back with infile.seek(0), which is rarely worth it.

c) lines = [line.strip() for line in infile if line.strip() != ''], inside the with block. It reads, cleans and filters in one pass, and it produces exactly what the rest of the program wants, a list of non empty strings. Note that the comprehension calls strip twice; a version that avoids it needs two lines, and at this size the readable one wins.

d) Neither notices, and it does not matter, because both hand over the last line as a string without a trailing newline and the strip that follows would have removed it anyway. It matters on the WRITING side: a program that appends to a file whose last line has no newline will glue its first line to that one. When appending, ending every line written with a newline is what keeps the file well formed.

e) The carriage return, written as backslash r, because Windows ends its lines with two characters, a carriage return and a newline. The two ways: strip, which removes every kind of whitespace at both ends, including that one, and is the reason strip is preferred to a hand written removal of the last character; or open the file in text mode with newline handling left to Python, which is the default and normally converts the pair for you. The symptom to recognise is a comparison that fails while the printed value looks right, since a carriage return does not show.

a) The second and the third produce the same file, since print adds the newline that write does not. The first, run three times in a loop, produces one single line reading Ana,8Ana,8Ana,8, because nothing separates the writes: write puts exactly the characters it is given and moves the position on. The habit that avoids it is to end every written line with a newline, or to use print with file, which also converts numbers on the way.

b) TypeError, write() argument must be str, not int. The two correct versions: outfile.write(str(8) + '\n'), which converts explicitly, or print(8, file=outfile), which converts on its own and adds the newline. This strictness is deliberate: a file is a stream of characters, and how a number should look in it is a decision the program must make, not the library.

c) It contains ONE line, the last student's, because opening with 'w' truncates and the open call is inside the loop, so every pass wipes what the previous one wrote. The mistake is the position of the open, not the mode: it belongs outside the loop, with open('report.txt', 'w') as outfile: and the loop inside the block. Mode 'a' inside the loop would also work and is the wrong fix, since it appends to whatever the previous RUN of the program left behind.

d) With 'w' the second run starts from an empty file, so the file always holds the results of the latest run only. With 'a' the second run adds to the first, so the file grows across runs. A log file needs 'a', which is exactly what a log is for; a report that describes the current state of the data needs 'w', so that it is never a mixture of two runs.

e) The last lines, or everything. Characters written are held in a buffer and reach the disk when the buffer fills, when flush is called, or when the file is closed; a reader opening the file before that sees only what has already gone through. The two fixes: close the file, which a with block does at the end of its block, or call outfile.flush() when the file must be readable while the program is still running. Writing and reading the same file at the same time is best avoided altogether.

a) try: value = int(text), except ValueError: then handle it. What must not be inside the try is everything that follows and depends on value: putting the whole rest of the program inside means a ValueError raised much later, by a completely different line, is caught by this handler and reported as a bad conversion. A try block holds the risky lines and nothing else, which is the same discipline as a guard clause holding only its guard.

b) A bare except catches KeyboardInterrupt, so the user can no longer stop the program with the interrupt key, and SystemExit, so a program that asked to end does not end. It also catches the NameError caused by a typo in the handler itself, which is how a bare except hides the very bug it was meant to survive. The correct form names the exception, except ValueError:, or at worst except Exception:, which is the base of the ordinary errors and does not catch the two above.

c) The else runs only when the try completed with no exception, and its purpose is to keep the try SHORT. Lines that cannot fail, or that could raise the same exception for a different reason, go in the else so that the handler cannot catch them by accident. It is the answer to the trap of question a), written as a clause instead of as a discipline.

d) with open('marks.txt', 'r') as infile: then the reading inside. The close is guaranteed on the way out, exception or not, which is what the finally was doing. What the with does NOT cover is the opening itself: a FileNotFoundError is raised by the open, before the block is entered, so it still needs a try around the with when the name is not trusted.

e) It is right when the exception is an EXPECTED event of the outside world that the program knows how to continue past: a line of the file that does not parse, a value the user mistyped, a file that is not there yet. It is the worst thing to do when the exception means the program's own assumptions are broken, because the program then carries on with a state it does not understand and produces wrong output instead of stopping. The test to apply: if the message would be I could not do this, so I skipped it, catching is right; if it would be something impossible happened, let it propagate.

a) float('n/a') raises ValueError, since the characters do not spell a number. values[10] raises IndexError, list index out of range. marks['Dan'] raises KeyError, with the key as the message. open('nope.txt') raises FileNotFoundError. And 'total: ' + 12 raises TypeError, can only concatenate str to str, which is the types chapter again. Naming the exception before writing the handler is what stops a handler from catching more than it understands.

b) It is wrong because the file can disappear, or its permissions change, between the test and the open: the two lines are not one operation, and something else may act in between. It is also two trips to the disk instead of one. The version that is not wrong asks forgiveness: try: with open(name) as f: ..., except FileNotFoundError: then handle it. The test has its uses for a message before starting a long job, but it is never a guarantee.

c) marks.get('Dan', 0), which returns the default when the key is absent and never raises. The same idea exists for a list only in the form of a length test, which is why the dictionary is the structure that makes look before you leap comfortable. Note that a default of 0 says the mark is zero, which may be false; a default of None says there is no mark, which is usually what is meant.

d) It should raise. Returning a special value, -1 or None, means every caller has to remember to test it, and the day one forgets, the wrong value travels on silently, which is exactly the failure of find returning -1 from the strings set. The raise is one line: raise ValueError('mark must be between 0 and 100, got ' + str(mark)). The message names the rule AND the offending value, which is what makes it useful in a traceback.

e) Either the raise happened outside the try, for instance in a line placed after the block or inside the except clause itself, in which case the handler cannot see it; or the exception is a ValueError raised again by the handler, since a handler that calls int on the same bad text raises the same error. A third and rarer one: the exception is a subclass caught elsewhere, or the try is inside a function whose caller re raises. In all three the fix starts by reading the traceback bottom up, since the LAST line names the place where it was raised.

a) def read_mark(): then while True: text = input('Mark: '), then try: value = int(text), except ValueError: print('That is not a whole number.'), continue, then if 0 <= value <= 100: return value, then print('The mark must be between 0 and 100.'). The order is the one that matters: read, convert with the conversion alone in the try, then test the range on a value that is now certainly an int, then return. The while True with a return is the honest form, since the loop ends exactly when a good value arrives.

b) Because the try must hold the risky line and nothing else. With the whole body inside, a ValueError raised by anything else, including a later change to the function, would be reported as that is not a whole number, and the real cause would be hidden. There is also a subtler reason: the range test cannot raise ValueError at all, so putting it inside says something false about it to whoever reads the code.

c) First, the user can no longer interrupt the program, because the interrupt key raises KeyboardInterrupt and the bare except swallows it inside an infinite loop: the program has to be killed. Second, any typo inside the loop, a misspelled variable raising NameError for instance, is caught too and reported as an invalid number, so the function loops for ever telling the user their input is wrong when the bug is in the code.

d) int('7.5') raises ValueError, so the function prints that it is not a whole number and asks again. The message is accurate, and it is not good enough on its own if the specification meant to accept 7.5: then the conversion is float and the message must change with it. A good message names what is expected rather than what was refused, for example please type a whole number between 0 and 100.

e) Change int to float and drop the range test, or replace it with whatever range makes sense. What must NOT be copied is the whole function under a new name: the two versions differ by a conversion and a test, so the version to write takes them as parameters, def read_number(prompt, lo, hi, convert=float), and the two uses become two calls. Copying a function to change two lines is how a program acquires two behaviours where it meant to have one, as the functions set said of the average computed twice.

a) def read_temps(filename): records = [], then with open(filename, 'r') as infile: then a first call, infile.readline(), to consume the header, then for line in infile: with the usual strip, skip of empty lines, split on the comma and conversion, appending [name, int(high), int(low)] to records; then return records. The header is skipped by that single readline BEFORE the loop, which is the cleanest place: skipping it inside the loop needs a counter or a test on every line, for one line out of many.

b) The highs are -6, -4, 2 and 11, whose sum is 3, so the mean is $3/4 = 0{,}75$ degree. The lows are -14, -13, -6 and 2, whose sum is -31, so the mean is $-31/4 = -7{,}75$ degrees. Note that both are computed with a division that returns a float, and that printing them with two decimals is a formatting decision, not a rounding of the data.

c) The gaps are 8, 9, 8 and 9 degrees for January, February, March and April, so the largest gap is 9 and it is reached TWICE, in February and in April. The question says the month, which is exactly the trap of the strings set: a maximum found by a running best returns the first of the two, and the honest answer lists both, [r for r in records if r[1] - r[2] == best].

d) int('') raises ValueError, on the conversion line, while the split itself succeeds and produces three fields of which one is empty. The two defensible answers: skip the record, with a try around the conversion and a continue, which is right when a missing value means no measurement; or record it as missing, storing None instead of a number, which is right when the month must still appear in the report. What is not defensible is converting it to 0, since a missing temperature is not a temperature of zero and it would drag every average down.

e) print(f'{name:<10}{high:>5}{low:>6}{high - low:>6}') inside the loop over the records, which lines the four columns up whatever the length of the month. The file must be closed first because the report is built from the STRUCTURE, not from the file: read once, close, then compute and print. Keeping the file open while printing works but ties the length of the report to the lifetime of the handle, and it is the habit that leads to reading a file twice and finding it empty the second time.

a) False. The truncation happens at the OPEN, not at the first write, so a file opened with 'w' is empty from that instant whatever follows. The counterexample: a file of a thousand lines, opened with 'w' and closed immediately, has zero lines. Correct version: use 'r' to look, 'a' to add, and reserve 'w' for a file whose previous content is genuinely to be discarded.

b) False. It is a STRING, ending with a newline, and it becomes a list only when something splits it. The counterexample: for the line Ana,8, line[0] is 'A', the first character, and not 'Ana', which is what a program written on this belief silently uses as a name. Correct version: line.strip().split(',') produces the list of fields, all of them strings.

c) False, it is the least safe. It catches KeyboardInterrupt, so the program cannot be stopped, and SystemExit, so it cannot end; and it catches the programming errors of the handler itself. The counterexample: a loop asking for input, with a bare except inside, cannot be interrupted and has to be killed. Correct version: name the exception expected, except ValueError:, and let everything else through.

d) False. write adds nothing at all, so three calls with no newline produce ONE line. The counterexample: three calls with 'Ana,8' give the file Ana,8Ana,8Ana,8. Correct version: write the newline yourself, outfile.write(line + '\n'), or use print(line, file=outfile), which adds it and converts non strings on the way.

e) False. The handle keeps its position and the first loop left it at the end, so the second loop sees nothing and, worse, raises nothing. The counterexample: two identical loops counting lines print 3 then 0. Correct version: read into a list once and walk that list as many times as needed, or move the position back with infile.seek(0) when re reading is really wanted.

a) with open('report.txt', 'w') as outfile: then for name, avg in records: outfile.write(name + ';' + str(avg) + '\n'). For Ana the string written is 'Ana;7.5\n', seven visible characters and the newline. Two conversions happen on that line and both are compulsory: str around the number, since write refuses anything else, and the newline, since write adds nothing.

b) with open('report.txt', 'r') as infile: then total = 0 and count = 0, then for line in infile: line = line.strip(), if line == '': continue, then name, text = line.split(';'), then total = total + float(text), then count = count + 1; the average is total / count. With 7.5, 6.0 and 8.0 the sum is 21.5 and the class average is $21{,}5/3 \approx 7{,}167$. The unpacking of the split into two names is the compact form, and it raises ValueError if a line has the wrong number of fields, which is a fair way to notice a corrupt file.

c) The digits beyond the second are lost, permanently, because the file becomes the only remaining copy: 7.166666 is written as 7.17 and comes back as 7.17. It matters as soon as the values are used for further arithmetic, since the rounding of each value accumulates in the total, exactly as in the floats exercise of the first set. The rule: round for the reader, keep full precision for the program, and if the file is data rather than a report, write the unrounded value.

d) It holds every student twice, once per run, and the reading loop computes the average of six lines, which happens to equal the average of three when the same data is written twice, and does not when the data has changed between the runs. The deeper problem is that nothing in the file says which run a line came from. A report file that describes the current state is opened with 'w'; if history is wanted, each line must carry a date.

e) The line splits into three fields instead of two, so the unpacking raises ValueError, not enough values to unpack, or a plain split gives a wrong name and a text that float refuses. Two preventions: choose a separator that cannot appear in the data, a tab or a character the names cannot contain, and validate it when writing; or use the csv module, which quotes any field containing the separator and unquotes it on the way back. The general rule of the strings set applies unchanged: a separator must be impossible in the data, or the data must be escaped.

a) No. parse_line knows that the line is malformed and knows nothing else: it cannot ask for a better line, cannot skip anything, and cannot decide whether one bad line among a thousand is acceptable. Its job is to state the problem as precisely as possible, which is what raising does, ideally with a message naming the offending text. A function that catches an error it cannot resolve turns information into silence.

b) try: record = parse_line(line), except ValueError as err: then count the bad line, record it with its number, and continue with the next one. read_file is the first caller that has a sensible answer, namely one line out of many is unusable and the rest of the file is still worth reading. Catching there and continuing is exactly the catch where you can do something sensible rule, and keeping the line NUMBER is what makes the summary usable.

c) In main, around the call to read_file, with except FileNotFoundError: then a message naming the file and a continue to the next one. main is the level that knows there are other files, so it is the level that can decide to carry on without this one. Note that the with statement inside read_file does not help here: the open is what raises, before any block is entered.

d) A list, or a dictionary keyed by filename whose values are the lists of problems: problems = {} in main, filled in by the two handlers of b) and c), and printed at the end. It is filled in exactly where the exceptions are caught, which is the argument for catching at a level that has somewhere to put the information. A summary that says 3 files unreadable, 12 lines skipped, with the names, is what turns a program that must not crash into a program that can be trusted.

e) Because it is a BUG, not an event of the outside world. Catching it would hide a division whose denominator is a count that should never have been zero, and the program would carry on producing averages computed from nothing. The correct response is to let it reach the top, read the traceback, find the empty list and fix the code that produced it. The line to hold on to: catch what the world does to you, never what you do to yourself.