Run Coverage on Tests

5 hours ago 1

A circular manhole cover that has had a white line painted over one edge, but then lifted up, and rotated 180 degrees before being replaced, so the line is discontinuous.

I recommend running coverage on your tests.

Here’s a couple of reasons why, from the past couple of months.

Example one #

When writing tests, it’s common to copy and paste test functions, but sometimes you forget to rename the new one (see also: the Last Line Effect).

For example:

def test_get_install_to_run_with_platform(patched_installs): i = installs.get_install_to_run("<none>", None, "1.0-32") assert i["id"] == "PythonCore-1.0-32" assert i["executable"].match("python.exe") i = installs.get_install_to_run("<none>", None, "2.0-arm64") assert i["id"] == "PythonCore-2.0-arm64" assert i["executable"].match("python.exe") def test_get_install_to_run_with_platform(patched_installs): i = installs.get_install_to_run("<none>", None, "1.0-32", windowed=True) assert i["id"] == "PythonCore-1.0-32" assert i["executable"].match("pythonw.exe") i = installs.get_install_to_run("<none>", None, "2.0-arm64", windowed=True) assert i["id"] == "PythonCore-2.0-arm64" assert i["executable"].match("pythonw.exe")

The tests pass, but the first one is never run because its name is redefined. This clearly shows up as a non-run test in the coverage report. In this case, we only need to rename one of them, and both are covered and pass.

But sometimes there’s a bug in the test which would cause it to fail, but we just don’t know because it’s not run.

Tip 2: pytest’s parametrize is a great way to combine similar test functions with different input data.

Example two #

This is more subtle:

im = Image.new("RGB", (1, 1)) for colors in (("#f00",), ("#f00", "#0f0")): append_images = (Image.new("RGB", (1, 1), color) for color in colors) im_reloaded = roundtrip(im, save_all=True, append_images=append_images) assert_image_equal(im, im_reloaded) assert isinstance(im_reloaded, MpoImagePlugin.MpoImageFile) assert im_reloaded.mpinfo is not None assert im_reloaded.mpinfo[45056] == b"0100" for im_expected in append_images: im_reloaded.seek(im_reloaded.tell() + 1) assert_image_similar(im_reloaded, im_expected, 1)

It’s not so obvious when looking at the code, but Codecov highlights a problem:

The same code, but Codecov has flagged the last two lines were not covered

The append_images generator is being consumed inside roundtrip(), so we have nothing to iterate over in the for loop – hence no coverage. The fix is to use a list instead of a generator.

Header photo: Misplaced manhole cover (CC BY-NC-SA 2.0 Hugo van Kemenade).

Read Entire Article