Ned Batchelder: Coverage goals

There’s a feature request to add a per-file threshold to coverage.py. I didn’t add the feature, I wrote a proof-of-concept: goals.py.

Coverage.py has a --fail-under option that will check the total coverage percentage, and exit with a failing status if it is too low. This lets people set a goal, and then check that they are meeting it in their CI systems.

The feature request is to check each file individually, rather than the project as a whole, to exert tighter control over the goal. That sounds fine, but I could see that it would actually be more complicated than that, because people sometimes have more complicated goals: 100% coverage in tests and 85% in product code, or whatever.

I suggested implementing it as a separate tool that used data from a JSON report. Then, I did just that.

The goals.py tool is flexible: you give it a percentage number, and then a list of glob patterns. It collects up the files that match the patterns, and checks the coverage of that set of files. You can choose to measure the group as a whole, or each file individually. Patterns can be negated to remove files from consideration.

For example:

# Check all Python files collectively, except in the tests/ directory.

$ python goals.py --group 85 '**/*.py' '!tests/*.py'



# We definitely want complete coverage of anything related to html.

$ python goals.py --group 100 '**/*html*.py'



# No Python file should be below 90% covered.

$ python goals.py --file 90 '**/*.py'

Each run of goals.py checks one set of files against one goal, but you can run it multiple times if you want to check multiple goals.

If you want to have more control over your coverage goals, give goals.py a try. It might turn into a full-fledged coverage.py feature, or maybe it’s enough as it is.

Feedback is welcome, either here or on the original feature request.

from Planet Python
via read more