While trying to reproduce an issue with coverage.py 5.0, I had a test suite that showed the problem, but it was inconvenient to run the whole suite repeatedly, because it took too long. I wanted to find just one test (or small handful of tests) that would demonstrate the problem.
But I knew nothing about these tests. I didn’t know what subset might be useful, or even what subsets there were, so I had to try random subsets and hope for the best.
I selected random subsets with a new trick: I used the -k option (select tests by a substring of their names) using single consonants. “pytest -k b” will run only the tests with a b in their name, for example. Then I tried “-k c”, “-k d”, “-k f”, and so on. Some will run the whole test suite (“-k t” is useless because t is in every test name), but some ran usefully small collections.
This is a mindless way to select tests, but I knew nothing about this test suite, so it was a quick way to run fewer than all of them. Running “-k q” was the best (only 16 tests). Then I looked at the test names, and selected yet smaller subsets with more thought. In the end, I could reduce it to just one test that demonstrated the problem.
from Planet Python
via read more
No comments:
Post a Comment