logilab-common #295922 pytest -t should randomize tests order [validation pending]
pytest -t runs tests in whichever order os.listdir() returned, which is what readdir(2) returned. Bottom line, this order is perfectly stable between consecutive runs. A little randomness should help spot tests that are not fully independent of one another. | |
| priority | normal |
|---|---|
| type | bug |
| done in | 1.0.2 |
| load | 0.500 |
| load left | 0.000 |
| closed by | #99881205d7c4 [pytest] Randomize tests order when using -t (closes #295922) |
| patch | [pytest] Randomize tests order when using -t (closes #295922) [applied] |
Comments
-
2015/07/09 08:16, written by rmeradi
-
2015/07/09 08:20, written by rcardona
-
2015/07/09 08:33, written by rmeradi
-
2015/07/09 09:06, written by jcristau
-
2015/07/09 09:16, written by rmeradi
add commentCan you show a case where this problem can happen?
run the following snippet and notice how the output doesn't change between invocations
python -c "import os; print os.listdir('.')"Sorry I didn't explain well my question. I don't see how the tests can't be fully independent of one another if they are run in separete environements.
they aren't run in separate environments, they're run in the same process (plus they may have side effects on the system)
I know that they are run in the same process but they don't share data between them that what's I mean by seperate environnements. So I though that they are independent. But I see the problem of the side effects on the system.