You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
According to the documentation context steps (setup and teardown) are specified once but applied to every test case. This is IMHO a little counter intuitive and also not consistent with the definition of automated fixtures which are executed once at the beginning of a spec.
Another issue is that, with "contexts", it is very difficult to discern the fixture part from the actual test case in the UI as illustrated below.
Thus in the event of a problem, it is unclear whether the test should be blocked (if the setup fails) or failed if the actual test case fails.
I think the term "fixture" should be used and its effect consistent, whether we are talking about manual or automated fixtures.
Spec Level Fixtures
The setup/teardown should be defined the same way it currently is with contexts, but executed only once (at the beginning of a spec and before executing the first test case, and at the end if required).
Furthermore, users should be able to set the outcome of a fixture. If the outcome fails, then automatically all test cases within that spec should be marked as blocked.
Case Level Fixture
It is sometimes useful to have a fixture at the test case level, but it should be clearly identified so that the user can appropriately decide whether the outcome of a test is blocked (fixture failed) or Pass/Fail.
I am not sure what is the use "contexts". I can't think of a scenario where I would need to execute the same thing over and over for every test case. I do understand that it makes sense in the context of automated tests (e.g. junit fixtures) but not so much for manual tests.
reacted with thumbs up emoji reacted with thumbs down emoji reacted with laugh emoji reacted with hooray emoji reacted with confused emoji reacted with heart emoji reacted with rocket emoji reacted with eyes emoji
-
According to the documentation context steps (setup and teardown) are specified once but applied to every test case. This is IMHO a little counter intuitive and also not consistent with the definition of automated fixtures which are executed once at the beginning of a spec.
Another issue is that, with "contexts", it is very difficult to discern the fixture part from the actual test case in the UI as illustrated below.
![image](https://user-images.githubusercontent.com/14916861/126734317-88f81426-7438-4bc3-ad9d-fa7dbb8b9c1e.png)
Thus in the event of a problem, it is unclear whether the test should be blocked (if the setup fails) or failed if the actual test case fails.
I think the term "fixture" should be used and its effect consistent, whether we are talking about manual or automated fixtures.
Spec Level Fixtures
The setup/teardown should be defined the same way it currently is with contexts, but executed only once (at the beginning of a spec and before executing the first test case, and at the end if required).
Furthermore, users should be able to set the outcome of a fixture. If the outcome fails, then automatically all test cases within that spec should be marked as blocked.
Case Level Fixture
It is sometimes useful to have a fixture at the test case level, but it should be clearly identified so that the user can appropriately decide whether the outcome of a test is blocked (fixture failed) or Pass/Fail.
I am not sure what is the use "contexts". I can't think of a scenario where I would need to execute the same thing over and over for every test case. I do understand that it makes sense in the context of automated tests (e.g. junit fixtures) but not so much for manual tests.
Beta Was this translation helpful? Give feedback.
All reactions