Our Azure Devops engineer set
up the Selenium UI Automation Tests in the Azure release pipeline. I asked him to set maximum
# of attempts =3.
The idea is that UI Automation
tests are not as robust as API tests. If the test fails 3 times, it means there
is something wrong with this test.
Maybe there is a bug.
Maybe developers update the UI
element names without letting me know.
Maybe there is the resolution
issue (1024*768) in Azure devops.
Maybe there is the account conflict
when doing the parallel runs.
When my colleagues see the Pipeline tab , they always feel
confused. Why it shows the error?
But in the Logs tab, it shows succeeded. So the test
result should show error or succeeded?
I always explain that it is the
design issue in the release pipeline. We set maximum # of attempts =3. For example,
this test has been run 2 times and it shows the Test Run
Failed messages twice. But the final test result is
succeeded.
Finally they understand they need
to look at succeeded text.
No comments:
Post a Comment