Our Azure Devops engineer set up the Selenium UI Automation Tests in the Azure release pipeline. I asked him to set maximum # of attempts =3.
The idea is that UI Automation tests are not as robust as API tests. If the test fails 3 times, it means there is something wrong with this test.
Maybe there is a bug.
Maybe developers update the UI element names without letting me know.
Maybe there is the resolution issue (1024*768) in Azure devops.
Maybe there is the account conflict when doing the parallel runs.
When my colleagues see the Pipeline tab , they always feel confused. Why it shows the error? But in the Logs tab, it shows succeeded. So the test result should show error or succeeded?
I always explain that it is the design issue in the release pipeline. We set maximum # of attempts =3. For example, this test has been run 2 times and it shows the Test Run Failed messages twice. But the final test result is succeeded.
Finally they understand they need to look at succeeded text.