Suites
Suites are great for grouping test cases to run them in parallel.
Suites are simply groups of multiple tests. Organise your tests into Suites to quickly run several tests at once.
By default BugBug has one suite called "All tests", where you will find all tests you created in a project. All newly created tests will be automatically added to this suite.
Here are some examples of suites that usually help with test automation workflow.
- you can have a full regression suite that you run manually before the release
- you can have a "feature branch" suite that has work in progress tests that are not yet ready to be run on production, but after the release you will add them to the production suite
- you can have suites that run the same tests with different profiles, for example checking your app on multiple languages
You can create custom suites in the Suites tab:
- 1.Click
Create new suitebutton - 2.Add
Suite name - 3.Choose tests that you want to include in a suite
- 4.Choose options for a suite
- 5.Save
If your suite sometimes randomly fails, but there is no certain reason for that, you may be annoyed by the "failed" false positive notifications. The industry term for tests that fail randomly for no reason is flaky tests.
Flakiness can be caused by many different factors:
- a slowdown in the internet connection
- busy server side
- temporary machine CPU overload
You probably don't want to get notifications for such randomly failed tests - your app works as it should, it was just a temporary problem that doesn't require attention. BugBug allows you to prevent such unnecessary alerts.
.png?alt=media&token=1f52e368-090d-43e5-a634-f2056c8f77ef)
When auto-retry is enabled:
- If one of the tests in a suite fails, it will be automatically run again
- if another attempt passes, we will mark the suite as "passed", but you can still see the failed test attempt in the test runs history, marked as "auto-retried"
- If the test fails for the second time, it will be again restarted
- if the test fails 3 times, we will mark the suite as "failed"
Example suite run history with auto-retry enabled. There were 3 attempts to run the same test.
When auto-retry is disabled:
- If any of the tests fail, the suite will continue running and after all the tests are finished it will be marked as "failed"
Example suite run history with auto-retry disabled. There was just one attempt to run the test.
Last modified 7mo ago