LogoLogo
BugBug HomepageContact SupportLoginSign up free
  • Documentation
  • Tutorial for beginners
  • Best practices
  • BugBug App FAQ
  • Quick Start
    • What is test automation?
    • Start for free
    • Create your first project
    • Install Chrome extension
    • Create and run the tests
  • Creating Tests
    • Your first test
    • Independent tests
    • Duplicating tests
    • New test from here
  • Recording tests steps
    • BugBug overlay
    • Recording clicks
    • Recording hover
    • Recording keyboard typing
    • Recording assertions
    • Recording drag & drop
    • Record from here
    • Re-recording steps
    • Recording pop-up window actions
    • Using keyboard shortcuts
  • Editing tests
    • Grouping steps
    • Components
    • Manually editing steps
    • Actions
    • Assertions
    • Tabs & iframes
    • Variables
    • Local variables
    • Profiles
    • Custom JavaScript actions
  • Running tests
    • Running the tests
    • Statuses
    • Run (locally)
    • Run and pause here
    • Run in cloud
    • Schedules
    • Parallel runs
    • Running via API
    • Test your local build or protected web page using ngrok
  • Preventing failed tests
    • Waiting conditions
    • Smart click
    • Smart scroll
    • Selectors
    • Timeout
    • Delay / Sleep
    • Project settings
  • Debugging Tests
    • Runs history
    • Screenshots
    • Debug in Chrome
    • Breakpoint (run step-by-step)
  • Organizing tests
    • Naming your tests
    • Searching tests
    • Suites
    • Components
    • Projects
  • Workflow Tips
    • Edit & Rewind
    • Changing the test screen size
    • Multiple environments
    • Testing registration & login
    • Integrating with build systems
    • Mobile version testing
  • Collaboration
    • Organizations
    • Inviting team members
    • Alerts
      • Sending email notification
      • Sending webhook
      • Sending Slack message
      • Sending Teams message
  • Integrations
    • CLI
    • Zapier
    • Slack
    • GitHub
    • Bitbucket
    • Gitlab
    • Trello
    • Jira
  • Your account
    • Account settings
    • Edit your name and email
    • Forgot password
    • Manage Subscriptions
    • Account FAQ
  • Troubleshooting
    • Prohibited behaviors
    • Updating Chrome extension
    • Clear cookies and site data for BugBug
    • Testing basic auth password protected websites
    • Common selectors issues
    • CAPTCHA in automation testing
    • Cloud tests sometimes failing
    • IPs list of cloud runners
    • VPN or a Firewall
    • A/B tests
    • Report a bug
  • IN-DEPTH GUIDES
    • Beginners tutorial to automation testing
    • XPath Selectors without coding
    • Tech Leader's Guide to Automation Testing
    • Test automation guides
  • FREE TOOLS
    • BugBug Testing Inbox
    • Example SaaS App
    • No-code XPath Selector Builder
  • Other links
    • BugBug Homepage
    • Pricing
    • Terms & conditions
    • Privacy Policy
Powered by GitBook
On this page
  • Why use suites?
  • What suites should I have?
  • Create new custom suite
  • Auto-retry failed cloud tests to prevent flaky tests notifications

Was this helpful?

  1. Organizing tests

Suites

PreviousSearching testsNextProjects

Last updated 11 months ago

Was this helpful?

Why use suites?

Suites are great for grouping test cases to run them in parallel. They are simply groups of multiple tests. Organise your tests into Suites to quickly run several tests at once.

By default BugBug has one suite called "All tests", where you will find all tests you created in a project. All newly created tests will be automatically added to this suite.

Suites can be to run in the cloud and are useful for .

What suites should I have?

Here are some examples of suites that usually help with test automation workflow.

  • you can have a suite that monitors only core features on your production, every hour

  • you can have a full regression suite that you run manually before the release

  • you can have a "feature branch" suite that has work in progress tests that are not yet ready to be run on production, but after the release you will add them to the production suite

  • you can have suites that run the same tests with different , for example checking your app on multiple languages

Create new custom suite

You can create custom suites in the Suites tab:

  1. Click Create new suite button

  2. Add Suite name

  3. Choose tests that you want to include in a suite

  4. Choose options for a suite

  5. Save

Auto-retry failed cloud tests to prevent flaky tests notifications

Flakiness can be caused by many different factors:

  • a slowdown in the internet connection

  • busy server side

  • temporary machine CPU overload

You probably don't want to get notifications for such randomly failed tests - your app works as it should, it was just a temporary problem that doesn't require attention. BugBug allows you to prevent such unnecessary alerts.

When auto-retry is enabled:

  • If one of the tests in a suite fails, it will be automatically run again

  • If the test fails for the second time, it will be again restarted

  • If the test fails the specified number of times, we will mark the suite as "failed"

When auto-retry is disabled:

  • If any of the tests fail, the suite will continue running and after all the tests are finished it will be marked as "failed"

If your suite sometimes randomly fails, but there is no certain reason for that, you may be annoyed by the "failed" false positive . The industry term for tests that fail randomly for no reason is flaky tests.

if another attempt passes, we will mark the suite as "passed", but you can still see the failed test attempt in the , marked as "auto-retried"

Important! Auto-retry only works in suites run in the BugBug cloud. is required.

notifications
test runs history
Pro subscription
scheduled
scheduled
working with different environments
profiles
Example suite run history with auto-retry enabled. There were 3 attempts to run the same test.
Example suite run history with auto-retry disabled. There was just one attempt to run the test.