Beginners tutorial to automation testing

Automation testing (whether e2e or unit testing) is the process of using software tools to execute test script on an application, comparing the actual results to the expected outcomes. It helps save time and effort by automating repetitive tasks and allows for quicker feedback on the application's quality.

Testing process can be done in test automation tool like BugBug or Selenium. The choice of automation testing tool depends on various factors like the type of application being tested, the programming language used, test data, testing framework, the team's expertise, and the testing requirements.

How to use BugBug?

This tutorial will guide you through the best practices of creating test automation with the BugBug testing tool, even if you're not a manual tester with technical knowledge. You're going to:

• Learn the basics of software testing concepts and terminology

• Follow the step-by-step screenshots

• Understand what to automate

This is not an ultimate guide with all there is to be known about test automation. We're going to explain everything assuming you have zero technical background but you're good with computers, unlike this guy:

Manual testing vs. automated testing

When software developers make changes in the code, they can break something in the app that has already been working. To make sure that nothing is broken, you need to test everything...

Manual testing

You just click everything in the app with your own computer and make sure there are no bugs. This is super boring and can take multiple days to complete!

It makes you feel like this:

Automated testing

You create step-by-step instructions for the computer to click everything for you automatically. You don't click on your own, but let the computer do the tedious work.

When you use this method you can feel like this:

You click "Run" and BugBug is clicking according to your instructions called test steps. Here's an example of an automated test where BugBug checks if the login works.

If BugBug is able to finish all the test steps, the test is marked as successfully finished & passed. It means that everything works as it should.

If BugBug is not able to finish, the test is failed. It means that something doesn't work right or your test steps are incorrect.

Keep reading to learn how to create such an automated test 💪

Create your first test via "recording"

Create your first automated test that checks if the login works. You will create a set of step-by-step instructions for BugBug that will tell it what exactly should be clicked or entered in the text fields.

Next steps would guide us to test execution:

1. Create a new project

2. Name it "Posters Galore" and enter a URL that will be the starting point of your app

In our example use https://marmelab.com/react-admin-demo/

3. Create a new test

We're going to create a test that checks if login to our app works, so we will name it simply "Login". In this case we are using functional testing - a type of software testing where the application under test is checked against its functional requirements.

4. Start the recording

Your test is empty for now. That means that it has no instructions - no test steps. The URL you entered before is already pre-filled, so you can click "start recording".

5. Wait 5-10 seconds until BugBug loads your page in an incognito window

On the right side of the page, you can see a BugBug panel with actions dedicated to recording. We call it "recording overlay". By default, BugBug records every click and keyboard typing, but using this panel you can also record additional special actions. BugBug is providing automation scripts as you click and navigate on the website.

6. Carefully enter login credentials using "demo" for login and password fields

BugBug shows "Saved" after each click or keyboard typing. Each of these recorded actions will become a "test step".

7. Click "Sign in"

You should now see the "Posters Galore" admin panel.

8. Add an assertion

We've successfully logged in, now we need to create an instruction for BugBug that will check if the page actually has logged in. What should appear on the page that clearly tells us that the login worked? What should we assert?

The simplest way of checking if the login has been successful is to check if the page shows some specific text. In our case we can assert if the text "Welcome to the react admin e-commerce demo" is visible. If it is visible everything is fine, if not, the test will fail.

Click "Assert" in the recording overlay

Click the text on the page that you want to check every time BugBug runs this test.

9. Finish recording the test

See that everything you clicked is now recorded as "test steps".

10. Run the test to see if it works

Observe how BugBug is repeating your actions.

11. Check if the test has passed

If your test has not passed and it's failed, it can be caused be several common problems which we will cover later.

12. Close the finished test

All your test steps are green! The login works as it should.

13. Achievement unlocked!

🎉 You have successfully automated your first test. Now to check if login works, you don't need to click on your own, you can just click "Run test" and BugBug will do it for you.

Does the test fail? Fix it with "Record from here"

Sometimes after you finished the initial recording, your test may still not pass. There might be several reasons for that:

• You forgot to record the mouse "hover"

• You clicked an incorrect element during the recording

• Your page has dynamic content, for example, you wanted to check if your homepage shows recommended products, but these products change every time you load the page

• Your page HTML is not built with automation testing in mind and uses unusual development patterns and solutions that are not industry-standard (see more advanced info on data-testid)

​ We're going to walk you through an example, where we record a test, but it doesn't work because we forgot to record mouse "hover". Then we're going to fix the test using the "record from here" action and add the missing hover step.

1. Create a new test and start recording (use the example app URL provided above)

2. Click "Utility"

3. Hover over a product and click the "Add to cart" button

Attention! This button only appears when you hover your mouse over the product. We deliberately skip recording this hover to demonstrate the consequences.

4. Record an assertion to the cart counter, checking if it shows "1"

5. Finish your recording

6. Run the test to see if it works... But it fails because we forgot to record the "hover".

Now it's time to fix the test by recording the missing "hover".

7. Click in between the test steps, just above the failed step

8. Click "Record from here"

9. Wait until BugBug reaches the right moment in your test

You can see that the test is ready for recording when the BugBug overlay shows the "REC" icon.

10. Click the "Add hover" button in the recording overlay

11. Click the product image

This tells BugBug that it should move the mouse to this element before attempting to click the "cart" icon inside it.

12. Finish your recording and confirm the recorded steps

You can see that we only added one step "hover".

13. Run the test to check if it works now

14. Achievement unlocked!

🎉 It worked! You've fixed a test with "record from here". You can use this technique for more advanced problems. Conclusion: if you see that your test fails, try to "record from here" and replace the failed steps with a new recording. It's the simplest way of fixing tests!

What to test?

You might think now: "Hold on, how can I test everything? Is it possible?"

In short: no, you will never be 100% sure if everything works as it should. But your goal is to lower the risk of app users encountering a bug. You should focus on testing features that are critical for your business.

Examples of features and consequences of their bugs:

How many tests should you create?

If you're just starting with test automation, your first goal is to create about 5 to 15 tests that check your core website functionality.

Later on, you can even have hundreds of tests, checking all the conditions and less critical features, but it's better to first focus on a small number of useful tests.

Here's an example list of tests that covers the most critical paths in a web app Software-as-a-Service product.

How many steps should a test have?

Shorter tests are better. Write automated tests that are short, avoiding long test cases.

Why? Because shorter tests are easier to maintain and fix.

Imagine a very long test that takes 10 minutes to run. But it fails at the end. Every time you want to fix it, you would need to wait these 10 minutes until the test goes through the previous steps. Test automation is supposed to make your work faster, so avoid such long tests.

How to make shorter tests?

Use multiple test accounts

Don't use your own personal login and password in automated tests. Create a dedicated test user for that. Each user can have a different configuration. For example, one of your test users can be a business user, and one can be a personal user. You create separate tests for each type of login.

Make a list of your test accounts and their params and store it somewhere in a spreadsheet, for example like this:

Split user paths to independent tests

Here's an example of a too long:

Split it to 2 independent tests:

Remember that tests need to be independent!

Tests should be independent of each other. You should not create 2 tests and then run them in a specific order. Every test needs to be self-contained. Learn more about it in our tips on atomic test cases.

What not to test?

A test automation framework is a set of guidelines, best practices, tools, and libraries that provide a structured approach to designing, organizing, and executing automated tests. It aims to standardize the testing process, increase test efficiency, and improve test maintenance.

That's why some features are known for being more difficult to automate. We recommend that you automate them later after you are done with the easier test cases.

• Registration: it's difficult to automate because you can't register the same email twice, so you need to generate a random email and verify it afterward.

• Collaboration: one user is adding something, and the second user sees the changes. You need one long test where you log in with the first user, then log out and log in again with the second user.

• Dynamic lists such as search results, product listings, exchange rates, and lists that change all the time. Automation requires a good understanding of how to make the list more predictable.

• Downloading files and checking their contents.

• Multi-language: you can have lots of tests for different languages, but it's hard to maintain unless your app supports selectors based on data-testid attributes.

• Dates and calendars: the current date changes every day, so it's not easy to record and repeat.

Suites: when do you need them?

Test suites are groups of individual tests. They allow you to run several tests at once with one click.

By default, BugBug creates an "All tests" suite for you. All new tests will be automatically added to this suite. If you're a beginner, you don't need to set up anything else for now, this one suite should cover the basic needs.

Run tests in the cloud

Check if your tests are passed when you run them in the cloud.

1. Go to "Suites"

2. Find "All tests"

3. Click "Run in cloud"

4. Wait until the suite is finished and passed

If the tests are passed locally but fail in the cloud, please contact our support. Usually, that should never happen.

Schedule automated tests

You can monitor your web app and run automated tests every hour and get notifications when your test fails. Learn more in our separate article about scheduling tests.

Take automation to a next level

This guide only covered the very beginning of what you can achieve with BugBug test coverage. Keep exploring and challenging yourself with more and more difficult test cases! Once you're comfortable with the basics, read our other in-depth guides, where we cover advanced techniques and a more strategic approach to automation testing in a startup.

If you have questions or comments on this guide, please contact us, our support team is very responsive and interested in your feedback.