Beginners tutorial to automation testing

Automation testing (whether e2e or unit testing) is the process of using software tools to execute test script on an application, comparing the actual results to the expected outcomes. It helps save time and effort by automating repetitive tasks and allows for quicker feedback on the application's quality.
Testing process can be done in test automation tool like BugBug or Selenium. The choice of automation testing tool depends on various factors like the type of application being tested, the programming language used, test data, testing framework, the team's expertise, and the testing requirements.

How to use BugBug?

This tutorial will guide you through the best practices of creating test automation with the BugBug testing tool, even if you're not a manual tester with technical knowledge. You're going to:
  • Learn the basics of software testing concepts and terminology
  • Follow the step-by-step screenshots
  • Understand what to automate
This is not an ultimate guide with all there is to be known about test automation. We're going to explain everything assuming you have zero technical background but you're good with computers, unlike this guy:

Manual testing vs. automated testing

When software developers make changes in the code, they can break something in the app that has already been working. To make sure that nothing is broken, you need to test everything...

Manual testing

You just click everything in the app with your own computer and make sure there are no bugs. This is super boring and can take multiple days to complete!
It makes you feel like this:
regression testing meme

Automated testing

You create step-by-step instructions for the computer to click everything for you automatically. You don't click on your own, but let the computer do the tedious work.
When you use this method you can feel like this:
automsted tests meme
You click "Run" and BugBug is clicking according to your instructions called test steps. Here's an example of an automated test where BugBug checks if the login works.
If BugBug is able to finish all the test steps, the test is marked as successfully finished & passed. It means that everything works as it should.
If BugBug is not able to finish, the test is failed. It means that something doesn't work right or your test steps are incorrect.
Keep reading to learn how to create such an automated test 💪

Create your first test via "recording"

Create your first automated test that checks if the login works. You will create a set of step-by-step instructions for BugBug that will tell it what exactly should be clicked or entered in the text fields.
Before you continue make sure you've set up your BugBug account and have access to a project and installed the BugBug Chrome extension.
Use the example app to follow each step on your own
Throughout this guide, we're using this open-source demo app called react admin Posters Galore.
You can learn by creating a new blank project and follow each step together with us.
Next steps would guide us to test execution:
  1. 1.
    Create a new project
Create a new project - BugBug test automation tool
2. Name it "Posters Galore" and enter a URL that will be the starting point of your app
Create a new project - BugBug test automation tool
3. Create a new test
We're going to create a test that checks if login to our app works, so we will name it simply "Login". In this case we are using functional testing - a type of software testing where the application under test is checked against its functional requirements.
Create a new test - BugBug test automation tool
4. Start the recording
Your test is empty for now. That means that it has no instructions - no test steps. The URL you entered before is already pre-filled, so you can click "start recording".
Start test recording - BugBug test automation tool
5. Wait 5-10 seconds until BugBug loads your page in an incognito window
Why incognito? Testing needs to always start from a clean state. There should be no cookies or browser cache for your page before starting the test.
Test recording - BugBug test automation tool
On the right side of the page, you can see a BugBug panel with actions dedicated to recording. We call it "recording overlay". By default, BugBug records every click and keyboard typing, but using this panel you can also record additional special actions. BugBug is providing automation scripts as you click and navigate on the website.
Test recording overlay - BugBug test automation tool
6. Carefully enter login credentials using "demo" for login and password fields
Why carefully? BugBug records every click, so it's better to be slow while recording and think about every click. You don't want to accidentally click in the empty background!
Test recording - BugBug test automation tool
Test recording - BugBug test automation tool
BugBug shows "Saved" after each click or keyboard typing. Each of these recorded actions will become a "test step".
7. Click "Sign in"
Test recording - BugBug test automation tool
You should now see the "Posters Galore" admin panel.
Test recording - BugBug test automation tool
8. Add an assertion
We've successfully logged in, now we need to create an instruction for BugBug that will check if the page actually has logged in. What should appear on the page that clearly tells us that the login worked? What should we assert?
Assertion = a test step that checks if something specific appears on the page, or if the page matches some specific expectations.
The simplest way of checking if the login has been successful is to check if the page shows some specific text. In our case we can assert if the text "Welcome to the react admin e-commerce demo" is visible. If it is visible everything is fine, if not, the test will fail.
Click "Assert" in the recording overlay
Adding assertion - BugBug test automation tool
Click the text on the page that you want to check every time BugBug runs this test.
Test assertion - BugBug test automation tool
9. Finish recording the test
Finish test recording - BugBug test automation tool
See that everything you clicked is now recorded as "test steps".
Test steps - BugBug test automation tool
10. Run the test to see if it works
Test Run - BugBug test automation tool
Observe how BugBug is repeating your actions.
When running the test:
  • Don't move your mouse cursor over the running test window.
  • Don't minimize the window, as Chrome may stop executing the test if it's not visible.
Test Run - BugBug test automation tool
11. Check if the test has passed
Test Run - BugBug test automation tool
If your test has not passed and it's failed, it can be caused be several common problems which we will cover later.
12. Close the finished test
Test Finish - BugBug test automation tool
All your test steps are green! The login works as it should.
Test Result - BugBug test automation tool
13. Achievement unlocked!
🎉 You have successfully automated your first test. Now to check if login works, you don't need to click on your own, you can just click "Run test" and BugBug will do it for you.

Does the test fail? Fix it with "Record from here"

Sometimes after you finished the initial recording, your test may still not pass. There might be several reasons for that:
  • You forgot to record the mouse "hover"
  • You clicked an incorrect element during the recording
  • Your page has dynamic content, for example, you wanted to check if your homepage shows recommended products, but these products change every time you load the page
  • Your page HTML is not built with automation testing in mind and uses unusual development patterns and solutions that are not industry-standard (see more advanced info on data-testid)
​ We're going to walk you through an example, where we record a test, but it doesn't work because we forgot to record mouse "hover". Then we're going to fix the test using the "record from here" action and add the missing hover step.
Use the example app to follow each step on your own
For this example, we're using this demo e-commerce app.
You can learn by creating a new blank project and following each step together with us.
  1. 1.
    Create a new test and start recording (use the example app URL provided above)
Start test recording - BugBug Test Automation Tool
2. Click "Utility"
Test recording - BugBug Test Automation Tool
3. Hover over a product and click the "Add to cart" button
Attention! This button only appears when you hover your mouse over the product. We deliberately skip recording this hover to demonstrate the consequences.
Test recording steps - BugBug Test Automation Tool
4. Record an assertion to the cart counter, checking if it shows "1"
Test recording steps - BugBug Test Automation Tool
Test recording steps - BugBug Test Automation Tool
5. Finish your recording
Test recording finish - BugBug Test Automation Tool
6. Run the test to see if it works... But it fails because we forgot to record the "hover".
Test run steps - BugBug Test Automation Tool
Test run - BugBug Test Automation Tool
Test recording run - BugBug Test Automation Tool
Now it's time to fix the test by recording the missing "hover".
7. Click in between the test steps, just above the failed step
Test fixing steps - BugBug Test Automation Tool
8. Click "Record from here"
Record from here test step - BugBug Test Automation Tool
9. Wait until BugBug reaches the right moment in your test
You can see that the test is ready for recording when the BugBug overlay shows the "REC" icon.
10. Click the "Add hover" button in the recording overlay
Recording overlay - BugBug Test Automation Tool
11. Click the product image
This tells BugBug that it should move the mouse to this element before attempting to click the "cart" icon inside it.
Test recording steps - BugBug Test Automation Tool
12. Finish your recording and confirm the recorded steps
You can see that we only added one step "hover".
Test recording finish - BugBug Test Automation Tool
Test recording confirm - BugBug Test Automation Tool
13. Run the test to check if it works now
Test run check - BugBug Test Automation Tool
Test run check - BugBug Test Automation Tool
14. Achievement unlocked!
🎉 It worked! You've fixed a test with "record from here". You can use this technique for more advanced problems. Conclusion: if you see that your test fails, try to "record from here" and replace the failed steps with a new recording. It's the simplest way of fixing tests!

What to test?

You might think now: "Hold on, how can I test everything? Is it possible?"
In short: no, you will never be 100% sure if everything works as it should. But your goal is to lower the risk of app users encountering a bug. You should focus on testing features that are critical for your business.
Checking if there are no new bugs is called regression testing. If you notice a bug in something that has been working before, it's called a regression.
Examples of features and consequences of their bugs:
What if it breaks:
Is it critical?
Users can't use your app
Homepage / Landing page
New clients can't register
New clients can't register
Creating new items
Users can't achieve their basic goal
Password reset
Only users who forgot their password can't access
Some checkbox in an obscure feature that nobody uses
Probably no-one will notice
Hell no
A list of things that you want to test is called a list of test cases. An individual test case should focus on just one process in your app.

How many tests should you create?

If you're just starting with test automation, your first goal is to create about 5 to 15 tests that check your core website functionality.
Later on, you can even have hundreds of tests, checking all the conditions and less critical features, but it's better to first focus on a small number of useful tests.
Here's an example list of tests that covers the most critical paths in a web app Software-as-a-Service product.
List of tests - BugBug Test Automation Tool

How many steps should a test have?

Shorter tests are better. Write automated tests that are short, avoiding long test cases.
Why? Because shorter tests are easier to maintain and fix.
Imagine a very long test that takes 10 minutes to run. But it fails at the end. Every time you want to fix it, you would need to wait these 10 minutes until the test goes through the previous steps. Test automation is supposed to make your work faster, so avoid such long tests.

How to make shorter tests?

Use multiple test accounts
Don't use your own personal login and password in automated tests. Create a dedicated test user for that. Each user can have a different configuration. For example, one of your test users can be a business user, and one can be a personal user. You create separate tests for each type of login.
Make a list of your test accounts and their params and store it somewhere in a spreadsheet, for example like this:
User with default configuration for business accounts
Trial account
Paying user, with a subscription active
Split user paths to independent tests
Here's an example of a too long:
Test name
Landing page
go to landing page URL
click "features"
assert all the features
click "contact"
assert contact information
Split it to 2 independent tests:
Test name
Landing page features
go to landing page URL
click "features"
assert all the features
Landing page contact
go to landing page URL
click "contact"
assert contact information
Remember that tests need to be independent!
Tests should be independent of each other. You should not create 2 tests and then run them in a specific order. Every test needs to be self-contained. Learn more about it in our tips on atomic test cases.

What not to test?

A test automation framework is a set of guidelines, best practices, tools, and libraries that provide a structured approach to designing, organizing, and executing automated tests. It aims to standardize the testing process, increase test efficiency, and improve test maintenance.
That's why some features are known for being more difficult to automate. We recommend that you automate them later after you are done with the easier test cases.
  • Registration: it's difficult to automate because you can't register the same email twice, so you need to generate a random email and verify it afterward.
  • Collaboration: one user is adding something, and the second user sees the changes. You need one long test where you log in with the first user, then log out and log in again with the second user.
  • Dynamic lists such as search results, product listings, exchange rates, and lists that change all the time. Automation requires a good understanding of how to make the list more predictable.
  • Downloading files and checking their contents.
  • Multi-language: you can have lots of tests for different languages, but it's hard to maintain unless your app supports selectors based on data-testid attributes.
  • Dates and calendars: the current date changes every day, so it's not easy to record and repeat.

Suites: when do you need them?

Test suites are groups of individual tests. They allow you to run several tests at once with one click.
By default, BugBug creates an "All tests" suite for you. All new tests will be automatically added to this suite. If you're a beginner, you don't need to set up anything else for now, this one suite should cover the basic needs.
You will need more suites later on, once you have more advanced tests that cover different testing environments.
Test suites - BugBug Test Automation Tool

Run tests in the cloud

You might think that if you created your 5 essential automated tests, now to run them you need to open up your computer and click "Run" and block your computer for several minutes. Good news - there's no need for that! You can run tests in the cloud and even schedule them to run automatically, even when you're asleep.
Run in cloud = run on some other machine in a data center, not on your own computer
Run locally = run on your own computer in your own browser
Check if your tests are passed when you run them in the cloud.
  1. 1.
    Go to "Suites"
  2. 2.
    Find "All tests"
  3. 3.
    Click "Run in cloud"
  4. 4.
    Wait until the suite is finished and passed
Test run in cloud - BugBug Test Automation Tool
If the tests are passed locally but fail in the cloud, please contact our support. Usually, that should never happen.

Schedule automated tests

You can monitor your web app and run automated tests every hour and get notifications when your test fails. Learn more in our separate article about scheduling tests.

Take automation to a next level

This guide only covered the very beginning of what you can achieve with BugBug test coverage. Keep exploring and challenging yourself with more and more difficult test cases! Once you're comfortable with the basics, read our other in-depth guides, where we cover advanced techniques and a more strategic approach to automation testing in a startup.
If you have questions or comments on this guide, please contact us, our support team is very responsive and interested in your feedback.
Happy (automated) testing!