Links

Beginners tutorial to automation testing

Super-detailed instructions describing every step in layman's terms with screenshots for every step.

How to use BugBug?

This tutorial will guide you through the process of creating test automation with the BugBug tool, even if you're not a technical person. You're going to:
  • Learn the basics of software testing concepts and terminology
  • Follow the step-by-step screenshots
  • Understand what to automate
We're going to explain everything assuming you have zero technical background but you're good with computers, unlike this guy:

Manual testing vs. automated testing

When software developers make changes in the code, they can break something in the app that has already been working. To make sure that nothing is broken, you need to test everything...

Manual testing

You just click everything in the app with your own computer and make sure there are no bugs. This is super boring and can take multiple days to complete!
It makes you feel like this:

Automated testing

You create step-by-step instructions to the computer to automatically click everything for you. You don't click on your own, but let the computer do the boring work.
When you use this method you can feel like this:
You click "Run" and BugBug is clicking according to your instructions called test steps. Here's an example of an automated test where BugBug checks if login works.
If BugBug is able to finish all the test steps, the test is marked as successfully finished & passed. It means that everything works as it should.
If BugBug is not able to finish, the test is failed. It means that something doesn't work right or your test steps are incorrect.
Keep reading to learn how to create such an automated test 💪

Create your first test via "recording"

Create your first automated test that checks if login works. You will create a set of step-by-step instructions to BugBug that will tell it what exactly should be clicked or entered in the text fields.
Before you continue make sure you've set up your BugBug account and have access to a project and installed BugBug Chrome extension.
Use the example app to follow each step on your own
  1. 1.
    Create a new project
2. Name it "Posters Galore" and enter a URL that will be the starting point of your app
3. Create a new test.
We're going to create a test that checks if login to our app works, so we will name it simply "Login".
4. Start the recording
Your test is empty for now. That means that it has no instructions - no test steps. The URL you entered before is already pre-filled, so you can just click "start recording".
5. Wait 5-10 seconds until BugBug loads your page in an incognito window.
Why incognito? Automated tests need to always start from a clean state. There should be no cookies or browser cache for your page before starting the test.
On the right side of the page you can see a BugBug panel with actions dedicated to recording. We call it "recording overlay". By default BugBug records every click and keyboard typing, but using this panel you can also record additional special actions.
6. Carefully enter login credentials using "demo" for login and password fields.
Why carefully? BugBug records every click, so it's better to be slow while recording and think about every click. You don't want to accidentaly click in the empty backgorund!
BugBug shows "Saved" after each click or keyboard typing. Each of these recorded actions will become a "test step".
7. Click "Sign in"
You should now see the "Posters Galore" admin panel.
8. Add an assertion
We've successfully logged in, now we need to create an instruction for BugBug that will check if the page actually has logged in. What should appear on the page that clearly tells us that the login worked? What should we assert?
Assertion = a test step that checks if something specific appears on the page, or the page matches some specific expectations.
The simplest way of checking if the login has been successful is to check if the page shows some specific text. In our case we can assert if text "Welcome to the react admin e-commerce demo" is visible. If it is visible everything is fine, if not, the test will fail.
Click "Assert" in the recording overlay
Click the text on the page that you want to check every time BugBug runs this test.
9. Finish recording the test
See that everything you clicked is now recorded as "test steps"
10. Run the test to see if it works
Observe how BugBug is repeating your actions.
When running the test:
  • Don't move your mouse cursor over the running test window
  • Don't minimize the window, as Chrome may stop executing the test if it's not visible
10. Check if the test has passed
If your test has not passed and it's failed, it can be caused be several common problems which we will cover later.
11. Close the finished test
All your test steps are green! The login works as it should.
12. Achievement unlocked!
🎉 You have successfully automated your first test. Now to check if login works, you don't need to click on your own, you can just click "Run test" and BugBug will do it for you.

Test fails? Fix it with "record from here".

Sometimes after you finished the initial recording, your test may still not pass. There might be several reasons for that:
  • You forgot to record mouse "hover"
  • You clicked an incorrect element during the recording
  • Your page has dynamic content, for example you wanted to check if your homepage shows recommended products, but these products change every time you load the page
  • Your page HTML is not built with automation testing in mind, uses unusual development patterns and solutions that are not industry-standard (see more advanced info on data-testid)
We're going to walk you through an example, where we record a test, but it doesn't work because we forgot to record mouse "hover". Then we're going to fix the test using "record from here" action and add the missing hover step.
Use the example app to follow each step on your own
  1. 1.
    Create a new test and start recording (use the example app URL provided above)
2. Click "Utility"
3. Hover over a product and click the "Add to cart" button
Attention! This button only appears on when you hover your mouse over the product. We deliberately skip recording this hover to demonstrate the consequences.
4. Record an assertion to the cart counter, checking if it shows "1"
5. Finish your recording
6. Run the test to see if it works... But it fails, because we forgot to record the "hover".
Now it's time to fix the test by recording the missing "hover".
7. Click in-between the test steps, just above the failed step
8. Click "Record from here"
9. Wait until BugBug reaches the right moment in your test
You can see that the test is ready for recording when BugBug overlay shows the "REC" icon
10. Click the "Add hover" button in the recording overlay
11. Click the product image
This tells BugBug that it should move mouse to this element before attempting to click the "cart" icon inside it.
12. Finish your recording and confirm the recorded steps
You can see that we only added one step "hover".
13. Run the test to check if it works now
14. Achievement unlocked!
🎉 It worked! You've fixed a test with "record from here". You can use this technique for more advanced problems. Conclusion: if you see that your test fails, try to "record from here" and replace the failed steps with a new recording. It's the simplest way of fixing tests!

What to test?

You might think now: "Hold on, how can I test everything? Is it possible?"
In short: no, you will never be 100% sure if everything works as it should. But your goal is to lower the risk of app users encountering a bug. You should focus on testing features that are critical for your business.
Checking if there are no new bugs is called regression testing. If you notice a bug in something that has been working before, it's called a regression.
Examples of features and consequences of their bugs:
Feature
What if it breaks:
Is it critical?
Login
Users can't use your app
Yes
Homepage / Landing page
New clients can't register
Yes
Registration
New clients can't register
Yes
Creating new items
Users can't achieve their basic goal
Yes
Password reset
Only users who forgot their password can't access
No
Some checkbox in an obscure feature that nobody uses
Probably no-one will notice
Hell no
A list of things that you want to test is called a list of test cases. An individual test case should focus on just one process in your app.

How many tests should you create?

If you're just starting with test automation, your first goal is to create about 5 to 15 tests that check your core website functionality.
Later on, you can even have hundreds of tests, checking all the conditions and less critical features, but it's better to first focus on small number of useful tests.
Here's an example list of tests that covers the most critical paths in a web app Software-as-a-Service product.

How many steps should a test have?

Shorter tests are better. It's better to have more tests that are short, avoiding long tests.
Why? Because shorter tests are easier to maintain and fix. Imagine a very long test that takes 10 minutes to run. But it fails at the end. Every time you want to fix it, you would need to wait these 10 minutes until the test goes through previous steps. Test automation is supposed to make your work faster, so avoid such long tests.

How to make shorter tests?

Use multiple test accounts
Don't use your own personal login and password in automated tests. Create a dedicated test users for that. Each user can have a different configuration. For example, one of your test users can be a business user, and one can be a personal user. You create separate tests for each type of login.
Make a list of your test accounts and their params and store it somewhere in a spreadsheet, for example like this:
Email
Password
Type
Parameters
qwe123qwe
Business
User with default configuration for business accounts
asd4rftha
Personal
Trial account
46dghbn90
Business
Paying user, with a subscription active
Split user paths to independent tests
Here's an example of a too long:
Test name
Steps
Landing page
go to landing page URL
click "features"
assert all the features
click "contact"
assert contact information
Split it to 2 independent tests:
Test name
Steps
Landing page features
go to landing page URL
click "features"
assert all the features
Landing page contact
go to landing page URL
click "contact"
assert contact information
Remember that tests need to be independent!
Tests should be independent of each other. You should not create 2 tests and then run them in a specific order. Every test needs to be self-contained. Learn more about it in our tips on atomic test cases.

What not to test?

Some features are known for being more difficult to automate. We recommend that you automate them later, after you are done with the easier test cases.
  • Registration: it's difficult to automate because you can't register the same email twice, so you need to generate a random email and verify it afterwards
  • Collaboration: one user is adding something, second user sees the changes. You need one long test where you log in with the first user, then log out and log in again with the second user.
  • Dynamic lists such as search results, product listings, exchange rates, lists that change all the time. Automation requires good understanding of how to make the list more predictable
  • Downloading files and checking their contents.
  • Multi-language: you can have lots of tests for different languages, but it's hard to maintain unless your app supports selectors based on data-testid attributes.
  • Dates and calendars: the current date changes every day, so it's not easy to record and repeat

Suites: when do you need them?

Test suites are groups of individual tests. They allow you to run several tests at once with one click.
By default BugBug creates an "All tests" suite for you. All new tests will be automatically added to this suite. If you're a beginner, you don't need to set up anything else for now, this one suite should cover the basic needs.
You will need more suites later on, once you have more advanced tests that cover different testing environments.

Run tests in the cloud

You might think that if you created your 5 essential automated tests, now to run them you need to open up your computer and click "Run" and block your computer for several minutes. Good news - there's no need for that! You can run tests in the cloud and even schedule them to run automatically, even when you're asleep.
🪄
Run in cloud = run on some other machine in a data center, not on your own computer
Run locally = run on your own computer in your own browser
Check if your tests are passed when you run them in the cloud.
  1. 1.
    Go to "Suites"
  2. 2.
    Find "All tests"
  3. 3.
    Click "Run in cloud"
  4. 4.
    Wait until the suite is finished and passed
If the tests are passed locally, but fail in the cloud, please contact our support. Usually that should never happen.

Schedule automated tests

You can monitor your web app and run automated tests every hour and get notifications when your test fail. Learn more in our separate article about scheduling tests.

Take automation to a next level

This guide only covered the very beginning of what you can achieve with BugBug. Keep exploring and challenging yourself with more and more difficult test cases! Once you're comfortable with the basics, read our other in-depth guides, where we cover advanced techniques and more strategic approach to automation testing in a startup.
If you have questions or comments to this guide, please contact us, our support team is very responsive and interested in your feedback.
Happy testing!
🍻