Beginners tutorial to automation testing
Last updated
Last updated
Automation testing (whether e2e or unit testing) is the process of using software tools to execute test script on an application, comparing the actual results to the expected outcomes. It helps save time and effort by automating repetitive tasks and allows for quicker feedback on the application's quality.
Testing process can be done in test automation tool like BugBug or Selenium. The choice of automation testing tool depends on various factors like the type of application being tested, the programming language used, test data, testing framework, the team's expertise, and the testing requirements.
This tutorial will guide you through the best practices of creating test automation with the BugBug testing tool, even if you're not a manual tester with technical knowledge. You're going to:
Learn the basics of software testing concepts and terminology
Follow the step-by-step screenshots
Understand what to automate
This is not an ultimate guide with all there is to be known about test automation. We're going to explain everything assuming you have zero technical background but you're good with computers, unlike this guy:
When software developers make changes in the code, they can break something in the app that has already been working. To make sure that nothing is broken, you need to test everything...
You just click everything in the app with your own computer and make sure there are no bugs. This is super boring and can take multiple days to complete!
It makes you feel like this:
You create step-by-step instructions for the computer to click everything for you automatically. You don't click on your own, but let the computer do the tedious work.
When you use this method you can feel like this:
You click "Run" and BugBug is clicking according to your instructions called test steps. Here's an example of an automated test where BugBug checks if the login works.
If BugBug is able to finish all the test steps, the test is marked as successfully finished & passed. It means that everything works as it should.
If BugBug is not able to finish, the test is failed. It means that something doesn't work right or your test steps are incorrect.
Keep reading to learn how to create such an automated test 💪
Create your first automated test that checks if the login works. You will create a set of step-by-step instructions for BugBug that will tell it what exactly should be clicked or entered in the text fields.
Before you continue make sure you've set up your BugBug account and have access to a project and installed the BugBug Chrome extension.
Next steps would guide us to test execution:
Create a new project
2. Name it "Posters Galore" and enter a URL that will be the starting point of your app
In our example use https://marmelab.com/react-admin-demo/
3. Create a new test
We're going to create a test that checks if login to our app works, so we will name it simply "Login". In this case we are using functional testing - a type of software testing where the application under test is checked against its functional requirements.
4. Start the recording
Your test is empty for now. That means that it has no instructions - no test steps. The URL you entered before is already pre-filled, so you can click "start recording".
5. Wait 5-10 seconds until BugBug loads your page in an incognito window
Why incognito? Testing needs to always start from a clean state. There should be no cookies or browser cache for your page before starting the test.
On the right side of the page, you can see a BugBug panel with actions dedicated to recording. We call it "recording overlay". By default, BugBug records every click and keyboard typing, but using this panel you can also record additional special actions. BugBug is providing automation scripts as you click and navigate on the website.
6. Carefully enter login credentials using "demo" for login and password fields
Why carefully? BugBug records every click, so it's better to be slow while recording and think about every click. You don't want to accidentally click in the empty background!
BugBug shows "Saved" after each click or keyboard typing. Each of these recorded actions will become a "test step".
7. Click "Sign in"
You should now see the "Posters Galore" admin panel.
8. Add an assertion
We've successfully logged in, now we need to create an instruction for BugBug that will check if the page actually has logged in. What should appear on the page that clearly tells us that the login worked? What should we assert?
Assertion = a test step that checks if something specific appears on the page, or if the page matches some specific expectations.
The simplest way of checking if the login has been successful is to check if the page shows some specific text. In our case we can assert if the text "Welcome to the react admin e-commerce demo" is visible. If it is visible everything is fine, if not, the test will fail.
Click "Assert" in the recording overlay
Click the text on the page that you want to check every time BugBug runs this test.
9. Finish recording the test
See that everything you clicked is now recorded as "test steps".
10. Run the test to see if it works
Observe how BugBug is repeating your actions.
When running the test:
Don't move your mouse cursor over the running test window.
Don't minimize the window, as Chrome may stop executing the test if it's not visible.
11. Check if the test has passed
If your test has not passed and it's failed, it can be caused be several common problems which we will cover later.
12. Close the finished test
All your test steps are green! The login works as it should.
13. Achievement unlocked!
🎉 You have successfully automated your first test. Now to check if login works, you don't need to click on your own, you can just click "Run test" and BugBug will do it for you.
Sometimes after you finished the initial recording, your test may still not pass. There might be several reasons for that:
You forgot to record the mouse "hover"
You clicked an incorrect element during the recording
Your page has dynamic content, for example, you wanted to check if your homepage shows recommended products, but these products change every time you load the page
Your page HTML is not built with automation testing in mind and uses unusual development patterns and solutions that are not industry-standard (see more advanced info on data-testid)
We're going to walk you through an example, where we record a test, but it doesn't work because we forgot to record mouse "hover". Then we're going to fix the test using the "record from here" action and add the missing hover step.
Create a new test and start recording (use the example app URL provided above)
2. Click "Utility"
3. Hover over a product and click the "Add to cart" button
Attention! This button only appears when you hover your mouse over the product. We deliberately skip recording this hover to demonstrate the consequences.
4. Record an assertion to the cart counter, checking if it shows "1"
5. Finish your recording
6. Run the test to see if it works... But it fails because we forgot to record the "hover".
Now it's time to fix the test by recording the missing "hover".
7. Click in between the test steps, just above the failed step
8. Click "Record from here"
9. Wait until BugBug reaches the right moment in your test
You can see that the test is ready for recording when the BugBug overlay shows the "REC" icon.
10. Click the "Add hover" button in the recording overlay
11. Click the product image
This tells BugBug that it should move the mouse to this element before attempting to click the "cart" icon inside it.
12. Finish your recording and confirm the recorded steps
You can see that we only added one step "hover".
13. Run the test to check if it works now
14. Achievement unlocked!
🎉 It worked! You've fixed a test with "record from here". You can use this technique for more advanced problems. Conclusion: if you see that your test fails, try to "record from here" and replace the failed steps with a new recording. It's the simplest way of fixing tests!
You might think now: "Hold on, how can I test everything? Is it possible?"
In short: no, you will never be 100% sure if everything works as it should. But your goal is to lower the risk of app users encountering a bug. You should focus on testing features that are critical for your business.
Checking if there are no new bugs is called regression testing. If you notice a bug in something that has been working before, it's called a regression.
Examples of features and consequences of their bugs:
Login
Users can't use your app
Yes
Homepage / Landing page
New clients can't register
Yes
Registration
New clients can't register
Yes
Creating new items
Users can't achieve their basic goal
Yes
Password reset
Only users who forgot their password can't access
No
Some checkbox in an obscure feature that nobody uses
Probably no-one will notice
Hell no
A list of things that you want to test is called a list of test cases. An individual test case should focus on just one process in your app.
If you're just starting with test automation, your first goal is to create about 5 to 15 tests that check your core website functionality.
Later on, you can even have hundreds of tests, checking all the conditions and less critical features, but it's better to first focus on a small number of useful tests.
Here's an example list of tests that covers the most critical paths in a web app Software-as-a-Service product.
Shorter tests are better. Write automated tests that are short, avoiding long test cases.
Why? Because shorter tests are easier to maintain and fix.
Imagine a very long test that takes 10 minutes to run. But it fails at the end. Every time you want to fix it, you would need to wait these 10 minutes until the test goes through the previous steps. Test automation is supposed to make your work faster, so avoid such long tests.
Use multiple test accounts
Don't use your own personal login and password in automated tests. Create a dedicated test user for that. Each user can have a different configuration. For example, one of your test users can be a business user, and one can be a personal user. You create separate tests for each type of login.
Make a list of your test accounts and their params and store it somewhere in a spreadsheet, for example like this:
test-automation-standard@xyz.com
qwe123qwe
Business
User with default configuration for business accounts
test-automation-trial@xyz.com
asd4rftha
Personal
Trial account
test-automation-subscribed@xyz.com
46dghbn90
Business
Paying user, with a subscription active
Split user paths to independent tests
Here's an example of a too long:
Landing page
go to landing page URL
click "features"
assert all the features
click "contact"
assert contact information
Split it to 2 independent tests:
Landing page features
go to landing page URL
click "features"
assert all the features
Landing page contact
go to landing page URL
click "contact"
assert contact information
Remember that tests need to be independent!
Tests should be independent of each other. You should not create 2 tests and then run them in a specific order. Every test needs to be self-contained. Learn more about it in our tips on atomic test cases.
A test automation framework is a set of guidelines, best practices, tools, and libraries that provide a structured approach to designing, organizing, and executing automated tests. It aims to standardize the testing process, increase test efficiency, and improve test maintenance.
That's why some features are known for being more difficult to automate. We recommend that you automate them later after you are done with the easier test cases.
Registration: it's difficult to automate because you can't register the same email twice, so you need to generate a random email and verify it afterward.
Collaboration: one user is adding something, and the second user sees the changes. You need one long test where you log in with the first user, then log out and log in again with the second user.
Dynamic lists such as search results, product listings, exchange rates, and lists that change all the time. Automation requires a good understanding of how to make the list more predictable.
Downloading files and checking their contents.
Multi-language: you can have lots of tests for different languages, but it's hard to maintain unless your app supports selectors based on data-testid attributes.
Dates and calendars: the current date changes every day, so it's not easy to record and repeat.
Test suites are groups of individual tests. They allow you to run several tests at once with one click.
By default, BugBug creates an "All tests" suite for you. All new tests will be automatically added to this suite. If you're a beginner, you don't need to set up anything else for now, this one suite should cover the basic needs.
You will need more suites later on, once you have more advanced tests that cover different testing environments.
Run in cloud = run on some other machine in a data center, not on your own computer
Run locally = run on your own computer in your own browser
Check if your tests are passed when you run them in the cloud.
Go to "Suites"
Find "All tests"
Click "Run in cloud"
Wait until the suite is finished and passed
If the tests are passed locally but fail in the cloud, please contact our support. Usually, that should never happen.
You can monitor your web app and run automated tests every hour and get notifications when your test fails. Learn more in our separate article about scheduling tests.
This guide only covered the very beginning of what you can achieve with BugBug test coverage. Keep exploring and challenging yourself with more and more difficult test cases! Once you're comfortable with the basics, read our other in-depth guides, where we cover advanced techniques and a more strategic approach to automation testing in a startup.
If you have questions or comments on this guide, please contact us, our support team is very responsive and interested in your feedback.
You might think that if you created your 5 essential automated tests, now to run them you need to open up your computer and click "Run" and block your computer for several minutes. Good news - there's no need for that! You can run tests in the cloud and even schedule them to run automatically, even when you're asleep.
Happy (automated) testing!
