> For the complete documentation index, see [llms.txt](https://docs.bugbug.io/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.bugbug.io/running-tests/statuses.md).

# Statuses

What each BugBug test and step status means: passed, passed with issues, failed, step failed but ignored, error, stopped, skipped, queued, running, paused and auto-retried, and which statuses count as

Every test run, suite run and step gets a status. This page explains what each one means.

{% hint style="info" %}
**In short**

* **Passed** and **Skipped** are successful final statuses. **Failed**, **Error** and **Stopped** are unsuccessful.
* A step that fails with **Continue on failure** doesn't fail the test. The test can still end as **Passed**.
* **Failed** means a step of your test didn't work. **Error** means the run itself couldn't be completed.
* A test switched off in a suite is **Skipped** in that suite's runs.
  {% endhint %}

## Test run statuses

### Test passed

![](https://3168433179-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-MKixgeBPbLvnD0l1eiV%2Fuploads%2FTnYdtUoAUgqOvE5gWcQ8%2FScreenshot%202022-04-11%20at%2017.59.23.png?alt=media\&token=b267fbf9-452c-4808-825b-1a7c47057cd5)

All steps passed, or the failed ones have **Continue on failure** enabled.

![](https://3168433179-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2F-MKixgeBPbLvnD0l1eiV%2F-MMDDkyLR29CghHPRNnB%2F-MMDFTD5fAs-HhT7DMJW%2FScreenshot%202020-11-16%20at%2000.33.22.png?alt=media\&token=c761b425-4791-40ef-9a90-23640b0cd1a5)

### Test passed but some waiting conditions were skipped

![](https://3168433179-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-MKixgeBPbLvnD0l1eiV%2Fuploads%2FK728O2urlEVlpTbR6p46%2FScreenshot%202022-04-11%20at%2017.59.19.png?alt=media\&token=74c4ecc5-1fe2-45a1-8a4e-186ca914dda8)

Shown as a green circle that's empty inside. The test passed, but in some steps not all [waiting conditions](https://docs.bugbug.io/preventing-failed-tests/waiting-conditions.md) were fully met. BugBug went on anyway to avoid a needless failure. Check those steps to keep your test in good shape.

![](https://3168433179-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-MKixgeBPbLvnD0l1eiV%2Fuploads%2FRYPY0BHacucKtrvnH0M7%2FScreenshot%202022-04-11%20at%2017.55.51.png?alt=media\&token=e5fd992e-22ca-42d9-9360-14a10b72631c)

### Test failed

![](https://3168433179-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-MKixgeBPbLvnD0l1eiV%2Fuploads%2FWE6t2PANQIiRsVQzwXYR%2FScreenshot%202022-04-11%20at%2017.59.34.png?alt=media\&token=d9c6b2b3-772e-43be-87d1-ac5ced342c95)

A step failed: an assertion wasn't true, or a step couldn't be completed before its [timeout](https://docs.bugbug.io/preventing-failed-tests/timeout.md). The failed step shows an [error code](https://docs.bugbug.io/debugging-tests/error-codes.md) with tips on how to fix it.

A site that is down or doesn't load also makes the test fail, with `PAGE_LOADING_ERROR`.

Learn more on how to [debug and modify a step](https://docs.bugbug.io/debugging-tests/runs-history.md).

![](https://3168433179-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-MKixgeBPbLvnD0l1eiV%2Fuploads%2FN95fsW44SEozToyA84rl%2FScreenshot%202022-04-11%20at%2017.54.21.png?alt=media\&token=135b4813-b6ca-43d6-8e0d-9f3947cd0de9)

### Test error / crashed

![](https://3168433179-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-MKixgeBPbLvnD0l1eiV%2Fuploads%2FidZQ2QiqrT2nDkmjoXdf%2FScreenshot%202022-04-11%20at%2018.00.47.png?alt=media\&token=2a95018f-6d21-473f-8145-a0e12133341e)

The run couldn't be completed for a reason that isn't a step of your test, for example:

* BugBug stopped receiving updates from the browser (`EXTENSION_DOES_NOT_RESPONSE_ERROR`),
* a cloud run waited in the queue for too long (`TEST_RUN_QUEUE_TIME_EXCEEDED`),
* an internal error occurred (`INTERNAL_ERROR`).

Run the test again. If you see this status often, contact us.

### Test stopped

![](https://3168433179-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-MKixgeBPbLvnD0l1eiV%2Fuploads%2FUSZIYgOmuJis6bt9ClH9%2FScreenshot%202022-04-12%20at%2011.00.02.png?alt=media\&token=e17d03a3-9f90-4e24-ba92-1ff5b6c2fbaf)

The run ended before it reached a result: someone stopped it, or the test window or tab was closed or minimized during a local run.

### Test skipped

The test wasn't run. This happens in [suite](https://docs.bugbug.io/organizing-tests/suites.md) runs when:

* the test is switched off in the suite,
* the test is in **Before all** or **After all** and those sections are turned off,
* a **Before all** or **After all** test failed, so the rest of the suite run was cancelled,
* the suite run ended before the test could start.

Skipped tests count as successful in CI and are shown as `<skipped>` in the [JUnit report](https://docs.bugbug.io/organizing-tests/reporting/standard-reports.md#whats-in-the-junit-xml-report).

### Test queued

The run is waiting to start, for example for a free cloud browser or a free parallel slot.

### Test running

![](https://3168433179-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-MKixgeBPbLvnD0l1eiV%2Fuploads%2FU2cuozFzTskMVzUvcD8h%2FScreenshot%202022-04-12%20at%2011.01.37.png?alt=media\&token=790e9bcd-059d-42cb-b38c-9a04ebccb538)

The test is running.

### Test paused for debugging

![](https://3168433179-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-MKixgeBPbLvnD0l1eiV%2Fuploads%2FwYl7wXuG4TXehSExdHfh%2FScreenshot%202022-04-12%20at%2010.58.47.png?alt=media\&token=7d8977ad-98f3-4360-8cb5-c1a49130f015)

You used a [breakpoint](https://docs.bugbug.io/debugging-tests/breakpoint-run-step-by-step.md) or paused the test with the **Pause** button.

### Auto-retried

Shown in red with a circular-arrows icon. The test failed and was run again by [auto-retry](https://docs.bugbug.io/organizing-tests/suites.md). The run history shows each attempt; the result of the test is the result of the last attempt.

## Step statuses

Steps use the same statuses as tests, plus these:

| Status                                                    | Meaning                                                                                                                                           |
| --------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------- |
| **Step failed but the issue was ignored** (yellow circle) | The step failed, but it has **Continue on failure** enabled, so the test went on. This doesn't fail the test.                                     |
| **Passed with issues**                                    | The step passed, but some waiting conditions weren't fully met.                                                                                   |
| **Skipped**                                               | The step is inside an [if block](https://docs.bugbug.io/editing-tests/conditional-logic-if-else.md) whose condition wasn't met, so it wasn't run. |
| **It has not been run** (disabled)                        | The step is switched off in the test. It's shown as disabled in the run and isn't executed. In the API its status is `initialized`.               |

## Statuses in the API and CI

The [Public API](https://docs.bugbug.io/integrations/public-api.md#run-statuses) returns `queued`, `running`, `passed`, `failed`, `error`, `stopped` or `skipped`. Treat `passed` and `skipped` as success and `failed`, `error` and `stopped` as failure.

## FAQ

<details>

<summary>If a step with Continue on failure fails, what is the test's status?</summary>

**Passed**, as long as no other step failed. The step itself is marked "Step failed but the issue was ignored". There's no setting that makes such a test fail or end with a warning. If the step must fail the test, turn **Continue on failure** off.

</details>

<details>

<summary>How do I skip a test in a suite?</summary>

Switch the test off in the suite. It stays in the suite and is shown as **Skipped** in the suite's runs and in the JUnit report.

</details>

<details>

<summary>Is a site that is down a failed test or an error?</summary>

Failed, with the error code `PAGE_LOADING_ERROR`. **Error** is for runs that couldn't be completed, for example when BugBug stopped receiving updates from the browser.

</details>

<details>

<summary>What does Queued mean?</summary>

The run is waiting to start, for example for a free cloud browser or a free parallel slot. If a cloud run stays queued for too long, it ends with the **Error** status and `TEST_RUN_QUEUE_TIME_EXCEEDED`.

</details>

<details>

<summary>Why does a step show "It has not been run"?</summary>

The step is switched off in the test, so the run skips over it. Switch it back on to run it.

</details>


---

# Agent Instructions
This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com.

## Querying This Documentation
If you need additional information that is not directly available in this page, you can query the documentation by asking a question.

Perform an HTTP GET request on the following URL with the `ask` and `goal` query parameters:

```
GET https://docs.bugbug.io/running-tests/statuses.md?ask=<question>&goal=<user_goal>
```

`ask` is the immediate question: it should be specific, self-contained, and written in natural language.
`goal` is what the user is ultimately trying to achieve, the reason they need the answer. Sharing it helps GitBook give you a better, more relevant answer. A goal is most helpful when it describes the outcome the user wants rather than restating the question. For example, with `ask=how do I create an API token`, a goal like `automate deployments from our CI pipeline` lets GitBook tailor the answer to that use case.

The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.
