> For the complete documentation index, see [llms.txt](https://docs.bugbug.io/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.bugbug.io/integrations/cli.md). # CLI Run BugBug tests and suites from the terminal and CI/CD with the BugBug CLI: install, sign in, run options, exit codes, JUnit reports, import and export, and configuration. The BugBug CLI (`@bugbug-io/cli`) runs and manages BugBug tests from the terminal. Use it to run tests or suites in CI/CD pipelines, download JUnit reports, export and import tests and projects, and install BugBug for AI agents. Use the CLI when you want ready-made commands. Use the [TypeScript SDK](/integrations/typescript-sdk.md) to build a Node.js or TypeScript integration, and the [Public API](/integrations/public-api.md) for direct HTTP calls. {% hint style="info" %} **Quick facts** * Package: `@bugbug-io/cli`, command: `bugbug`. * Requires Node.js 24 or newer and a **Pro** plan or higher. * Sign in with `bugbug login` locally, or use the project API token (`BUGBUG_API_TOKEN`) in CI. * `bugbug tests run ` and `bugbug suites run ` wait for the result and exit with `0` only when the run passed. * Runs happen in the BugBug cloud. {% endhint %} {% embed url="" %} ## Install You need [Node.js](https://nodejs.org/en/download/) 24 or newer and npm 11 or newer. ```bash npm install -g @bugbug-io/cli bugbug --help ``` Or run it without a global install: ```bash npx @bugbug-io/cli --help ``` ## Sign in ### On your own machine: browser login ```bash bugbug login ``` This opens BugBug in the browser and stores a user token in `~/.bugbug/config.yaml`. You don't need to copy a token by hand. `bugbug logout` removes it. ### In CI and scripts: project API token Use the project API token wherever a browser login isn't possible: CI/CD, shared runners, scripts. 1. Open your project in BugBug. 2. Go to **Integrations → API Token**. 3. Copy the **API Token**. The **Integrations → CLI** tab shows the same token inside a complete command you can copy. Pass it in an environment variable, which is the usual way in CI: ```bash export BUGBUG_API_TOKEN= bugbug tests list ``` or as an option: ```bash bugbug --token tests list ``` The token belongs to one project, so commands run with it always work on that project. {% hint style="warning" %} Treat the token like a password. Keep it in your CI's secret store, and don't commit it to the repository, including in `.env` or `bugbug.yaml`. {% endhint %} ### Link a directory to a project Run `bugbug init` in your repository to pin it to one BugBug project: ```bash bugbug init ``` `bugbug project init` is the same command. In a non-interactive shell, pass the token and the project ID: ```bash bugbug --token --project-id init ``` `init` writes the project ID to `bugbug.yaml` in the current directory and stores the token separately in `~/.bugbug/config.yaml`. Commands run from that directory or its subdirectories then use that project. See [bugbug.yaml](#bugbug.yaml). ## Find a test or suite ID In the app, open the test or suite, open its menu and choose **Run via CLI or webhook**. Or list them in the terminal: ```bash bugbug tests list bugbug suites list --search "checkout" ``` `bugbug tests show ` and `bugbug suites show ` print the details of one item. ## Run tests and suites ```bash bugbug tests run bugbug suites run ``` The run starts in the cloud. The CLI prints the run ID, shows status changes, and waits for the final result. | Option | What it does | | ------------------------ | --------------------------------------------------------------------------------------------------------------------------------------------- | | `-p, --profile ` | Run with this [profile](/editing-tests/variables.md), by name. Without it, the default profile is used. | | `--variable ` | Override a variable for this run only. Repeatable. The variable must already exist in the project. | | `--tag ` | Assign a [tag](/organizing-tests/tags.md) to the run. Repeatable. Missing tags are created. | | `--reporter junit` | Save a JUnit report when the run ends. The default reporter, `inline`, prints the result in the terminal. | | `--output-path ` | Where to save the JUnit report. Only with `--reporter junit`. Default: `bugbug-test--junit.xml` or `bugbug-suite--junit.xml`. | | `--timeout ` | How long to wait for the result. Default: 5 minutes for a test, 10 minutes for a suite. | | `--no-wait` | Start the run and exit right away, without waiting for the result. | ```bash bugbug suites run \ --profile "Staging" \ --variable email=qa+ci@example.com \ --tag "release-2.4" \ --reporter junit --output-path reports/bugbug.xml \ --timeout 30 ``` {% hint style="info" %} If the run takes longer than `--timeout`, the CLI stops waiting and exits with an error, but the run keeps going in the cloud. For long suites, set `--timeout` to more than the suite usually takes. {% endhint %} ### Exit codes | Code | Meaning | | ---- | ------------------------------------------------------------------------------------------------------------------------------ | | `0` | The run passed. | | `2` | The run failed. | | `3` | The run was stopped. | | `1` | Any other result or error: the run ended with an error, the wait timed out, authentication failed, or the command was invalid. | Any code other than `0` fails a CI step, so the build fails when a test fails. ## Run in CI/CD ```bash npm install -g @bugbug-io/cli bugbug --ci suites run --reporter junit --output-path report.xml ``` * Store the project API token as a secret named `BUGBUG_API_TOKEN`. Don't use `bugbug login` in CI. * `--ci` gives plain, non-interactive output. It's on automatically when the `CI` environment variable is `true`, which most CI services set. * Use `--json` to get machine-readable output instead. * Publish `report.xml` with your CI's JUnit test report step. ## Work with existing runs ```bash bugbug testruns show # result of a test run bugbug testruns logs # logs of a test run bugbug testruns stop bugbug testruns report -o report.xml # JUnit; printed to stdout without -o bugbug suiteruns show # result of a suite run bugbug suiteruns show --compact # totals and only the failed tests bugbug suiteruns stop bugbug suiteruns report -o report.xml ``` `testruns show` and `suiteruns show` also accept `--reporter junit --output-path `. ### Tags and notes ```bash bugbug tests tags set --tag flaky --tag draft bugbug suites tags set --tag nightly bugbug testruns tags set --tag "Version:1.23" bugbug suiteruns tags set --tag "Version:1.23" ``` `tags set` replaces the current tags. Leave out `--tag` to clear them. See [Tags](/organizing-tests/tags.md). ```bash bugbug testruns notes set "Failed after the staging deployment" bugbug suiteruns notes set "Nightly smoke run" bugbug testruns notes set # clears the note ``` ## Export and import ```bash bugbug tests export # YAML, saved as bugbug-test-.yaml bugbug tests export --format zip -o test.zip bugbug project export -o bugbug-project.zip ``` ```bash bugbug tests import test.yaml --conflict-mode overwrite bugbug tests import test.zip --folder-id bugbug project import bugbug-project.zip ``` | Option | What it does | | --------------------------- | ---------------------------------------------------------------------------------- | | `--conflict-mode overwrite` | If a test or component with the same slug already exists, update it. | | `--conflict-mode skip` | If it already exists, leave it unchanged. | | `--folder-id ` | Put the imported test in this test folder. Overrides the folder recorded in a ZIP. | * Without `--conflict-mode`, a test whose slug already exists isn't imported. Always set it in scripts. * The file must end in `.yaml`, `.yml` or `.zip`. * `project import` always creates a new project. For the file format, ZIP layout and how slugs match existing items, see [YAML schema](/export-import-and-backup/yaml-schema.md) and [Importing](/export-import-and-backup/importing.md). {% hint style="warning" %} Import changes project data. Review exported or generated files before you import them. {% endhint %} ## Use the CLI with AI agents The BugBug Plugin is the recommended way to connect an AI coding agent to BugBug. It installs [BugBug MCP](/ai-testing/mcp.md) and [BugBug Skills](/ai-testing/skills.md) together: ```bash npx @bugbug-io/cli plugin --agent= ``` Supported `--agent` values are `cursor`, `claude`, `vscode`, `codex` and `copilot`. Add `--dry-run` to see what would be installed without writing files. The plugin installer has no `--token` option: the AI client signs in to BugBug MCP on its own. Read more: [BugBug Plugin](/ai-testing/plugin.md). ## Configuration The CLI looks for the token, the project and the API address in this order, and uses the first value it finds: 1. Command-line options: `--token`, `--project-id`. 2. Environment variables: `BUGBUG_API_TOKEN`, `BUGBUG_PROJECT_ID`, `BUGBUG_API_URL`. 3. `bugbug.yaml` in the current directory or a parent directory. 4. `.env` in the current directory or a parent directory. 5. The global config `~/.bugbug/config.yaml`, written by `bugbug login` and `bugbug init`. The [TypeScript SDK](/integrations/typescript-sdk.md) uses the same order, with its constructor options in place of command-line options. | Environment variable | Description | | -------------------- | ----------------------------------------------------------------------------------------------- | | `BUGBUG_API_TOKEN` | API token used to authenticate. | | `BUGBUG_PROJECT_ID` | Project ID. Needed with a user token from `bugbug login` when no project is linked or selected. | | `BUGBUG_TELEMETRY` | `false` turns telemetry off, `true` turns it on. Overrides `telemetry` in `bugbug.yaml`. | | `DO_NOT_TRACK` | Set to `1` or `true` to turn telemetry off. | | `CI` | When `true`, forces plain, non-interactive output. | ### bugbug.yaml `bugbug.yaml` links a directory, usually a repository, to one BugBug project. `bugbug init` creates it, and you can also write it yourself: ```yaml projectId: '' ``` * **Where it's read from.** The CLI, the [TypeScript SDK](/integrations/typescript-sdk.md) and BugBug Skills look for `bugbug.yaml` in the current directory, then in each parent directory, and use the first one they find. * **Order.** A value from `bugbug.yaml` is used only when there's no command-line option or environment variable for it. See the list above. * **Commit it.** `bugbug.yaml` from `bugbug init` contains only the project ID, so you can commit it, and everyone who clones the repository works with the same project. `init` stores the token in `~/.bugbug/config.yaml`, not in this file. * **When it's there, the project is fixed.** In the interactive navigator, switching projects isn't offered in a directory with `bugbug.yaml`. #### Fields | Field | What it does | | ----------- | --------------------------------------------------------------------------------------------------------------------------------------------- | | `projectId` | The project the directory works with. | | `telemetry` | `false` turns off the CLI's anonymous telemetry (usage analytics and error reports) for this directory. Default: on. Read only by the CLI. | | `logLevel` | `debug` turns on verbose output, like `--verbose`. | | `apiUrl` | Overrides the BugBug API address. You don't need it with BugBug's cloud. | | `token` | API token. Supported, but don't use it in a file you commit: keep the token in an environment variable or in `~/.bugbug/config.yaml` instead. | Other keys are ignored. ```yaml projectId: '' telemetry: false ``` #### Telemetry The CLI sends anonymous telemetry by default. It decides whether telemetry is on in this order, and the first setting it finds wins: 1. `--disable-telemetry` turns it off. 2. `DO_NOT_TRACK=1` or `DO_NOT_TRACK=true` turns it off. 3. `BUGBUG_TELEMETRY=false` or `true`. 4. `telemetry` in `bugbug.yaml`. 5. On. ### Global options | Option | Description | | ------------------------------ | ------------------------------------------------------------- | | `-t, --token ` | API token. Overrides every other source. | | `-p, --project-id ` | Project ID, when the token isn't tied to one project. | | `-v, --verbose` | Verbose output, including HTTP requests. | | `--ci` | Plain, non-interactive output for CI. | | `--json` | JSON output. | | `--disable-telemetry` | Disable telemetry. | | `-h, --help` | Show help. Works on every command: `bugbug tests run --help`. | ## All commands Run `bugbug` without arguments in a terminal to open the interactive navigator. | Command | Description | | ------------------------------------------------------ | --------------------------------------------------------------------------------------------------------------------------- | | `bugbug login` / `bugbug logout` | Sign in with the browser / remove the stored login. | | `bugbug init` | Link the current directory to a project. Same as `bugbug project init`. | | `bugbug tests list` / `show ` | List tests / show one test. | | `bugbug tests run ` | Run a test. | | `bugbug tests export ` | Export a test as YAML or ZIP. | | `bugbug tests import ` | Import a test from YAML or ZIP. | | `bugbug tests tags set ` | Replace a test's tags. | | `bugbug suites list` / `show ` | List suites / show one suite. | | `bugbug suites run ` | Run a suite. | | `bugbug suites tags set ` | Replace a suite's tags. | | `bugbug testruns show/logs/stop/report ` | Show, get logs, stop, or download the JUnit report of a test run. | | `bugbug testruns tags set` / `notes set ` | Set a test run's tags / note. | | `bugbug suiteruns show/stop/report ` | Show, stop, or download the JUnit report of a suite run. | | `bugbug suiteruns tags set` / `notes set ` | Set a suite run's tags / note. | | `bugbug profiles list` | List run profiles. | | `bugbug tags list` | List project tags with colors and usage counts. | | `bugbug files list` / `upload ` | List files in the [Files](/organizing-tests/files.md) library / upload a file (max 10 MB, optional `--folder `). | | `bugbug files rename ` | Rename a file. | | `bugbug files move ` | Move a file with `--folder ` or `--root`. | | `bugbug files download ` / `delete ` | Download / delete a file. | | `bugbug project settings` | Show project settings. | | `bugbug project export` / `import ` | Export the project as ZIP / import a project ZIP. | | `bugbug plugin` | Install the BugBug Plugin for AI agents. | Shorthands: | Shorthand | Same as | | ----------------------------------------------------- | ---------------------------------------------- | | `bugbug init` | `project init` | | `bugbug run test ` / `run suite ` | `tests run` / `suites run` | | `bugbug list test` / `suite` / `profile` | `tests list` / `suites list` / `profiles list` | | `bugbug stop test ` / `stop suite ` | `testruns stop` / `suiteruns stop` | | `bugbug logs test ` | `testruns logs` | | `bugbug export test ` / `export project` | `tests export` / `project export` | | `bugbug import test ` / `import project ` | `tests import` / `project import` | ## FAQ
Where do I find the API token for the CLI, and is it per project or per user? In **Integrations → API Token** in your project. The **Integrations → CLI** tab shows it inside a ready-to-copy command. It belongs to the project, not to a user. On your own machine you can skip it and use `bugbug login`, which stores a user token instead.
What is bugbug.yaml, and can I commit it? It links a directory to one BugBug project with its `projectId`. `bugbug init` creates it with only the project ID, so you can commit it. Keep tokens out of it. See [bugbug.yaml](#bugbug.yaml).
How do I turn off CLI telemetry? Add `telemetry: false` to `bugbug.yaml`, set `BUGBUG_TELEMETRY=false` or `DO_NOT_TRACK=1`, or pass `--disable-telemetry`. See [Telemetry](#telemetry).
I used bugbug config set-token and bugbug remote run. What's the equivalent now? Those commands come from `@testrevolution/bugbug-cli`, which is deprecated. Install `@bugbug-io/cli` instead. The equivalents are: * `bugbug config set-token ` → `bugbug login`, `bugbug init`, or the `BUGBUG_API_TOKEN` variable. * `bugbug remote run test ` → `bugbug tests run `. * `bugbug remote run suite ` → `bugbug suites run `. * `--result-timeout ` → `--timeout `.
Why does my CI job stop waiting before the suite finishes? By default the CLI waits 5 minutes for a test and 10 minutes for a suite. Set `--timeout ` to wait longer. The run itself keeps going in the cloud.
Which plan do I need for the CLI? Pro or higher.
How do I import a test that already exists in the project? Add `--conflict-mode overwrite` to update the existing test, or `--conflict-mode skip` to leave it unchanged. BugBug matches tests and components by slug. See [Importing](/export-import-and-backup/importing.md).
--- # Agent Instructions This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com. ## Querying This Documentation If you need additional information that is not directly available in this page, you can query the documentation by asking a question. Perform an HTTP GET request on the following URL with the `ask` and `goal` query parameters: ``` GET https://docs.bugbug.io/integrations/cli.md?ask=&goal= ``` `ask` is the immediate question: it should be specific, self-contained, and written in natural language. `goal` is what the user is ultimately trying to achieve, the reason they need the answer. Sharing it helps GitBook give you a better, more relevant answer. A goal is most helpful when it describes the outcome the user wants rather than restating the question. For example, with `ask=how do I create an API token`, a goal like `automate deployments from our CI pipeline` lets GitBook tailor the answer to that use case. The response will contain a direct answer to the question and relevant excerpts and sources from the documentation. Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.