---
name: buildkite-plugin-testing
description: >-
  Test, lint, and set up CI for Buildkite plugins using Bats, the
  buildkite/plugin-tester image, the buildkite/plugin-linter, and shellcheck.
  Use when writing or fixing .bats tests for plugin hooks, stubbing commands
  like docker/git in tests, validating plugin.yml, or creating the plugin's own
  .buildkite/pipeline.yml CI. Triggers on "bats", "plugin-tester",
  "plugin-linter", or "test a Buildkite plugin".
allowed-tools: Read, Write, Edit, Grep, Glob, Bash
---

# Buildkite plugin testing

Plugins are shell, so they're tested with **Bats** (Bash Automated Testing
System) run inside the **`buildkite/plugin-tester`** image, linted with
**`buildkite/plugin-linter`**, and statically checked with **shellcheck**.

> Verify current image tags / behavior via WebSearch on `buildkite.com`:
> - Writing/testing plugins: https://buildkite.com/docs/pipelines/integrations/plugins/writing
> - Plugin Tester: https://buildkite.com/buildkite/plugin-tester
> - Bats: https://github.com/bats-core/bats-core

## The three checks every plugin needs

1. **shellcheck** every hook script (catches unquoted vars, bad tests, etc.).
2. **plugin-linter** validates `plugin.yml` + README against your plugin id.
3. **Bats** exercises hook behavior, including failure paths.

## Running locally

The tester image mounts your plugin at `/plugin` and runs `bats tests/` by
default:

```bash
docker run --rm -v "$PWD:/plugin:ro" buildkite/plugin-tester:latest
```

Or via `docker-compose.yml` (preferred, keeps lint + tests together):

```yaml
services:
  tests:
    image: buildkite/plugin-tester:latest
    volumes: [".:/plugin:ro"]
  lint:
    image: buildkite/plugin-linter:latest
    command: ["--id", "your-org/your-plugin"]
    volumes: [".:/plugin:ro"]
```
```bash
docker-compose run --rm tests
docker-compose run --rm lint
```

## Writing good Bats tests

Bats basics:
```bash
@test "description" {
  run "$PWD/hooks/command"      # captures $status and $output
  [ "$status" -eq 0 ]
  [ "$output" = "expected" ]
  [[ "$output" == *"substring"* ]]
}
```

Rules for plugin hook tests:
- **Set the exact env vars** the hook reads (`BUILDKITE_PLUGIN_<NAME>_<KEY>`).
  Getting the names right is most of the battle — see the `buildkite-plugin-dev`
  `reference/env-var-mapping.md`.
- **Test the failure paths**: missing required settings, invalid values. Assert
  both the non-zero `$status` and the error message.
- **Cover array settings** (`_0`, `_1`) and defaults (var unset).
- **Don't call the network or real tools.** Stub them (see below).
- Use `setup()` / `teardown()` for per-test fixtures and cleanup.

### Stubbing external commands (docker, git, aws, curl)

Put a fake executable earlier on `PATH` so the hook calls your stub:

```bash
setup() {
  STUB_DIR="$(mktemp -d)"
  cat > "${STUB_DIR}/docker" <<'EOF'
#!/bin/bash
echo "docker $*"          # record the call; or branch on "$1" to fake output
EOF
  chmod +x "${STUB_DIR}/docker"
  export PATH="${STUB_DIR}:$PATH"
}
teardown() { rm -rf "${STUB_DIR:-}"; }
```

### Richer stubbing with `bats-mock` + `bats-assert` (the standard pattern)

The `buildkite/plugin-tester` image bundles `bats-support`, `bats-assert`, and
`bats-mock`, exposed via `${BATS_PLUGIN_PATH}/load.bash`. This is how mature
plugins (e.g. `docker-compose-buildkite-plugin`) test — `stub`/`unstub` assert
the *exact* commands and arguments, in sequence:

```bash
#!/usr/bin/env bats

load "${BATS_PLUGIN_PATH}/load.bash"   # bats-support + bats-assert + bats-mock
load '../lib/plugin.bash'

setup() {
  export BUILDKITE_PLUGIN_MY_PLUGIN_IMAGE="alpine:3"
  export BUILDKITE_COMMAND="echo hello"
}

@test "runs docker with the expected args" {
  # Each line is one expected invocation: "args : command-to-run-as-the-stub"
  stub docker \
    "run --rm alpine:3 /bin/sh -e -c 'echo hello' : echo ran-it"

  run "$PWD/hooks/command"

  assert_success
  assert_output --partial "ran-it"

  unstub docker          # also FAILS the test if the stub wasn't called as specified
}
```

Key points:
- `stub <cmd> "<expected-args> : <stub-body>"` — multiple strings = multiple
  expected calls, matched in order.
- `unstub <cmd>` verifies every expected call happened (and no more). Always
  `unstub` what you `stub`.
- `assert_success` / `assert_failure` / `assert_output --partial "..."` /
  `refute_output` read better than raw `[ ]` checks.
- Use `stub`/`unstub` when you must assert *how* a command was called; use the
  simple PATH-stub (above) or plain `[ ]` checks for output-only assertions.

> These tests require the tester image (or a local install of the bats helper
> libs). Pure-logic tests (e.g. `lib/plugin.bash` helpers) need none and run
> under plain `bats` — keep those separate so they're trivially runnable.

## CI: the plugin's own `.buildkite/pipeline.yml`

```yaml
steps:
  - label: ":bash: shellcheck"
    plugins:
      - shellcheck#v1.3.0:
          files: hooks/**

  - label: ":sparkles: lint plugin"
    plugins:
      - plugin-linter#v3.3.0:
          id: your-org/your-plugin

  - label: ":bats: tests"
    command: bats tests/
    plugins:
      - docker#v5.11.0:
          image: "buildkite/plugin-tester:latest"
          mount-checkout: true
```

> Plugin version tags above are examples — pin to the latest stable tags and
> confirm them in the Buildkite plugins directory before committing.

## Compiled (Go) plugins: layered testing

When the plugin is a compiled binary (see `buildkite-plugin-dev`
`reference/compiled-binary-plugins.md`), test in layers:

1. **`go test ./...`** — unit-test the real logic. Inject `getenv func(string)
   string` so config parsing is hermetic (no real environment). This is where
   most coverage should live.
2. **Bats** — smoke-test the `hooks/command` *shim*: assert it fails clearly when
   `download:false` and no binary is on PATH, and (optionally, with a stubbed
   `curl`) that it builds the right release URL. Don't hit the network.
3. **`e2e/`** (optional) — run the built binary against fixture inputs.

CI runs all of these (the scaffolded `go` style wires up a `go test` step, a
`shellcheck` step, and a bats step). Keep `gofmt`/`go vet` clean too.

## Checklist before publishing
- [ ] `shellcheck` passes on all hooks.
- [ ] (Go) `go test ./...`, `go vet`, and `gofmt -l` are clean.
- [ ] `plugin-linter` passes (`additionalProperties: false`, README documents
      every setting).
- [ ] Bats covers each setting, each default, and each failure path.
- [ ] No real network/tool calls in tests (everything stubbed).
- [ ] CI pipeline runs all three checks.