--- name: jmeter-load-testing description: "Authors Apache JMeter `.jmx` test plans (Thread Groups + HTTP samplers + assertions + listeners) in the JMeter GUI, runs them headlessly via `jmeter -n -t plan.jmx -l results.jtl`, generates an HTML dashboard with `-e -o`, and gates CI on JTL parsing. Use when the project has an existing JMeter investment, needs JVM-native load tooling, or works in domains with strong JMeter community support (banking, telecom, enterprise)." rating: 25 d6: 4 --- # jmeter-load-testing ## Overview Apache JMeter is the long-running JVM-native load-testing tool. Tests are XML `.jmx` files authored in JMeter's GUI; CI runs them headless via the `jmeter` CLI ([jmeter-getstarted][getstarted]). [getstarted]: https://jmeter.apache.org/usermanual/get-started.html The official guidance is explicit: > "GUI mode should only be used for creating the test script, CLI mode > (NON GUI) must be used for load testing." ([jmeter-getstarted][getstarted]) This skill covers the CLI / CI side. Authoring is GUI-driven and out of scope here - see the JMeter user manual for the Thread Group / HTTP Sampler / Assertion authoring flow. ## When to use - The project already has `.jmx` test plans. - The team is on the JVM and prefers JMeter's mature ecosystem (BlazeMeter, Taurus, plugin manager, distributed mode). - A specific protocol is needed that JMeter has first-class support for: JDBC, JMS, MQTT, FTP, LDAP, SOAP - k6's plugin landscape is weaker for niche protocols. If the team is starting fresh, evaluate [`k6-load-testing`](../k6-load-testing/SKILL.md) (developer-friendly JS), [`gatling-load-testing`](../gatling-load-testing/SKILL.md) (JVM DSL), or [`locust-load-testing`](../locust-load-testing/SKILL.md) (Python) before adopting JMeter - XML authoring has a steep learning curve. ## Install JMeter requires **Java 8 or higher** with `JAVA_HOME` set. Download the latest release from [jmeter.apache.org](https://jmeter.apache.org/) and extract - there is no installer ([jmeter-getstarted][getstarted]). For Docker-based CI, official images are at `apache/jmeter`. Pin to a specific tag rather than `latest`. ## Running ### Canonical CLI invocation Per [jmeter-getstarted][getstarted]: ```bash jmeter -n -t test.jmx -l results.jtl ``` | Flag | Purpose | |------|---------------------------------------------------------| | `-n` | Non-GUI mode. **Required for load tests.** | | `-t` | Path to the `.jmx` test plan. | | `-l` | Output file for raw sample results (JTL - CSV-shaped). | ### With HTML dashboard Per [jmeter-getstarted][getstarted]: ```bash jmeter -n -t test.jmx -l results.jtl -e -o report_folder ``` | Flag | Purpose | |------|----------------------------------------------------------| | `-e` | Generate the HTML dashboard report automatically. | | `-o` | Output folder (must be empty or non-existent). | The dashboard shows percentiles, throughput, error rates, response- time graphs, and per-sampler breakdowns - the canonical JMeter output for human review. ### Other useful flags | Flag | Purpose | |-----------------------------------|---------------------------------------------------------| | `-J=` | Override a JMeter property at the JVM level. | | `-G=` | Override a property in distributed (remote) mode. | | `-Jjmeter.save.saveservice.output_format=csv` | Force CSV JTL (vs. XML). | | `-q ` | Additional properties file. | | `-r` | Start the test on remote slaves (distributed mode). | | `-X` | Exit JMeter when the test finishes. | For multi-environment runs, parameterize the test plan with `${__P(api.base.url, default)}` and override at the CLI: ```bash jmeter -n -t orders.jmx -l results.jtl \ -Japi.base.url=https://staging.example.com \ -Japi.token=$API_TOKEN ``` ## Parsing results The JTL file is the structured artifact for CI gating. Default JTL columns (CSV): `timeStamp`, `elapsed`, `label`, `responseCode`, `responseMessage`, `threadName`, `dataType`, `success`, `failureMessage`, `bytes`, `sentBytes`, `grpThreads`, `allThreads`, `URL`, `Latency`, `IdleTime`, `Connect`. Quick-and-dirty pass/fail with `awk`: ```bash # Count errors awk -F',' 'NR>1 && $8=="false"' results.jtl | wc -l # Compute mean response time awk -F',' 'NR>1 { sum+=$2; n++ } END { print sum/n }' results.jtl ``` For richer parsing, use the JMeter HTML report's `statistics.json` under the report folder - it contains the percentiles per sampler in JSON form. ## CI integration ```yaml # .github/workflows/jmeter.yml name: load-test on: pull_request: paths: ['tests/load/**.jmx'] schedule: - cron: '0 4 * * *' jobs: jmeter: runs-on: ubuntu-latest steps: - uses: actions/checkout@v5 - name: Run JMeter via Docker env: API_TOKEN: ${{ secrets.STAGING_API_TOKEN }} run: | docker run --rm \ -v "$PWD:/work" \ -w /work \ apache/jmeter \ -n -t tests/load/orders.jmx \ -l results.jtl \ -e -o report \ -Japi.token=$API_TOKEN \ -Japi.base.url=https://staging.example.com - name: Pass/fail gate run: | ERRORS=$(awk -F',' 'NR>1 && $8=="false"' results.jtl | wc -l) if [ "$ERRORS" -gt 10 ]; then echo "::error::Got $ERRORS errors (>10 threshold)" exit 1 fi - name: Upload report if: always() uses: actions/upload-artifact@v4 with: name: jmeter-report path: | report/ results.jtl retention-days: 14 ``` The Docker invocation pattern keeps the CI runner clean - no Java / JMeter install on the runner. ## Anti-patterns | Anti-pattern | Why it fails | Fix | |-----------------------------------------------------------|----------------------------------------------------------------------|-----| | Running tests in GUI mode "just for this one quick run" | GUI runner adds overhead; metrics are skewed; per [jmeter-getstarted][getstarted] this is explicitly forbidden for load tests. | Always `-n`. GUI is for authoring only. | | Hard-coding URLs / tokens in the `.jmx` XML | Test plan binds to one environment. | Parameterize: `${__P(api.base.url, ...)}`; override with `-J`. | | Saving JTL in XML format | XML JTL files are 5-10x larger than CSV; slow to parse. | Force CSV with `-Jjmeter.save.saveservice.output_format=csv`. | | Listeners enabled in CI runs | UI listeners (View Results Tree, Aggregate Report) consume RAM proportional to sample count; OOMs at scale. | Disable all listeners in `.jmx`; emit JTL only; generate HTML report post-run via `-e -o`. | | One mega-test-plan with 100 samplers | Failure attribution is hard; runtime dominated by one slow endpoint. | Split into per-domain `.jmx` files; run as separate CI jobs. | | Threshold gates only on error rate | A 30-second response that succeeds passes the rate gate but breaks UX. | Pair error-rate with percentile gates parsed from `statistics.json`. | ## Limitations - **XML authoring.** `.jmx` files are not human-friendly; reviews of test-plan changes are awkward. - **Memory hungry.** JMeter's per-sample memory overhead is high vs. k6 / Gatling. - **Single-machine VU limits.** ~1000 threads per machine before JVM contention; for higher loads, use distributed mode (master + remote slaves) or move to a tool with a smaller per-VU footprint. - **Plugin ecosystem fragmentation.** Many JMeter plugins are abandoned; vet plugin maintenance status before depending on one. ## References - [jmeter-getstarted][getstarted] - canonical CLI flags, GUI-vs-CLI guidance, dashboard generation. - Apache JMeter user manual - https://jmeter.apache.org/usermanual/ - [`k6-load-testing`](../k6-load-testing/SKILL.md), [`gatling-load-testing`](../gatling-load-testing/SKILL.md), [`locust-load-testing`](../locust-load-testing/SKILL.md) - alternatives by stack. - [`perf-budget-gate`](../perf-budget-gate/SKILL.md) - downstream gate aggregating multiple runner verdicts.