---
name: benchmonkey
description: ambil output dari chatgpt -> masukkan ke file sample.json -> jalankan scripts -> ambil data dari results.md -> masukkan ke chatgpt -> ambil hasil analysis -> berikan ke boss untuk laporan
user-invocable: true
---
## Before You Start

- Akses chatgpt `https://chatgpt.com` dengan playwright ( non-headless )/ headless:false
- Masuk ke direktori `~/CIVIL_GROUP/Projects/API_CIVIL_GROUP` sebelum menjalankan `python benchmark_runner.py`

## Workflow

1. **Tentukan capability + prompt** — BOSS chat ke gw:
   ```
   "Goblin, jalanin categorize. prompt: buatin 20 input 500 kata topik=UI"
   ```

2. **Smart batching** — gw jalanin script:
   - **≤150 kata** → 1x kirim ke ChatGPT, ambil output, simpan
   - **>150 kata** → auto-pecah per batch (max 5 item/batch)
     - Contoh: 20 item × 500 kata → 4 batch × 5 item
     - Loop: kirim prompt → ambil output → append → delay 3s → batch berikutnya
   - Semua output digabung ke `~/CIVIL_GROUP/Projects/API_CIVIL_GROUP/[capability]/sample.json`

3. **Jalankan benchmark**
   ```bash
   cd ~/CIVIL_GROUP/Projects/API_CIVIL_GROUP
   python benchmark_runner.py
   ```

4. **Upload hasil** ke `ANALYZER OUTPUT AI`
   ```bash
   bash scripts/upload-to-chat.sh "ANALYZER OUTPUT AI" "tests/[capability]/results.md" "tests/[capability]/benchmark_context.md"
   ```

5. **Minta analisis** + ambil hasilnya buat laporan BOSS

## Cara Pake (Chat ke Goblin)

> **BOSS:** "Goblin, jalanin categorize. prompt: buatin 20 input 500 kata topik=UI"
>
> Gw otomatis:
> 1. Parse prompt → 500 kata > 150 → 4 batch × 5 item
> 2. Loop 4x: kirim ke ChatGPT → ambil output → append ke `categorize/sample.json`
> 3. Jalanin benchmark → upload → analisis → lapor

**Kata ≤150 langsung 1x, >150 auto-batch (max 5 item/batch):**
- 50 kata × 20 item → 1 batch × 20 item (langsung)
- 500 kata × 20 item → 4 batch × 5 item

## Cara Pake (Terminal Manual)

**Step 1 — Kirim prompt + ambil output (auto-batch):**
```bash
bash scripts/pull-from-chat.sh "GENERATOR INPUT" categorize "sample.json" "buatin 20 input 500 kata topik=UI"
```

**Step 2 — Jalanin benchmark:**
```bash
cd ~/CIVIL_GROUP/Projects/API_CIVIL_GROUP
python benchmark_runner.py
```

**Step 3 — Upload hasil:**
```bash
bash scripts/upload-to-chat.sh "ANALYZER OUTPUT AI" "tests/benchmark/results.md" "tests/benchmark/benchmark_context.md"
```

## Catatan

- Scripts ada di folder `scripts/`
- Semua script pake **persistent profile** — login ChatGPT tinggal gas ✅
- `pull-from-chat.sh`: `$1 = CHAT_NAME`, `$2 = CAPABILITY`, `$3 = OUTPUT_FILE`, `$4 = PROMPT`
- **Smart batching**: >150 kata → auto-pecah (max 5 item/batch). ≤150 kata → langsung 1x
- Delay 3 detik antar batch biara ga rate-limit
- `upload-to-chat.sh`: `$1 = CHAT_NAME`, `$2 = FILE1`, `$3 = FILE2` — pure path, no capability logic

---

### 🔧 Yang Bisa Lo Edit

**Di `pull-from-chat.sh`:**
- `BASE_DIR` — root project (default: `~/CIVIL_GROUP/Projects/API_CIVIL_GROUP`)
- `CHAT_NAME` — nama chat sidebar (default: `GENERATOR INPUT`)
- `OUTPUT_FILE` — nama file output (default: `sample.json`)

**Di `upload-to-chat.sh`:**
- `CHAT_NAME` — default chat tujuan (default: `ANALYZER OUTPUT AI`)
- `FILE1` / `FILE2` — path file yang diupload