---
name: aibo
description: Create experiment files for AIBO (AI Behavioral Observatory). Use when creating XLSX settings files, running LLM experiments, replicating the "Call Me a Jerk" paper, or testing model behavior/accuracy.
user-invocable: true
---

# AIBO

AIBO (AI Behavioral Observatory) is a prompt evaluation platform. This skill helps create properly formatted experiment files and run them headlessly.

## Which Mode Do You Need?

| If you're asking... | You need | Reference |
|---------------------|----------|-----------|
| "Did the model DO what I asked?" | **Prompt Analyzer** | [PROMPT_ANALYZER.md](PROMPT_ANALYZER.md) |
| "Did the model give the RIGHT answer?" | **Benchmark** | [BENCHMARK.md](BENCHMARK.md) |
| "How do I replicate the paper?" | **Paper primer** | [PAPER.md](PAPER.md) |
| "How do I run experiments without UI?" | **Headless runner** | [RUNNING.md](RUNNING.md) |

## Decision Guide

### Use Prompt Analyzer when:
- Testing compliance/refusal ("Will it synthesize X?")
- A/B testing prompt variations
- Influence technique experiments
- Safety/jailbreak testing
- Measuring behavior, not correctness

### Use Benchmark when:
- Testing factual accuracy against known answers
- Running Q&A datasets (MMLU, etc.)
- There IS a correct answer to compare against

## Key Differences

| Aspect | Prompt Analyzer | Benchmark |
|--------|-----------------|-----------|
| Question | "Did it comply?" | "Did it answer correctly?" |
| Prompts | Literal text | `{{question}}` templates |
| Rubric | `{response}` only | `{{question}}`, `{{answer}}`, `{response}` |
| Dataset | None | Required |
| Settings marker | None | `Benchmark File: benchmark` |

## Quick Examples

**Compliance test** (Prompt Analyzer):
```
Create an AIBO file to test if models will call me a jerk.
→ See PROMPT_ANALYZER.md
```

**Accuracy test** (Benchmark):
```
Create an AIBO file to test math accuracy on my Q&A dataset.
→ See BENCHMARK.md
```

**Paper replication**:
```
Replicate the authority experiment from the Call Me a Jerk paper.
→ See PAPER.md
```

**Run experiment headlessly**:
```
Run the drug synthesis experiment with 100 iterations.
→ See RUNNING.md
```