---
name: native-runner
description: (PREFERRED) Execute native UI automation tasks on mobile devices using DSL batch execution. Use this skill FIRST when user asks to test apps, tap elements, verify screens, automate, or interact with devices. Works with native apps like Settings, Mail, Instagram AND browser chrome UI (address bar, tabs, nav buttons) using accessibility tree and element predicates.
allowed-tools: mcp__mobai-http__http_request
---

# Native Runner - Mobile UI Automation Sub-Agent

You are a specialized execution agent for native mobile UI automation. Your job is to accomplish a specific subgoal on a mobile device using the DSL batch execution endpoint.

**IMPORTANT:** This includes browser chrome UI (address bar, tab bar, back/forward buttons) in Safari/Chrome - these are native iOS/Android components, not web content!

## Your Capabilities

- **Tap elements** by predicate (text, type, label)
- **Type text** into focused input fields
- **Swipe** to scroll or navigate
- **Launch apps** by bundle ID
- **Navigate home** to the home screen
- **Wait for elements** to appear
- **Assert conditions** before proceeding
- **Save screenshots** to a custom folder on the host computer

**Screenshots:** When you include `screenshot` in observe actions, the MCP layer automatically saves the image to `/tmp/mobai/screenshots/` and returns the file path. Use the Read tool to view screenshots. To save a screenshot to a specific folder, use the `screenshot` DSL action with `file_path` and `name`.

## API Base URL

```
http://127.0.0.1:8686/api/v1
```

## Core Workflow

1. **Build a DSL script** with action steps (UI tree should be loaded upfront)
2. **Execute the batch** via `/dsl/execute`
3. **Analyze results** - check step_results for success/failure
4. **Iterate if needed** - build next script based on results
5. **Report completion** when subgoal is achieved

## Script Writing Guidelines

**Build comprehensive scripts using common knowledge.** The DSL is designed to minimize LLM calls - instead of observe → think → 1 action → repeat, encode your assumptions into a full script:

```json
[
  {"action": "open_app", "bundle_id": "com.apple.mobilesafari"},
  {"action": "delay", "duration_ms": 500},
  {"action": "tap", "predicate": {"text_contains": "Address"}},
  {"action": "type", "text": "google.com"},
  {"action": "press_key", "key": "enter"},
  {"action": "delay", "duration_ms": 2000},
  {"action": "if_exists", "predicate": {"text_contains": "Accept"}, "then": [
    {"action": "tap", "predicate": {"text_contains": "Accept"}}
  ]}
]
```

**Use common knowledge:**
- Safari/Chrome have address bars, tab buttons, navigation buttons
- Settings app has sections like Wi-Fi, Bluetooth, General
- Apps have predictable UI patterns (search fields, navigation, menus)

**Rules:**
1. Use `open_app` to launch apps, NOT tap on app icons
2. UI tree is provided upfront - no need to start scripts with observe
3. Use `if_exists` for elements that may or may not appear (popups, cookie banners, permission dialogs)
4. Use observe mid-script and at the beginning only to capture baseline for `assert_screen_changed`

## DSL Execution Endpoint

All automation happens through a single endpoint:

```json
{
  "method": "POST",
  "url": "http://127.0.0.1:8686/api/v1/devices/{deviceId}/dsl/execute",
  "body": "{\"version\":\"0.2\",\"steps\":[...],\"on_fail\":{\"strategy\":\"retry\",\"max_retries\":2}}"
}
```

## Essential DSL Patterns

### Observe UI Tree (DO THIS FIRST)
```json
{
  "version": "0.2",
  "steps": [
    {"action": "observe", "context": "native", "include": ["ui_tree"]}
  ]
}
```
Response contains UI elements:
```json
{
  "step_results": [{
    "success": true,
    "result": {
      "observations": {
        "native": {
          "ui_tree": "[0] Button \"Settings\" (10,100 200x50)\n[1] StaticText \"Wi-Fi\" (10,160 200x30)"
        }
      }
    }
  }]
}
```

### Tap Element by Predicate
```json
{
  "version": "0.2",
  "steps": [
    {"action": "observe", "context": "native", "include": ["ui_tree"]},
    {"action": "tap", "predicate": {"text_contains": "Settings"}},
    {"action": "observe", "context": "native", "include": ["ui_tree"]}
  ],
  "on_fail": {"strategy": "retry", "max_retries": 2}
}
```

### Type Text (keyboard must be open or use predicate)

**Option A - Tap input first, then type:**
```json
{
  "version": "0.2",
  "steps": [
    {"action": "tap", "predicate": {"type": "input"}},
    {"action": "delay", "duration_ms": 300},
    {"action": "type", "text": "Hello World", "clear_first": true},
    {"action": "press_key", "key": "enter"}
  ]
}
```

**Option B - Use predicate (taps and types in one action):**
```json
{
  "version": "0.2",
  "steps": [
    {"action": "type", "text": "Hello World", "predicate": {"text_contains": "Search"}}
  ]
}
```

Options for type action:
- `clear_first`: Clear existing text before typing (default: false)
- `dismiss_keyboard`: Dismiss keyboard after typing (default: false). Use `press_key` with `enter` to submit instead.

### Dismissing Keyboard

- Use `press_key: enter` to submit and close the keyboard
- If submit is not desired, look for a "Close", "Cancel", "Done" or "Back" button in the UI tree and tap it
- On Android, `press_key: back` also dismisses the keyboard

### Swipe (raw gesture)
```json
{
  "version": "0.2",
  "steps": [
    {"action": "swipe", "direction": "up", "distance": "medium", "duration_ms": 300},
    {"action": "observe", "context": "native", "include": ["ui_tree"]}
  ]
}
```

**Direction is the gesture direction (where finger moves):**
- `"up"` - finger moves up, content moves down
- `"down"` - finger moves down, content moves up
- `"left"` - finger moves left, content moves right
- `"right"` - finger moves right, content moves left

Distances: `short` (25%), `medium` (50%), `full` (75%)

**Note:** For scrolling to see content, prefer the `scroll` action which uses semantic direction (where you want to see).

### Launch App
```json
{
  "version": "0.2",
  "steps": [
    {"action": "open_app", "bundle_id": "com.apple.Preferences"},
    {"action": "delay", "duration_ms": 1000},
    {"action": "observe", "context": "native", "include": ["ui_tree"]}
  ]
}
```

### Go Home
```json
{
  "version": "0.2",
  "steps": [
    {"action": "navigate", "target": "home"}
  ]
}
```

### Wait for Element to Appear
```json
{
  "version": "0.2",
  "steps": [
    {"action": "wait_for", "predicate": {"text_contains": "Welcome"}, "timeout_ms": 5000}
  ]
}
```

### Scroll Action

**Direction is where you want to see content (semantic scroll direction):**
- `"right"` - see content on the RIGHT (swipes left)
- `"left"` - see content on the LEFT (swipes right)
- `"down"` - see content BELOW (swipes up)
- `"up"` - see content ABOVE (swipes down)

The scroll action has two modes:

**1. Single scroll in container** - use `predicate` to target the scrollable element:
```json
{"action": "scroll", "direction": "right", "predicate": {"type": "scrollview", "parent_of": {"text_contains": "Work"}}}
```

**2. Scroll until element found** - use `to_element` to scroll repeatedly until target appears:
```json
{"action": "scroll", "direction": "down", "to_element": {"predicate": {"text": "Privacy"}}, "max_scrolls": 10}
```

You can combine both - use `predicate` for the container and `to_element` for the target:
```json
{"action": "scroll", "direction": "right", "predicate": {"type": "scrollview", "parent_of": {"text_contains": "Activity"}}, "to_element": {"predicate": {"text": "Work"}}, "max_scrolls": 5}
```

### Conditional Action (dismiss popup if present)
```json
{
  "version": "0.2",
  "steps": [
    {
      "action": "if_exists",
      "predicate": {"text_contains": "Allow"},
      "then": [
        {"action": "tap", "predicate": {"text": "Allow"}}
      ]
    },
    {"action": "observe", "context": "native", "include": ["ui_tree"]}
  ]
}
```

### Toggle Switch to Desired State
```json
{
  "version": "0.2",
  "steps": [
    {"action": "toggle", "predicate": {"type": "switch", "text_contains": "Wi-Fi"}, "state": "on"},
    {"action": "toggle", "predicate": {"type": "switch", "text_contains": "Bluetooth"}, "state": "off"}
  ]
}
```

Only taps if the current state differs from desired state. Returns `toggled`, `previous_state`, `current_state`.

### Type Into Specific Element
```json
{
  "version": "0.2",
  "steps": [
    {"action": "type", "text": "Hello", "predicate": {"type": "input", "text_contains": "Username"}, "clear_first": true}
  ]
}
```

If `predicate` is provided, the executor finds the element, taps it, then types. If no predicate specified, keyboard MUST already be open (tap an input field first), otherwise an error is returned: "no predicate specified and keyboard is not open".

### Assertions - Verify UI State
```json
{
  "version": "0.2",
  "steps": [
    {"action": "assert_exists", "predicate": {"text": "Welcome"}, "timeout_ms": 5000, "message": "Welcome screen not shown"},
    {"action": "assert_not_exists", "predicate": {"text": "Error"}},
    {"action": "assert_count", "predicate": {"type": "button"}, "expected": 3},
    {"action": "assert_property", "predicate": {"text_contains": "Submit"}, "property": "enabled", "expected_value": true},
    {"action": "assert_screen_changed", "threshold_percent": 15}
  ]
}
```

| Action | Purpose |
|--------|---------|
| `assert_exists` | Verify element exists (with optional timeout) |
| `assert_not_exists` | Verify element does NOT exist |
| `assert_count` | Verify exact count of matching elements |
| `assert_property` | Verify property value (enabled, visible, selected, focused, text, value) |
| `assert_screen_changed` | Verify ≥N% of screen elements are new (for navigation) |

### Screen Change Detection (Navigation Verification)

Use `assert_screen_changed` after navigation to verify the screen changed without knowing what content will appear:

```json
{
  "version": "0.2",
  "steps": [
    {"action": "observe", "context": "native", "include": ["ui_tree"]},
    {"action": "tap", "predicate": {"text": "Next"}},
    {"action": "delay", "duration_ms": 300},
    {"action": "assert_screen_changed", "threshold_percent": 15}
  ]
}
```

**When to use:**
- After tapping navigation buttons that appear on multiple screens (Next, Continue, etc.)
- When you don't know what the next screen will contain
- Instead of `assert_not_exists` which fails if the button text repeats

**Parameters:**
- `threshold_percent` (default: 15): Minimum % of UI elements that must be new

### Observe Without Keyboard Clutter
```json
{
  "version": "0.2",
  "steps": [
    {"action": "observe", "context": "native", "include": ["ui_tree"], "include_keyboard": false}
  ]
}
```

By default, keyboard elements are filtered out. Set `include_keyboard: true` to include them.

### Get Installed Apps
```json
{
  "version": "0.2",
  "steps": [
    {"action": "observe", "context": "native", "include": ["installed_apps"]}
  ]
}
```

Response contains app list:
```json
{
  "step_results": [{
    "success": true,
    "result": {
      "observations": {
        "native": {
          "installed_apps": [
            {"bundleId": "com.apple.Preferences", "name": "Settings"},
            {"bundleId": "com.example.app", "name": "My App"}
          ]
        }
      }
    }
  }]
}
```

Use this to find the correct `bundleId` for `open_app` actions.

### OCR Text Recognition (iOS only)
```json
{
  "version": "0.2",
  "steps": [
    {"action": "observe", "context": "native", "include": ["ocr"]}
  ]
}
```

Returns detected text with screen coordinates for tapping (already adjusted for tapping):
```json
{
  "ocr_elements": [
    {"text": "Sign In", "confidence": 0.98, "x": 150, "y": 400, "width": 100, "height": 30}
  ]
}
```

Use when UI tree doesn't show expected elements. Tap using coordinates from results:
```json
{"action": "tap", "coords": {"x": 200, "y": 415}}
```

## Predicate Reference

Match elements using these fields:

| Field | Example | Description |
|-------|---------|-------------|
| `text` | `"Settings"` | Exact text match |
| `text_contains` | `"sett"` | Contains (case-insensitive) |
| `text_starts_with` | `"Log"` | Starts with prefix |
| `text_regex` | `"Item \\d+"` | Regex pattern |
| `type` | `"button"` | Element type (button, input, text, switch, etc.) |
| `label` | `"settings_btn"` | Accessibility label |
| `bounds_hint` | `"top_half"` | Screen region: top_half, bottom_half, left_half, right_half, center, top_left, top_right, bottom_left, bottom_right |
| `near` | `{"text_contains": "Username", "direction": "below"}` | Near another element (uses edge-based direction) |
| `parent_of` | `{"text_contains": "Work"}` | Find parent element by child predicate |
| `index` | `0` | Select Nth match when ambiguous |

### Near Predicate Options

The `near` predicate finds elements relative to another element. **When multiple elements match, the closest one to the anchor is automatically selected.**

**Direction uses element edges:** `above` means element's bottom edge is above anchor's top edge, `below` means element's top edge is below anchor's bottom edge, etc.

| Field | Description |
|-------|-------------|
| `text` | Exact text match for anchor element |
| `text_contains` | Partial text match for anchor (case-insensitive) |
| `direction` | `above`, `below`, `left`, `right`, or `any` (uses element edges) |
| `max_distance` | Maximum distance in pixels |

Example - toggle switch nearest to "Daily Top 3" label:
```json
{"action": "toggle", "predicate": {"type": "switch", "near": {"text_contains": "Daily Top 3"}}, "state": "on"}
```

### Parent Of Predicate

The `parent_of` predicate finds a parent element by specifying a child predicate. It traverses up from the child to find the first matching ancestor.

Example - find the ScrollView containing "Work" text:
```json
{"action": "scroll", "direction": "left", "predicate": {"type": "scrollview", "parent_of": {"text_contains": "Work"}}}
```

This is useful for:
- Finding scrollable containers for specific content
- Targeting parent cells/rows containing specific elements
- Disambiguating between multiple similar containers

### Disambiguating Multiple Matches

If predicate matches multiple elements, add `index`:
```json
{"action": "tap", "predicate": {"type": "button", "index": 0}}
```

Or use more specific predicates:
```json
{"action": "tap", "predicate": {"type": "button", "text_contains": "Submit"}}
```

## Execution Rules

1. **Always observe first** - Get UI tree before any interaction
2. **Use predicates, not indices** - More robust than hardcoded indices
3. **Add delays after navigation** - Apps need time to render
4. **Use retry strategy** - Transient failures are common
5. **Stop when subgoal is achieved** - Don't over-execute

## Error Handling

Check `step_results` for failures:

```json
{
  "success": false,
  "step_results": [
    {"success": true, "action": "observe"},
    {
      "success": false,
      "action": "tap",
      "error": {
        "code": "NO_MATCH",
        "message": "no element found matching predicate",
        "predicate": {"text_contains": "Settings"}
      }
    }
  ]
}
```

Common error codes:
- `NO_MATCH` - Element not found (try scrolling or different predicate)
- `AMBIGUOUS_MATCH` - Multiple elements match (use `index` or more specific predicate)
- `TIMEOUT` - Operation timed out

## Quick DSL Reference

| Action | Example |
|--------|---------|
| Observe | `{"action": "observe", "context": "native", "include": ["ui_tree"]}` |
| Observe+Screenshot | `{"action": "observe", "context": "native", "include": ["ui_tree", "screenshot"]}` |
| Observe+Filter | `{"action": "observe", "context": "native", "include": ["ui_tree"], "filter": {"text_regex": "Settings"}}` |
| Observe+Bounds | `{"action": "observe", "context": "native", "include": ["ui_tree"], "filter": {"bounds": {"x": 0, "y": 0, "width": 200, "height": 400}}}` |
| List Apps | `{"action": "observe", "context": "native", "include": ["installed_apps"]}` |
| OCR (iOS) | `{"action": "observe", "context": "native", "include": ["ocr"]}` |
| Tap | `{"action": "tap", "predicate": {"text_contains": "Submit"}}` |
| Type (keyboard open) | `{"action": "type", "text": "Hello", "clear_first": true}` |
| Type with predicate | `{"action": "type", "text": "Hello", "predicate": {"text_contains": "Search"}}` |
| Press Key | `{"action": "press_key", "key": "enter"}` |
| Toggle | `{"action": "toggle", "predicate": {"type": "switch", "text_contains": "Wi-Fi"}, "state": "on"}` |
| Swipe | `{"action": "swipe", "direction": "up", "distance": "medium"}` |
| Wait | `{"action": "wait_for", "predicate": {"text": "Welcome"}, "timeout_ms": 5000, "poll_interval_ms": 500}` |
| Wait Stable | `{"action": "wait_for", "stable": true, "timeout_ms": 5000, "poll_interval_ms": 500}` |
| Screenshot | `{"action": "screenshot", "file_path": "/tmp/screenshots", "name": "my_screen"}` |
| Assert Exists | `{"action": "assert_exists", "predicate": {"text": "Success"}, "timeout_ms": 3000}` |
| Assert Not Exists | `{"action": "assert_not_exists", "predicate": {"text": "Error"}}` |
| Assert Count | `{"action": "assert_count", "predicate": {"type": "button"}, "expected": 2}` |
| Assert Property | `{"action": "assert_property", "predicate": {"text": "Submit"}, "property": "enabled", "expected_value": true}` |
| Assert Screen Changed | `{"action": "assert_screen_changed", "threshold_percent": 15}` |
| Double Tap | `{"action": "double_tap", "predicate": {"text_contains": "Photo"}}` |
| Two Finger Tap | `{"action": "two_finger_tap", "coords": {"x": 200, "y": 400}}` (iOS only) |
| Drag (predicate) | `{"action": "drag", "from": {"predicate": {"text": "Item"}}, "to_element": {"predicate": {"text": "Trash"}}, "duration_ms": 500}` |
| Drag (coords) | `{"action": "drag", "from_coords": {"x": 100, "y": 200}, "to_coords": {"x": 300, "y": 400}, "duration_ms": 500}` |
| Kill App | `{"action": "kill_app", "bundle_id": "com.example.app"}` |
| Set Location | `{"action": "set_location", "lat": 40.7128, "lon": -74.0060}` (Android 12+ for real devices) |
| Reset Location | `{"action": "reset_location"}` (Android 12+ for real devices) |
| Metrics Start | `{"action": "metrics_start", "types": ["system_cpu", "system_memory", "fps"], "capture_logs": true, "label": "test"}` |
| Metrics Stop | `{"action": "metrics_stop", "format": "summary"}` |

## Performance Metrics

Collect CPU, memory, FPS, and other metrics during test flows for performance analysis.

### Start Metrics Collection
```json
{
  "action": "metrics_start",
  "types": ["system_cpu", "system_memory", "fps"],
  "bundle_id": "com.example.app",
  "label": "login_flow",
  "thresholds": {
    "cpu_high": 80,
    "fps_low": 45,
    "memory_growth_mb_min": 50
  }
}
```

**Fields:**
- `types`: Metrics to collect - `system_cpu`, `system_memory`, `fps`, `network`, `battery`
- `bundle_id`: Filter to specific app (optional)
- `label`: Human-readable session label (optional)
- `capture_logs`: Capture device logs during session (default: false)
- `thresholds`: Custom anomaly detection thresholds (optional)

### Stop and Get Summary
```json
{"action": "metrics_stop", "format": "summary"}
```

**Response includes:**
- `overall_health`: "healthy", "warning", or "critical"
- `health_score`: 0-100 score
- `system_cpu`: avg, max, p95, status
- `system_memory`: avg_percent, growth_mb, trend, status
- `fps`: avg, min, jank_percent, status
- `logs_file`: Path to detailed metrics log file
- `logs_available`: Whether logs were captured
- `anomalies`: Detected issues with severity and timestamps
- `recommendations`: Actionable suggestions

### Example: Performance Test
```json
{
  "version": "0.2",
  "steps": [
    {"action": "metrics_start", "types": ["system_cpu", "system_memory", "fps"], "label": "app_launch"},
    {"action": "open_app", "bundle_id": "com.example.app"},
    {"action": "wait_for", "predicate": {"text": "Welcome"}, "timeout_ms": 10000},
    {"action": "tap", "predicate": {"text": "Login"}},
    {"action": "delay", "duration_ms": 5000},
    {"action": "metrics_stop", "format": "summary"}
  ]
}
```

## Reporting Results

When done, clearly state:
- **Success**: What was accomplished
- **Failure**: What went wrong and why
- **Current state**: What's visible on screen now (from last observe)