--- name: launch-vast-instance description: >- Launch (or destroy) a Vast.ai GPU instance pre-provisioned with the Open Superintelligence Lab LLM research kit (universe-lm) via a saved public template. Invoke when the user says "launch a vast box", "spin up a gpu", "rent a gpu for the research kit", "start a research box", "get me a ready-to-train box", or "destroy the vast box". This skill is written FOR an AI agent: it self-installs the vastai CLI and asks the user for their Vast API key if one isn't configured, then rents a box that auto-clones the repo, installs deps, downloads the dataset, and verifies CUDA — ready to train in ~5 min. Costs real money — always confirm GPU + spend before renting, and only ever destroy instances this skill created. --- # launch-vast-instance A self-contained runbook for an **AI agent** to rent a Vast.ai GPU box that self-provisions the `universe-lm` LLM research kit and is ready to `python run_experiment.py --config 10m` in ~5 minutes. You do **not** need to know anything about the user's machine in advance — the steps below install the CLI and obtain credentials interactively. ## CONSTANTS (the shared public template — same for everyone) | Thing | Value | |---|---| | Template name | `research-kit (universe-lm)` | | Template hash | `718634dced67a38b7fea70c28c157f50` (public — anyone can launch from it) | | Public share link | `https://cloud.vast.ai?ref_id=220350&template_id=718634dced67a38b7fea70c28c157f50` | | Base image | `pytorch/pytorch:2.4.1-cuda12.1-cudnn9-runtime` | | Offer filter | `reliability>0.98 num_gpus=1 gpu_ram>=12 rentable=true disk_space>=60 inet_down>=200 cuda_vers>=12.1` | | Ready signal in logs | `BOX READY` + `CUDA OK` | | Repo on box | `/workspace/universe-lm` · dataset `processed_data/pretrain_1B` · HF cache `/workspace/hf` | Every `vastai` call prints a harmless LibreSSL `NotOpenSSL` warning to stderr — strip it with `2>&1 | grep -viE 'NotOpenSSL|warnings.warn'`. ## STEP 0 — Prerequisites (do these first, automatically) ### a) Install the vastai CLI if missing ```bash vastai --version 2>/dev/null || python3 -m pip install --user vastai ``` The CLI needs **Python ≥ 3.10** (it uses `match`). On macOS, Apple's system `python3` is 3.9 and will fail with `SyntaxError: invalid syntax` — use `python3.11`/`3.12`/`3.13` (e.g. `brew install python`) for the install. The binary may land in `~/Library/Python//bin/vastai` or `~/.local/bin/vastai`; add that to PATH or call it by full path. Verify: `vastai --version`. ### b) Get the API key (ASK THE USER if it isn't set) ```bash vastai show user 2>&1 | grep -viE 'NotOpenSSL|warnings.warn' | head -1 ``` If this returns `403 ... requires login` (or any auth error), the key isn't configured. **Ask the user** — do not invent or guess one: > "I need your Vast.ai API key to rent a GPU. Get it at **vast.ai → Account → > API Key → Copy** and paste it here. It's stored locally in > `~/.config/vastai/vast_api_key` and never committed anywhere." Then set it and confirm: ```bash vastai set api-key vastai show user --raw 2>/dev/null | python3 -c "import sys,json;d=json.load(sys.stdin);print('credit: $'+str(round(d.get('credit',0),2)))" ``` If credit is ~0, tell the user they must add a payment method / credits at vast.ai before renting. ## STEP 1 — List existing instances (never touch what you didn't create) ```bash vastai show instances --raw 2>/dev/null | python3 -c " import sys,json for d in json.load(sys.stdin): print(d['id'], d.get('gpu_name'), d.get('actual_status'), f\"\${d.get('dph_total',0):.3f}/hr\", d.get('label')) " ``` Remember these ids — the user may have other boxes running. You will only ever destroy the id you launch below. ## STEP 2 — Find the cheapest matching offer ```bash vastai search offers 'reliability>0.98 num_gpus=1 gpu_ram>=12 rentable=true disk_space>=60 inet_down>=200 cuda_vers>=12.1' \ -o dph_total --raw 2>/dev/null | python3 -c " import sys,json o=json.load(sys.stdin)[0] print('PICK', o['id'], o['gpu_name'], f\"\${o['dph_total']:.3f}/hr\", o['geolocation']) " ``` Prefer Ampere+ GPUs (RTX 3060/3090/4090). The run is pinned to **bf16**, which Pascal cards (e.g. Titan Xp) lack. The CUDA 12.1+ filter ensures the host driver matches the cu121 image. ## STEP 3 — Confirm spend, then rent FROM the template **Confirm the GPU + $/hr with the user before this step.** Then, using the offer id from Step 2: ```bash vastai create instance --template_hash 718634dced67a38b7fea70c28c157f50 --disk 60 --raw 2>&1 \ | grep -viE 'NotOpenSSL|warnings.warn' # -> {"new_contract": , "success": true} ``` ## STEP 4 — Wait until running, then verify provisioning ```bash # 4a. Poll until running (image pull + boot ~1-2 min) for i in $(seq 1 8); do sleep 20 ST=$(vastai show instance --raw 2>/dev/null | python3 -c "import sys,json;print(json.load(sys.stdin).get('actual_status') or 'loading')") echo "$ST"; [ "$ST" = running ] && break; done # 4b. Provisioning (pip + dataset) runs AFTER 'running'. Watch the on-start log # for BOX READY (no SSH key needed). ~3-5 more min. vastai logs 2>&1 | grep -iE 'Saved dataset|CUDA OK|BOX READY|Error|Traceback|NO CUDA' | tail ``` Green when the log shows `✅ Saved dataset to processed_data/pretrain_1B`, `CUDA OK …`, and `BOX READY -> cd /workspace/universe-lm && python run_experiment.py --config 10m`. ## STEP 5 — SSH in / run (optional) ```bash vastai ssh-url # -> ssh://root@: # then on the box: cd /workspace/universe-lm && python run_experiment.py --config 10m ``` Launch long runs inside `tmux` so they survive disconnect. ## STEP 6 — Destroy when done (stop billing) ```bash vastai destroy instance -y 2>&1 | grep -viE 'NotOpenSSL|warnings.warn' ``` Afterward run `vastai show instances` and confirm ONLY your box is gone and the user's other boxes still run. ## SAFETY (non-negotiable) - Renting costs real money. Always confirm GPU + rough $/hr before Step 3. - Never destroy or modify an instance you did not create this session. - Never guess an API key. If it's missing, ask the user (Step 0b). ## What the box runs (for reference) The template's on-start script does, unattended: install `git` if missing → `git clone https://github.com/vukrosic/universe-lm` → `pip install -r requirements.txt` → `python data/download_hf_data.py` (1B pretrain set) → assert `torch.cuda.is_available()`. Source: `https://github.com/vukrosic/universe-lm` (`scripts/vast_provision.sh`).