--- name: openscientist-hypothesis description: Use when running or wiring OpenScientist for AI Gene Review gene-function hypotheses, especially blinded comparisons against local bioinformatics analyses. metadata: short-description: AIGR OpenScientist hypothesis workflow --- # OpenScientist Hypothesis Workflow Use this skill for AIGR work that runs, stages, or maintains OpenScientist hypothesis jobs for gene function review. ## Core Rules - Treat OpenScientist as an independent bioinformatics scientist exploring a hypothesis: gene `G` has function `F`. - For existing annotations, prefer the neutral function-assignment mode: `--as-function-hypothesis`. - Do not feed existing local `*-bioinformatics/RESULTS.md` analyses into OpenScientist prompts. Those are holdout evidence for post-run comparison. - It is acceptable to pass bounded non-holdout review context, especially the selected term, original evidence reference, and a small number of relevant literature/deep-research references. - Avoid sending every function for genes with large reviews; select the specific hypothesis being tested. ## Running Jobs - Use the `just` recipes, not raw `deep-research-client`, for AIGR hypothesis jobs. The justfile carries project defaults such as OpenScientist iteration count. - Be patient with OpenScientist. Jobs may run quietly for a long time before the local report is written. - Do not interrupt a real OpenScientist run merely because the local wrapper is silent or the child process is sleeping. - Check completed and running jobs at before deciding a job is stuck. - One-iteration OpenScientist runs are smoke tests only. Real hypothesis jobs should use the project default of at least 3 iterations unless the user explicitly asks otherwise. ## Timeouts - Give analyses ample time. Real 3-iteration runs commonly take ~50-70 minutes, and structural / fold-discovery jobs can run longer; they are cancelled mid-analysis if the timeout is too short (observed: a job cancelled at the upstream 3600s default after producing nothing). - The `just` recipes now inject the maximum OpenScientist job timeout (`--param timeout=7200`, i.e. 2h) alongside `--param max_iterations=3`, and the wrapper's subprocess wall (`--timeout-seconds`, default 8100s) is kept above the job timeout so the child process is not killed first. If you ever bypass `just` and call the script directly, pass both: e.g. `-- --param max_iterations=3 --param timeout=7200` plus `--timeout-seconds 8100`. The two timeouts are independent — raising one without the other still cancels the run. - 7200s is the API ceiling: `OpenScientistParams.timeout` is validated `le=7200`, so a larger value fails fast with a pydantic `less_than_equal` error and writes nothing. Do not exceed it. - Do not lower these to "save time"; a too-short timeout wastes the whole run. - When a job *still* hits the 7200s ceiling (common for human proteins asked a multi-faceted question), the fix is **scope, not time**: re-run with a single decisive analysis in the hypothesis (e.g. "Foldseek fold assignment" alone, not fold + disorder + motif + targeting) and/or `--param max_iterations=2`. Observed: C18orf21 7200s timeout -> 2708s when narrowed to Foldseek-only; HSPA12A/DNAJC28 likewise completed once narrowed. The paralog of a job that *did* finish (e.g. HSPA12B at 5208s) will usually finish on a plain retry — variance, not a hard limit. - Scoping each hypothesis to one computable question up front (rather than a broad "characterize this gene") both avoids timeouts and gives cleaner, more decisive provenance. ## Failure Modes - A job can exit 0 yet write no report. Always confirm `openscientist.md` (and the `openscientist_artifacts/` dir) actually exist before treating a run as done; an empty hypothesis directory means it failed. - Two distinct failure signatures seen, both leaving an empty output dir: - `detail=... timed out after Ns ... has been cancelled` -> raise the timeout (see above) and re-run. - `detail=... Request ID: req_...` -> a transient OpenScientist-internal LLM error (their server-side agent call failed; the id is for their debugging). Re-run; if the same gene fails this way repeatedly, treat it as a persistent upstream issue, move on, and report it rather than retrying indefinitely. - Remove the empty `GENE-hypotheses//` directory before re-running so a failed attempt does not masquerade as a completed one. ## After Completion - Verify the generated report does not include withheld local bioinformatics paths or `RESULTS.md` content. - Compare the OpenScientist conclusion against the held-out local bioinformatics result after the run. - Keep operational lessons in skills or agent instructions, not in user-facing project pages. ## Wiring Verdicts Into Reviews - **A refuted *proposal* must be deleted, not set to `REMOVE`.** Validation enforces that annotations absent from GOA carry `action: NEW` (they are AI proposals, identifiable by having no `original_reference_id`). If a run refutes such a proposal, delete the annotation block *and* any `core_functions` entry built on it, and record the refutation in the `description` + an OpenScientist `file:` reference. `action: REMOVE` is only for real GOA annotations and will fail validation on a not-in-GOA term ("Found N annotations not in GOA"). - **Gate findings that depend on uncacheable references.** When a run's conclusion rests on very recent papers not in `publications/` (so their `supporting_text` cannot be verbatim-verified) and that you cannot otherwise confirm, record it as a clearly-labelled *lead* — keep the conservative existing annotation, note "not yet verified / PMIDs not cached", and cite the OpenScientist `file:` report — rather than asserting a new annotation. Verify, don't trust. (Example: C18orf21 -> RMP24 / RNase MRP, left as a lead pending primary-literature caching.) - **Don't overstate disagreements.** Before wiring "the run overturns the prior review", check the primary data: the two analyses often *agree on the facts* (e.g. residues present) and differ only on *interpretation* (e.g. a PROSITE signature failure). Wire genuine standoffs as `UNDECIDED` with both lines of evidence and the decisive experiment, and do not import the run's secondary claims (e.g. a named degenerate paralog) unless corroborated by repo data. - **Verbatim `supporting_text` gotchas:** plain (unquoted) YAML scalars cannot contain `": "` — single-quote any quote containing a colon-space (e.g. "Foldseek: zero ..."). The pre-edit hook catches these. ## Job Hygiene - Background jobs are killed on container restart; the harness reports which. Check the `GENE-hypotheses//` dir — if empty, clean it and re-launch. - Check whether the API key is already set before assuming it is not: `printenv OPENSCIENTIST_API_KEY` (it is exported in some environments, so the `just` recipes work directly). Only if it is absent, pass `OPENSCIENTIST_API_KEY` inline when calling the script directly, and wire it into the environment/setup config for reproducible `just` runs.