--- name: colm-author-response description: Use when writing a COLM rebuttal in the OpenReview discussion phase — triaging reviews released in late May, running cache-based follow-up experiments inside the roughly two-and-a-half-week window, answering contamination and baseline-fairness objections with evidence, and writing for the area chair who decides in July. --- # COLM Author Response In 2026 the COLM rebuttal ran May 22 to June 8: reviews and the response window opened the same day, giving about eighteen days between first sight of the reviews and the last word you get before the July 8 decision. A rebuttal in that geometry is a project with a plan, not a reply-all reflex. ## Budget the eighteen days ```text days 1-2 read only; draft nothing; log every reviewer claim into the four bins (misreading / evidence gap / scope fight / correct hit — colm-review-process) days 3-9 run the winnable experiments — cache-based analyses first, single-GPU or small-API-budget runs second; anything needing full retraining is out of scope, say so plainly days 10-13 draft one response per reviewer + a short common preamble for cross-cutting points; every paragraph anchored to a page, table, appendix section, or new number days 14-16 co-author cold read: does each response open with its conclusion? days 17-18 post with margin; OpenReview deadline-hour uploads are self-harm ``` The highest-yield experiments are the ones your artifact discipline pre-paid for: a cached-responses archive lets you re-slice results, add a metric, or test a reviewer hypothesis in hours without touching a model (`colm-artifact-evaluation`). ## The objections COLM reviewers actually raise These four LM-specific objections dominate borderline cases; each has a right and a wrong answer shape: | Objection | Weak answer | Strong answer | |---|---|---| | "Benchmark is likely contaminated for these models." | "This benchmark is widely used." | Overlap analysis numbers, or dated-item split showing the effect survives on post-cutoff items | | "Baselines are under-tuned; better prompting closes the gap." | "We used standard prompts." | The tried-ledger: k prompt variants per system, matched budgets; if the gap narrows, report the narrowed gap | | "Results are on one model family; is this about LMs or about model X?" | "We expect it generalizes." | One added family inside the window, claim rescoped in the revision either way | | "API model results won't reproduce." | "We used the public API." | Version strings + query dates already in App. A, plus commitment to release the response cache | Notice the pattern: every strong answer is either *evidence you can generate this week* or *a scope concession that makes the paper truer*. Both move ACs; indignation moves nothing. ## Writing mechanics - **Conclusion first, then support.** "The gap survives decontamination (new Table R1): on items post-dating every model's cutoff, the effect is 7.9 points." Then the method sentence. - **Number new artifacts** R1, R2, ... and reference them consistently; the AC skims all responses in one sitting and consistent labels are navigation. - **Quote the reviewer's own words** for each point you answer — it prevents the "they dodged my question" reading — and answer *the words*, not a nastier paraphrase. - **Concede in the open.** A visible "R2 is right; we overclaimed X and have rescoped to Y" is the single most credibility-generating sentence a rebuttal can contain. - **Stay anonymous.** The discussion is inside double-blind: no lab names, no "in our previous paper," no links that resolve to you. - **Keep LLM-policy discipline** — if a response experiment used LLM-based evaluation, it is disclosable under the same 2026 policy as the paper. ## Handling the difficult cases - **The low-engagement review** (two lines, generic, possibly LLM-drafted without disclosure): respond substantively and briefly, then note factually in a comment visible to the AC that the review does not engage the paper's content. Flag, do not accuse. - **The scope-fight reviewer** (wants a different paper): one crisp paragraph on why the paper's question is the right size, pointing at the venue's own genre — measurement and analysis papers are COLM's award lineage — then stop. Repetition reads as anxiety. - **The correct hit you cannot fix in eighteen days:** say what the fix is, why it does not fit the window, and how the revision will scope claims meanwhile. ACs reward authors who can tell the difference between a flesh wound and a broken spine. ## Per-reviewer response skeleton One response per reviewer, same skeleton, filled with their words and your numbers: ```text Thank you for the careful reading. We address each point below. > "" > "" ... Summary of changes committed for the revision: 1. 2. ... We believe the remaining disagreement (if any) is , which we leave to the AC's judgment. ``` Length discipline: reviewers are reading many rebuttals in the same week you are writing yours. A response that fits on one screen per reviewer, with bolded conclusions and numbered artifacts, gets read twice; a 2,000-word essay gets skimmed once. If a point genuinely needs depth, put the depth in a clearly labeled final section so the main thread stays scannable. ## After June 8 Nothing. The reviewer-AC discussion through the July 8 decision has no author channel — resist the urge to post updates. Spend the month preparing both branches: the camera-ready plan (due August 7 if accepted) and the resubmission map (`colm-review-process`). ## Output format ```text [Window] days remaining: of ~18 [Per reviewer] R: bins A/B/C/D counts + the one thing they must hear [Experiment queue] [Concessions] [AC preamble] <3-sentence summary of what changed during discussion> ```