--- name: geospatial-project-setup description: Set up geospatial web platforms (CesiumJS, DroneDB, GDAL/PDAL stacks): format compatibility research, risk register, MVP scoping, sprint planning. Covers drone data, GIS formats, tile serving pipelines. --- # Geospatial Project Setup Playbook for standing up geospatial web platforms built on CesiumJS, DroneDB, GDAL/PDAL, and related open-source GIS stacks. ## When to Load This Skill - New project involves CesiumJS viewer, drone data, GIS formats, tile serving - User asks about format compatibility, pipeline risks, MVP scoping for geospatial - Stack includes any of: DroneDB, GDAL, PDAL, Cesium Ion, COG, EPT, COPC, 3D Tiles, GeoTIFF, LAS/LAZ, OBJ, GeoJSON, DXF --- ## The GX-GeoShare Stack (Reference Implementation) Proven stack for drone data management + sharing platforms: | Layer | Choice | Why | |-------|--------|-----| | Data engine | DroneDB Core 1.7.4 | C++ OSS — EXIF/GDAL/PDAL/COG/EPT/STAC out of the box | | Data server | DroneDB Registry 2.3.1 | .NET 9, JWT, REST, tile serving, Hangfire jobs | | 3D viewer | CesiumJS 1.141 + Resium 1.21.1 | Industry standard; React bindings via Resium | | Frontend | Vite + React 18 + TypeScript | Fastest dev loop for Cesium SPA (NOT Next.js — SSR adds no value) | | UI | shadcn/ui + Tailwind | Copy-in MIT components, no lock-in | | State | Zustand + React-Query | Minimal; Redux unnecessary | | Backend | FastAPI Python 3.12 | Async, OpenAPI, PostGIS-friendly | | Database | PostgreSQL 16 + PostGIS 3.4 | Annotations/comments/shares are relational + geospatial | | Storage | MinIO | S3-compatible OSS backing DroneDB Registry | | Real-time | FastAPI WebSocket + Yjs CRDT | Conflict-free collaborative annotation | | Auth | DroneDB Registry JWT | Re-use Registry auth; Keycloak overkill for <50-user teams | | Proxy | Caddy 2 | Auto-TLS, simpler than Nginx | | Deploy | Docker Compose | Single `docker compose up` | See `references/gx-geoshare-mvp-summary.md` for full MVP format scope, limitations table, and sprint plan. See `references/dronedb-registry-ops.md` for DroneDB Registry v2.3.1 auth endpoint details, default credentials, user creation, Google OAuth gap, Caddy proxy routing, and database inspection patterns. See `references/sprint-infra-patterns.md` for Sprint 1 QA gate, GDAL version mismatch fix, credentials-in-compose blocker, legacy codebase assessment framework, and docker-compose security checklist. See `references/ux-review-bug-patterns.md` for UX bug patterns and fix recipes: component-built-but-never-wired, router double-prefix, share URL internal hostname, auth mode mismatch, mailto email sharing, integrity baseline drift. See `references/async-auth-fix-patterns.md` for the full fix catalogue: SQLAlchemy MissingGreenlet + selectinload, passlib/bcrypt incompatibility bypass, slowapi rate-limiter removal, authClient.ts JSON login, Docker named-container env injection, annotation double-prefix, Playwright stdin timeout. See `references/sprint-crew-crash-patterns.md` for crash patterns: max_iter exhaustion, prefill 400, f-string syntax errors, bash grep false negatives, partial re-run strategies, integration seam review checklist, stale uvicorn pattern, agent exploration loop fix, **components-without-pages (Crash 15)**, and **chain resume after manual block fix (Crash 16)**. See `references/gx-ux-review-crew-pattern.md` for the full UX Review Crew architecture (3-agent Planner→Tester→Reviewer), DroneDB test-user creation, complete API route map (Sprint 14), all frontend pages, Playwright setup, user journey step order, and UX pitfalls including the double-prefix annotation route bug. See `templates/playwright_journey_template.ts` — validated 8-test Playwright user journey spec with self-healing patterns: fresh login per test, exact selectors (`getByRole('link', { name: 'Open' })` for project cards, `{ exact: true }` for ambiguous buttons), self-healing project navigation (find existing card or fall back to cancel+reuse), self-healing share URL fallback (API-create if cross-test variable is empty), email input uses `.last()` for disambiguation. Updated Sprint 17. See `ai-native-dev-workflow` → `references/qa-and-review-patterns.md` for the QA script pattern and Claude Code write-first prompt pattern used in sprint crews. See `templates/qa_sprint_template.sh` — copy to `scripts/qa_sprintN.sh` for each sprint; uses python3 regex (not bash grep) to avoid `\|` false negatives. Includes Docker-dep handling pattern. See `templates/qa_sprint15_foundation_fixes.sh` — QA template for wiring/config/polish sprints; demonstrates correct dynamic baseline measurement, curl-based auth verify, and QA_STATUS= exit contract. See `scripts/measure_baselines.sh` — run before writing any new sprint QA script to get real current Pyrefly + Biome counts; never copy prior sprint baselines blindly. See `templates/crew_sprint_template.py` — base crew script with crash-safe agent patterns (QA=1 script call, Reviewer=1 Claude Code call, prefill-safe model). See `templates/crew_sprint_tight_pattern.py` — the tight pattern that prevents the exploration trap (Crash 13): commands in task.description, max_iter=8, backstory says DO NOT explore. Use this as the base for all sprint scripts. See `templates/run_chain_template.sh` — chain runner with correct uvicorn restart (system Python 3.10, not .venv), frontend rebuild+deploy, and verdict checking. Prevents Crash 14 (stale uvicorn BLOCKs). See `templates/playwright.config.ts` — minimal Playwright config for GX-GeoShare (headless, screenshot-on, 60s timeout, 1 retry). ## Sprint Scope Patterns (GX-GeoShare reference implementations) When planning sprints for drone/geospatial platforms, use these as starting points: ### Geotagged JPEG Gallery (Sprint 7 pattern) Key deliverables for photo management: - `jpeg_process.py` Celery task: EXIF GPS via Pillow, 256×256 thumbnail, upload to MinIO `gx-thumbnails` - `photos.py` router: `GET /datasets/{id}/photos`, thumbnail proxy, full image proxy - `photo_manager.py`: trigger Celery task on upload, generate presigned URLs - `PhotoLayer.tsx`: BillboardCollection at GPS coordinates (NOT EntityCollection — performance) - `PhotoGalleryPanel.tsx`: grid thumbnails + lightbox + "Jump to map" button - Non-geotagged JPEGs: accepted with `preview_only=True`, NOT shown on map - DB migration: `photos` table with `lat, lon, alt, thumbnail_url, original_url, preview_only` - Fixture: real JPEG with GPS EXIF via `piexif + Pillow` (not empty file — real binary ≥1KB) ### Sharing + Polish (Sprint 8 pattern) Key deliverables for sharing and deployment: - `sharing.py` router: POST create (UUID v4 token), GET resolve (public, no auth), DELETE revoke - Share tokens MUST be UUID v4 (not sequential — prevents enumeration). Expired/revoked → 404 (not 401) - `permissions.py`: `OWNER / EDITOR / VIEWER` enum, `has_permission(user_id, project_id, role, db)` - `SharedViewPage.tsx`: public read-only viewer, no upload/annotation/delete buttons - `ShareDialog.tsx`: copy-to-clipboard URL, expiry selector (Never/7/30/90 days) - UI polish: `Spinner.tsx`, `ErrorBoundary.tsx`, `EmptyState.tsx` as shared components - `DEPLOYMENT.md` ≥200 lines: Docker setup, MinIO buckets, Caddy TLS, alembic migrations, air-gapped mode - DB migration: `share_links(id, project_id, token UUID, created_by, expires_at nullable, revoked_at nullable)` --- ## Pre-Development Research (MANDATORY for Geospatial) Geospatial stacks have high silent failure risk — wrong CRS, missing nodata mask, wrong tile projection. Always run full format compatibility research BEFORE any sprint dev crews. Use the **3-stage chained pipeline** from `ai-native-dev-workflow` references/chained-research-pipeline.md: ### Stage 1 Research Prompt Must Cover 1. **Format × pipeline matrix** — for each format: does DroneDB ingest it? What does it output? Can CesiumJS render it? What conversion is required? 2. **Live GitHub bug query** — `curl -s "https://api.github.com/repos/DroneDB/DroneDB/issues?state=open&labels=bug&per_page=30"` (real bugs, not just docs) 3. **CesiumJS rendering specifics** — what provider type per format, known version-specific bugs, WebGL memory limits 4. **Silent failure modes** — file loads without error but renders in wrong location / wrong scale / missing nodata 5. **Output:** `FORMAT_COMPATIBILITY.md`, `RISK_REGISTER.md`, `SCOPE_ESTIMATE.md` ### MVP Format Scoping (Hunter Lin preference) Do NOT try to support every format in MVP. Pick the highest-value, lowest-pipeline-complexity set. For drone platforms, standard MVP: | ✅ IN MVP | ❌ OUT (defer) | |-----------|----------------| | GeoTIFF (RGB orthophoto, Float32 DEM) | LAS / LAZ / COPC (complex pipeline) | | OBJ + MTL + single texture | glTF / GLB direct upload | | 3D Tiles (B3DM, PNTS) — pre-built ZIP | E57 / PLY / PTS / PTX (scanner formats) | | GeoJSON | KML / KMZ / SHP / GeoPackage | | DXF (2D only, read-only) | BigTIFF >4GB | | Geotagged JPEG (browse/preview) | Multi-band/NIR/thermal GeoTIFF | | | Non-WGS84 CRS input | | | MP4 video, 360° panoramas | --- ## Critical Format-Pipeline Risks (Geospatial) These are the top risks that cause silent data corruption — embed all in RISK_REGISTER.md and ensure every sprint gate tests for them: ### RISK-01: Vertical CRS Silent Data Corruption ⚠️ HIGHEST PRIORITY - **Scenario:** GeoTIFF uploaded in orthometric height (EGM2008). GDAL's `gdalwarp` strips vertical datum when reprojecting. Heights rendered in Cesium using ellipsoidal height — up to 70m error globally, 30m in CONUS. - **Mitigation:** Mandatory Vertical Datum Declaration dialog that blocks DEM upload completion until user selects datum. Store in PostGIS. Display persistent banner in viewer. - **Detection test:** Upload Float32 DEM → verify `requires_vertical_datum: true` in response → verify datum stored in DB → verify viewer shows yellow warning banner. ### RISK-02: Non-WGS84 GeoTIFF Renders in Wrong Location - **Scenario:** GeoTIFF in OSGB36, GDA2020, or local projection. Accepted silently, tiles served, renders 100s of km from correct location with no error. - **Mitigation:** Reject non-WGS84 input at upload with HTTP 422. Include detected CRS WKT in error response. Only accept EPSG:4326 or WGS84-based UTM zones (EPSG:32601–32760). - **Detection test:** Upload OSGB36 GeoTIFF → expect HTTP 422 with `non_wgs84_crs` error code. ### RISK-03: DroneDB COG Nodata Mask Regression - **Issue:** DroneDB using GDAL 3.11.x may produce black opaque borders on orthophoto edges instead of transparent nodata. Bug in GDAL mask handling. - **Mitigation:** Pin DroneDB Registry to 2.3.1. After COG job completes, fetch a tile at known-nodata coordinate and verify alpha channel is 0 (transparent), not RGB=0 (black). - **Detection test:** `img.getpixel((0,0))[3] == 0` for corner pixel of nodata tile. ### RISK-04: JWT Algorithm Mismatch (Registry RS256 vs FastAPI HS256) - **Scenario:** DroneDB Registry issues RS256 (asymmetric) JWTs. If FastAPI is configured for HS256 (shared secret), ALL auth fails with 401. - **Mitigation:** Sprint 1 — inspect `GET /.well-known/jwks.json` on Registry. If RS256, configure FastAPI with JWKS fetch + cache. Document in env file. - **Detection test:** `curl -s http://registry:5000/.well-known/jwks.json | jq '.keys[0].alg'` — configure to match. ### RISK-05: OBJ Texture Path Failures (Windows → Linux) - **Scenario:** OBJ from Windows DJI Terra / Pix4D uses backslash paths in MTL (`map_Kd textures\atlas.png`). Linux `obj2gltf` fails to find texture files → GLB has no textures (renders grey). - **Mitigation:** Pre-flight normalises all MTL path separators to forward slashes before conversion. - **Detection test:** Upload OBJ with backslash MTL paths → verify GLB has texture (not grey mesh). ### RISK-06: COPC → Cesium Pipeline Gap - **Scenario:** DroneDB outputs COPC internally from LAS input. CesiumJS 1.141 has NO native COPC reader. Invisible to users — they upload LAS, it processes "successfully," but nothing renders. - **Mitigation:** Reject LAS/LAZ at upload with HTTP 422 in MVP. Workaround: pre-convert to 3D Tiles PNTS externally. - **Detection test:** Upload LAS file → expect HTTP 422 with `format_not_supported: las`. ### RISK-07: DXF Coordinate System Assumption - **Scenario:** DXF files have no CRS metadata. GX assumes WGS84 decimal degrees. DXF in local/site grid coordinates (common in civil engineering) renders thousands of km from correct location with no error. - **Mitigation:** Document clearly in upload UI and limitations doc. No validation possible (no CRS in format). - **Detection test:** Upload DXF in WGS84 → verify renders at correct location. Warn user in UI. --- ## Reference Files - `references/geokernel-architecture.md` — Full GeoKernel field software architecture: service map, IPC protocol, camera back-projection math, wse-sdk dependency boundary, new AI module sprint plans, Plane project link. --- ## Project Scaffold Checklist For new geospatial platforms: ``` project/ ├── docs/ │ ├── PRD.md # Product requirements │ ├── STACK_DECISION.md # Tech stack with justifications │ ├── FORMAT_COMPATIBILITY.md # Format × pipeline × CesiumJS matrix │ ├── RISK_REGISTER.md # All risks with mitigation + detection test │ ├── SCOPE_ESTIMATE.md # Sprint estimates with complexity ratings │ ├── MASTER_PLAN.md # Planning crew output │ ├── MVP_PLAN.md # Scoped sprint plan (authoritative) │ ├── MVP_LIMITATIONS.md # Stakeholder limitations doc │ ├── MVP_QA_GATES.md # Binary go/no-go gates per sprint │ └── MEASUREMENT_LIMITATIONS.md # Flat-globe measurement error docs ├── services/ │ ├── gx-api/ # FastAPI backend │ ├── frontend/ # React + Vite + CesiumJS │ ├── gx-worker/ # OBJ→GLB conversion, async tasks │ └── caddy/ # Reverse proxy config ├── infra/ │ ├── postgres/init.sql # PostGIS init │ └── minio/init.sh # Bucket creation ├── tests/ │ ├── fixtures/ # Test files: orthophoto, dem, obj, 3dtiles, geojson, dxf │ ├── api/ # pytest API tests │ └── e2e/ # Playwright E2E tests ├── dev-output/ # Agent logs, test results, review notes ├── graphify-out/ # Knowledge graph (after Phase 0 ingestion) ├── docker-compose.yml ├── CLAUDE.md # Claude Code project context └── .env # Secrets (gitignored) ``` ### .env Required Variables (Geospatial Platform) ```bash ANTHROPIC_API_KEY= CESIUM_ION_TOKEN= # From ion.cesium.com/tokens LARK_APP_ID= # For project tracking LARK_APP_SECRET= LARK_BASE_APP_TOKEN= GITHUB_TOKEN= # Classic PAT with repo scope POSTGRES_URL= MINIO_ACCESS_KEY= MINIO_SECRET_KEY= DRONEDB_REGISTRY_URL= DRONEDB_JWT_SECRET= # Or JWKS URL if RS256 ``` --- ## Active Projects Reference - **InfraInspect** (TWC drone bridge inspection): sprint plan + architecture → `references/infra-inspect-sprint-plan.md` - **InfraInspect v2 design** (Cesium + revised UX): → `references/infrainspect-v2-stack-decisions.md` - **GeoKernel field software** (DataXquad drone mapping + AI inspection): → `references/geokernel-architecture.md` - **GeoKernel** (app-geokernel Electron field app): full architecture, IPC protocol, wse-sdk map, AI module template, camera back-projection → `references/geokernel-electron-architecture.md` - **GeoKernel / Wakemapper** (Electron desktop photogrammetry + drone app, Wakesight): full architecture, service inventory, wse-sdk gap analysis, AI integration pattern, porting analysis → `references/geokernel-electron-photogrammetry-architecture.md` - **GeoKernel AI modules** (inspection + emergency, June 2026): IPC pattern, camera back-projection math, wse-sdk dependency map, touch screen adaptation, Plane.so project GEO-2 to GEO-13 → `references/geokernel-drone-inspection-architecture.md` ## InfraInspect v2 Stack Decisions (2026-05) Decided after full gap audit + library freshness check: | Use | Skip | Reason | |-----|------|--------| | CesiumJS 1.141 + Resium 1.21 | Leaflet, OpenLayers, Potree | Single lib covers map+GeoTIFF+3D mesh+future point cloud | | react-zoom-pan-pinch 4.0 | OpenSeadragon | 2D ortho photo zoom only; Cesium cannot do 2D photo annotation | | CSS + SVG | Any annotation lib | Defect bbox overlay already 90% done with position:absolute | **Leaflet is outdated** — v1.9.4 frozen since 2022, v2.0 still alpha. Do not recommend. **OpenSeadragon** is for gigapixel pathology slides, overkill for drone JPEGs. **Potree** npm package (0.1.0) is dead — real Potree is GitHub-only, no React support. Cesium white-theme config for TWC (government bright aesthetic — NOT dark space): ```typescript viewer.scene.backgroundColor = Cesium.Color.fromCssColorString('#F0F5FA') viewer.scene.skyBox.show = false viewer.scene.skyAtmosphere.show = false viewer.scene.sun.show = false viewer.scene.moon.show = false new Cesium.OpenStreetMapImageryProvider({ url: 'https://tile.openstreetmap.org/' }) ``` **Cesium as lazy-loaded modal pattern**: load Cesium ONLY on /map route — do not bundle it into main chunk. Prevents 3MB+ cold-load penalty on Dashboard/BridgeList pages. ## Gap Audit Pattern Before planning geospatial sprints, always run a full gap audit: 1. `find src -name '*.tsx' | xargs grep -l 'mock\|Mock\|picsum'` — find pages still on mock data 2. `cat src/router.tsx` — find PlaceholderPage routes 3. `grep -r 'cesium\|leaflet\|openlayers' package.json` — confirm what libs are actually installed 4. Check backend models for missing GPS fields (lat/lon/alt) before planning map sprint 5. Cross-check PRD screen list vs actual routes — every PlaceholderPage is a sprint ## Reference Files - `references/geokernel-app-architecture.md` — Full GeoKernel (app-geokernel) architecture: Electron 40 + Vue 3 + CesiumJS stack, theme tokens, IPC pattern, wse-sdk gap, land-seg service (SAM3/TensorRT sm_86), camera back-projection math, Emergency Survey module design, GPS accuracy table, Plane project IDs. ## Pitfalls 1. **Never skip format research** — geospatial format bugs are silent. A GeoTIFF that uploads "successfully" can render 10,000 km from the correct location. 2. **CesiumJS cannot stream COPC natively** (as of 1.141) — the LAS→COPC→Cesium pipeline does not exist in the browser. Defer point clouds to post-MVP. 3. **DroneDB Registry JWT algorithm** — check RS256 vs HS256 in Sprint 1 before writing any auth code. Fixing this late means rewriting all JWT validation middleware. 4. **Vertical datum is mandatory metadata for DEMs** — GDAL strips vertical datum on reproject. Store it explicitly. Display it prominently. Never let a DEM upload complete without declaring it. 5. **DXF has no CRS** — document this prominently. Users from civil engineering backgrounds often use local coordinate systems in DXF. 6. **3D Tiles 1.0 vs 1.1** — CesiumJS 1.141 has experimental 1.1 support only. Validate and only guarantee 1.0 in MVP. 7. **OBJ must be uploaded with all textures in same batch** — MTL references texture filenames; if any are missing the conversion silently produces an untextured grey mesh. 8. **GeoJSON is always loaded entirely into browser memory** — no server-side tiling in MVP. Warn users for files >50 MB. 9. **`EllipsoidTerrainProvider` vs `CesiumTerrainProvider`** — default to Ellipsoid (flat globe, no Ion dependency) for air-gapped deployments. Offer Ion terrain as opt-in via env var. 71. **CesiumJS is wrong for photo annotation viewers and government/enterprise UI** — CesiumJS is the right tool for 3D terrain + georeferenced raster draping. It is the WRONG tool for: (a) drone photo viewers with defect bbox overlays (use OpenSeadragon + Canvas/SVG), (b) simple bridge/asset location maps (use Leaflet.js — 45KB vs 3MB, white-themed, no Ion dependency), (c) GeoTIFF 2D overlay without terrain (use OpenLayers + geotiff.js). Government/enterprise products with bright white design systems (#FFFFFF backgrounds, no dark mode) must not embed Cesium's dark WebGL canvas in the main layout — it violates every contrast and aesthetic guideline. Pattern: **Cesium as a lazily-loaded modal** (open on explicit user click) rather than as the page shell. This avoids 3-5s cold-start on field tablets, prevents design pollution, and scopes the 3D feature correctly for MVP. See `references/infrainspect-viewer-technology-decision.md` for full analysis. 72. **Component port over codebase merge when two products share a tech stack** — When Product B needs viewer components that Product A already has working (e.g. GeoTiffLayer.tsx, MeshLayer.tsx), copy the individual files rather than merging the two products. Merging is high-risk when: different domain models, different auth systems, different brand/compliance requirements, separate infrastructure stacks, working E2E test suites that a merge would break. The correct pattern: identify the 3-5 self-contained component files that solve the hard problem (they are usually small, 50-100 LOC each), copy them into the target project, adapt props to the target's domain context. This is "borrowing a library", not "merging products". 10. **Nexus (.nxs) files from DroneDB Hub** — CesiumJS has no Nexus loader. There is no conversion path from .nxs back to OBJ without data loss. Reject explicitly with clear message. 11. **LAS/LAZ can be added to MVP if owner insists** — the default is to defer point clouds because the pipeline (PDAL → py3dtiles → 3D Tiles PNTS) adds a dedicated worker container. When the user adds LAS/LAZ back, add Sprint N dedicated to that pipeline and add a `gx-worker` service with Python 3.12 + Node.js 20 (for obj2gltf) + PDAL. COPC stays out regardless — CesiumJS 1.141 has no native COPC reader. 12. **DroneDB Registry image tag** — always pin to a specific version (`dronedb/registry:2.3.1`), never `:latest`. The Registry's GDAL version affects COG mask behaviour (nodata transparency bug in GDAL 3.11.x). 13. **Integration seam failures are the #1 Sprint 2 blocker** — the backend logic (CRS validation, format rejection, DEM detection) tends to be correct. What breaks is the contract between layers: (a) frontend calls wrong API URL paths, (b) error code strings don't match between validator and tests (`unsupported_format` vs `format_not_supported`), (c) vertical datum dialog defaults to a value instead of null so the Confirm button is never truly blocked, (d) vertical datum option list in frontend doesn't match backend whitelist (e.g., frontend has MSL, backend doesn't). Review these seams explicitly — they won't show up in unit tests. 14. **VerticalDatumDialog must default to null — not a value** — the whole point of the dialog is to force explicit user selection. If `selectedDatum` initialises to any value (EGM96, WGS84, anything), the Confirm button is enabled from the start, reintroducing the silent datum failure that RISK-03 mitigates. Reviewer must check: `useState(null)` and `disabled={selectedDatum === null}`. 15. **Frontend URL paths diverge from backend routes silently** — `gxClient.ts` is written by the frontend agent without reading the actual router files. The agent invents plausible-sounding paths (`/api/v1/uploads/...`) that don't exist in the backend. The reviewer must be explicitly instructed to cross-reference `gxClient.ts` URLs against the actual router files (`upload.py`, `vertical_datum.py`, `datasets.py`). Add this as an explicit check in the sprint's Claude Code review prompt. 16. **App.tsx needs React Router wired — frontend pages are unreachable otherwise** — the dev agent creates LoginPage, ProjectListPage, DatasetPage as components, but leaves App.tsx as a placeholder. Without React Router, none of the pages are accessible from the running frontend. Always include App.tsx router wiring in the frontend dev task description. 17. **`docker compose up` workflow scope** — pushing `.github/workflows/ci.yml` requires the `workflow` PAT scope. If the token lacks it, push CI config separately after adding the scope. See `github-repo-management` → `references/pat-scope-pitfalls.md`. 42. **`check_integrity.sh` baselines drift silently sprint-over-sprint** — the hardcoded baseline values in `check_integrity.sh` (`baseline=4`, `baseline=2`) fall behind the real error counts as new code is added. Before writing a new sprint's QA script, always re-measure the actual current counts: ```bash cd services/gx-api && pyrefly check 2>&1 | python3 -c "import sys,re; m=re.search(r'INFO (\d+) errors', sys.stdin.read()); print('PYREFLY:', m.group(1))" cd ../.. && ./node_modules/.bin/biome check services/frontend/src/ 2>&1 | python3 -c "import sys,re; m=re.search(r'Found (\d+) errors', sys.stdin.read()); print('BIOME:', m.group(1))" ``` Bake those actual numbers into the new sprint's QA script. Never copy the prior sprint's baseline blindly — it will show false REGRESSION on the first run. 43. **Crew-based UX review produces static analysis only if Playwright never runs** — when a CrewAI UX review crew's Tester agent fails to execute Playwright (stdin timeout, missing config, missing test user), the Reviewer agent will still produce a polished bug report — but every verdict is source-code inspection, not browser evidence. The reviewer MUST label this explicitly as "static analysis only" and treat the absent Playwright run as a CRITICAL bug itself (same severity as a missing feature). A "code looks correct" verdict is meaningless for demo readiness. Always verify `ux-screenshots/` is non-empty before accepting any PASS verdict. 44. **Playwright stdin timeout when passing large prompts to Claude Code** — a Tester agent that constructs a 12KB+ Playwright task and passes it via `$()` shell substitution will silently fail: Claude Code prints `"no stdin data received in 3s, proceeding without it"` and produces no output. Fix: write the Playwright spec to a file via Claude Code first (`--allowedTools Write --max-turns 20`), then run `npx playwright test` as a separate bash call. Never pass large prompts via shell substitution inside a CrewAI tool call. 45. **Pre-wire check before writing Playwright specs** — before a Tester agent writes any journey spec, the Planner must verify features are actually reachable (not just implemented in isolation): ```bash grep -r "ShareDialog" services/frontend/src/pages/ # must return hits grep -r "AnnotationPanel\|annotMode" services/frontend/src/pages/DatasetPage.tsx ``` A fully-built component that is never imported by any page is unreachable — tests for it will always fail and the spec will mislead the reviewer. Flag these as CRITICAL in the test plan. 19. **Caddy `handle_path /api/*` strips the prefix — use `handle /api/*` instead** — `handle_path` in Caddy strips the matched prefix before proxying. So `handle_path /api/*` causes FastAPI to receive `/v1/projects` instead of `/api/v1/projects`, producing 404 on every API call. Every route would appear broken in Docker even though the code is correct. Fix: use `handle /api/*` (no path stripping) and keep router prefixes as `/api/v1/...`. Add a specific check in every Sprint 3+ reviewer prompt: `grep "handle /api" services/caddy/Caddyfile` must NOT show `handle_path`. 20. **Graphify write-first pattern for large codebases** — standard `--max-turns 60` Graphify builds fail on repos with 50+ files: agent spends all turns reading and never writes the report (result: 4-line skeleton file). Fix: use a **section-by-section write-first prompt** with `--max-turns 80`. Each step must say "Read X → immediately update section Y in the file". Key instruction: "After EVERY bash read command, immediately write findings to GRAPH_REPORT.md. Never batch reads." Confirm success by checking file line count: `wc -l graphify-out/GRAPH_REPORT.md` — should be 200+ lines for a real report. 21. **Graphify reveals critical Sprint N bugs before the crew runs** — run Phase 0 ingestion (GitNexus + Graphify) after every sprint commit, not just at project start. The post-Sprint-2 graph surfaced 4 real bugs (DatasetPage prop vs useParams, Caddy prefix strip, dead projectStore, upload_validator extension gap) that would have been Sprint 3 blockers if not documented in CLAUDE.md and GRAPH_REPORT.md before the crew started. This is the primary value of the graph: finding integration seam failures between layers. 22. **Pyrefly + Biome as CrewAI pipeline guardrails** — wire both static checkers into all 3 layers of every sprint crew: Dev agent (Claude Code prompt includes `bash scripts/check_integrity.sh` instruction), QA script (mandatory Section 2 with baseline gate), Reviewer agent (reads integrity log, regression = BLOCKER). Biome baseline improves over time as code quality improves — update `check_integrity.sh` and the QA script after each sprint. See `ai-native-dev-workflow` → `references/sprint-crew-review-patterns.md` for baseline update pattern. 23. **Pyrefly config: flat TOML in `pyrefly.toml`, not `[tool.pyrefly]`** — Pyrefly reads flat keys (`python-interpreter`, `project-includes`, `project-excludes`) directly in `pyrefly.toml`. Using `[tool.pyrefly]` in `pyproject.toml` causes silent config ignore (`WARN: Extra keys`). Run `pyrefly init` inside the service directory (`services/gx-api/`) to generate a valid config, then customise. Run pyrefly FROM that directory. 24. **Biome v2 schema keys differ from v1** — Biome 2.x: `files.includes` (not `include`), `files.experimentalScannerIgnores` (not `ignore`), no `noVar` in `style` rules, `organizeImports` under `assist.actions.source`. Check the installed version first: `./node_modules/.bin/biome --version`. For v2, run `biome init` in a temp dir and use its generated skeleton as the base. 25. **Crew script SPRINT_SCOPE triple-quote pitfall** — avoid using Python triple-quoted strings (`"""..."""`) for SPRINT_SCOPE when the content includes em-dashes (`—`), Unicode arrows (`→`), or nested docstrings. Python raises `SyntaxError: invalid character` at runtime. Fix: use string concatenation (`SPRINT_SCOPE = ("line1\n" "line2\n" ...)`) with ASCII substitutes for special characters. Always verify with `python3 -m py_compile crew_sprintN.py` before launching. 26. **LAS/LAZ magic-byte check belongs in the API validator, not only the worker** — the worker pipeline (PDAL) validates deeply, but a lightweight LASF magic-byte check (`file_header[:4] == b"LASF"`) must also run in the FastAPI validator to fail fast before the file reaches the queue. `.laz` files compress the LAS header — magic check for LAZ is deferred to PDAL. Signature: `validate_las(filename, file_size, file_header=None)` — `file_header` optional so tests can omit it. 27. **py3dtiles CLI for LAS → PNTS conversion** — `py3dtiles convert {input.las} --out {output_dir} --overwrite`. Preceded by `pdal translate {input.laz} {decompressed.las}` for LAZ files. Both are subprocess calls (Docker-only). All tests must mock `subprocess.run`. Import `py3dtiles` inside `try/except` with `# pyrefly: ignore[missing-import]` just like osgeo. Output: `{output_dir}/tileset.json` + `{output_dir}/*.pnts`. 28. **Biome error counts improve sprint-over-sprint** — new TypeScript code written by Claude Code agents tends to be cleaner than Sprint 2 legacy code. Biome baseline went 22→11→11→2 across Sprints 2-5. Always run `bash scripts/check_integrity.sh` before writing a new sprint's QA script to get the real current Biome count. Never copy the prior sprint's baseline blindly. 29. **Python `5_000` numeric separator breaks bash grep** — QA scripts that check for polling intervals using `grep "5000"` will false-FAIL when source code uses `5_000`. Always use Python regex via the `check_regex()` helper: pattern `5[_]?000` matches both forms. The same applies to any numeric constant check: `10_000`, `80_000`, etc. 18. **Sprint 1 infra gate: verify JWT algo before writing auth code** — run `scripts/verify-jwt-algo.sh` against a live Registry before Sprint 2. DroneDB Registry may issue RS256 (JWKS) or HS256 (shared secret) tokens; the FastAPI middleware must match. Getting this wrong means all auth fails silently or accepts forged tokens. 30. **DroneDB Registry Docker Hub tag uses `v` prefix** — `dronedb/registry:2.3.1` returns "not found". Correct tag: `dronedb/registry:v2.3.1`. Always verify tags via Docker Hub API before committing. See `ai-native-dev-workflow` → `references/docker-first-deploy-pitfalls.md` for all first-deploy version pitfalls. 31. **GDAL Python wheel must match system `libgdal-dev` version** — `python:3.12-slim-bookworm` (Debian 12) ships GDAL 3.6.2 via apt. Pinning `gdal==3.8.5` in `requirements.txt` causes `Getting requirements to build wheel: error` because the wheel compiles against `gdal-config --version`. Fix: run `docker run --rm python:3.12-slim-bookworm bash -c 'apt-get install -y -qq libgdal-dev && gdal-config --version'` to find the system version, then pin to match. 32. **PDAL is not in Debian trixie apt** — `apt-get install pdal` returns `E: Unable to locate package pdal` on Debian 13. `pip install pdal` also fails (requires `libpdal-dev` C++ headers). For MVP demo deploys, skip PDAL — LAS/LAZ conversion task queues but never runs; upload still works and the format is accepted. For production, use `FROM pdal/pdal:latest AS pdal-base` multi-stage build. 33. **Vite `VITE_*` env vars baked at build time, not runtime** — setting `VITE_API_URL` under `docker-compose.yml environment:` does nothing to the bundled JS. Vite reads env vars when `npm run build` runs. The Dockerfile must have `ARG VITE_API_URL` + `ENV VITE_API_URL=$VITE_API_URL` BEFORE `RUN npm run build`. The compose file must pass them under `build.args:` (not `environment:`). Without this, `baseURL` in `gxClient.ts` falls back to its default and ALL API calls go to the wrong host. 34. **Rollup import errors not caught by `tsc --noEmit`** — `export default Foo` + `import { Foo }` is a TypeScript error that `tsc --noEmit` misses but Rollup (Vite's bundler) catches at build time. The only way to detect this is to actually run `npx vite build`. Run it locally before pushing, or discover it during Docker build. Fix by adding `export { Foo };` alongside `export default Foo;` in the source file. 35. **PostgreSQL image does not auto-create the app database** — even with `POSTGRES_USER` and `POSTGRES_PASSWORD` set, the image only creates the database named by `POSTGRES_DB` (default: `postgres`). If the app's `DATABASE_URL` points to `gx`, Alembic will fail with `FATAL: database "gx" does not exist`. Post-deploy manual step: `docker exec gx-postgres psql -U $POSTGRES_USER -c "CREATE DATABASE gx;"`. Or set `POSTGRES_DB=gx` in the compose file to have it auto-created. 37. **Alembic ENUM creation fails on partial previous run** — if a migration that creates a PostgreSQL ENUM type (e.g. `dataset_status`) was run previously and failed mid-transaction, the type may already exist. `sa.Enum(...).create(checkfirst=True)` and `create_type=False` both still fail in SQLAlchemy 2.0.36 because the `before_create` event fires regardless. Fix: replace both patterns with an `op.execute()` DO block + `postgresql.ENUM(create_type=False)`: ```python op.execute( "DO $$ BEGIN " "CREATE TYPE dataset_status AS ENUM ('pending','processing','completed','failed'); " "EXCEPTION WHEN duplicate_object THEN null; " "END $$;" ) dataset_status = postgresql.ENUM("pending","processing","completed","failed", name="dataset_status", create_type=False) ``` Also: always connect as `$POSTGRES_USER` (not `postgres`) and target the `postgres` maintenance DB: `docker exec gx-postgres psql -U $POSTGRES_USER -d postgres -c "CREATE DATABASE gx;"`. 38. **FastAPI 0.115.0 raises `AssertionError: Status code 204 must not have a response body` at startup** — this fires when a `@router.delete` with `status_code=HTTP_204_NO_CONTENT` has a non-trivial return type or response_class. The simplest fix: change all delete endpoints to `status_code=HTTP_200_OK` and return `{"deleted": True}`. Batch-fix with Python regex across all router files. Then `sudo docker cp services/gx-api/app/routers/. gx-api:/app/app/routers/ && sudo docker restart gx-api`. 39. **Overnight orchestrator notifications via Feishu** — for multi-sprint overnight builds, write a standalone `dev-output/feishu_notify.py` script that uses the Feishu IM API to send messages to the project group chat. Key points: - Get the `CHAT_ID` from `~/.hermes/processes.json` → `watcher_chat_id` field - Always hardcode `DOMAIN = "https://open.larksuite.com"` — never use `os.environ.get("FEISHU_DOMAIN")` (that env var is a connection mode tag like `"lark"`, not a URL) - Use `urllib.request` (stdlib) so no extra deps needed - Send a test message before launching the overnight build to confirm it works - In the overnight shell orchestrator, call `python3 dev-output/feishu_notify.py "message"` at each milestone (sprint start, sprint approved/blocked, health audit, final complete) 40. **DroneDB Registry `/users/authenticate` is multipart/form-data, PascalCase fields** — NOT JSON, NOT camelCase. `{"username":"admin"}` returns `{"error":"Value cannot be null."}`. Correct curl: `-F "Username=admin" -F "Password=pass"`. The Swagger schema at `/swagger/v1/swagger.json` shows `multipart/form-data` encoding for this endpoint. Any frontend agent generating a JSON `axios.post` to this endpoint will produce silent 500s. Always specify form encoding explicitly in the sprint task for the login API call. See `references/dronedb-registry-ops.md`. 41. **`gxClient.ts` login path diverges from actual Registry route** — the frontend agent commonly generates `/registry/api/login` as the login URL (inventing a plausible-sounding path). The actual endpoint after Caddy strips `/registry` is `/users/authenticate` (so frontend path = `/registry/users/authenticate`). Reviewer must cross-check `gxClient.ts` login call against the Registry Swagger at `/swagger/v1/swagger.json`. This is a silent 500 that looks like an auth failure. 42. **DroneDB Registry default admin password is `password123`** — seeded at first container launch via `appsettings.json` `DefaultAdmin.Password`. Change immediately in production. User exists as `admin` / `admin@example.com`. No self-registration in GeoShare frontend was built (Sprints 1–9); users must be created by admin via Registry API or the DroneDB Hub UI at `http://HOST/registry/`. See `references/dronedb-registry-ops.md`. 43. **Annotation router double-prefix bug (`/api/v1/api/v1/...`)** — confirmed in Sprint 14 openapi.json, **fixed Sprint 16**. When the annotations router is mounted with `include_router(..., prefix="/api/v1")` inside a router that is ALREADY prefixed with `/api/v1`, the result is `/api/v1/api/v1/datasets/{id}/annotations`. Fix: change router prefix from `/api/v1/datasets` to just `/datasets` — main.py provides the `/api/v1` prefix. Detection: `curl http://localhost:8000/openapi.json | python3 -c "import sys,json; [print(p) for p in json.load(sys.stdin)['paths'] if 'annot' in p]"` — if any path starts with `/api/v1/api/v1`, the bug is present. 49. **Frontend list endpoint response-shape mismatch — always verify backend return type before typing** — A list endpoint that returns a bare `[]` array MUST be typed as `gxClient.get()` reading `response.data`. If typed as `gxClient.get<{ items: T[] }>()` reading `response.data.items`, the result is always `undefined` with no runtime error — just a permanently empty list. Sprint 16 instance: `listAnnotations` was reading `.annotations` from a bare array response. Rule: before writing any frontend API method for a list endpoint, read the exact backend `return` statement and match the TypeScript generic to the actual shape. See `references/ux-review-bug-patterns.md` Pattern 6. 44. **`AUTH_PROVIDER=dronedb` disables `/auth/register` silently** — when `AUTH_PROVIDER=dronedb` (the default), `POST /auth/register` returns `{"detail": "Not Found"}` and `POST /auth/login` returns `{"detail": "Local auth is not enabled."}`. Test users for Playwright/E2E must be created via DroneDB Registry admin API (`POST http://localhost:5000/users` with admin Bearer token). Alternative: set `AUTH_PROVIDER=dual` in `.env` and restart gx-api. The front-end RegisterPage still renders a form but all submissions fail until local auth is re-enabled. 42. **CrewAI UX/browser crew: Playwright spec must be written to file, not passed via stdin** — When a CrewAI tester agent passes a large Playwright spec to Claude Code via `$()` shell substitution, Claude Code prints "no stdin data received in 3s" and executes nothing. The only reliable pattern: have Claude Code WRITE the spec files first (`Write` tool), then run `npx playwright test` in a separate bash call. All "PASS" verdicts from a crew that never ran Playwright are static code analysis, not runtime evidence. See `references/sprint-crew-crash-patterns.md` Crash 6. 43. **passlib 1.7.4 + bcrypt ≥4.x incompatibility crashes every hash_password call** — passlib's internal self-test sends a >72-byte secret which bcrypt 4+ rejects in strict mode. Symptom: HTTP 500 on every registration/login with `AttributeError: module 'bcrypt' has no attribute '__about__'`. Fix: remove passlib, call `bcrypt` directly. See `references/sprint-crew-crash-patterns.md` Crash 7. 44. **SQLAlchemy MissingGreenlet when lazy relationship accessed after async commit** — `issue_token_pair()` accesses `user.organisation_memberships`. If the User object was returned after `await db.commit()` without eager loading, the lazy loader fires a sync DB call and crashes with `MissingGreenlet`. Fix: reload the user with `selectinload(User.organisation_memberships)` after commit. Applies to any code path where a User is committed then passed to a function that reads relationships. See `references/sprint-crew-crash-patterns.md` Crash 8. 45. **slowapi rate limiter + asyncpg = MissingGreenlet on first real request** — `slowapi 0.1.9` with Redis backend uses sync storage internally. When `@limiter.limit("5/minute")` decorates an async FastAPI handler, it triggers a sync DB ping inside the async event loop. Symptom: 500 on every request to the decorated endpoint, even when the endpoint code is correct. Fix for dev: `RATE_LIMIT_ENABLED=false` in `docker-compose.yml` environment block. See `references/sprint-crew-crash-patterns.md` Crash 9. 46. **docker-compose.yml env var changes don't reach running containers without full kill+rm+up** — `docker compose restart` and `docker compose up -d` do NOT recreate containers that are already running with the same config-hash. New env vars silently never appear. Fix: `sudo docker kill && sudo docker rm && docker compose up -d `. Also: `${VAR}` in environment blocks resolves from the root `.env` at compose read-time — if the variable isn't in `.env`, the value is empty string even if the var is in the environment block. See `references/sprint-crew-crash-patterns.md` Crash 10. 47. **check_integrity.sh baselines become stale sprint-over-sprint** — the baselines in `check_integrity.sh` must be updated after each sprint that introduces new code. Stale baselines cause false REGRESSION reports that block the QA agent. Pattern: after every sprint, run `bash scripts/check_integrity.sh` manually, read the actual counts, and patch the baseline values in the script. Baselines for GX-GeoShare as of Sprint 15: Pyrefly=24, Biome=25. 48. **UX reviewer without live browser evidence produces misleading verdicts** — when Playwright never actually ran (due to stdin timeout or missing config), the reviewer produces "code review" verdicts labelled as PASS/FAIL. These are not user-journey evidence. Always check `dev-output/ux-screenshots/` for real screenshot files and `ux-test-results.txt` for non-empty Playwright output before trusting any UX review verdict. 43. **authClient.ts login must send JSON `{email, password}`, not URLSearchParams** — FastAPI `LoginRequest` is a Pydantic model reading JSON body. Sending `Content-Type: application/x-www-form-urlencoded` with `username=` field returns 422. Always cross-reference authClient's Content-Type and field names against the backend schema. This breaks Playwright login tests silently — the alert on screen reads "Input should be a valid dictionary or object to extract fields from". 44. **Frontend `serve -s dist` container does NOT hot-reload** — If the container runs `node serve -s dist` (check with `docker exec gx-frontend ps aux`), it serves a static pre-built bundle. Editing `src/` files has zero effect. Any authClient, component, or page change requires `docker compose build frontend` + stop + rm + up. Never waste a debugging session on this — check the process first. 45. **`docker compose.override.yml` environment block may not apply on restart** — `docker compose restart` never recreates containers; env vars added to override are ignored. `docker compose up -d` skips recreation if config-hash matches. The only reliable sequence: `docker kill + rm + docker compose -f compose.yml -f override.yml up -d`. Also: `${VAR}` values in `environment:` resolve from the root `.env` at compose read-time — if the var is not in `.env`, the value is empty string even if `environment:` looks correct. 46. **Playwright strict-mode violation on button selectors** — `/create/i` regex matches multiple buttons (e.g., both "Create" and "Create Project"). Always use `{ name: 'Create', exact: true }` when the button label is a common word. Use exact placeholder text `input[placeholder="Project name"]` not fuzzy `placeholder*=name`. Strict-mode violations abort the test with no screenshot evidence. 47. **Playwright test cascade: fix each test to be self-contained** — Tests that depend on state created by a prior test (project created in test 3, share URL captured in test 4) will all cascade-fail when test 3 fails. Fix: each test logs in fresh; test 7 (anonymous share) has a self-healing fallback that calls the API directly to create a real share link if `shareUrl` is empty. 54. **`handleCreate()` must call `navigate()` after project creation — React omission** — `createProject()` API call succeeds and project is added to local state, but the form closes and user stays on the list page. Playwright `waitForURL(/\/projects\/.+/)` then times out — yet the project was created successfully. Fix: add `useNavigate` hook and call `navigate(\`/projects/${created.id}\`)` immediately after `setCreateOpen(false)`. **Diagnosis shortcut:** take a screenshot at the `waitForURL` timeout point — if the project card is visible on the list page, navigation was never triggered (API worked fine). Root cause is always a missing `navigate()` call. 55. **`docker compose up -d ` may be blocked — use `sudo docker restart ` directly** — Hermes blocks `docker compose up -d` as a long-lived process. Use `sudo docker restart ` instead. Container names differ from compose service names — run `sudo docker ps --format "{{.Names}}"` first to find the exact name (e.g. `gx-frontend` not `gx-geoshare-frontend-1`). After restart, verify: `curl -s -o /dev/null -w "%{http_code}" http://HOST/` should return 200. 56. **GX-GeoShare auth router is at `/auth/...` not `/api/v1/auth/...`** — `app.include_router(auth_router, prefix="/auth")` with no `/api/v1` prefix. Login URL is `http://HOST/auth/login`, register is `http://HOST/auth/register`. When writing test curl commands or Playwright API calls, verify first: `curl -s http://localhost:8000/openapi.json | python3 -c "import sys,json; [print(p) for p in json.load(sys.stdin)['paths'] if 'auth' in p]"`. 57. **Diagnose Playwright `waitForURL` timeout by visually inspecting the failure screenshot** — the timeout error message is useless. The PNG at `dev-output/playwright-test-results//test-failed-1.png` shows exactly what was on screen. Use `vision_analyze()` on it before reading the code — in Sprint 17 the screenshot revealed the project card was already visible on the list page, immediately pointing to missing `navigate()` rather than an API failure. Always look at the screenshot first. 36. **Use port 80 when GCP firewall scope is unavailable** — if the VM's service account lacks `compute.firewalls` permission, `gcloud compute firewall-rules create` fails with `insufficient authentication scopes`. Workaround: change Caddy to listen on port 80 (which is already open via the `http-server` network tag) instead of any custom port. Update both the `Caddyfile` (`:80 {`) and `docker-compose.yml` (`- "80:80"`). Access the app at `http://VM_IP` with no port suffix. 42. **`passlib 1.7.4 + bcrypt ≥4.x` incompatibility crashes every `CryptContext` instantiation** — passlib's internal bug-detection test passes a >72-byte secret which bcrypt 4+ rejects in strict mode, causing `ValueError: password cannot be longer than 72 bytes` on every `hash_password()` call. Fix: bypass passlib entirely and call `bcrypt` directly. In `security.py`: `import bcrypt as _bcrypt_lib`, then `_bcrypt_lib.hashpw(plain.encode("utf-8"), _bcrypt_lib.gensalt(rounds=rounds))` for hash and `_bcrypt_lib.checkpw(plain.encode(), hashed.encode())` for verify. Drop `from passlib.context import CryptContext` and `_get_pwd_context()` entirely. 43. **SQLAlchemy `MissingGreenlet` from lazy-loaded ORM relationships in async FastAPI endpoints** — accessing a lazy-loaded relationship (e.g. `user.organisation_memberships`) after `await db.refresh(user)` triggers a sync connection attempt inside an async context, crashing with `sqlalchemy.exc.MissingGreenlet: greenlet_spawn has not been called`. Fix: replace `await db.refresh(user)` with an explicit selectinload query: `result2 = await db.execute(select(User).options(selectinload(User.organisation_memberships)).where(User.id == user.id)); user = result2.scalar_one()`. Apply this pattern in EVERY endpoint that returns a User and then passes it to `issue_token_pair()` (register, login, refresh). Never use `db.refresh()` on a model that has lazy relationships accessed downstream. 44. **`slowapi` rate limiter + asyncpg = `MissingGreenlet` in async FastAPI** — `slowapi 0.1.9` with a Redis storage backend performs sync Redis calls inside async endpoints via the `limits` library. This triggers `MissingGreenlet` on any endpoint decorated with `@limiter.limit()`. Fix for dev: set `RATE_LIMIT_ENABLED=false` in `docker-compose.yml` environment block (not just `.env` — the env file isn't loaded by the container). Also remove `@limiter.limit()` decorators from new auth endpoints (`/register`, `/login`) that touch the DB. Note: setting env vars in `.env` alone does NOT propagate to the container — add them explicitly to the `environment:` block in `docker-compose.yml`. 45. **FastAPI local JWT (HS256) doesn't populate `request.state.org_id` / `user_id`** — all routers downstream read `request.state.org_id` and `request.state.user_id` directly (not from the returned payload dict). The DroneDB path populates these via the existing JWT middleware, but the local DUAL-auth path in `get_current_user()` only returns the payload dict without writing to `request.state`. Fix: in the DUAL path, after successful `decode_access_token(token)`, add: `request.state.user_id = payload.get("sub", ""); request.state.org_id = payload.get("org_id", ""); request.state.email = payload.get("email", "")` before returning the payload. Without this, every router call returns HTTP 500 `'State' object has no attribute 'org_id'`. 46. **`authClient.ts` login must use JSON, not URLSearchParams** — the FastAPI `LoginRequest` Pydantic model reads a JSON body `{email, password}`. Sending `application/x-www-form-urlencoded` with `username=` returns HTTP 422 `Input should be a valid dictionary or object`. The frontend `authClient.ts` was initially written with `URLSearchParams + body.append("username", email)` — this must be changed to `axios.post("/auth/login", { email, password }, { headers: { "Content-Type": "application/json" } })`. Field name must be `email` not `username`. 47. **Frontend static bundle is NOT hot-reloaded — changes to `.ts` source require a Docker rebuild** — the production frontend container runs `serve -s dist` (not Vite dev server). The `services/frontend/src` volume mount only serves hot-reload during `vite dev`, not when the container runs the pre-built bundle. Any change to `authClient.ts`, `gxClient.ts`, or any other TS file requires: `sudo docker compose build frontend` followed by a force-recreate (`sudo docker stop gx-frontend && sudo docker rm gx-frontend && sudo docker compose up -d frontend`). Use `sudo docker inspect gx-frontend --format '{{json .Config.Env}}'` to verify new env vars are present in the running container. 48. **`docker compose up -d --force-recreate` silently fails when `container_name` conflicts** — if a named container already exists and is running, `--force-recreate` may not apply without an explicit stop+rm first. Pattern that reliably works: `sudo docker stop && sudo docker rm && sudo docker compose up -d `. Verify by checking the container ID changed: `sudo docker ps | grep `. 49. **Playwright `waitForURL` timeouts caused by login returning 500** — if the login endpoint is broken (e.g. `MissingGreenlet`, wrong content-type, missing JWT secret), the browser stays on `/login` and `waitForURL(/\/projects/)` times out with no useful error message. Always diagnose by reading `error-context.md` in `dev-output/playwright-test-results/` — the YAML page snapshot shows any error alert rendered on screen (e.g. `alert: Input should be a valid dictionary or object`). 50. **Playwright strict-mode violation from duplicate button names** — `getByRole('button', { name: /create/i })` resolves to multiple elements when the page has both a `Create` button (exact) and a `Create Project` button. Fix: use `{ name: 'Create', exact: true }` for the exact match. Always read the strict-mode error message — it lists all matching elements with their exact text. 51. **Playwright `shareUrl` cross-test variable is empty when upstream test failed** — if test N captures a URL into a module-level `let shareUrl = ''` variable and test M relies on it, test M gets an empty string when test N failed. Fix: make test M self-healing — check `if (!shareUrl)` and call the API directly (login → list projects → create share link) to get a real URL before proceeding. Never use a fallback like `/share/testtoken` — it will 404 and the test will produce misleading failures. 52. **Playwright email locator ambiguity in ShareDialog** — after share links are generated, there may be multiple `input[type=email]` elements on the page. Use `.last()` not `.first()` for the notify-email field, since the login email field (from a previous form) may still be in the DOM. `page.locator('input[placeholder*=email i]').last()` is more reliable. 53. **`check_integrity.sh` baselines must be updated after each sprint** — the script ships with Pyrefly baseline=4, Biome baseline=2 (Sprint 2 era). As the codebase grows these become stale and the script reports false regressions every sprint. After any sprint that introduces new code, run the actual checkers, read the counts, and patch `check_integrity.sh` to set `PYREFLY_BASELINE` and `BIOME_BASELINE` to the current values. Never copy a prior sprint's baseline blindly. See `references/multi-spa-subpath-serving.md` for: Vite `base:` config, Caddy `handle_path` + `file_server` + `try_files`, volume mount pattern, docker-compose.yml changes, and deploy workflow for a second SPA on the same GCP VM. See `references/drone-inspection-standards.md` for the corrected drone-survey standards framework (ISO 13822 L1-2, PAS 125, V1-V5 severity scale, water aqueduct defect taxonomy). Use this any time a product involves drone-based infrastructure inspection — CNS 12891 requires physical contact and does NOT apply to drone-only surveys. See `references/twinspect-competitive-analysis.md` for Twinspect by Twinsity competitive intelligence (features, gaps, adoption recommendations) scraped May 2025 — use when building any drone inspection platform to identify differentiators and features to borrow. See `references/sprint-chain-infra-workarounds.md` for docker socket / alembic port-mapping / BashTool .env injection patterns when running a sprint chain on a VM without docker group access. See `references/sprint-chain-block-recovery.md` for the BLOCK recovery pattern — when the dev agent builds components but misses page files / router wiring, Hermes writes them directly and restarts the chain rather than re-running the full sprint crew. See `references/twc-infra-inspect-domain-notes.md` for Taiwan Water Corporation domain notes (standards, severity system, brand). See `references/infrainspect-twc-architecture.md` for: TWC brand tokens, V1–V5 severity scale, defect types, 4-view ortho system, 10 mock bridges, all page routes/components/types, sprint plan, and competitive position vs Twinspect. See `references/infrainspect-v2-design-decisions.md` for: final geo/viewer library decisions (CesiumJS replaces Leaflet/OpenLayers/Potree), Cesium white-theme init code, v2 screen inventory (13 routes), user journeys per role, OrthoPhotoViewer + DefectBboxOverlay architecture, RBAC roles, sprint 9-16 execution order, **exact TypeScript type schema for Defect/ViewDirection/OrthoPhotoViewer props (Sprint 12 authoritative)**, and CRITICAL backend runtime identity (system Python 3.10, not .venv). See `references/infrainspect-viewer-technology-decision.md` for: CesiumJS vs Leaflet/OpenSeadragon/OpenLayers analysis for InfraInspect, why Cesium clashes with TWC design language, right-tool-per-job table, component-port vs merge decision, and recommended sprint sequencing for geospatial features (S9–S12). See `references/upload-and-container-pitfalls.md` for: Axios timeout three-layer fix, pending-dataset diagnosis flowchart, stopped-container conflict (`docker rm -f` fix), SQLAlchemy MissingGreenlet on `/auth/refresh`, and CrewAI `output_file=` silent failure + log-extraction recovery pattern. See `references/gx-geoshare-local-auth-bugs.md` for the full diagnostic trace of bugs 42-46. See `references/dronedb-registry-ops.md` for DroneDB Registry v2.3.1 auth endpoints (form-encoded, NOT JSON), dataset creation, file upload with required `path=` field, default credentials, SPA 500 behaviour, and correct tile URL patterns. See `references/docker-build-deploy-pitfalls.md` for Vite `--build-arg` requirement, `docker restart` keeping old image, network alias loss after stop+rm+run, and container name vs compose service name patterns. 42. **SQLAlchemy async + lazy-load = `MissingGreenlet` — MUST use `selectinload`** — Any SQLAlchemy ORM model with `lazy="select"` relationships (e.g. `organisation_memberships`) will crash with `sqlalchemy.exc.MissingGreenlet: greenlet_spawn has not been called` when accessed outside an async context. This fires when `db.refresh(user)` is followed by code that touches a lazy relationship (e.g. `user.organisation_memberships` in `issue_token_pair`). The fix — apply to EVERY `db.refresh()` call that is followed by relationship access: ```python from sqlalchemy.orm import selectinload result2 = await db.execute( select(User) .options(selectinload(User.organisation_memberships)) .where(User.id == user.id) ) user = result2.scalar_one() # do NOT call await db.refresh(user) — the selectinload query replaces it ``` Apply this pattern in `register_user`, `authenticate_user`, and any other service function that returns a User to code that accesses relationships. The symptom is always a `MissingGreenlet` traceback going through `sqlalchemy/orm/session.py _connection_for_bind` and `sqlalchemy/orm/strategies.py _emit_lazyload`. 43. **`passlib 1.7.4 + bcrypt ≥4.x` = crash on every `CryptContext` instantiation** — passlib's internal self-test passes a >72-byte secret to bcrypt 4+, which now enforces strict mode and raises `ValueError: password cannot be longer than 72 bytes`. Every call to `CryptContext(schemes=["bcrypt"], ...)` crashes, including at import time. The fix: bypass passlib entirely and call bcrypt directly: ```python import bcrypt as _bcrypt_lib def hash_password(plain: str) -> str: if not plain: raise ValueError("Password must not be empty.") rounds = getattr(settings, "BCRYPT_ROUNDS", 12) salt = _bcrypt_lib.gensalt(rounds=rounds) return _bcrypt_lib.hashpw(plain.encode("utf-8"), salt).decode("utf-8") def verify_password(plain: str, hashed: str) -> bool: try: return _bcrypt_lib.checkpw(plain.encode("utf-8"), hashed.encode("utf-8")) except Exception: return False ``` Remove `from passlib.context import CryptContext` and `_get_pwd_context()` entirely. The `bcrypt` package is already installed as a passlib dependency — no new pip install needed. 44. **`slowapi` rate limiter + async FastAPI + Redis = `MissingGreenlet`** — `slowapi 0.1.9` with a Redis storage backend performs a sync storage check inside `@limiter.limit()` decorator even when `RATE_LIMIT_ENABLED=False`. In an async FastAPI endpoint, this triggers `MissingGreenlet`. The fix for dev/demo deployments: remove `@limiter.limit("N/minute")` from any endpoint that hits an async database path. Set `RATE_LIMIT_ENABLED=false` in `docker-compose.yml` environment block (not just `.env` — env_file is not always mounted into the container). Do NOT rely on `docker restart` to pick up new environment variables; must `docker stop + docker rm + docker compose up -d` to recreate the container. 45. **Docker Compose named containers (`container_name:`) ignore `docker compose restart` env changes** — when a service has `container_name: gx-api`, `docker compose restart` and even `docker compose up -d --force-recreate` frequently reuse the existing container and its original env. The only reliable way to inject new environment variables: `sudo docker stop && sudo docker rm && sudo docker compose -f docker-compose.yml -f docker-compose.override.yml up -d `. Always verify with `docker inspect --format '{{json .Config.Env}}'` after the recreation. New env vars in `docker-compose.yml environment:` block need the container fully removed first. 46. **`authClient.ts` login: must send JSON `{email, password}`, NOT `URLSearchParams {username}`** — FastAPI's `LoginRequest` Pydantic model reads a JSON body. Sending `Content-Type: application/x-www-form-urlencoded` with `username=` field returns 422. The correct authClient.ts login: ```typescript const response = await authAxios.post( "/auth/login", { email, password }, { headers: { "Content-Type": "application/json" } }, ); ``` This is a common mistake when copying OAuth2 patterns (which use form-encoded `username`). The GX-GeoShare backend uses a plain Pydantic `LoginRequest`, not FastAPI's `OAuth2PasswordRequestForm`. 47. **`check_integrity.sh` baselines drift sprint-over-sprint — update after every sprint** — The script's hardcoded baseline numbers (Pyrefly N errors, Biome N errors) become stale as new code is added. When the script reports false regressions on a clean sprint, update the thresholds immediately. Get current counts: `pyrefly check 2>&1 | grep 'INFO.*errors'` and `biome check services/frontend/src/ 2>&1 | grep 'Found.*errors'`. Update the `if [ "$COUNT" -gt N ]` lines in the script. Never copy the prior sprint's QA script baseline blindly. 48. **Playwright large spec via `$()` shell substitution = stdin timeout** — passing a >5KB prompt via `$(cat file.md)` or inline `$()` to Claude Code causes `"no stdin data received in 3s, proceeding without it"` and the prompt is never received. The fix: write the spec file via Claude Code's `Write` tool directly to disk, then run `npx playwright test` in a separate bash call. Pattern: ```python # WRONG — stdin timeout on large prompts: bash(command=f'{CLAUDE} -p "$(cat /tmp/big_prompt.md)" ...') # CORRECT — write first, run second: bash(command=f'{CLAUDE} -p "Write this spec file to {SPEC_PATH}: ..." --allowedTools Write,Bash --max-turns 40') bash(command=f'npx playwright test {SPEC_PATH} --reporter=list | tee {RESULTS}') ``` 58. **`_org_id()` must return `""` (lenient), not raise 403 for empty org** — In single-tenant / no-org deployments, users register without org membership. JWT `org_id` claim is `""`. If `_org_id()` raises HTTP 403 when `org_id` is empty, EVERY dataset/upload/tile/annotation API call fails for all locally-registered users. Fix: return `""` — `Project.org_id == ""` matches projects created by no-org users (which also have `org_id=""`). Do NOT fall back to `user_id` — projects have `org_id=""` and `Project.org_id == user_id` never matches. Apply to ALL 11 routers that have this guard. 59. **`Dataset.status` must be explicitly set to `completed` after successful upload** — after `registry.upload_file()` returns 201, `dataset.status` stays `pending` forever. Fix: add `dataset.status = DatasetStatus.completed` before `await db.commit()` in the non-DEM upload path. Import `DatasetStatus` explicitly from `app.models.project`. 60. **CesiumJS `h-[600px]` hardcode breaks full-viewport layout** — Fix: change to `h-full`. Requires `AppLayout
` to use `overflow-hidden` (not `overflow-y-auto`) so `h-full` propagates down. Page-level wrappers (ProjectListPage, ProjectDetailPage) need `h-full overflow-y-auto` on their own wrapper divs so those pages scroll. 61. **Project card "Open" is a `` rendered as ``, not a `