Handoff · AI3D-382 — open the PR, then file the follow-ups
Continuation target
Open the pull request for feat/AI3D-382-returns-recall against master on
Bitbucket (git@bitbucket.org:ioholding/3dai.iolabs.pointcloud.verticalsigns.git), with the PR
body drafted below, and file the seven follow-up tickets. Miro pushes the branch himself — do not
push, and do not rebase.
The detection work is finished and gated. Nothing in the algorithm is left to do; this is the landing sequence.
Current state
| branch | feat/AI3D-382-returns-recall, worktree /home/ai/dev/wt-vs-382, HEAD 02f59e5 “AI3D-382: Final summary page”. Never pushed. |
|---|---|
| size | 146 commits ahead of local master — 100 AI3D-382: commits plus 46 AI3D-373: commits, because this branch was cut from feat/AI3D-373-tree-recall @ ad41427 (PR #7, not merged). All messages are one line, prefix AI3D-382: / AI3D-373: . |
| history | The AI3D-382 range was rewritten (twice) to strip Cursor Co-authored-by trailers, so those hashes differ from anything referenced in older notes. The AI3D-373 range is untouched and ad41427 is already on origin, so the push is a plain new-branch push, not a force-push. |
| master | Local master = c25d64c (“AI3D-379 Skip guardrail_support/top_rail companions in rail_halfpost model loading”) and is already merged into the branch (git merge-base --is-ancestor master HEAD passes). origin/master = 4c3042f, i.e. behind local master by one unpushed commit as of this handoff. Re-check before opening the PR — see step 1. |
| tests | 1 811 collected, green. Ruff clean. |
| config | Recommended new-data config committed at configs/a1nr_returns_recommended.json with a README section. Run-shaped twin: /home/ai/veg373_work/returns/cfg/r9.json. |
| evidence | Final summary page: ai3d-382-summary-20260907 (source docs/pages/ai3d-382-summary-20260907.html, committed). |
Next steps, in order
1. Check master has not moved, and merge (never rebase)
cd /home/ai/dev/wt-vs-382
git fetch origin
git log --oneline -1 origin/master
git merge-base --is-ancestor origin/master HEAD && echo "up to date" || echo "master moved"
# only if it moved:
git merge origin/master # merge commit is fine; DO NOT rebase (published-ish rewritten history)
uv run pytest -q # re-gate after any merge
Expected conflict surface if AI3D-379-style config work lands again:
src/verticalsigns/config.py, src/verticalsigns/_model_treedetect.py,
src/verticalsigns/trees.py — the same three files that conflicted on the previous master merge.
2. Miro pushes
# MIRO runs this, not the agent:
git push -u origin feat/AI3D-382-returns-recall
House rule: agents never push. Ask, then wait.
3. Open the PR
Use the bitbucket-pr skill (Bitbucket Cloud 2.0 API, workspace ioholding,
repo 3dai.iolabs.pointcloud.verticalsigns). Source
feat/AI3D-382-returns-recall, destination master, title:
AI3D-382: number_of_returns — tree recall and foliage-vs-structure on the no-scan-angle-filter data
Decide with Miro first: this PR carries 46 AI3D-373 commits as well, because PR #7
(feat/AI3D-373-tree-recall → master) never merged. Either (a) land PR #7 first and retarget this
PR at feat/AI3D-373-tree-recall, or (b) open this one against master and let it
supersede PR #7. Option (b) is the default assumption below; say so explicitly in the PR description.
4. File the follow-up tickets
Seven of them, listed in Follow-ups. Two are decisions for Miro, one belongs to seg3d, one is already handed to tablecloth.
PR body — draft
Copy-paste draft (Markdown)
## Summary
Uses the new per-point `number_of_returns` on the no-scan-angle-filter dataset to
(1) fix tree recall — the old 10 m candidate slab and the v6 hedge gate were tuned on
scan-angle-filtered clouds with the tree tops cut off — and (2) separate foliage from
signs, guardrails and walls.
Nine detection passes, each ≤ 3 segments + visual check, discarding anything that did
not help. Final gate on **six validation segments never used for tuning**
(003/022/023/087/093/094), base config vs shipped config:
| metric | base | shipped | delta |
|---|---|---|---|
| canopy recall > 10 m | 0.018 | **0.769** | +0.752 (6/6 segments) |
| canopy recall 4–10 m | 0.750 | **0.903** | +0.153 |
| foliage-cell recall (segment-wide) | 0.494 | **0.860** | +0.366 |
| hard-contamination cells | 111 | **26** | points −75 % |
| truth-spot recall | 1.000 | **1.000** | both `not_tree` spots bit-identical |
| signs / delineators | 4 / 88 | **5 / 88** | +1 real sign, 0 lost |
| tree detections | 331 | 322 | −9, all rendered: road / mast / barrier, 0 vegetation |
| worst-segment runtime | 308 s | **447 s (7.4 min)** | +45 %, budget is 20 min |
085 human ground truth (ReCap annotations, 1.87 M points, 100 % matched @ 5 cm):
precision 0.489 → **0.543**, veg-tolerant precision 0.741 → **0.853**, recall 0.755
**unchanged** — every precision gain was bought with zero true-tree loss.
**Old-data parity with the packaged defaults is identical**: `clusters.csv`,
`tree_clusters.csv` and `point_masks.npz` match row-for-row and bit-for-bit against the
pre-change reference; the only differences are append-only columns and one crown-circle
float inside the documented P2 overlay tolerance. Every new knob defaults OFF.
## What this adds (all default-off knobs)
| knob | effect |
|---|---|
| `tree_detection.canopy_extension_*` (0.20 / 0.02) | paints the canopy above the 10 m slab from per-cell multi/triple-return fractions — the whole recall win |
| `returns_stripper.*` (tau 0.30, r 0.75, `reject_promoted_poles`) | drops foliage returns from sign candidates before the shape features; guards: min height 1.5 m, min 20 dropped, keep-only sparse fallback, min remaining 0.5, max surviving mrf 0.25, neighbour rules on unstripped geometry, promoted-pole guard |
| `sign_post.min_points_per_m = 100` | sparse-sign guard, kills airborne ghost signs (reason `sparse_sign`) |
| `tree_detection.hard_blob_unpaint_enabled` + `reject_instance` + survivor test + `cell_hysteresis 0.5` + `low_band_min_elev_m 0.3` with canopy-column guard | removes hard flat structure (billboards, lane markings, moving vehicles, road slivers) from tree paint, and drops whole instances that are mostly structure |
| `tree_detection.guardrail_mask_unpaint_enabled` | point-level unpaint of tree paint claimed by the seg3d guardrail masks; reaches rails inside mega-instances that no per-cluster rule can touch |
| perf: lazy crown-overlay fits, voxel-weighted lobe DBSCAN, convex-hull MEC prefilter | `crown_circles()` was 86 % of runtime; segment 002 1 355 s → 198 s (6.8×), decisions and masks bit-identical |
| `tcs_ground.*` per-segment ground | implemented, reviewed, tested — **shipped `per_segment = false`**; the code path stays for the tablecloth work |
## What was tried and removed
- hedge-gate returns veto — fired on nothing corpus-wide; tree/hedge AUC is 0.70. **Code removed.**
- returns stripper on the rail half-post path — inert, emitted half-posts byte-identical. **Code removed.**
- hard-blob mrf hysteresis 0.02 — cut rectangles out of a live shrub bank, never fired on its target. **Default 0.0.**
- per-segment TCS ground (naive / supported-seeding / ELM variants) — none reached per-record parity. **`per_segment = false`.**
- DEM (Digital Elevation Model, the per-cell ground surface) percentile / cell-size / noise-trim changes, lower hard-cell point floor, elongation gate — each measured, each loses more true tree than it gains, or captures almost nothing.
## Evidence
Final summary: https://miro-plans-battlebox.pages.dev/pages/ai3d-382-summary-20260907
Per-pass reports: /pages/ai3d-382-pass2-3-report-20260906 · pass4 · pass5 · pass6 · pass7 (all -20260906)
· pass8 · pass9 (-20260907) · DEM-bias explainer /pages/dem-bias-explainer-20260906
## Config
Recommended new-data config: `configs/a1nr_returns_recommended.json` (+ README section).
Requires `rail_halfpost.models_dir` pointing at a seg3d guardrails output root for the
guardrail-mask unpaint; without it the hook is a no-op.
## Tests
1 811 tests green (`uv run pytest -q`), `uv tool run ruff check src tests scripts` clean.
## Known untested on the real corpus
These landed as review fixes and are byte-neutral on every segment run so far, i.e. no
run exercised their branch. They are covered by unit tests only:
- canopy provenance check — extend canopies only for tree-path trees (`ae988ba`)
- hard-blob reject **survivor test** anchored at `min_elev` (`c51bca6`, docs `7bd8792`)
- returns-stripper **pole guard on `sign_post`** clusters (docs `2483578`)
- **`marker_mask` keying** of the marker return-column skip (`740854e`)
- hard-blob **hysteresis cap fallback** to strict cells (`18c9d73`, relaxation `b3dcc24`)
## Open items (tickets filed separately)
1. product definition: do attached shrubs/grass under a tree count as “tree”? (67 % of residual FPs)
2. mega-instance splitting — seg3d
3. pole-vs-trunk discriminator (lamp/camera masts, gantry columns painted tree)
4. tree vs hedge needs labels (returns AUC 0.70)
5. `return_number` export — available in the LAS on N:, not needed so far
6. per-segment TCS ground — handed to tablecloth
7. seg3d fuse: add `tree` to `PAINT_FIRST_CLASSES` in `3dsegmentation/masks.py`
Follow-up tickets
| # | ticket | owner / kind | content |
|---|---|---|---|
| 1 | Product definition: are attached shrubs and grass under a tree “tree”? | Miro decision | 67 % of the remaining false positives against the 085 human GT are annotator classes 74 (low veg) / 75 (medium veg) inside our tree paint. If they count, precision is 0.853 rather than 0.543. Nothing should be tuned before this is answered. |
| 2 | Split mega-instances | seg3d | 78–100 m verge instances swallow rails, road and vehicles together with every true positive (085 instance 3: 6.9 M points, zero GT trees). Unreachable by per-cluster rules. |
| 3 | Pole-vs-trunk discriminator | new ticket | Lamp/camera masts and gantry columns are painted tree and survive every unpaint knob because they are genuine vertical columns. Reference case: 003 at (−3329, −3671), 20 082 points, identical before and after. Render: analysis/val_renders/hardcell_003_-3329_-3671.png. |
| 4 | Tree vs hedge needs labels | data | Measured multi-return AUC 0.70 — returns cannot separate them. Miro's hedge definition is a size/band property, not a foliage property. |
| 5 | return_number export | backlog | Present in the LAS on N: but not in the exported NPZ; would enable first/last-echo logic. Nothing in AI3D-382 needed it — do not spend the export effort speculatively. |
| 6 | Per-segment TCS ground | handed over | Already handed to the tablecloth repo with the seeding / clamp / noise-rejection findings: handoff-tablecloth-segment-ground-default-20260906-1420. |
| 7 | seg3d fuse: add tree to PAINT_FIRST_CLASSES | one-liner | In 3dsegmentation/masks.py: guardrail classes are already PAINT_LAST (they override tree), but tree (LAS 5) has rank 0, so road and vehicle rows we paint tree still leak through the fuse. Low/medium vegetation are already PAINT_FIRST. |
Verification — what was run, and how to re-run it
cd /home/ai/dev/wt-vs-382
uv run pytest -q # 1811 tests, green
uv tool run ruff check src tests scripts # clean
# one detector run with the shipped config
R=/home/ai/veg373_work/returns
uv run verticalsigns-detect \
--data-dir $R/data/A1NR/lane_points \
--segments 085 \
--out $R/out/<name> \
--config $R/cfg/r9.json \
--dump-point-masks --log-level INFO
# the full validation gate (6 segments, base + shipped, 4-way parallel, ~55 min)
bash $R/run_val.sh # writes out/val_base, out/val_r9, logs/val_*
| check | result |
|---|---|
| final gate on six untouched validation segments | pass eval/val_judgement.md — 12 runs rc=0, 0 errors, mask_file_missing = 0 |
| 085 human-GT progression | pass eval/r9_judgement.md |
| old-data parity with packaged defaults | identical eval/val/parity_diff.txt, reference out/plumb_old_002/segment_002 |
| test suite / lint | pass 1 811 tests, ruff clean |
| test-split (reserve) segments | not run the 14 reserve segments in split.json were deliberately never touched |
| corpus-wide run with the shipped config | not run only the 5 training + 6 validation segments |
| the five review-fix branches listed above | unit tests only byte-neutral on every real run |
Risks and open uncertainty
| risk | detail |
|---|---|
| Runtime +45 % accepted | Mostly the returns-stripper KD-trees plus the canopy cell pass and mask matching. Worst validation segment 447 s = 7.4 min against a 20 min/segment budget; worst wall clock 493 s under 4-way concurrency. Not a budget risk today, but it is the biggest single cost of the change. |
hard_blob_reject_min_survivor_points = 2000 is an absolute count monitor | All nine validation rejects had 70–1 771 survivors and were all correct, but a genuine sapling with fewer than 2 000 points above 0.3 m would be killed by the same rule. Never observed on the training corpus or on validation — monitor, do not retune on zero evidence. |
| Guardrail-mask unpaint depends on seg3d masks deployment | Requires rail_halfpost.models_dir to point at a seg3d guardrails output root (/home/ai/seg3d_out/260905/guardrails_A1B0 for A1NR). With no masks the hook is a silent no-op — the run now warns if the mask's record_names match none of the segment's run3 records. The knob defaults OFF, so that is safe. |
| Per-record TCS cache directory is per dataset data corruption | The TCS cache is keyed by record stem, and old and new data share stems. Pointing a new-data run at tcs_cache_A1B0 silently feeds new-data ground to old data (measured: 448 vs 514 clusters, z_ground off by 4.7 m). Old-data parity runs must use /home/ai/veg373_work/recall/phase3/cfg/v6mv.json; new-data runs use their own tcs_cache_A1NR. |
| Hedge growing through a guardrail loses its rail-embedded foliage | 3 500 GT low/medium_veg points on 085 = 0.6 % of that segment's vegetation paint. Structural, bounded, and the bush stays detected. Escape hatch: guardrail_mask_unpaint_classes = ["w_beam"] + mask erosion. |
| PR scope | Carries 46 AI3D-373 commits because PR #7 never merged — see step 3. |
House rules that applied
- Never push. Miro pushes. Agents commit locally only.
- Never rebase this branch — its AI3D-382 range was already rewritten to strip Cursor
Co-authored-bytrailers. Integrate master withgit merge. - No co-author trailers in commit messages, and no “Generated with Claude Code” footer on commits. (PR bodies do carry the generated-with footer.)
- Commit often, one-line messages, prefixed
AI3D-382:. - Work in a worktree, not in the main checkout.
- The loop rule: ≤ 3 segments per pass, visual check on renders, measure whether the last change helped, throw the code away ruthlessly if it did not. Stop when detection is perfect, needs a human decision or annotation, or needs better data.
- Never hard-gate on returns — they are evidence, not a veto.
- Tune on train, check on validation, never touch the reserve split
(
/home/ai/veg373_work/returns/split.json).
Key artifacts
| what | where |
|---|---|
| worktree / branch | /home/ai/dev/wt-vs-382 · feat/AI3D-382-returns-recall @ 02f59e5 |
| main checkout | /home/ai/dev/3dai.iolabs.pointcloud.verticalsigns (master c25d64c) |
| recommended config | configs/a1nr_returns_recommended.json · run-shaped twin /home/ai/veg373_work/returns/cfg/r9.json |
| final summary page | /pages/ai3d-382-summary-20260907 · source docs/pages/ai3d-382-summary-20260907.html |
| pass reports | pass 2–3 · 4 · 5 · 6 · 7 · 8 · 9 · DEM bias |
| judgements | /home/ai/veg373_work/returns/eval/r1_judgement.md … r9_judgement.md, val_judgement.md |
| diagnoses | /home/ai/veg373_work/returns/diag/DIAG_treepaint.md · diag2/DIAG_dem_hardcells.md |
| renders | /home/ai/veg373_work/returns/analysis/{r1..r9}_renders/, val_renders/, dem_bias/ |
| detector outputs | /home/ai/veg373_work/returns/out/ — val_base, val_r9, r1…r9, parity_final_old_002, plumb_old_002 |
| data | /home/ai/veg373_work/returns/data/A1NR/lane_points (14 truth segments, no scan-angle filter, with number_of_returns) |
| seg3d guardrail masks | /home/ai/seg3d_out/260905/guardrails_A1B0 |
| run scripts | /home/ai/veg373_work/returns/run_val.sh, run_corpus.sh, run_baseline.sh |
| page builders | /home/ai/veg373_work/returns/analysis/build_summary.py, build_report_p{23,4,5,6,7,8,9}.py |
Suggested skills
bitbucket-pr— for creating the PR on Bitbucket Cloud.html-page— if any further report or explainer page is needed.