Handoff · AI3D-382 — open the PR, then file the follow-ups

7 Sep 2026 · continuation brief for the next agent · repo 3dai.iolabs.pointcloud.verticalsigns · worktree /home/ai/dev/wt-vs-382 · branch feat/AI3D-382-returns-recall @ 02f59e5

Continuation target

Open the pull request for feat/AI3D-382-returns-recall against master on Bitbucket (git@bitbucket.org:ioholding/3dai.iolabs.pointcloud.verticalsigns.git), with the PR body drafted below, and file the seven follow-up tickets. Miro pushes the branch himself — do not push, and do not rebase.

The detection work is finished and gated. Nothing in the algorithm is left to do; this is the landing sequence.

Current state

branchfeat/AI3D-382-returns-recall, worktree /home/ai/dev/wt-vs-382, HEAD 02f59e5 “AI3D-382: Final summary page”. Never pushed.
size146 commits ahead of local master — 100 AI3D-382: commits plus 46 AI3D-373: commits, because this branch was cut from feat/AI3D-373-tree-recall @ ad41427 (PR #7, not merged). All messages are one line, prefix AI3D-382: / AI3D-373: .
historyThe AI3D-382 range was rewritten (twice) to strip Cursor Co-authored-by trailers, so those hashes differ from anything referenced in older notes. The AI3D-373 range is untouched and ad41427 is already on origin, so the push is a plain new-branch push, not a force-push.
masterLocal master = c25d64c (“AI3D-379 Skip guardrail_support/top_rail companions in rail_halfpost model loading”) and is already merged into the branch (git merge-base --is-ancestor master HEAD passes). origin/master = 4c3042f, i.e. behind local master by one unpushed commit as of this handoff. Re-check before opening the PR — see step 1.
tests1 811 collected, green. Ruff clean.
configRecommended new-data config committed at configs/a1nr_returns_recommended.json with a README section. Run-shaped twin: /home/ai/veg373_work/returns/cfg/r9.json.
evidenceFinal summary page: ai3d-382-summary-20260907 (source docs/pages/ai3d-382-summary-20260907.html, committed).

Next steps, in order

1. Check master has not moved, and merge (never rebase)

cd /home/ai/dev/wt-vs-382
git fetch origin
git log --oneline -1 origin/master
git merge-base --is-ancestor origin/master HEAD && echo "up to date" || echo "master moved"
# only if it moved:
git merge origin/master        # merge commit is fine; DO NOT rebase (published-ish rewritten history)
uv run pytest -q               # re-gate after any merge

Expected conflict surface if AI3D-379-style config work lands again: src/verticalsigns/config.py, src/verticalsigns/_model_treedetect.py, src/verticalsigns/trees.py — the same three files that conflicted on the previous master merge.

2. Miro pushes

# MIRO runs this, not the agent:
git push -u origin feat/AI3D-382-returns-recall

House rule: agents never push. Ask, then wait.

3. Open the PR

Use the bitbucket-pr skill (Bitbucket Cloud 2.0 API, workspace ioholding, repo 3dai.iolabs.pointcloud.verticalsigns). Source feat/AI3D-382-returns-recall, destination master, title:

AI3D-382: number_of_returns — tree recall and foliage-vs-structure on the no-scan-angle-filter data

Decide with Miro first: this PR carries 46 AI3D-373 commits as well, because PR #7 (feat/AI3D-373-tree-recall → master) never merged. Either (a) land PR #7 first and retarget this PR at feat/AI3D-373-tree-recall, or (b) open this one against master and let it supersede PR #7. Option (b) is the default assumption below; say so explicitly in the PR description.

4. File the follow-up tickets

Seven of them, listed in Follow-ups. Two are decisions for Miro, one belongs to seg3d, one is already handed to tablecloth.

PR body — draft

Copy-paste draft (Markdown)
## Summary

Uses the new per-point `number_of_returns` on the no-scan-angle-filter dataset to
(1) fix tree recall — the old 10 m candidate slab and the v6 hedge gate were tuned on
scan-angle-filtered clouds with the tree tops cut off — and (2) separate foliage from
signs, guardrails and walls.

Nine detection passes, each ≤ 3 segments + visual check, discarding anything that did
not help. Final gate on **six validation segments never used for tuning**
(003/022/023/087/093/094), base config vs shipped config:

| metric | base | shipped | delta |
|---|---|---|---|
| canopy recall > 10 m | 0.018 | **0.769** | +0.752 (6/6 segments) |
| canopy recall 4–10 m | 0.750 | **0.903** | +0.153 |
| foliage-cell recall (segment-wide) | 0.494 | **0.860** | +0.366 |
| hard-contamination cells | 111 | **26** | points −75 % |
| truth-spot recall | 1.000 | **1.000** | both `not_tree` spots bit-identical |
| signs / delineators | 4 / 88 | **5 / 88** | +1 real sign, 0 lost |
| tree detections | 331 | 322 | −9, all rendered: road / mast / barrier, 0 vegetation |
| worst-segment runtime | 308 s | **447 s (7.4 min)** | +45 %, budget is 20 min |

085 human ground truth (ReCap annotations, 1.87 M points, 100 % matched @ 5 cm):
precision 0.489 → **0.543**, veg-tolerant precision 0.741 → **0.853**, recall 0.755
**unchanged** — every precision gain was bought with zero true-tree loss.

**Old-data parity with the packaged defaults is identical**: `clusters.csv`,
`tree_clusters.csv` and `point_masks.npz` match row-for-row and bit-for-bit against the
pre-change reference; the only differences are append-only columns and one crown-circle
float inside the documented P2 overlay tolerance. Every new knob defaults OFF.

## What this adds (all default-off knobs)

| knob | effect |
|---|---|
| `tree_detection.canopy_extension_*` (0.20 / 0.02) | paints the canopy above the 10 m slab from per-cell multi/triple-return fractions — the whole recall win |
| `returns_stripper.*` (tau 0.30, r 0.75, `reject_promoted_poles`) | drops foliage returns from sign candidates before the shape features; guards: min height 1.5 m, min 20 dropped, keep-only sparse fallback, min remaining 0.5, max surviving mrf 0.25, neighbour rules on unstripped geometry, promoted-pole guard |
| `sign_post.min_points_per_m = 100` | sparse-sign guard, kills airborne ghost signs (reason `sparse_sign`) |
| `tree_detection.hard_blob_unpaint_enabled` + `reject_instance` + survivor test + `cell_hysteresis 0.5` + `low_band_min_elev_m 0.3` with canopy-column guard | removes hard flat structure (billboards, lane markings, moving vehicles, road slivers) from tree paint, and drops whole instances that are mostly structure |
| `tree_detection.guardrail_mask_unpaint_enabled` | point-level unpaint of tree paint claimed by the seg3d guardrail masks; reaches rails inside mega-instances that no per-cluster rule can touch |
| perf: lazy crown-overlay fits, voxel-weighted lobe DBSCAN, convex-hull MEC prefilter | `crown_circles()` was 86 % of runtime; segment 002 1 355 s → 198 s (6.8×), decisions and masks bit-identical |
| `tcs_ground.*` per-segment ground | implemented, reviewed, tested — **shipped `per_segment = false`**; the code path stays for the tablecloth work |

## What was tried and removed

- hedge-gate returns veto — fired on nothing corpus-wide; tree/hedge AUC is 0.70. **Code removed.**
- returns stripper on the rail half-post path — inert, emitted half-posts byte-identical. **Code removed.**
- hard-blob mrf hysteresis 0.02 — cut rectangles out of a live shrub bank, never fired on its target. **Default 0.0.**
- per-segment TCS ground (naive / supported-seeding / ELM variants) — none reached per-record parity. **`per_segment = false`.**
- DEM (Digital Elevation Model, the per-cell ground surface) percentile / cell-size / noise-trim changes, lower hard-cell point floor, elongation gate — each measured, each loses more true tree than it gains, or captures almost nothing.

## Evidence

Final summary: https://miro-plans-battlebox.pages.dev/pages/ai3d-382-summary-20260907
Per-pass reports: /pages/ai3d-382-pass2-3-report-20260906 · pass4 · pass5 · pass6 · pass7 (all -20260906)
· pass8 · pass9 (-20260907) · DEM-bias explainer /pages/dem-bias-explainer-20260906

## Config

Recommended new-data config: `configs/a1nr_returns_recommended.json` (+ README section).
Requires `rail_halfpost.models_dir` pointing at a seg3d guardrails output root for the
guardrail-mask unpaint; without it the hook is a no-op.

## Tests

1 811 tests green (`uv run pytest -q`), `uv tool run ruff check src tests scripts` clean.

## Known untested on the real corpus

These landed as review fixes and are byte-neutral on every segment run so far, i.e. no
run exercised their branch. They are covered by unit tests only:

- canopy provenance check — extend canopies only for tree-path trees (`ae988ba`)
- hard-blob reject **survivor test** anchored at `min_elev` (`c51bca6`, docs `7bd8792`)
- returns-stripper **pole guard on `sign_post`** clusters (docs `2483578`)
- **`marker_mask` keying** of the marker return-column skip (`740854e`)
- hard-blob **hysteresis cap fallback** to strict cells (`18c9d73`, relaxation `b3dcc24`)

## Open items (tickets filed separately)

1. product definition: do attached shrubs/grass under a tree count as “tree”? (67 % of residual FPs)
2. mega-instance splitting — seg3d
3. pole-vs-trunk discriminator (lamp/camera masts, gantry columns painted tree)
4. tree vs hedge needs labels (returns AUC 0.70)
5. `return_number` export — available in the LAS on N:, not needed so far
6. per-segment TCS ground — handed to tablecloth
7. seg3d fuse: add `tree` to `PAINT_FIRST_CLASSES` in `3dsegmentation/masks.py`

Follow-up tickets

#ticketowner / kindcontent
1Product definition: are attached shrubs and grass under a tree “tree”?Miro decision67 % of the remaining false positives against the 085 human GT are annotator classes 74 (low veg) / 75 (medium veg) inside our tree paint. If they count, precision is 0.853 rather than 0.543. Nothing should be tuned before this is answered.
2Split mega-instancesseg3d78–100 m verge instances swallow rails, road and vehicles together with every true positive (085 instance 3: 6.9 M points, zero GT trees). Unreachable by per-cluster rules.
3Pole-vs-trunk discriminatornew ticketLamp/camera masts and gantry columns are painted tree and survive every unpaint knob because they are genuine vertical columns. Reference case: 003 at (−3329, −3671), 20 082 points, identical before and after. Render: analysis/val_renders/hardcell_003_-3329_-3671.png.
4Tree vs hedge needs labelsdataMeasured multi-return AUC 0.70 — returns cannot separate them. Miro's hedge definition is a size/band property, not a foliage property.
5return_number exportbacklogPresent in the LAS on N: but not in the exported NPZ; would enable first/last-echo logic. Nothing in AI3D-382 needed it — do not spend the export effort speculatively.
6Per-segment TCS groundhanded overAlready handed to the tablecloth repo with the seeding / clamp / noise-rejection findings: handoff-tablecloth-segment-ground-default-20260906-1420.
7seg3d fuse: add tree to PAINT_FIRST_CLASSESone-linerIn 3dsegmentation/masks.py: guardrail classes are already PAINT_LAST (they override tree), but tree (LAS 5) has rank 0, so road and vehicle rows we paint tree still leak through the fuse. Low/medium vegetation are already PAINT_FIRST.

Verification — what was run, and how to re-run it

cd /home/ai/dev/wt-vs-382
uv run pytest -q                                  # 1811 tests, green
uv tool run ruff check src tests scripts          # clean

# one detector run with the shipped config
R=/home/ai/veg373_work/returns
uv run verticalsigns-detect \
  --data-dir $R/data/A1NR/lane_points \
  --segments 085 \
  --out $R/out/<name> \
  --config $R/cfg/r9.json \
  --dump-point-masks --log-level INFO

# the full validation gate (6 segments, base + shipped, 4-way parallel, ~55 min)
bash $R/run_val.sh                                # writes out/val_base, out/val_r9, logs/val_*
checkresult
final gate on six untouched validation segmentspass eval/val_judgement.md — 12 runs rc=0, 0 errors, mask_file_missing = 0
085 human-GT progressionpass eval/r9_judgement.md
old-data parity with packaged defaultsidentical eval/val/parity_diff.txt, reference out/plumb_old_002/segment_002
test suite / lintpass 1 811 tests, ruff clean
test-split (reserve) segmentsnot run the 14 reserve segments in split.json were deliberately never touched
corpus-wide run with the shipped confignot run only the 5 training + 6 validation segments
the five review-fix branches listed aboveunit tests only byte-neutral on every real run

Risks and open uncertainty

riskdetail
Runtime +45 % acceptedMostly the returns-stripper KD-trees plus the canopy cell pass and mask matching. Worst validation segment 447 s = 7.4 min against a 20 min/segment budget; worst wall clock 493 s under 4-way concurrency. Not a budget risk today, but it is the biggest single cost of the change.
hard_blob_reject_min_survivor_points = 2000 is an absolute count monitorAll nine validation rejects had 70–1 771 survivors and were all correct, but a genuine sapling with fewer than 2 000 points above 0.3 m would be killed by the same rule. Never observed on the training corpus or on validation — monitor, do not retune on zero evidence.
Guardrail-mask unpaint depends on seg3d masks deploymentRequires rail_halfpost.models_dir to point at a seg3d guardrails output root (/home/ai/seg3d_out/260905/guardrails_A1B0 for A1NR). With no masks the hook is a silent no-op — the run now warns if the mask's record_names match none of the segment's run3 records. The knob defaults OFF, so that is safe.
Per-record TCS cache directory is per dataset data corruptionThe TCS cache is keyed by record stem, and old and new data share stems. Pointing a new-data run at tcs_cache_A1B0 silently feeds new-data ground to old data (measured: 448 vs 514 clusters, z_ground off by 4.7 m). Old-data parity runs must use /home/ai/veg373_work/recall/phase3/cfg/v6mv.json; new-data runs use their own tcs_cache_A1NR.
Hedge growing through a guardrail loses its rail-embedded foliage3 500 GT low/medium_veg points on 085 = 0.6 % of that segment's vegetation paint. Structural, bounded, and the bush stays detected. Escape hatch: guardrail_mask_unpaint_classes = ["w_beam"] + mask erosion.
PR scopeCarries 46 AI3D-373 commits because PR #7 never merged — see step 3.

House rules that applied

Key artifacts

whatwhere
worktree / branch/home/ai/dev/wt-vs-382 · feat/AI3D-382-returns-recall @ 02f59e5
main checkout/home/ai/dev/3dai.iolabs.pointcloud.verticalsigns (master c25d64c)
recommended configconfigs/a1nr_returns_recommended.json · run-shaped twin /home/ai/veg373_work/returns/cfg/r9.json
final summary page/pages/ai3d-382-summary-20260907 · source docs/pages/ai3d-382-summary-20260907.html
pass reportspass 2–3 · 4 · 5 · 6 · 7 · 8 · 9 · DEM bias
judgements/home/ai/veg373_work/returns/eval/r1_judgement.mdr9_judgement.md, val_judgement.md
diagnoses/home/ai/veg373_work/returns/diag/DIAG_treepaint.md · diag2/DIAG_dem_hardcells.md
renders/home/ai/veg373_work/returns/analysis/{r1..r9}_renders/, val_renders/, dem_bias/
detector outputs/home/ai/veg373_work/returns/out/val_base, val_r9, r1r9, parity_final_old_002, plumb_old_002
data/home/ai/veg373_work/returns/data/A1NR/lane_points (14 truth segments, no scan-angle filter, with number_of_returns)
seg3d guardrail masks/home/ai/seg3d_out/260905/guardrails_A1B0
run scripts/home/ai/veg373_work/returns/run_val.sh, run_corpus.sh, run_baseline.sh
page builders/home/ai/veg373_work/returns/analysis/build_summary.py, build_report_p{23,4,5,6,7,8,9}.py

Suggested skills