Tree recall - tile-and-score results (29 Aug)

29 Aug 2026 · for Miro · AI3D-373 tree recall

Summary

Truth set /home/ai/veg373_work/recall/phase0/truth_v2.json: 103 + 10 spots (two Opus judges calibrated on Miro’s examples; gate 43/43 big/mid rf-rejected spots judged tree). Variants: v0 = today’s detector, v1 = tile-and-score 8 m tiles, v2 = v1 + relaxed RF 0.5 for units with robust height ≥ 3.5 m. Best by weighted score (recall − 2 × FP rate): v2 (recall 0.92 vs v0 0.41, FP rate 0.83 vs 0.00). Before/after judges on 085/060/093/086/087/002: {"better":15,"same":25,"worse":15,"nonTree":12}.

Eval table

variantrecallfp_rateweightedTPFNFPTNaccepted ptsrf-rejected ptsn_tiles
v00.4100.0000.4104159012767,2068,965,3030
v10.9100.833-0.7579191026,910,5452,757,62240
v20.9200.833-0.7479281026,945,4252,722,74240

Per-KIND recall (tree spots only)

kindn_tree spotsv0v1v2
bigrej310/31 = 0.0031/31 = 1.0031/31 = 1.00
midrej125/12 = 0.427/12 = 0.588/12 = 0.67
lowrej234/23 = 0.1719/23 = 0.8319/23 = 0.83
acclow1615/16 = 0.9416/16 = 1.0016/16 = 1.00
acc1615/16 = 0.9416/16 = 1.0016/16 = 1.00
anchor22/2 = 1.002/2 = 1.002/2 = 1.00

Flips

v1 vs v0: 60 flips, ALL unclaimed → claimed (0 claimed → unclaimed). Format: name[T=tree|N=not_tree] n_tree v0>variant.

lowrej_002_c0_s0[N] 0>43293; lowrej_002_c0_s1[N] 0>20811; lowrej_002_c0_s2[N] 0>36437; bigrej_003_c7_s0[T] 0>41165; bigrej_003_c7_s1[T] 0>36288; bigrej_003_c7_s2[T] 0>49350; lowrej_003_c1_s0[T] 37>39643; lowrej_003_c1_s1[T] 0>36450; lowrej_003_c1_s2[T] 0>38586; lowrej_018_c0_s0[T] 0>10620; lowrej_018_c0_s1[T] 0>135159; lowrej_018_c0_s2[T] 0>29459; bigrej_022_c0_s0[T] 0>87961; bigrej_022_c0_s1[T] 0>57841; bigrej_022_c0_s2[T] 0>1484; acc_023_c11_s0[T] 75>3002; bigrej_023_c7_s0[T] 0>21207; bigrej_023_c7_s1[T] 0>2114; bigrej_023_c7_s2[T] 0>6441; midrej_023_c25_s0[T] 0>1602; bigrej_031_c1_s0[T] 0>29448; bigrej_031_c1_s1[T] 0>82785; bigrej_031_c1_s2[T] 0>28654; lowrej_060_c12_s0[N] 0>8647; lowrej_060_c12_s1[N] 0>2301; lowrej_060_c12_s2[N] 0>5979; lowrej_060_c5_s0[N] 0>5883; lowrej_060_c5_s1[N] 0>1480; lowrej_060_c5_s2[N] 0>3278; miss_060_tall_s0[N] 0>9458; acclow_066_c4_s0[T] 27>1059; lowrej_066_c15_s0[T] 0>286; lowrej_066_c15_s1[T] 0>34510; lowrej_066_c15_s2[T] 0>34439; lowrej_066_c8_s2[T] 0>24497; bigrej_086_c0_s0[T] 0>36758; bigrej_086_c0_s1[T] 0>42923; bigrej_086_c0_s2[T] 0>52912; bigrej_086_c2_s0[T] 0>32309; bigrej_086_c6_s0[T] 0>28712; midrej_086_c3_s0[T] 0>46347; bigrej_087_c2_s0[T] 0>24527; bigrej_087_c2_s1[T] 0>27505; bigrej_087_c2_s2[T] 0>44619; bigrej_087_c5_s0[T] 0>39005; bigrej_087_c6_s0[T] 0>39247; lowrej_087_c3_s0[T] 0>291; bigrej_090_c0_s0[T] 0>8846; bigrej_090_c0_s1[T] 0>3993; bigrej_090_c0_s2[T] 0>44311; lowrej_090_c1_s0[T] 0>322; bigrej_093_c1_s0[T] 0>123087; bigrej_093_c1_s1[T] 0>119347; bigrej_093_c1_s2[T] 0>110915; bigrej_094_c0_s0[T] 0>123612; bigrej_094_c0_s1[T] 0>117488; bigrej_094_c0_s2[T] 0>102978; lowrej_094_c1_s0[T] 0>102713; lowrej_094_c1_s1[T] 0>63319; lowrej_094_c1_s2[T] 0>61471

v2 vs v0: 61 flips, ALL unclaimed → claimed. Same list as v1 plus one: midrej_087_c40_s0[T] 120>1651.

v2 vs v1: exactly 1 flip — midrej_087_c40_s0[T] 120>1651 (unclaimed → claimed). No spot regressed in any variant.

Before / after at the truth spots

32 spots that changed or where a judge flagged worse / not-tree paint. Left column = today’s detector (v0), right column = v2.

acc_085_c2_s0 worse

segment 085 · kind acc · truth tree · crown cover 0.95 → 0.85

v2 demotes ~10k pts of the left dense crown mass (1-3 m) to medium_vegetation teal; tree 75,985 -> 65,924, no new false paint but real crown lost.

before · az045
before az045
after (v2) · az045
after az045
before · az225
before az225
after (v2) · az225
after az225

anchor_085_fp1_MIRO_TREE_s0 worse

segment 085 · kind anchor · truth tree · crown cover 0.85 → 0.5

Miro's TREE anchor: tree 45,441 -> 26,924 with 13,089 pts flipped to medium_vegetation plus guardrail/unclassified growth; large loss of exactly the crowned verge greenery Miro calls tree.

before · az045
before az045
after (v2) · az045
after az045
before · az225
before az225
after (v2) · az225
after az225

lowrej_085_c4_s0 worse

segment 085 · kind lowrej · truth tree · crown cover 0.9 → 0.78

tree 66,753 -> 57,335; a mid-height crown block becomes medium_vegetation (119 -> 6,544) and unclassified triples; true crown paint lost, no new not-tree paint.

before · az045
before az045
after (v2) · az045
after az045
before · az225
before az225
after (v2) · az225
after az225

midrej_060_c7_s0 better

segment 060 · kind midrej · truth tree · crown cover 0 → 0.05

Baseline tree=0; v2 adds only 249 mint pts (of ~7.4k veg) on the sparse low fringe at the cluster edge, main crown mass still teal - tiny gain, no obvious not-tree paint.

before · az045
before az045
after (v2) · az045
after az045
before · az225
before az225
after (v2) · az225
after az225

lowrej_060_c12_s0 worse not-tree painted

segment 060 · kind lowrej · truth not_tree · crown cover 0 → 0

v2 paints 10,298 pts mint over a large slab of the continuous roadside vegetation bank that the judges call not-tree - big new false positive.

before · az045
before az045
after (v2) · az045
after az045
before · az225
before az225
after (v2) · az225
after az225

lowrej_060_c12_s1 worse not-tree painted

segment 060 · kind lowrej · truth not_tree · crown cover 0 → 0

3,580 mint pts appear on the outer skirt of the same not-tree veg bank (incl. low <1 m fringe); smaller but still a clear false positive.

before · az045
before az045
after (v2) · az045
after az045
before · az225
before az225
after (v2) · az225
after az225

lowrej_060_c12_s2 worse not-tree painted

segment 060 · kind lowrej · truth not_tree · crown cover 0 → 0

10,144 mint pts cover roughly the whole right half of the not-tree vegetation carpet, much of it below 1 m in az225 - large false positive.

before · az045
before az045
after (v2) · az045
after az045
before · az225
before az225
after (v2) · az225
after az225

lowrej_060_c5_s0 worse not-tree painted

segment 060 · kind lowrej · truth not_tree · crown cover 0 → 0

7,788 mint pts on the not-tree verge vegetation, including low ground-hugging scrub in az225 - false positive.

before · az045
before az045
after (v2) · az045
after az045
before · az225
before az225
after (v2) · az225
after az225

lowrej_060_c5_s1 worse not-tree painted

segment 060 · kind lowrej · truth not_tree · crown cover 0 → 0

3,644 mint pts on the far/low edge of the not-tree veg mass; moderate false positive, baseline had none.

before · az045
before az045
after (v2) · az045
after az045
before · az225
before az225
after (v2) · az225
after az225

lowrej_060_c5_s2 worse not-tree painted

segment 060 · kind lowrej · truth not_tree · crown cover 0 → 0

4,442 mint pts spread over the low outer scrub of the not-tree bank (well under 1 m in places) - false positive.

before · az045
before az045
after (v2) · az045
after az045
before · az225
before az225
after (v2) · az225
after az225

miss_060_tall_s0 worse not-tree painted

segment 060 · kind tallmiss · truth not_tree · crown cover 0 → 0

12,611 mint pts newly cover a big block of the not-tree tall vegetation wall, reaching down to the yellow-green low veg at ground level - largest false positive of the set.

before · az045
before az045
after (v2) · az045
after az045
before · az225
before az225
after (v2) · az225
after az225

bigrej_093_c1_s0 better

segment 093 · kind bigrej · truth tree · crown cover 0 → 0.92

Baseline painted the whole 0-4 m wooded verge medium_vegetation (tree=0); v2 paints 162k pts mint over the crown mass, grass strip stays low_vegetation.

before · az045
before az045
after (v2) · az045
after az045
before · az225
before az225
after (v2) · az225
after az225

bigrej_093_c1_s1 better

segment 093 · kind bigrej · truth tree · crown cover 0 → 0.9

RGB shows real crowns to 4 m along the verge; before tree=0, after tree=164k covering nearly all of it, only thin low_veg fringe (7.7k) left at the toe.

before · az045
before az045
after (v2) · az045
after az045
before · az225
before az225
after (v2) · az225
after az225

bigrej_093_c1_s2 better

segment 093 · kind bigrej · truth tree · crown cover 0 → 0.9

Wooded slope beside the carriageway: baseline all medium_vegetation, v2 paints 147k mint; ground/asphalt/vehicle unchanged, no spill onto road.

before · az045
before az045
after (v2) · az045
after az045
before · az225
before az225
after (v2) · az225
after az225

bigrej_086_c0_s0 better

segment 086 · kind bigrej · truth tree · crown cover 0 → 0.95

Baseline painted the whole 0-4 m woody thicket teal (tree 0); v2 paints 67,363 tree covering nearly the entire crown mass, only fringe ground/low_veg left.

before · az045
before az045
after (v2) · az045
after az045
before · az225
before az225
after (v2) · az225
after az225

bigrej_086_c0_s1 better

segment 086 · kind bigrej · truth tree · crown cover 0 → 0.95

tree 0 -> 61,294; mint covers the full vegetated bank up to 4 m and stops cleanly at the brown ground slope.

before · az045
before az045
after (v2) · az045
after az045
before · az225
before az225
after (v2) · az225
after az225

bigrej_086_c0_s2 better

segment 086 · kind bigrej · truth tree · crown cover 0 → 0.9

tree 0 -> 82,819; whole crown block painted, adjacent road/verge strip correctly left ground/low_veg.

before · az045
before az045
after (v2) · az045
after az045
before · az225
before az225
after (v2) · az225
after az225

bigrej_086_c2_s0 better

segment 086 · kind bigrej · truth tree · crown cover 0 → 0.85

tree 0 -> 50,023 over the tall canopy; asphalt and verge grass untouched, ~10k medium_veg still unpainted at the crown edges.

before · az045
before az045
after (v2) · az045
after az045
before · az225
before az225
after (v2) · az225
after az225

bigrej_086_c6_s0 better

segment 086 · kind bigrej · truth tree · crown cover 0 → 0.6

tree 0 -> 40,557 but the dense 2-3.5 m block stays medium_vegetation; recall gained on the lower thicket only, partial fix.

before · az045
before az045
after (v2) · az045
after az045
before · az225
before az225
after (v2) · az225
after az225

midrej_086_c3_s0 better

segment 086 · kind midrej · truth tree · crown cover 0 → 0.95

Clear trunked trees; tree 0 -> 62,975 covering crowns and trunks, medium_veg residue only 1,753.

before · az045
before az045
after (v2) · az045
after az045
before · az225
before az225
after (v2) · az225
after az225

bigrej_087_c2_s0 better

segment 087 · kind bigrej · truth tree · crown cover 0 → 0.75

tree 0 -> 39,166: crown mass (2-4 m, leafy in RGB) now mint; left teal blob still unpainted.

before · az045
before az045
after (v2) · az045
after az045
before · az225
before az225
after (v2) · az225
after az225

bigrej_087_c2_s1 better

segment 087 · kind bigrej · truth tree · crown cover 0.08 → 0.8

tree 3,341 -> 47,188: the whole verge crown belt painted, ground/road strip untouched.

before · az045
before az045
after (v2) · az045
after az045
before · az225
before az225
after (v2) · az225
after az225

bigrej_087_c2_s2 better

segment 087 · kind bigrej · truth tree · crown cover 0 → 0.9

tree 0 -> 72,258: near-complete paint of the 4 m crown wall, only a small teal patch left.

before · az045
before az045
after (v2) · az045
after az045
before · az225
before az225
after (v2) · az225
after az225

bigrej_087_c5_s0 better

segment 087 · kind bigrej · truth tree · crown cover 0 → 0.6

tree 0 -> 40,823: central broad 4 m crown painted; left-hand crowns stay medium_vegetation.

before · az045
before az045
after (v2) · az045
after az045
before · az225
before az225
after (v2) · az225
after az225

bigrej_087_c6_s0 better

segment 087 · kind bigrej · truth tree · crown cover 0 → 0.7

tree 0 -> 40,823: the isolated foliage tree now mint; surrounding low yellow-green verge left alone.

before · az045
before az045
after (v2) · az045
after az045
before · az225
before az225
after (v2) · az225
after az225

midrej_087_c40_s0 same not-tree painted

segment 087 · kind midrej · truth tree · crown cover 0.3 → 0.42

3,359 -> 4,910; the added mint is a low ~0-1.5 m bushy clump at ground level, gain on real crowns negligible.

before · az045
before az045
after (v2) · az045
after az045
before · az225
before az225
after (v2) · az225
after az225

lowrej_087_c3_s0 same not-tree painted

segment 087 · kind lowrej · truth tree · crown cover 0 → 0.02

0 -> 652: the 4 m veg mass stays teal; only a small sub-1 m blob by the delineator turned mint.

before · az045
before az045
after (v2) · az045
after az045
before · az225
before az225
after (v2) · az225
after az225

acc_087_c1_s0 worse

segment 087 · kind acc · truth tree · crown cover 0.95 → 0.65

tree 62,991 -> 42,181: large blocks of a dense real tree stand demoted to unclassified/medium.

before · az045
before az045
after (v2) · az045
after az045
before · az225
before az225
after (v2) · az225
after az225

acc_087_c1_s1 worse

segment 087 · kind acc · truth tree · crown cover 0.95 → 0.88

tree 72,327 -> 67,375: holes punched into an already-correct canopy, no compensating gain.

before · az045
before az045
after (v2) · az045
after az045
before · az225
before az225
after (v2) · az225
after az225

lowrej_002_c0_s0 worse not-tree painted

segment 002 · kind lowrej · truth not_tree · crown cover 0 → 0

tree 0 -> 57,199: the entire continuous roadside scrub/hedge bank is painted mint, and vehicle (2,871 -> 347) plus unclassified (7,333 -> 128) points are swallowed into tree too.

before · az045
before az045
after (v2) · az045
after az045
before · az225
before az225
after (v2) · az225
after az225

lowrej_002_c0_s1 worse not-tree painted

segment 002 · kind lowrej · truth not_tree · crown cover 0 → 0

tree 0 -> 42,635 over the 2-3 m clipped hedge bank behind the guardrail; also eats vehicle (1,031 -> 358) and ground (293,829 -> 285,231).

before · az045
before az045
after (v2) · az045
after az045
before · az225
before az225
after (v2) · az225
after az225

lowrej_002_c0_s2 worse not-tree painted

segment 002 · kind lowrej · truth not_tree · crown cover 0 → 0

tree 0 -> 55,005 covering the whole scrub carpet plus a big slice of ground (25,041 -> 11,445) and all unclassified (6,097 -> 0).

before · az045
before az045
after (v2) · az045
after az045
before · az225
before az225
after (v2) · az225
after az225

Contested spot for Miro

acclow_087_c42_s0 contested

judges: botanist: tree 0.6; skeptic: unsure 0.45 (h95 2.1 m)

Tree or not?

az045
az045
az225
az225

Runtime

All 7 review items done in /home/ai/dev/wt-vs-recall (branch feat/AI3D-373-tree-recall, HEAD d824b4c), TDD (failing test first) for every behaviour change. Full suite 654 passed; uvx ruff check src tests clean.

  1. medium scripts/tree_recall_eval.py: ACCEPT_OUTCOMES now {vegetation_rf, vegetation_rf_relaxed, conic_rule, conifer_rule}, with a comment tying it to detect._TREE_OUTCOME_TYPE and stating why medium_vegetation stays out. It was the only “accepted” derivation in the script (corpus_summary line ~223). New test tests/test_tree_recall_eval.py::test_relaxed_rf_accepts_count_as_accepted.
  2. medium New knob tree_detection.tile_context_siblings (DetectorConfig.tree_tile_context_siblings, default True = today’s behaviour). detect.assign_context_features gained an optional group_ids arg (one id per cluster, negative = ungrouped): the forest loop skips same-group neighbours, and the ring builds owner_group = group-of-owning-cluster so a whole sibling set is dropped with one comparison — a proper exclusion, not a post-hoc subtraction. trees.detect_trees passes unit_parent as group_ids ONLY when the knob is down, so the default call is byte-identical (injected assign_context doubles unaffected). owner_group is only built when something is actually grouped, so the default path gains no extra pass over the candidate cloud. Tested both settings on the elongated belt fixture (tests/test_tree_tiles.py::test_sibling_tiles_can_be_excluded_from_a_tiles_own_context) … [source note truncated here]

Per-item detail for review items 3–7 was not supplied for this page.

Open questions for Miro

  1. Contested spot acclow_087_c42 — tree or not?
  2. Accept the v2 knobs as production default for A1?
  3. FN spots still missed: midrej_002_c17_s2, midrej_003_c3_s0, midrej_003_c3_s1, lowrej_022_c5_s0, midrej_060_c7_s0, lowrej_066_c8_s0, lowrej_066_c8_s1, lowrej_086_c5_s0
  4. FP spots: lowrej_002_c0_s0, lowrej_002_c0_s1, lowrej_002_c0_s2, lowrej_060_c12_s0, lowrej_060_c12_s1, lowrej_060_c12_s2, lowrej_060_c5_s0, lowrej_060_c5_s1, lowrej_060_c5_s2, miss_060_tall_s0