Handoff: tablecloth — classify ground on the combined segment cloud, make it the default, make it explicit

2026-09-06 · for an implementation agent in /home/ai/dev/3dai.iolabs.pointcloud.tablecloth (battlebox) · generated by Claude Fable 5.1 from the AI3D-382 session

Summary

Continuation target. Tablecloth (TCS) currently classifies ground per record: runner.process_segment loops process_record, and each record builds its own SMRF surface. Miro's rule: "Correctly, it should run on the combined point cloud from all the records in the segment." Records are overlapping drive passes; per-record surfaces disagree at overlaps and starve the min-Z seeds on sparse verges.

Make segment-wide classification the default config option, keep per-record as an explicit opt-out, and make the "segment cloud" an explicit, named concept in the code (a module + function + summary field), not an implicit loop. The per-record output contract (<stem>_tablecloth_masks.npz, cleaned <stem>_run3_points.npz, per-record run_summary.json entries) must not change, because seg3d (3dsegmentation/ground.py) and the asphaltedge rasters consume those files by name.

Update 2026-09-06 17:30 — measured on the verticalsigns copy of this change (A/B on segments 018/060/066/085). A naive segment-wide union with plain min-Z seeding is worse than per record: the union keeps only 0.37 / 0.52 / 0.63 / 0.75 of the ground points on 066 / 085 / 060 / 018, the DEM (Digital Elevation Model, the per-cell ground surface) tail is biased downward (z_ground p5 −3.0 m, min −8.1 m on 066), road surface becomes candidates, a real sign is lost on a 27 mm ground shift, and a truth crown gets a tile-shaped hole. Cause: below-ground noise (multi-echo / mirror artefacts) is sparse per record but its density scales with the number of overlapping passes, and min-Z per 0.2 m cell takes the lowest noise point. Real gains exist (human-GT tree recall on 085 0.755 → 0.887, sparse verges classified consistently), so the segment scope is right, the seeding is not.

Requirement added: segment scope must use noise-robust seeding, e.g. supported min-Z — per cell take the lowest 5 cm z-bin that holds ≥ k points (k ≈ 3, bins relative to the cell minimum, streamable as a uint16 histogram of ~20 bins per cell), or a streamed low percentile; and the union surface must never sit below the per-record surface by more than the elev tolerance (clamp or per-tile fallback where the local kept fraction is poor, e.g. record071 on 066 keeps 0.089). Acceptance canaries from the verticalsigns A/B: kept fraction per segment within ±0.05 of per record; 018 sign at (−3212.87, −2195.74) retained; no z_ground shift below −0.5 m at truth spots. Report: /home/ai/veg373_work/returns/eval/r3_judgement.md, renders /home/ai/veg373_work/returns/analysis/r3_renders/.

Update 2026-09-06 23:30 — what was tried in verticalsigns (rounds 5-7) and what to carry over. Measured on A1NR 018/060/066/085 against the per-record reference, judgements in /home/ai/veg373_work/returns/eval/r4_judgement.md … r6_judgement.md, code in worktree /home/ai/dev/wt-vs-382 (tcs_ground.py, knobs tcs_ground.*):

  • Supported min-Z seeding (lowest 5 cm bin with ≥3 points, cumulative or per-bin, sparse histogram ≈ 12-19 MB per segment): does not reach per-record parity on its own. Per cell, isolated below-ground noise and sparse ground under canopy are indistinguishable (2 ground + 40 canopy returns: per-bin support seeds 2.8 m above ground; plain min-Z seeds at noise up to 8 m below).
  • ELM-style neighbourhood test (reject a cell min > 0.5 m below the 11×11 median of neighbouring mins; re-seed or empty the cell): made things worse on corridors — the median sits on the verge/barrier, carriageway minima get rejected, 15 % of cells on 066/085 emptied and nearest-filled (066 union kept 0.40 → 0.38, z_ground p5 −2.7 → −4.0 m). A road-aware or slope-corrected reference is required before this can work.
  • Clamp to the per-record surface (eff_z = max(segment_z, record_z − 0.05)): the only mechanism that held every canary (sign retained, no crown holes, kept fractions within 0.02, z_ground p5 −0.15 m). But it reduces the union to "per record with holes filled" and buys third-decimal gains for +10-20 % runtime; still loses one segment-edge tree where the union surface is thinnest.
  • Decision in verticalsigns: per_segment=false, elm_enabled=false, clamp default on (inert). The segment scope remains the right architecture; the open problem is robust seeding of the union. Suggested next attempts for tablecloth: (a) low-outlier rejection against a road-aware reference (e.g. the per-record surfaces' median, or a slope-corrected local plane fit, not a raw median of cell mins); (b) reject noise per record before the union (each record's own isolated low points are rare and easy to flag), then union the cleaned records — this keeps the union's density benefit without the noise-density penalty; (c) a per-point "isolated low return" flag (no neighbour within 0.3 m in 3-D) as a pre-filter.

Current state (verified 2026-09-06)

Design (what "explicit" should look like)

  1. Config. New field ground_scope: Literal["segment", "record"] = "segment" in TableclothConfig + "ground_scope": "segment" in tablecloth.default.json. "record" reproduces today's behaviour bit-for-bit (keep a test that asserts this on the synthetic fixtures). Overridable with --set ground_scope=record.
  2. An explicit segment-cloud abstraction. New module segment_cloud.py (or segment_ground.py) with, e.g.: 
    @dataclass(frozen=True)
class SegmentCloud:
    segment_id: str
    records: tuple[Path, ...]          # discover_record_paths order
    n_points_per_record: tuple[int, ...]
    def iter_chunks(self, config) -> Iterator[np.ndarray]   # all records, chunked, in record order

def build_segment_surface(cloud: SegmentCloud, config) -> GroundSurface | None
def classify_segment(cloud: SegmentCloud, surface, config) -> list[np.ndarray]   # one kept mask per record, row-aligned
    Surface building is the existing two streamed passes, fed by cloud.iter_chunks() instead of one record's chunks (pass 1 bounds over ALL records, pass 2 min-Z seeds over ALL records, one _finalize_smrf_surface). Classification is classify_mask_from_chunks per record against the shared surface. Peak memory stays one chunk + the grid (a 100 m × ~150 m segment at 0.2 m is ~500 × 750 cells — trivial), so record_chunk_points keeps its meaning and the 10 GB soft budget is untouched. Log lines should say segment_018: segment surface from 7 records, 31.2 M points, grid 512x760 so the scope is visible in every run log.
  3. Runner. process_segment() becomes: discover records → build SegmentCloud → (scope=segment) build the surface once → for each record: classify against it, then the unchanged tail of process_record (cleaned npz, masks npz, overlay PNG, summary entry). Factor the per-record tail out of process_record so both scopes share it; process_record in scope=record keeps calling _classify_record.
  4. CSF. _csf_classify needs the whole cloud in memory. For scope=segment either concatenate all records (≤ ~1.1 GB of float64 xyz for 45 M points, acceptable under the budget — log the size) or refuse with a clear TableclothConfigError. Recommend: concatenate, warn above memory_budget_gb, and document. ground_percentile > 0 already cannot stream; keep that error and mention scope in its message.
  5. Edge padding / surface mesh. smrf_edge_pad_enabled now extrapolates only at the segment's outer boundary, which is the intended behaviour. With surface_mesh_enabled in segment scope write one segment_NNN_tablecloth_surface.ply instead of one per record (per-record PLYs would be identical copies). Keep the per-record PLY name in record scope.
  6. Summary (append-only). Add ground_scope and, in segment scope, segment_surface (records, n_points, grid shape, build seconds) to every per-record entry. Do not remove or rename existing keys.
  7. Public API for other repos. Export SegmentCloud, build_segment_surface, classify_segment from the package __init__ and document them in the README so verticalsigns can drop its own copy.

Steps

  1. cd /home/ai/dev/3dai.iolabs.pointcloud.tablecloth && git switch -c feat/segment-ground-scope && uv sync --extra dev. Commit small, one-line messages; ask Miro for the Jira tag if one is required (the AI3D-382 session used AI3D-382:; this change is not that ticket).
  2. Config field + JSON + tests/test_config.py case (--set ground_scope=record parses; ground_scope=foo rejected).
  3. segment_cloud.py with unit tests on synthetic data: two overlapping records over one plane with an elevated slab; assert (a) segment-scope masks per record equal the slices of a classification of the concatenated cloud, (b) a verge sampled sparsely in record A and densely in record B gets the same decision in both records, (c) record scope is byte-identical to the current outputs (reuse tests/test_ground_smrf.py fixtures).
  4. Runner refactor (shared per-record tail), CLI test in tests/test_runner_cli.py for both scopes, summary fields, README (Configuration table, run_summary schema, a "Scope" paragraph in the mechanism section, public API section).
  5. Real-data A/B on the new dataset, segment 018 and 086 (largest): see Verification. Runtime and peak RSS must stay within the same order as per-record.
  6. If the defaults change behaviour for seg3d (they will, slightly), say so in the README changelog and tell Miro that /home/ai/seg3d_out/260905/ground_A1B0 should be regenerated before the next fuse (11 segments took 955 s).
  7. Publishing to Nexus / version bump is Miro's call (wrap-up skill exists); do not publish unasked.

Verification

# reference (per record) already exists:
ls /home/ai/seg3d_out/260905/ground_A1B0/segment_018/

# segment scope (default after the change); --out must not nest inside --data-dir
DATA=/mnt/d/a123-data/Abschnitt_1_no_angle_filter_returns/branch_000
OUT=/home/ai/veg373_work/tablecloth_seg   # scratch: never /tmp, never /mnt/d
mkdir -p $OUT/logs
cd /home/ai/dev/3dai.iolabs.pointcloud.tablecloth
/usr/bin/time -v uv run python -m iolabs_point_cloud_tablecloth.runner \
  --data-dir $DATA/lane_points --segments 018,086 --out $OUT/ground_seg \
  --log-level INFO > $OUT/logs/ground_seg.log 2>&1
# opt-out must reproduce the reference bit-for-bit
/usr/bin/time -v uv run python -m iolabs_point_cloud_tablecloth.runner \
  --data-dir $DATA/lane_points --segments 018 --out $OUT/ground_rec \
  --set ground_scope=record --log-level INFO > $OUT/logs/ground_rec.log 2>&1

Report per record: kept fraction (reference vs segment scope), number of points whose decision changed, and where they are (bin the changed points into 2 m cells and list the top cells — expected: record overlaps and sparse verges, not the carriageway). Compare ground_rec masks to the reference with np.array_equal. Check run_summary.json peak RSS and runtime. Then confirm seg3d still loads the masks: python -c "from iolabs_point_cloud_segmentation_3d.ground import load_ground_mask" path unchanged, file names unchanged.

Not run yet: nothing of this is implemented; no A/B exists. The only related measurement is from verticalsigns, where mixing ground from another dataset shifted z_ground by up to 4.7 m and changed 66/514 clusters — i.e. the detectors are sensitive to the ground surface, so the A/B matters.

Risks and open questions

Suggested skills