AI3D-382 number_of_returns in run3 NPZ — fleet diff report

Generated 2026-09-02T04:26:40.872676+00:00. Window: 2026-09-01 00:00 to now.

Order: LLM-applied order

Summary

AI3D-382 adds the per-point LAS number_of_returns field to the run3 segment NPZ contract and threads it through the LIDAR highway pipeline: producer (Step 3 segmentationtrajectory and the orchestrator's range-split writer), the shared schema in iolabs-common, and the two consumers that read run3 records (Step 5 maskclustering, Step 6 3dsegmentation). 25 commits across 6 repos, all authored 2026-09-01, all on remote refs. No fetch warnings.

Contract: number_of_returns is uint8, always written on save, optional on load (legacy NPZs are zero-filled; 0 means unknown and is never defaulted to 1, since 0 is not a legal LAS return count). Boolean arrays are rejected, out-of-range values raise, and no 1–7 domain check is applied because LAS point formats 6–10 carry 4-bit counts up to 15.

Structure: the work landed in three rounds the same day. Round 1 added the field end to end (3c981c5, 74a972e, e0c28bc). Round 2 replaced hand-rolled per-key code with a schema registry (POINT_RECORD_SCHEMA in common, 639d755) so producer and consumers iterate one field tuple; adding a future optional field is now one registry entry plus one extraction line. Round 3 was an adversarial Sol + Fable review pass; its one real defect fix is 2a79902, where Step 5's cluster NPZ writer was silently dropping the new channel at its output boundary.

Release gating (nothing is live yet): published iolabs-common 0.7.0 predates all of this. The producer venv resolves common 0.3.2, so it mirrors the schema locally rather than importing it, and its parity test skips until the floor is raised. Consumers on 0.7.0 drop the unknown key on load without crashing, so new 7-key NPZs cannot break old consumers. Pending: publish common 0.7.1, raise floors, repin the orchestrator conda env. The Step 5 filteringintensity change (31a4ceb) was reverted (ab5a38b) because it broke that repo against its pinned common; it nets to zero in this report and can be re-applied after the release.

The final orchestrator entry (3dd6817) is an unrelated AI3D-226 merge that fell inside the time window; it is ranked last and can be ignored.

Matching Commits

  1. iolabs-common 3c981c5: AI3D-382 Add number_of_returns to the run3 point-NPZ contract

    Miroslav Simko <ms@iolabs.ch> 2026-09-01T10:43:42+02:00

    2 files · 9 snippets · score 75

    Origin of the contract. segment_points_io gains NUMBER_OF_RETURNS_KEY/NUMBER_OF_RETURNS_DTYPE (uint8); save_points_npz always writes it, load_points_npz zero-fills it when absent (0 = unknown). Every other commit in this report exists to feed or honour this.

  2. Step 3 segmentationtrajectory 74a972e: AI3D-382 Write number_of_returns into run3 segment NPZ outputs

    Miroslav Simko <ms@iolabs.ch> 2026-09-01T10:48:35+02:00

    1 files · 13 snippets · score 75

    Producer side: divide_las_file_by_planes now extracts number_of_returns per chunk and writes it into each per-segment NPZ. LAS files without the field degrade to zeros via the fallback helper rather than raising, unlike RGB/intensity which stay mandatory.

  3. iolabs-common 639d755: AI3D-382 Drive the point-record contract from a schema registry

    Miroslav Simko <ms@iolabs.ch> 2026-09-01T14:54:43+02:00

    2 files · 12 snippets · score 75

    Largest structural change. Replaces per-key special-casing in common with a POINT_RECORD_SCHEMA registry (key, dtype, required/optional, fill) and adds generic mask_record/concat_records helpers driven by it. Later producer/consumer commits iterate this registry instead of hardcoding names. Big diff but mostly mechanical; the registry table and the load/save loops are the parts worth reading.

  4. Orchestrator e0c28bc: AI3D-382 Write number_of_returns through the range-split run3 NPZ writer

    Miroslav Simko <ms@iolabs.ch> 2026-09-01T10:58:14+02:00

    1 files · 8 snippets · score 75

    Orchestrator's own range-split run3 writer (the non-chunked path in s3_segment_mapper.py) mirrored the producer change so both write paths emit the same 7-key record. Note the orchestrator conda env still pins the producer at 0.7.2, so this path emits legacy records until repinned. No pytest infrastructure there; untested, flagged as follow-up.

  5. Step 5 maskclustering f9f1b08: AI3D-382 Load run3 records through common segment_points_io and carry channels by key

    Miroslav Simko <ms@iolabs.ch> 2026-09-01T14:53:04+02:00

    1 files · 6 snippets · score 75

    Step 5 stops parsing run3 NPZs by hand and loads them through common's segment_points_io, carrying ancillary channels by key. This is what makes the consumer schema-agnostic; REQUIRED_ARRAYS becomes a deprecated alias of POINT_RECORD_KEYS (member order changed, gains optional keys under new common).

  6. Step 5 maskclustering 2a79902: AI3D-382 Forward extra ColorIntensityData channels into cluster NPZs

    Miroslav Simko <ms@iolabs.ch> 2026-09-01T23:36:12+02:00

    2 files · 9 snippets · score 85

    Round-3 review find. write_cluster_npz only wrote a fixed member list, so the new channel was silently dropped at Step 5's output boundary. Fix is generic: any extra ColorIntensityData field discovered via dataclasses.fields() is forwarded, with a warn-and-skip guard on collision with fixed member names. Downstream tablecloth reader verified key-generic, so the extra member is safe.

  7. Step 6 3dsegmentation de39aa0: AI3D-382 Carry unknown run3 record keys through SegmentCloud

    Miroslav Simko <ms@iolabs.ch> 2026-09-01T14:51:34+02:00

    2 files · 7 snippets · score 75

    Step 6 consumer: SegmentCloud gains an extra mapping so unknown run3 record keys survive load → fuse → save instead of being discarded. Known gap left as follow-up: writer.py's ReCap export still hardcodes number_of_returns=1.

  8. Step 3 segmentationtrajectory 4e54928: AI3D-382 Thread run3 NPZ per-point fields as one schema-driven dict

    Miroslav Simko <ms@iolabs.ch> 2026-09-01T14:54:28+02:00

    2 files · 13 snippets · score 75

    Producer refactor: the per-point ancillary arrays travel as one dict[str, ndarray] keyed by ANCILLARY_FIELD_NAMES through extraction, angle mask, dtype registration, writer and memmap. Large line churn but proven byte-identical NPZ output old-vs-new. Cuts the cost of adding a field to one tuple entry plus one extraction line.

  9. iolabs-common 270b56b: AI3D-382 Carry number_of_returns through ColorIntensityData

    Miroslav Simko <ms@iolabs.ch> 2026-09-01T10:43:42+02:00

    2 files · 11 snippets · score 75

    ColorIntensityData (the in-memory record shared by consumers) gains an optional number_of_returns member; select_by_mask/append carry it.

  10. iolabs-common 14f041e: AI3D-382 Make ColorIntensityData.number_of_returns keyword-only and uint8-enforced

    Miroslav Simko <ms@iolabs.ch> 2026-09-01T23:39:27+02:00

    4 files · 15 snippets · score 75

    Hardening from round 3: the member becomes keyword-only (fleet grep showed all 10 construction sites already use keywords) and is coerced to uint8 with bool rejection via a new shared _dtype_coercion module that segment_points_io now delegates to. Error strings kept byte-identical.

  11. iolabs-common 1bb1991: AI3D-382 Coerce number_of_returns to uint8 on load and reject bool arrays

    Miroslav Simko <ms@iolabs.ch> 2026-09-01T11:09:17+02:00

    2 files · 6 snippets · score 75

    Load-side strictness: stored counts are coerced to uint8, boolean arrays rejected (a mask is not a count), non-integral and out-of-range values raise. Deliberately no 1–7 domain check (4-bit LAS formats go to 15).

  12. iolabs-common a733b88: AI3D-382 Derive ColorIntensityData mask and append from its fields

    Miroslav Simko <ms@iolabs.ch> 2026-09-01T14:55:33+02:00

    2 files · 6 snippets · score 75

    select_by_mask and append are derived from dataclasses.fields() instead of listing members, so a future field needs no edits there.

  13. iolabs-common cbbb210: AI3D-382 Harden generic record helpers and field mapping

    Miroslav Simko <ms@iolabs.ch> 2026-09-01T15:08:19+02:00

    4 files · 12 snippets · score 75

    Follow-up hardening of the generic helpers (shape/dtype checks in mask_record/concat_records, _map_fields edge cases) plus tests.

  14. Step 3 segmentationtrajectory 014ab30: AI3D-382 Build ColorIntensityData kwargs by feature detection so old iolabs-common works

    Miroslav Simko <ms@iolabs.ch> 2026-09-01T15:11:22+02:00

    2 files · 10 snippets · score 85

    Cross-version compatibility: the producer's load_color_intensity_data builds kwargs filtered by the installed dataclass's declared fields, so it works against pinned common 0.3.2 (no such field) and against 0.7.1+ (field carried). Zero production callers today; latent.

  15. Step 3 segmentationtrajectory 45eb56e: AI3D-382 Generalize the LAS field fallback helper and freeze DEFAULT_FIELD_DTYPES

    Miroslav Simko <ms@iolabs.ch> 2026-09-01T15:11:02+02:00

    1 files · 4 snippets · score 75

    Makes the LAS fallback helper generic over key/dtype (_record_field_or_zeros) and freezes DEFAULT_FIELD_DTYPES as a MappingProxyType so the schema can't be mutated process-wide.

  16. Step 5 maskclustering 3bd1d4e: AI3D-382 Derive separation channel dtypes from the record schema

    Miroslav Simko <ms@iolabs.ch> 2026-09-01T14:53:04+02:00

    1 files · 2 snippets · score 75

    Small: separation-channel dtypes in the maskclustering pipeline come from the record schema instead of a local table. Removes 6 lines of duplication.

  17. Step 5 filteringintensity ab5a38b: AI3D-382 Revert bright-filter number_of_returns until new common release

    Miroslav Simko <ms@iolabs.ch> 2026-09-01T11:07:39+02:00

    1 files · 8 snippets · score 75

    Revert of 31a4ceb. Step 5 bright-points I/O is outside the run3 contract and the change broke filteringintensity master against its pinned common 0.3.2. Net effect of this pair in the report is zero. Re-apply after the common release and floor bump if wanted.

  18. Step 5 filteringintensity 31a4ceb: AI3D-382 carry number_of_returns through step5 bright-points NPZ I/O

    Miroslav Simko <ms@iolabs.ch> 2026-09-01T10:49:18+02:00

    1 files · 8 snippets · score 75

    Original attempt to carry the field through the bright-points NPZ I/O; reverted the same day (see ab5a38b). Kept in history only.

  19. Step 3 segmentationtrajectory 60bcec1: AI3D-382 Pin zero-fill semantics in the schema parity test and document the fallback cast

    Miroslav Simko <ms@iolabs.ch> 2026-09-01T23:35:31+02:00

    2 files · 5 snippets · score 75

    Test/doc only. Parity test now pins zero-fill semantics and a round-trip; fallback helper docstring warns that its plain astype would wrap a wider future field silently, so range checks belong at the call site.

  20. Step 3 segmentationtrajectory 5d20310: AI3D-382 Guard the run3 NPZ schema against drift from iolabs.common.segment_points_io

    Miroslav Simko <ms@iolabs.ch> 2026-09-01T15:11:49+02:00

    2 files · 5 snippets · score 85

    Adds the schema-parity test against common's segment_points_io. It importorskips, so it is inert on the pinned 0.3.2 and activates by itself once the floor is raised. CLAUDE.md documents the schema duplication.

  21. Step 3 segmentationtrajectory d09a6ce: AI3D-382 Cover number_of_returns write path and zero fallback in tests

    Miroslav Simko <ms@iolabs.ch> 2026-09-01T10:48:35+02:00

    2 files · 11 snippets · score 55

    Tests for the producer write path and the zero fallback, plus fixture updates for the extra key.

  22. iolabs-common b9e3427: AI3D-382 Document the extra-key policy of the point-record helpers

    Miroslav Simko <ms@iolabs.ch> 2026-09-01T23:39:27+02:00

    1 files · 4 snippets · score 75

    Docstrings only: states the extra-key policy (unknown NPZ members are ignored on load, never propagated) on the four record helpers. No behavior change.

  23. Step 3 segmentationtrajectory 27008e7: AI3D-382 Document the number_of_returns NPZ field

    Miroslav Simko <ms@iolabs.ch> 2026-09-01T10:50:28+02:00

    3 files · 5 snippets · score 85

    Docs only (AGENTS.md / CLAUDE.md / knowledge.md) describing the new field.

  24. Step 3 segmentationtrajectory 30dc2f2: AI3D-382 Document the schema-driven NPZ field tuple

    Miroslav Simko <ms@iolabs.ch> 2026-09-01T14:54:28+02:00

    1 files · 1 snippets · score 85

    Docs only: CLAUDE.md describes the schema-driven field tuple.

  25. Orchestrator 3dd6817: Merged in ai3d-two-stage-step7 (pull request #4)

    Miroslav Simko <ms@iolabs.ch> 2026-09-01T06:32:53Z

    0 files · 0 snippets · score 10

    Unrelated to AI3D-382: merge of an AI3D-226 helper-pipeline YAML that happened to fall inside the window. Ignore.

Context Commits

No matching commits.