fogfish / iq / 20679682425

31%

push

github

feat: implement ArrayCollector processor for batch document collection (ADR-0007 Phase 2) (#73)

* feat: implement ArrayCollector processor for batch document collection (ADR-0007 Phase 2)

Implement ArrayCollector processor that collects all input documents
and emits them as an array when EOF signal is received. This enables
batch processing mode for the --array CLI flag.

Key features:
- Collects documents in memory during pipeline execution
- Returns empty slice during collection to prevent premature propagation
- Emits collected documents as array upon EOF signal
- Automatically resets after EOF for potential reuse
- Pre-allocates buffer for reasonable performance

Implementation:
- Add ArrayCollector processor in collector.go
- Add comprehensive unit tests covering:
  * Collection and emission behavior
  * Empty collection handling
  * Multiple documents in single call
  * Reset behavior after EOF
- Add integration test with Conduit pipeline

Testing:
- All unit tests pass (4/4)
- Integration test passes
- No regressions in existing tests

Resolves #65 (ADR-0007 Phase 2)
Related to ADR /doc/adr/0007_array_as_input.md

* update license

17 of 21 new or added lines in 1 file covered. (80.95%)

1228 of 3526 relevant lines covered (34.83%)

0.37 hits per line