fogfish / iq / 20697941198

31%

Build Type

push

github

Committed by web-flow

Commit Message

Implement #81: Pipeline integration with storage sink and emit execution (#87)

* Implement #81: Pipeline integration with storage sink and emit execution

Added storage-based sink with emit context support for pipeline integration:

- Created StorageSink that wraps Storage interface with emit support
- Updated all step types (Agent, Router, Foreach, Run) to set emit context
- ForeachStep now manages iteration counters for nested loops
- Each step propagates emit prefix through execution context
- Storage sink applies emit prefix and counters to output keys
- Added Storage() method to sink builder for emit-aware output
- Added comprehensive tests for StorageSink with emit and counters

This enables emit-based output control where workflow steps can specify
output key prefixes via 'emit' attribute. Foreach loops add iteration
counters (e.g., file.000001.txt, file.000002.txt) for multi-item processing.

The implementation:
- Preserves emit context across step execution using context.Context
- Supports nested foreach with stacked counters
- Maintains backward compatibility (emit is optional)
- Uses existing Storage interface for filesystem operations

Example usage:
  steps:
    - uses: prompts/summarize.md
      emit: summaries  # Output to summaries/doc.txt

    - foreach:
        selector: document
        emit: processed  # Output to processed/doc.000001.txt, etc.

* Fix emit context propagation using EmitContextCapture pattern

Issue: Emit context set in workflow steps wasn't reaching StorageSink because Go contexts are immutable - workflow execution creates new contexts that don't propagate back.

Solution: Implemented EmitContextCapture pattern - a mutable struct passed via context that workflow steps can write to during execution.

Changes:
- Added EmitContextCapture struct and helpers (WithEmitCapture, GetEmitCapture) to compiler/context.go
- Updated all workflow steps (AgentStep, RouterStep, ForeachStep, RunStep) to capture emit in EmitContextC... (continued)

Run Details

73 of 198 new or added lines in 9 files covered. (36.87%)

3 existing lines in 3 files now uncovered.

1655 of 4214 relevant lines covered (39.27%)

0.43 hits per line