fogfish / iq / 20696400416 / 1
32%
main: 32%

DEFAULT BRANCH: main
Ran
Files 41
Run time 1s
Badge
Embed ▾
README BADGES
x

If you need to use a raster PNG badge, change the '.svg' to '.png' in the link

Markdown

Textile

RDoc

HTML

Rst
coverage: 39.293% (-1.7%) from 41.026%
20696400416.1

push

github

web-flow 
feat: Implement skip-if-exists flag for incremental processing (ADR-0… (#86)

* feat: Implement skip-if-exists flag for incremental processing (ADR-0006 Phase 3)

This commit implements the --skip-if-exists CLI flag to enable incremental
processing and recovery from failures. The flag checks if anchor output
exists before processing each document, allowing expensive LLM operations
to be skipped when results already exist.

Implementation Details:

1. CLI Flag Integration (cmd/opts.go):
   - Added skipIfExists field to optsAgent struct
   - Registered --skip-if-exists flag with description
   - Integrated skip logic into agent build pipeline
   - Validates that --output-dir is specified when using skip-if-exists

2. Anchor Key Computation (internal/blueprint/compiler/anchor.go):
   - Created AnchorKeyComputer to calculate expected output keys
   - Supports all step types: AgentStep, RouterStep, ForeachStep, RunStep
   - Handles emit attribute to compute prefixed output paths
   - For foreach steps, anchor is the array file (not individual elements)
   - Defaults to input key when no emit is specified

3. Skip Checking Logic (internal/blueprint/compiler/skip.go):
   - Created SkipChecker using Storage interface to check file existence
   - Uses AnchorKeyComputer to determine expected output location
   - Reports skipped documents via progress reporter
   - Returns early if anchor file already exists

4. Pipeline Integration (internal/service/worker/worker.go):
   - Added workflow field to Builder to store compiled workflow
   - Created SkipIfExists() builder method
   - Integrates storage, anchor computer, and progress reporter
   - Adds skip processor to pipeline before agent execution

5. Processor Implementation (internal/iosystem/processor/skipifexists.go):
   - Created SkipIfExists processor implementing Processor interface
   - Filters documents based on anchor existence check
   - Passes through EOF markers and empty document sets
   - Implements... (continued)

1578 of 4016 relevant lines covered (39.29%)

0.43 hits per line

Source Files on job 20696400416.1