stacklok / toolhive / 23595496600

65%

Build Type

push

github

Committed by web-flow

Commit Message

Add forEach step type for VMCP composite tool workflows (#4375)

* Add forEach step type for VMCP composite tool workflows

Workflows today require statically defined steps -- you cannot dynamically
spawn steps based on a previous step's output. This makes patterns like
"for each package in the image, query for vulnerabilities" impossible
without hardcoding every item.

Add a third step type `forEach` that iterates over a collection produced
by a previous step and executes an inner tool step for each item, with
configurable parallelism and error handling.

Key design decisions:
- forEach is a DAG-opaque macro: the DAG sees one node, internal
  parallelism is self-managed via errgroup + semaphore
- Single inner step (tool type only) keeps it simple; multi-step
  iteration bodies and elicitation inner steps can come later
- Nested forEach is explicitly rejected at validation time
- Collection resolves via Go template expansion then JSON parse,
  reusing existing template infrastructure
- Aggregated output has a well-known shape (iterations, count,
  completed, failed) so downstream templates have a predictable
  structure to reference

Safety limits:
- maxIterations: default 100, hard cap 1000
- maxParallel: default 10 (DAG default), hard cap 50
- itemVar cannot shadow the reserved "index" key
- Step and workflow timeouts apply as normal

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* Address review feedback on forEach step implementation

Write a "cancelled" sentinel to iteration results when errgroup
cancels the context, preventing zero-value entries from producing
misleading aggregated output. Add MaxParallel() accessor to
dagExecutor to avoid coupling forEach to DAG struct internals.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

Run Details

255 of 322 new or added lines in 7 files covered. (79.19%)

36 existing lines in 4 files now uncovered.

52383 of 80144 relevant lines covered (65.36%)

64.61 hits per line