lsm / neokai / 28218871896

82%

Build Type

push

github

Committed by web-flow

Commit Message

feat(space-runtime): auto-continue on terminal error results (#2180)

* feat(space-runtime): auto-continue on terminal error results (#673)

A node-agent session that ends on a terminal error result (e.g. Codex 400,
error_during_execution) left the node silently idle with a *live* session and
no recovery firing — every result message was classified terminal, so the
alive-stuck, non-terminal-idle, crash-retry, and blocked-run sweeps all bailed.
The worker was revivable only by a manual 'continue'.

Add handleTerminalErrorIdleExecutions, a catch-all sweep in processRunTick
(right after handleNonTerminalIdleExecutions) that injects a bounded number of
'continue' messages via the same primitive a manual continue uses
(tam.injectRuntimeRecoveryMessage), then escalates to 'blocked' so
attemptBlockedRunRecovery takes over with its own MAX_BLOCKED_RUN_RETRIES cap.

Guards (task #673):
- Only retryable subtypes: error_during_execution, error_max_turns.
  Cost (error_max_budget_usd) and structured-output exhaustion are skipped.
- Prompt-too-long results skipped (deferred to #670 compaction), detected via
  terminal_reason='prompt_too_long' or errors[] signature.
- Only when task is in_progress with no reportedStatus and session alive.
- Per-execution cap MAX_TERMINAL_ERROR_CONTINUE_RETRIES=2, then blocked.
- Deterministic repeats (identical error signature) escalate immediately.
- Grace cooldown between attempts (mirrors alive-stuck nag grace); applied on
  failure too so a transiently-failing injection cannot tight-loop.
- Re-spawn (new session id after blocked recovery) resets the budget; total
  stays bounded by MAX_BLOCKED_RUN_RETRIES.

Escalation mirrors handleAliveStuckExecutions' blocked path: execution/run/task
-> blocked with blockReason 'execution_failed' + task_blocked/workflow_run_blocked
notifications, deduplicated via notifiedTaskSet.

* fix(space-runtime): harden terminal-error recovery per review (#673)

Address codex review findings on PR #2180... (continued)

Coverage Stats

11423 of 16451 branches covered (69.44%)

Branch coverage included in aggregate %.

200 of 310 new or added lines in 2 files covered. (64.52%)

226 existing lines in 1 file now uncovered.

96470 of 115191 relevant lines covered (83.75%)

298.58 hits per line