stacklok / toolhive / 20856154197

push

github

Add Recovery Middleware to Prevent Server Crashes from Panics (#3219)

This PR implements dedicated panic recovery middleware to ensure all toolhive servers recover gracefully from panics instead of crashing. Previously, panic recovery was inconsistently handled within the telemetry middleware, which is optional and may not be enabled.

Closes #3107

Changes
New Recovery Package (pkg/recovery/)

Created RecoveryMiddleware that catches panics, logs them, and returns HTTP 500
Implements the types.Middleware interface for factory pattern integration
Includes unit tests for panic recovery scenarios
Proxy Runner Integration (pkg/runner/)

Registered recovery middleware factory in GetSupportedMiddlewareFactories()
Added recovery middleware to both PopulateMiddlewareConfigs() and WithMiddlewareFromFlags()
Positioned as the outermost wrapper to catch panics from all other middleware
thv API Server Integration (pkg/api/server.go)

Added recovery middleware as the first middleware in the chi router chain
vMCP Server Integration (pkg/vmcp/server/server.go)

Applied recovery middleware as the outermost wrapper in the MCP handler chain
Telemetry Middleware Cleanup (pkg/telemetry/middleware.go)

Removed redundant panic recovery code (outer handler recover, span.End recover, WriteHeader recover)
Recovery is now handled by the dedicated middleware
Testing
Added unit tests for recovery middleware covering:
Normal requests pass through correctly
Panics are recovered and return 500 Internal Server Error
Request context is preserved through the middleware

---------

Signed-off-by: Jeremy Drouillard <jeremy@stacklok.com>

13 of 46 new or added lines in 6 files covered. (28.26%)

29145 of 58234 relevant lines covered (50.05%)

58.11 hits per line