Morry98 / fastapi-task-manager / 22868571260

98%

push

github

Improve task management with Redis Streams, leader election and  management API (#2)

## Summary

  Major overhaul of fastapi-task-manager introducing a more robust and scalable
  architecture:

  - **Redis Streams mode**: Replaced the legacy polling mode with Redis Streams +
  consumer groups for distributed task execution, ensuring reliable message
  delivery and at-least-once processing
  - **Leader election**: Added distributed leader election via Redis so only one
  instance schedules tasks across multiple app replicas
  - **Reconciler & heartbeat**: New reconciler component detects and recovers
  stale/failed tasks; tasks now emit heartbeats to track liveness
  - **Exponential backoff retry**: Failed tasks are automatically retried with
  configurable exponential backoff
  - **Dynamic task CRUD API**: New endpoints for creating, updating, and deleting
  tasks at runtime with Redis persistence
  - **Revamped management API**: Async bulk operations for task groups and tasks
  (enable/disable, trigger, health checks)
  - **Statistics via Redis Streams**: Migrated statistics storage from Redis Lists
  to Streams for better performance and queryability
  - **Worker identity**: Each worker instance now has a unique identity tracked via
   `WorkerIdentity` schema
  - **Centralized Redis keys**: All Redis key patterns consolidated in
  `redis_keys.py`
  - **Simplified config**: Removed redundant TTL config fields; renamed `app_name`
  → `redis_key_prefix` and `log_level` config
  - **Comprehensive test suite**: Added 12 test modules covering all new components
   (~3,500+ lines of tests)
  - **Full documentation site**: MkDocs Material docs with getting started guide,
  architecture overview, API reference, deployment guide, and dynamic tasks
  tutorial
  - **CI workflow**: Added GitHub Actions CI pipeline with Python 3.14 support
  - **Pre-commit improvements**: Migrated from mypy+vermin to Astral's `ty`, added
  bandit security linter

3182 of 3264 new or added lines in 32 files covered. (97.49%)

3342 of 3425 relevant lines covered (97.58%)

4.86 hits per line