Alan-Jowett / CoPilot-For-Consensus / 20352533977

78%

push

github

Fix Service Metrics dashboard by adding push() calls to metrics collectors (#410)

* Initial plan

* Add metrics push() calls to all services and enhance dashboard

- Added push() method calls in parsing, embedding, chunking, summarization, and orchestrator services
- Services now push metrics to Pushgateway after collection
- Enhanced Grafana dashboard with new panels:
  * Chunking Throughput
  * Summarization Latency
  * Queue Depths (RabbitMQ)
  * Error Rates by Service
  * Processing Latency for Parsing & Chunking
- Fixed Vector Store Size panel to use qdrant_collection_vectors_count metric
- All panels now have proper queries and thresholds

Co-authored-by: Alan-Jowett <20480683+Alan-Jowett@users.noreply.github.com>

* Document service metrics in SERVICE_MONITORING.md

- Added comprehensive Service Metrics Reference section with all metric names and labels
- Documented metric types (Counter, Histogram, Gauge) for each service
- Included dashboard query examples for each metric
- Added troubleshooting section for "No data" issues in Service Metrics Dashboard
- Enhanced Available Dashboards section with detailed panel descriptions
- Provided quick fix checklist for common metric issues

Co-authored-by: Alan-Jowett <20480683+Alan-Jowett@users.noreply.github.com>

* Fix duplicate ID fields in Grafana dashboard panels

- Removed duplicate id fields from gridPos objects
- Panel IDs now only appear once at panel level
- Prevents panel rendering issues in Grafana

Co-authored-by: Alan-Jowett <20480683+Alan-Jowett@users.noreply.github.com>

* Changes before error encountered

Co-authored-by: Alan-Jowett <20480683+Alan-Jowett@users.noreply.github.com>

* Refactor: Use safe_push() helper to reduce code duplication

- Replaced duplicate push logic with safe_push() helper method in all services
- Updated embedding, chunking, summarization, and orchestrator services
- Reduces code duplication and ensures consistent error handling
- Addresses PR review comment ... (continued)

9 of 19 new or added lines in 6 files covered. (47.37%)

4642 of 5582 relevant lines covered (83.16%)

0.94 hits per line