Alan-Jowett / CoPilot-For-Consensus / 20282926157

78%

push

github

Implement idempotent processing across all pipeline stages (#286)

* Initial plan

* Implement idempotency for all processing stages

- Updated vectorstore interface and Qdrant implementation to use upsert semantics
- Modified embedding service to handle duplicate embeddings gracefully
- Added idempotency checks to summarization service (skip if summary exists)
- Added idempotency checks to orchestrator service (skip if summary exists)
- Added comprehensive idempotency tests for embedding, summarization, and orchestrator
- Documented idempotency patterns and requirements in CONTRIBUTING.md

Co-authored-by: Alan-Jowett <20480683+Alan-Jowett@users.noreply.github.com>

* Fix summarization idempotency test - provide messages for context

The test needs to return messages when querying the messages collection
so the summarization process can retrieve context before calling the
summarizer. Fixed side effect to handle both collections properly.

Co-authored-by: Alan-Jowett <20480683+Alan-Jowett@users.noreply.github.com>

* Address code review feedback - improve exception handling

- Use specific exception types (ConnectionError, OSError, TimeoutError) for database connectivity issues
- Log unexpected exceptions with full traceback for debugging
- Clarify that status updates are idempotent (safe to repeat)
- Make error handling more precise to avoid masking programming errors

Co-authored-by: Alan-Jowett <20480683+Alan-Jowett@users.noreply.github.com>

* Fix summarization tests to be collection-aware for idempotency

The idempotency changes added a check for existing summaries before processing.
Updated mock_document_store fixture to return collection-specific data:
- Returns empty list for "summaries" collection (no duplicates by default)
- Returns messages for "messages" collection
- Updated tests that override query_documents to use side_effect instead of return_value

This fixes all 9 failing tests in the summarization service.

Co-authored-by: Alan-Jowet... (continued)

17 of 26 new or added lines in 4 files covered. (65.38%)

4092 of 4909 relevant lines covered (83.36%)

0.95 hits per line