Alan-Jowett / CoPilot-For-Consensus / 20241438647

77%

push

github

Implement archives collection write in ingestion service (#245)

* Initial plan

* Add archives collection write capability to ingestion service

- Added document store dependency to ingestion configuration schema
- Updated ingestion main.py to initialize and connect to document store
- Updated IngestionService to accept DocumentStore parameter
- Added _write_archive_record method to write archive metadata to document store
- Updated docker-compose.yml with correct DOC_STORE_TYPE env var
- Updated tests to include document store in fixtures
- Added test to verify archives collection is populated after ingestion

Co-authored-by: Alan-Jowett <20480683+Alan-Jowett@users.noreply.github.com>

* Add archive status update to parsing service

- Updated parsing service to set archive status to 'processed' after successful parsing
- Updated parsing service to set archive status to 'failed' on parsing errors
- Added _update_archive_status method with best-effort error handling

Co-authored-by: Alan-Jowett <20480683+Alan-Jowett@users.noreply.github.com>

* Update integration test to verify archives collection

- Updated end-to-end integration test to verify archives collection is populated
- Added assertions to check archive document structure
- All tests pass successfully

Co-authored-by: Alan-Jowett <20480683+Alan-Jowett@users.noreply.github.com>

* Address code review feedback and add verification script

- Add defensive check for error_reporter before calling report()
- Fix bare except clause in verification script
- Add scripts/verify_archives_collection.py for manual verification
- All tests still passing

Co-authored-by: Alan-Jowett <20480683+Alan-Jowett@users.noreply.github.com>

* Add implementation summary document

- Created comprehensive INGESTION_ARCHIVES_IMPLEMENTATION.md
- Documents problem, solution, testing, and deployment
- Includes verification steps and future enhancements
- All changes complete and tested

Co-authored-by: Alan-Jowett <2048068... (continued)

22 of 28 new or added lines in 2 files covered. (78.57%)

4029 of 4845 relevant lines covered (83.16%)

0.94 hits per line