Alan-Jowett / CoPilot-For-Consensus / 20556275760

78%

push

github

Fix Ollama read timeout on CPU by making LLM timeout and citation length configurable (#610)

* Initial plan

* Add configurable LLM timeout and citation text max length

- Add llm_timeout_seconds config (default: 300s) to address CPU inference timeouts
- Add citation_text_max_length config (default: 300, reduced from 500) to reduce prompt size
- Update LocalLLMSummarizer and LlamaCppSummarizer to accept timeout from factory
- Update SummarizerFactory to pass timeout parameter to summarizers
- Update main.py to read timeout and citation_text_max_length from config
- Add comprehensive tests for timeout configuration
- All 55 adapter tests passing

Co-authored-by: Alan-Jowett <20480683+Alan-Jowett@users.noreply.github.com>

* Add test for configurable citation_text_max_length

- Add test_format_citations_custom_max_length to verify custom max length works
- Update test comment to clarify default behavior in fixture

Co-authored-by: Alan-Jowett <20480683+Alan-Jowett@users.noreply.github.com>

* Update summarization README with new config options

- Document LLM_TIMEOUT_SECONDS and CITATION_TEXT_MAX_LENGTH
- Add guidance for CPU performance optimization
- Note default values and troubleshooting tips for timeouts

Co-authored-by: Alan-Jowett <20480683+Alan-Jowett@users.noreply.github.com>

* Address code review feedback

- Update main.py default to match schema (300 instead of 500)
- Clarify test comment about fixture vs schema defaults
- Remove historical context from schema description

Co-authored-by: Alan-Jowett <20480683+Alan-Jowett@users.noreply.github.com>

* Refactor config attribute access to use getattr

- Replace hasattr checks with getattr for cleaner code
- Improves consistency with Python best practices

Co-authored-by: Alan-Jowett <20480683+Alan-Jowett@users.noreply.github.com>

* Align service default citation_text_max_length with schema

- Update SummarizationService default from 500 to 300
- Update test expectations to match new default
-... (continued)

8 of 8 new or added lines in 1 file covered. (100.0%)

5912 of 7757 relevant lines covered (76.22%)

0.85 hits per line