berserkhmdvhb / agentic-scraper / 17350471794

87%

main: 88%

Build Type

push

github

Committed by berserkhmdvhb

Commit Message

refactor(agents.llm_fixed): improve safety, logging, and test coverage

Main code changes:
- Moved `Settings` and `ScrapeRequest` imports into TYPE_CHECKING blocks
  to satisfy ruff TC001 and reduce runtime dependencies.
- Wrapped OpenAI `AsyncOpenAI` import in a safe try/except ImportError with a
  minimal stub (`Chat`, `Completions`) for monkeypatch-based testing.
- Unified `AsyncOpenAI` binding to avoid mypy redefinition errors and support
  test stubbing consistently.
- Replaced inline log/raise strings with constants from config/messages
  (empty response, validation failure, unexpected errors, success).
- Cleaned up stub classes: used CapWords naming (`Chat`, `Completions`) and
  explicit parameters instead of `*args`/`**kwargs`, resolving ruff ANN401/N801.
- Raised exceptions using constants/variables instead of inline literals,
  fixing ruff EM101/TRY003.
- Structured `_extract_impl` flow with logging and return inside `else` block,
  resolving ruff TRY300.

Test changes:
- Added `tests/backend/scraper/agents/test_llm_fixed.py` with async unit tests
  covering success, screenshot enabled, empty content, and error handling cases.
- Implemented fake AsyncOpenAI client, chat, completions, and response objects
  to simulate OpenAI SDK behavior deterministically.
- Patched `capture_optional_screenshot` safely (non-temp path) to verify
  screenshot integration without triggering filesystem warnings.
- Used caplog/capsys assertions consistently to check logger output when
  responses are empty or errors are raised.
- Treated injected `_BoomError` as `OpenAIErrorT` to test exception handling
  branch in `extract_structured_data`.

Run Details

14 of 14 new or added lines in 1 file covered. (100.0%)

1 existing line in 1 file now uncovered.

2686 of 4540 relevant lines covered (59.16%)

4.73 hits per line