rm-hull / news-barge / 37159892085

93%

push

github

perf(scraper): reuse shared Playwright BrowserContext

Reuse a single `BrowserContext` for all Playwright fetches (listings and
articles) rather than creating a new context per article. This avoids
the
expensive ~2.8s per-context setup cost on every fetch (issue #49).

```mermaid
sequenceDiagram
    participant Main as main_async
    participant Browser as Playwright Browser
    participant Context as Shared BrowserContext
    participant Fetcher as fetch_html_playwright

    Main->>Browser: launch(headless=True)
    Main->>Browser: new_context(...)
    Browser-->>Main: shared_context
    Note over Main,Context: Created once per run (avoids ~2.8s per-fetch
 overhead)

    loop For each article/listing
        Main->>Fetcher: url, shared_context, semaphore
        Fetcher->>Context: new_page()
        Context-->>Fetcher: page (cheap)
        Fetcher->>Fetcher: goto(url), extract HTML
        Fetcher->>Context: page.close()
    end

    Main->>Context: close()
    Main->>Browser: close()
```

Fixes #49

12 of 13 new or added lines in 3 files covered. (92.31%)

733 of 800 relevant lines covered (91.63%)

0.92 hits per line