rm-hull / dot-block / 29107864483

62%

Build Type

push

github

Committed by web-flow

Commit Message

fix: optimize memory usage for blocklist loading (#182)

* feat: optimize memory usage for blocklist loading

Refactored blocklist loading to stream files from disk, significantly
reducing memory overhead by calculating exact Bloom filter size before
allocation.

```mermaid
sequenceDiagram
    participant App
    participant Downloader
    participant FileSystem
    participant BloomFilter

    App->>Downloader: Download blocklists
    Downloader-->>FileSystem: Store temp files
    App->>FileSystem: Count entries in files
    FileSystem-->>App: Total count
    App->>BloomFilter: Initialize with size (totalCount)
    App->>FileSystem: Stream files
    FileSystem-->>BloomFilter: Add entries to filter
    App->>FileSystem: Cleanup temp files
```

* refactor: simplify blocklist initialization

Remove conditional check for nil items in `NewBlockList` and allow
`Load` to handle empty or nil slices gracefully to prevent potential
runtime panics during initialization. Added regression tests to ensure
`IsBlocked` remains stable with empty input.

* refactor: ensure blocklist temp files are cleaned up

Move the slice initialization and the `defer` cleanup logic before the
download loop. This ensures that any successfully downloaded temporary
files are tracked and removed even if a subsequent download fails.

```mermaid
sequenceDiagram
    participant B as BlocklistUpdater
    participant D as Downloader
    B->>B: Init files slice
    B->>B: Setup defer cleanup
    loop Each URL
        B->>D: Download file
        D-->>B: Return path/isTemp
        B->>B: Append to files
    end
    B->>B: Process counts
    B->>B: Trigger deferred cleanup
```

* refactor: improve blocklist parsing logic

- Trim whitespace from scanned lines to handle inconsistent formatting.
- Explicitly ignore double-octothorpe comment lines during scan.

* fix: prevent panic when initializing bloom filter

Added a safety check to ensure the bloom filter capacity is at least 1
when totalCoun... (continued)

Coverage Stats

65 of 107 new or added lines in 4 files covered. (60.75%)

4 existing lines in 2 files now uncovered.

1478 of 2364 relevant lines covered (62.52%)

731.8 hits per line