rm-hull / dot-block

63%

main: 62%

LAST BUILD ON BRANCH fix/stream-blocklists

branch: fix/stream-blocklists

CHANGE BRANCH

x

Reset

Sync Branches

• fix/stream-blocklists
• chore/correct-package-name
• chore/slog
• chore/update-deps
• chore/update-go-deps
• dependabot/docker/golang-1.26-alpine
• dependabot/github_actions/actions/cache-5
• dependabot/github_actions/actions/cache-6
• dependabot/github_actions/actions/checkout-6
• dependabot/github_actions/actions/checkout-7
• dependabot/github_actions/actions/upload-artifact-6
• dependabot/github_actions/docker/build-push-action-7
• dependabot/github_actions/docker/login-action-4
• dependabot/github_actions/docker/metadata-action-6
• dependabot/github_actions/docker/setup-buildx-action-4
• dependabot/github_actions/golangci/golangci-lint-action-9
• dependabot/github_actions/mikepenz/action-junit-report-6
• dependabot/go_modules/github.com/Depado/ginprom-1.8.3
• dependabot/go_modules/github.com/axiomhq/hyperloglog-0.2.6
• dependabot/go_modules/github.com/caddyserver/certmagic-0.25.2
• dependabot/go_modules/github.com/caddyserver/certmagic-0.25.3
• dependabot/go_modules/github.com/caddyserver/certmagic-0.25.4
• dependabot/go_modules/github.com/cockroachdb/errors-1.14.0
• dependabot/go_modules/github.com/getsentry/sentry-go-0.37.0
• dependabot/go_modules/github.com/getsentry/sentry-go-0.38.0
• dependabot/go_modules/github.com/getsentry/sentry-go-0.39.0
• dependabot/go_modules/github.com/getsentry/sentry-go-0.40.0
• dependabot/go_modules/github.com/getsentry/sentry-go-0.41.0
• dependabot/go_modules/github.com/getsentry/sentry-go-0.42.0
• dependabot/go_modules/github.com/getsentry/sentry-go-0.44.1
• dependabot/go_modules/github.com/getsentry/sentry-go-0.45.0
• dependabot/go_modules/github.com/getsentry/sentry-go-0.45.1
• dependabot/go_modules/github.com/getsentry/sentry-go-0.46.0
• dependabot/go_modules/github.com/getsentry/sentry-go-0.46.1
• dependabot/go_modules/github.com/getsentry/sentry-go-0.46.2
• dependabot/go_modules/github.com/getsentry/sentry-go-0.47.0
• dependabot/go_modules/github.com/getsentry/sentry-go/gin-0.37.0
• dependabot/go_modules/github.com/getsentry/sentry-go/gin-0.38.0
• dependabot/go_modules/github.com/getsentry/sentry-go/gin-0.39.0
• dependabot/go_modules/github.com/getsentry/sentry-go/gin-0.40.0
• dependabot/go_modules/github.com/getsentry/sentry-go/gin-0.41.0
• dependabot/go_modules/github.com/getsentry/sentry-go/gin-0.42.0
• dependabot/go_modules/github.com/getsentry/sentry-go/gin-0.44.1
• dependabot/go_modules/github.com/getsentry/sentry-go/gin-0.45.0
• dependabot/go_modules/github.com/getsentry/sentry-go/gin-0.45.1
• dependabot/go_modules/github.com/getsentry/sentry-go/gin-0.46.1
• dependabot/go_modules/github.com/getsentry/sentry-go/gin-0.46.2
• dependabot/go_modules/github.com/getsentry/sentry-go/gin-0.47.0
• dependabot/go_modules/github.com/gin-contrib/pprof-1.5.4
• dependabot/go_modules/github.com/gin-gonic/gin-1.12.0
• dependabot/go_modules/github.com/miekg/dns-1.1.69
• dependabot/go_modules/github.com/miekg/dns-1.1.72
• dependabot/go_modules/github.com/pires/go-proxyproto-0.13.0
• dependabot/go_modules/github.com/pires/go-proxyproto-0.14.0
• dependabot/go_modules/github.com/rm-hull/godx-0.0.10
• dependabot/go_modules/github.com/rm-hull/godx-0.0.7
• dependabot/go_modules/github.com/rm-hull/godx-0.0.8
• dependabot/go_modules/github.com/rm-hull/godx-0.2.1
• dependabot/go_modules/github.com/rm-hull/godx-0.2.2
• dependabot/go_modules/github.com/rm-hull/godx-0.2.3
• dependabot/go_modules/github.com/rm-hull/godx-0.2.4
• dependabot/go_modules/github.com/samber/slog-gin-1.19.1
• dependabot/go_modules/github.com/samber/slog-gin-1.20.0
• dependabot/go_modules/github.com/samber/slog-gin-1.20.1
• dependabot/go_modules/github.com/samber/slog-gin-1.21.0
• dependabot/go_modules/github.com/samber/slog-gin-1.21.1
• dependabot/go_modules/github.com/spf13/cobra-1.10.2
• dependabot/go_modules/github.com/tavsec/gin-healthcheck-1.7.12
• dependabot/go_modules/github.com/tavsec/gin-healthcheck-1.7.13
• dependabot/go_modules/github.com/tavsec/gin-healthcheck-1.7.14
• dependabot/go_modules/github.com/tavsec/gin-healthcheck-1.7.15
• dependabot/go_modules/github.com/tavsec/gin-healthcheck-1.7.16
• dependabot/go_modules/go.uber.org/zap-1.28.0
• dependabot/go_modules/golang.org/x/crypto-0.44.0
• dependabot/go_modules/golang.org/x/crypto-0.45.0
• dependabot/go_modules/golang.org/x/crypto-0.46.0
• dependabot/go_modules/golang.org/x/net-0.47.0
• dependabot/go_modules/golang.org/x/net-0.48.0
• dependabot/go_modules/golang.org/x/net-0.49.0
• dependabot/go_modules/golang.org/x/net-0.50.0
• dependabot/go_modules/golang.org/x/net-0.52.0
• dependabot/go_modules/golang.org/x/net-0.53.0
• dependabot/go_modules/golang.org/x/net-0.55.0
• dependabot/go_modules/golang.org/x/net-0.56.0
• dependabot/go_modules/golang.org/x/net-0.57.0
• dependabot/go_modules/golang.org/x/sync-0.20.0
• dependabot/go_modules/golang.org/x/sync-0.21.0
• dependabot/go_modules/golang.org/x/sync-0.22.0
• doc/browser-config
• feat/acme-DNS-01-challenge
• feat/cache-reaper
• feat/cache-ttl-floor
• feat/connection-pool
• feat/dispatcher-source
• feat/dns-listener
• feat/dns-metrics-skill
• feat/doh-server
• feat/ecs
• feat/geoblock
• feat/ip2location-cronjob
• feat/latency-based-routing
• feat/log-latency
• feat/log-noise-filter
• feat/mobileconfig
• feat/multiple-blocklists
• feat/newly-registered-domains
• feat/otel
• feat/proxy-protocol
• feat/reduce-info-level-logging
• feat/reload
• feat/reload-blocklist
• feat/semver-release
• feat/sentry-integration
• feat/sentry-logging
• feat/server-release
• feat/swap-to-maxmind
• feat/top-blocked-domains
• feat/top-domains
• feat/upstream-latency
• feat/upstream-ttl-metrics
• fix/blocklist-age-gauge
• fix/data-races
• fix/hot-paths
• fix/improved-error-reporting
• fix/no-NXDOMAIN-errors
• fix/read-and-dial-timeouts
• fix/revert-to-UDP
• fix/round-robin-error-observability
• fix/round-robin-retry
• fix/use-spacesaver-error
• gemini/patch-issue-129-b1f017a
• gemini/patch-issue-131-4aba170
• gemini/patch-issue-145-7c41ad9
• main
• refactor/error-wrap
• refactor/handle-multiple-questions
• refactor/log-adapters
• refactor/p99-latency-spikes
• refactor/telemetry-goroutine
• test/integration-test
• test/pi-agent
• testing/integration

Build # 29051970230

Build Type

Pull #182

github

Committed by rm-hull

Commit Message

feat: optimize memory usage for blocklist loading

Refactored blocklist loading to stream files from disk, significantly
reducing memory overhead by calculating exact Bloom filter size before
allocation.

```mermaid
sequenceDiagram
    participant App
    participant Downloader
    participant FileSystem
    participant BloomFilter

    App->>Downloader: Download blocklists
    Downloader-->>FileSystem: Store temp files
    App->>FileSystem: Count entries in files
    FileSystem-->>App: Total count
    App->>BloomFilter: Initialize with size (totalCount)
    App->>FileSystem: Stream files
    FileSystem-->>BloomFilter: Add entries to filter
    App->>FileSystem: Cleanup temp files
```

Pull Request Pull Request #182: feat: optimize memory usage for blocklist loading

Coverage Stats

67 of 109 new or added lines in 4 files covered. (61.47%)

4 existing lines in 2 files now uncovered.

1478 of 2362 relevant lines covered (62.57%)

741.23 hits per line