rm-hull / dot-block

67%

main: 67%

LAST BUILD ON BRANCH feat/async-initial-load

branch: feat/async-initial-load

CHANGE BRANCH

x

Reset

Sync Branches

• feat/async-initial-load
• chore/correct-package-name
• chore/slog
• chore/update-deps
• chore/update-go-deps
• dependabot/docker/golang-1.26-alpine
• dependabot/github_actions/actions/cache-5
• dependabot/github_actions/actions/cache-6
• dependabot/github_actions/actions/checkout-6
• dependabot/github_actions/actions/checkout-7
• dependabot/github_actions/actions/download-artifact-8
• dependabot/github_actions/actions/setup-go-7
• dependabot/github_actions/actions/setup-node-7
• dependabot/github_actions/actions/upload-artifact-6
• dependabot/github_actions/docker/build-push-action-7
• dependabot/github_actions/docker/login-action-4
• dependabot/github_actions/docker/login-action-4.5.2
• dependabot/github_actions/docker/metadata-action-6
• dependabot/github_actions/docker/setup-buildx-action-4
• dependabot/github_actions/golangci/golangci-lint-action-9
• dependabot/github_actions/mikepenz/action-junit-report-6
• dependabot/go_modules/github.com/Depado/ginprom-1.8.3
• dependabot/go_modules/github.com/axiomhq/hyperloglog-0.2.6
• dependabot/go_modules/github.com/caddyserver/certmagic-0.25.2
• dependabot/go_modules/github.com/caddyserver/certmagic-0.25.3
• dependabot/go_modules/github.com/caddyserver/certmagic-0.25.4
• dependabot/go_modules/github.com/cockroachdb/errors-1.14.0
• dependabot/go_modules/github.com/getsentry/sentry-go-0.37.0
• dependabot/go_modules/github.com/getsentry/sentry-go-0.38.0
• dependabot/go_modules/github.com/getsentry/sentry-go-0.39.0
• dependabot/go_modules/github.com/getsentry/sentry-go-0.40.0
• dependabot/go_modules/github.com/getsentry/sentry-go-0.41.0
• dependabot/go_modules/github.com/getsentry/sentry-go-0.42.0
• dependabot/go_modules/github.com/getsentry/sentry-go-0.44.1
• dependabot/go_modules/github.com/getsentry/sentry-go-0.45.0
• dependabot/go_modules/github.com/getsentry/sentry-go-0.45.1
• dependabot/go_modules/github.com/getsentry/sentry-go-0.46.0
• dependabot/go_modules/github.com/getsentry/sentry-go-0.46.1
• dependabot/go_modules/github.com/getsentry/sentry-go-0.46.2
• dependabot/go_modules/github.com/getsentry/sentry-go-0.47.0
• dependabot/go_modules/github.com/getsentry/sentry-go-0.48.0
• dependabot/go_modules/github.com/getsentry/sentry-go/gin-0.37.0
• dependabot/go_modules/github.com/getsentry/sentry-go/gin-0.38.0
• dependabot/go_modules/github.com/getsentry/sentry-go/gin-0.39.0
• dependabot/go_modules/github.com/getsentry/sentry-go/gin-0.40.0
• dependabot/go_modules/github.com/getsentry/sentry-go/gin-0.41.0
• dependabot/go_modules/github.com/getsentry/sentry-go/gin-0.42.0
• dependabot/go_modules/github.com/getsentry/sentry-go/gin-0.44.1
• dependabot/go_modules/github.com/getsentry/sentry-go/gin-0.45.0
• dependabot/go_modules/github.com/getsentry/sentry-go/gin-0.45.1
• dependabot/go_modules/github.com/getsentry/sentry-go/gin-0.46.1
• dependabot/go_modules/github.com/getsentry/sentry-go/gin-0.46.2
• dependabot/go_modules/github.com/getsentry/sentry-go/gin-0.47.0
• dependabot/go_modules/github.com/getsentry/sentry-go/gin-0.48.0
• dependabot/go_modules/github.com/gin-contrib/pprof-1.5.4
• dependabot/go_modules/github.com/gin-gonic/gin-1.12.0
• dependabot/go_modules/github.com/miekg/dns-1.1.69
• dependabot/go_modules/github.com/miekg/dns-1.1.72
• dependabot/go_modules/github.com/pires/go-proxyproto-0.13.0
• dependabot/go_modules/github.com/pires/go-proxyproto-0.14.0
• dependabot/go_modules/github.com/pires/go-proxyproto-0.15.0
• dependabot/go_modules/github.com/prometheus/client_golang-1.24.0
• dependabot/go_modules/github.com/rm-hull/godx-0.0.10
• dependabot/go_modules/github.com/rm-hull/godx-0.0.7
• dependabot/go_modules/github.com/rm-hull/godx-0.0.8
• dependabot/go_modules/github.com/rm-hull/godx-0.2.1
• dependabot/go_modules/github.com/rm-hull/godx-0.2.2
• dependabot/go_modules/github.com/rm-hull/godx-0.2.3
• dependabot/go_modules/github.com/rm-hull/godx-0.2.4
• dependabot/go_modules/github.com/samber/slog-gin-1.19.1
• dependabot/go_modules/github.com/samber/slog-gin-1.20.0
• dependabot/go_modules/github.com/samber/slog-gin-1.20.1
• dependabot/go_modules/github.com/samber/slog-gin-1.21.0
• dependabot/go_modules/github.com/samber/slog-gin-1.21.1
• dependabot/go_modules/github.com/spf13/cobra-1.10.2
• dependabot/go_modules/github.com/tavsec/gin-healthcheck-1.7.12
• dependabot/go_modules/github.com/tavsec/gin-healthcheck-1.7.13
• dependabot/go_modules/github.com/tavsec/gin-healthcheck-1.7.14
• dependabot/go_modules/github.com/tavsec/gin-healthcheck-1.7.15
• dependabot/go_modules/github.com/tavsec/gin-healthcheck-1.7.16
• dependabot/go_modules/go.uber.org/zap-1.28.0
• dependabot/go_modules/golang.org/x/crypto-0.44.0
• dependabot/go_modules/golang.org/x/crypto-0.45.0
• dependabot/go_modules/golang.org/x/crypto-0.46.0
• dependabot/go_modules/golang.org/x/net-0.47.0
• dependabot/go_modules/golang.org/x/net-0.48.0
• dependabot/go_modules/golang.org/x/net-0.49.0
• dependabot/go_modules/golang.org/x/net-0.50.0
• dependabot/go_modules/golang.org/x/net-0.52.0
• dependabot/go_modules/golang.org/x/net-0.53.0
• dependabot/go_modules/golang.org/x/net-0.55.0
• dependabot/go_modules/golang.org/x/net-0.56.0
• dependabot/go_modules/golang.org/x/net-0.57.0
• dependabot/go_modules/golang.org/x/sync-0.20.0
• dependabot/go_modules/golang.org/x/sync-0.21.0
• dependabot/go_modules/golang.org/x/sync-0.22.0
• dependabot/npm_and_yarn/web/babel/core-8.0.1
• dependabot/npm_and_yarn/web/recharts-3.10.0
• dependabot/npm_and_yarn/web/recharts-3.10.1
• dependabot/npm_and_yarn/web/tanstack/react-query-5.101.3
• dependabot/npm_and_yarn/web/tanstack/react-router-1.170.18
• dependabot/npm_and_yarn/web/tanstack/router-cli-1.167.19
• dependabot/npm_and_yarn/web/types/node-26.1.1
• dependabot/npm_and_yarn/web/typescript-eslint-8.64.0
• dependabot/npm_and_yarn/web/typescript-eslint-8.65.0
• doc/browser-config
• feat/acme-DNS-01-challenge
• feat/admin-routes
• feat/admin-ui
• feat/asn-lookup
• feat/blocked-domains-cause
• feat/blocklist-schedules
• feat/cache-reaper
• feat/cache-ttl-floor
• feat/connection-pool
• feat/disable-blocklist
• feat/dispatcher-source
• feat/dns-listener
• feat/dns-metrics-skill
• feat/doh-server
• feat/ecs
• feat/ede-codes
• feat/geoblock
• feat/ip2location-cronjob
• feat/latency-based-routing
• feat/log-latency
• feat/log-noise-filter
• feat/mobileconfig
• feat/multiple-blocklists
• feat/navbar
• feat/newly-registered-domains
• feat/otel
• feat/proxy-protocol
• feat/reduce-info-level-logging
• feat/reload
• feat/reload-blocklist
• feat/rolling-average
• feat/semver-release
• feat/sentry-integration
• feat/sentry-logging
• feat/separate-blocklists
• feat/server-release
• feat/sse-events
• feat/swap-to-maxmind
• feat/top-blocked-domains
• feat/top-domains
• feat/ui-scaffolding
• feat/upstream-latency
• feat/upstream-ttl-metrics
• feat/user-avatar
• feat/yaml-config
• fix/blocklist-age-gauge
• fix/chart-realtime-scroll
• fix/data-races
• fix/event-sequence
• fix/hot-paths
• fix/improved-error-reporting
• fix/issue-225
• fix/issue-250
• fix/no-NXDOMAIN-errors
• fix/nodata-caching
• fix/read-and-dial-timeouts
• fix/remove-asn-from-event-stream
• fix/revert-to-UDP
• fix/round-robin-error-observability
• fix/round-robin-retry
• fix/stream-blocklists
• fix/use-spacesaver-error
• gemini/patch-issue-129-b1f017a
• gemini/patch-issue-131-4aba170
• gemini/patch-issue-145-7c41ad9
• main
• refactor/error-wrap
• refactor/handle-multiple-questions
• refactor/log-adapters
• refactor/p99-latency-spikes
• refactor/telemetry-goroutine
• refactor/use-jasengo
• test/benchmark
• test/integration-test
• test/pi-agent
• testing/integration

Build # 30497775305

Build Type

Pull #258

github

Committed by rm-hull

Commit Message

refactor: optimize blocklist loader with streaming and buffering

- Replaced line counting with a fast `io.Reader` buffer scan to estimate
 bloom filter size.
- Refactored `stream` to accept `io.Reader`, decoupling file I/O from
parsing logic.
- Reduced memory overhead by processing blocklists in a single pass.

```mermaid
sequenceDiagram
    participant B as BlockList.processFile
    participant F as File (io.Reader)
    participant C as countNewlines
    participant S as stream

    B->>C: Read file to estimate size
    C->>B: Return line count
    B->>B: Seek(0, io.SeekStart)
    B->>S: Stream file with scanner
    S->>B: Return parsed metadata
    B->>B: Apply Bloom Filter
```

Pull Request Pull Request #258: refactor: fetch blocklists asynchronously on startup

Coverage Stats

51 of 59 new or added lines in 3 files covered. (86.44%)

4 existing lines in 2 files now uncovered.

2141 of 3176 relevant lines covered (67.41%)

551.35 hits per line