supabase / supavisor / 23064604059

82%

Build Type

push

github

Committed by web-flow

Commit Message

perf: implement peep metrics storage/exporter for high cardinality metrics (#875)

Implements several optimizations:

- Tag maps deduplicated into a shared table with integer IDs:
per-scheduler keys are {int, int} instead of {int, map}, eliminating
N_schedulers copies of every tag combination
  - Formatted Prometheus label strings cached in ETS by tag ID
- Distribution bucket boundaries precomputed once per metric definition,
not once per {metric, tags} series
- Bucket values extracted as raw integer lists instead of
`%{boundary_string => count}` maps, which eliminates String.to_float
parsing, map allocation, and sorting during export

With a sample benchmark:
```
Name                                              ips        average  deviation         median         99th %
custom (PeepStorage + PrometheusCached)          0.38         2.65 s    ±33.80%         2.61 s         3.54 s
upstream (Striped + Prometheus)                0.0536        18.64 s     ±0.00%        18.64 s        18.64 s

Comparison:
custom (PeepStorage + PrometheusCached)          0.38
upstream (Striped + Prometheus)                0.0536 - 7.02x slower +15.99 s

Memory usage statistics:

Name                                       Memory usage
custom (PeepStorage + PrometheusCached)         1.98 GB
upstream (Striped + Prometheus)                 7.08 GB - 3.58x memory usage +5.10 GB
```

Run Details

111 of 238 new or added lines in 5 files covered. (46.64%)

1 existing line in 1 file now uncovered.

2437 of 3072 relevant lines covered (79.33%)

56254.68 hits per line