supabase / supavisor
81%

DEFAULT BRANCH: main
Repo Added
Files 146
Badge
Embed ▾
README BADGES
x

If you need to use a raster PNG badge, change the '.svg' to '.png' in the link

Markdown

Textile

RDoc

HTML

Rst

LAST BUILD ON BRANCH main
branch: SELECT
CHANGE BRANCH
x

coverage: 81.32% (+0.1%) from 81.179%
37803781336

push

github

web-flow 
Clear the secrets for validation on a pool shutdown (#1208)

Invalidate client authentication globally on a pool shutdown

It only invalidates the `:secrets_for_validation` if there's no sibling
pool for the same tenant and DB user; i.e. the cache is NOT cleared if
there's another pool for the same tenant and DB user but different
mode/db/upstream_tls/search_path settings.

The PR also adds a missing logs.

Caveats:
* on each pool shutdown RCP multicall is made once or twice: to retrieve
pools' PIDs and then optionally to invalidate the cache globally

EDIT 1:

New approach has been proposed: instead of checking and invalidating
globally via `:erpc`, we take advantage of the `:syn`'s `:tenants`
process group to which we join every pool keyed by `{tenant, user}`.
When a pool shuts down, we check the group for members — if there're
some we keep the cache, otherwise, we invalidate. The invalidation is
run locally.

The process still involves inter-cluster comms on the shutdown as we
check the remote pools liveness via `erpc` but the impact is way lower
as we already have a list of pools and we don't have to check each node.

The downside of invalidating locally is that it might fail (e.g. due to
the erpc-base liveness check) at one node while succeed on the other.
But we'd be facing the same risk with the previous approach.

EDIT 2:

We slightly changed the approach by using a separate `tenant_pools`
scope as the `Supavisor.Terminator` unregisters a pool being drained and
shutdown early from the `:tenants` scpoe to allow for the pool to be
created on another node.



EDIT 3 (race-freedom of the last-member check):

We dropped the `:erpc` liveness check. It isn't needed, and it broke the
node-down case.

`:syn` calls `on_process_left/5` serially in each node's scope process,
after removing the leaving pid from the group, and every node gets a
callback for every leaving pool. `invalidate_local/2` only clears that
node's cache, so each node decides on its o... (continued)

29 of 30 new or added lines in 3 files covered. (96.67%)

4 existing lines in 4 files now uncovered.

3191 of 3924 relevant lines covered (81.32%)

48564.55 hits per line

Relevant lines Covered
3924 RELEVANT LINES 3191 COVERED LINES
48564.55 HITS PER LINE
Source Files on main

Recent builds

Builds Branch Commit Type Ran Committer Via Coverage
37803781336 main Clear the secrets for validation on a pool shutdown (#1208) Invalidate client authentication globally on a pool shutdown It only invalidates the `:secrets_for_validation` if there's no sibling pool for the same tenant and DB user; i.e. the cache... push web-flow github
81.32
37801668836 fix/transaction-pipelining-edge-cases Merge 3e13f87d1 into 25e5b0d3b Pull #1207 web-flow github
81.65
37761956415 szymonmentel/pooler-694-clear-the-secrets_for_validation-on-a-pool-shutdown Merge d095178cd into 25e5b0d3b Pull #1208 web-flow github
81.35
37630327040 main Merge pull request #1233 from supabase/feat/pooler-822-cgroup-memory-metrics feat: add cgroup v2 memory metrics push web-flow github
81.18
37627671385 feat/pooler-822-cgroup-memory-metrics Merge 54efc1bd7 into 065280b2b Pull #1233 web-flow github
81.23
37488509561 feat/pooler-822-cgroup-memory-metrics Merge c4ef41d50 into 065280b2b Pull #1233 web-flow github
81.13
37480509142 main chore: remove unused proxy listener on PROXY_PORT (#1232) The :pg_proxy listener (PROXY_PORT, default 5412) accepted clients in :proxy mode, but nothing connects to it. Cross-node proxying uses the pool's internal shard listener from get_pool_ran... push web-flow github
81.08
37478957978 remove-dead-listener Merge b7b726759 into 381e4756a Pull #1232 web-flow github
81.11
37477917102 main fix: client disconnecting before the allow_list check no longer crashes the handler (#1231) Fixes POOLER-393 push web-flow github
81.0
37475986159 main feat: add client error counter tagged by error code and mode (#1230) This adds a counter of client connection errors, [:supavisor, :client, :errors], tagged with the error code and the pooling mode. It is emitted from `ClientHandler.Error.termina... push web-flow github
82.16
See All Builds (1327)