supabase / supavisor / 15590033421

65%

Build Type

push

github

Committed by web-flow

Commit Message

feat: JSON output format for log messages (#653)

Output logs in line JSON format (JSON object per line) in format that is
as similar to the format that is expected by the Logflare as possible.

The idea there is to log to the Vector which will then take care of
logging to file(s) and to dispatch the logs to Logflare.

It provides 3 new ENV variables that we need to set in deployment:
- `SUPAVISOR_LOG_FILE_PATH=<path>` - path to file where we want to store
our logs. These logs are automatically rotated and up to 5 files (each
max 8 MiB, though I think we can make it configurable or larger) are
stored
- `SUPAVISOR_LOG_FORMAT=json` to output logs in Logflare JSON format
- `SUPAVISOR_ACCESS_LOG_FILE_PATH=<path>` - path to access log for
`fail2ban` processing

That is current approach. I also started using `SUPAVISOR_` prefix to
all (new) environment variables that are used to configure Supavisor, to
easily scan for variables that we are defining. Current uses of Logflare
will work as is, that behaviour is not removed, but the Supavisor will
log to stdout (or file if specified) by default now. I wonder whether we
should always log to stdout and log to file in addition to logging to
stdout. That is open and that is almost one line change if needed.

This module
https://github.com/supabase/supavisor/pull/653/files#diff-fcb4c5de2167e1c43638c936caa9684b4
would probably need to be exported to some new Logflare library (cc
@Ziinc), but for simplicity it is currently in Supavisor. I haven't yet
written proper test for it.

I also have added new metadata field for logs in `ClientHandler` -
`state`. It contains the connection state at the time of log. It is
mostly useful for logging access logs (as these are heavily limited)
where I log only events in `exchange` state (as it is where all
connection failures happen). I also set logger metadata much earlier in
the connection setup to be able to see all that info ASAP.

This code also u... (continued)

Run Details

47 of 75 new or added lines in 4 files covered. (62.67%)

9 existing lines in 2 files now uncovered.

1036 of 2058 relevant lines covered (50.34%)

132.81 hits per line