cybertec-postgresql / pgwatch / 40330496377

88%

Build Type

push

github

Committed by web-flow

Commit Message

[-] fix `PostgresWriter.flush()` hang on unmarshalable measurements (#1593)

* fix `PostgresWriter.flush()` hang on unmarshalable measurements

Port of #1591 to master.

A measurement that fails to marshal (e.g. a `NaN` value) aborted
`CopyFrom`
mid-envelope, leaving `copyFromMeasurements` mid-metric with a sticky
`Err()`.
The rest of the failed metric's rows went to the next metric's table,
and when
the failed metric was the last one `MetricName()` returned an empty
identifier,
so `flush()` looped forever.

`copyFromMeasurements.SkipMetric()` now drops the rest of the failed
metric
(or the next one if the copy failed before reading it) and clears the
error;
`flush()` calls it after a failed `CopyFrom`.

Master has no pgxmock CopyFrom support, so the flush regression tests
use a
small fake sink that mimics pgx `CopyFrom` instead of the mock-based
ones.

* remove `PostgresWriter.lastError` channel

The channel was unbuffered and both the send in `flush()` and the receive
in `Write()` were non-blocking, so no error was ever delivered. The only
consumer logged the error, which `flush()` already does. Partition
creation errors are now logged in `flush()` instead.

* fix SkipMetric() dropping the next metric after a server-side COPY error

pgx drains the CopyFromSource concurrently with sending COPY data, so when
the server reports an error mid-copy (e.g. a 23514 check violation) the
iterator has usually already reached the metric boundary. The old
SkipMetric() then saw no current metric and skipped the following one,
losing its measurements.

SkipMetric now takes the identifier that was passed to CopyFrom and only
advances while the upcoming envelopes still carry that metric name.

The copy-sink tests now assert row counts per metric instead of comparing
jsoniter output, whose map key order is random and made the test flaky.

Add TestFlush_ServerSideCopyErrorKeepsNextMetric.

Coverage Stats

19 of 21 new or added lines in 1 file covered. (90.48%)

10 existing lines in 2 files now uncovered.

5479 of 6233 relevant lines covered (87.9%)

1.0 hits per line