tarantool / crud
89%
master: 89%

LAST BUILD BRANCH: add-s3-coverage-upload
DEFAULT BRANCH: master
Repo Added
Files 68
Badge
Embed ▾
README BADGES
x

If you need to use a raster PNG badge, change the '.svg' to '.png' in the link

Markdown

Textile

RDoc

HTML

Rst

LAST BUILD ON BRANCH rebuilt2
branch: rebuilt2
CHANGE BRANCH
x

coverage: 88.556%. First build
33089793479

push

github

Satbek 
fix: retry a call only after a recovery action

call_with_retry_and_recovery() retried every failed call once, no
matter what the error was, and for a single call it followed the
destination reported in a bucket ref error. Two problems came out of
that.

First, the destination redirect was unsafe. During a rebalancing the
source storage rejects a request with a bucket ref error and reports
the transfer destination; the router then sent the retry straight to
that destination, which may not have received the bucket yet. In fast
mode the request was applied there against an empty space, so the same
key could end up on two storages at once.

Second, a call that failed for an unrelated reason -- a storage error,
a timeout -- was retried blindly, which could silently double the
effective request timeout the user had asked for.

Rework the recovery into an explicit step: recover_from_err() performs
an action for the errors it knows and only then reports whether a
retry makes sense.

  * MISSING_MASTER -- the master is not discovered yet, so it is
    discovered explicitly and the same replicaset is retried.
  * NON_MASTER -- the cached master is stale, but the bucket did not
    move, so the master is updated and the same replicaset is retried.
  * WRONG_BUCKET, BUCKET_IS_LOCKED, TRANSFER_IS_IN_PROGRESS -- the
    route is stale, so the bucket route cache is reset and the retry
    target is re-discovered via vshard:route() instead of being taken
    from the error. If the error covers more than one bucket, there is
    no single target to retry on, so the error is returned as is.
  * Anything else is returned to the caller without a retry.

The retry no longer extends the request timeout: a deadline is taken
before the first attempt, and the retry gets only the time left before
it (request_timeout is clamped to the same value).

Map calls do not go through the recovery at all: they are async, so
storage errors arrive later in the future payload and there is... (continued)

37 of 43 new or added lines in 1 file covered. (86.05%)

5440 of 6143 relevant lines covered (88.56%)

11967.1 hits per line

Relevant lines Covered
6143 RELEVANT LINES 5440 COVERED LINES
11967.1 HITS PER LINE
Source Files on rebuilt2

Recent builds

Builds Branch Commit Type Ran Committer Via Coverage
33089793479 rebuilt2 fix: retry a call only after a recovery action call_with_retry_and_recovery() retried every failed call once, no matter what the error was, and for a single call it followed the destination reported in a bucket ref error. Two problems came out of... push Satbek github
88.56
See All Builds (1521)