tarantool / tarantool / 8774

91%

master: 88%

push

travis-ci

vinyl: use LSN from WAL instead of index_opts->lsn in vylog

An index can be dropped and then recreated with the same space/index id.
To discriminate between different incarnations of the same index during
recovery, we use index LSN stored in index options, as it is supposed to
be unique. However, the uniqueness property doesn't always hold:

 - If two indexes are created from different fibers, they might receive
   the same LSN.

 - If an index is created by inserting a record into _index system space
   directly, without using the public API, as it is the case in case of
   logical backup, its LSN might conflict with the LSN of an existing
   index or a previous incarnation of the same index stored in vylog.

These exceptions can result in unrecoverable errors during local
recovery, like this one:

  F> can't initialize storage: Invalid VYLOG file: Duplicate index id 3

Besides, storing LSN in index options is ugly, because LSN isn't a
user-defined option - it's a part of the implementation.

To fix this issues, let's use the LSN passed to Index::commitCreate,
i.e. the actual LSN received by the row that created the index. There's
one problem though: snapshot rows don't store LSNs. However, it doesn't
mean we can't find the index in vylog corresponding to a snapshot row:
we just need to look up the index by space_id/index_id instead of LSN
and then compare the snapshot LSN with the LSN of the last index
incarnation stored in vylog - if the latter turns out to be less, then
we need to load the index, otherwise the index is going to be dropped
and we need to load a dummy index. For more details, see the comment to
vy_recovery_load_index().

Another issue that needs a clarification is backward compatibility. The
thing is the LSN written to the index options lags behind the actual LSN
assigned to the row that created the index by 1. So to preserve backward
compatibility, we use LSN from index options for legacy indexes that
have it, while for indexes cr... (continued)

130 of 130 new or added lines in 3 files covered. (100.0%)

29863 of 34577 relevant lines covered (86.37%)

1117368.83 hits per line