skeema / skeema / 6830565347

92%

Build Type

push

github

Committed by evanelias

Commit Message

diff/push: improve --verify perf by changing order of operations

When running `skeema diff` or `skeema push`, by default the --verify option
is enabled, which confirms the correctness of all generated ALTER TABLE
statements by running them on empty shadow tables in a workspace. This commit
improves the performance of the verification step by changing the order of
operations to filter out unnecessary verifications.

This change provides a major performance benefit primarily for users with
many auto_increment tables, and/or many range-partitioned tables.

At a high level, the previous approach ordered its steps this way:
1. Generate full diff between actual state (from live DB) and desired
   state (from filesystem)
2. For all alterations to existing tables in the diff, verify correctness
   if --verify is enabled (as it is by default). Halt if verification fails.
3. Develop concrete execution plan which ignores unnecessary differences from
   things which typically aren't tracked in version control (mainly
   auto_increment counters or range partition rotation, but also things like
   FK name drift from use of pt-osc)
4. Output plan (if `skeema diff`) or run plan (if `skeema push`)

This commit conceptually swaps steps 2 and 3, so that unnecessary differences
are filtered out earlier and don't need to be handled in the verification
step.

The actual implementation of this is deceptively complex, due to the nuances
of some edge-cases where a given table modification must be split into
multiple separate ALTER TABLEs, and run in a particular order.

This commit also refactors some bookkeeping for how unsupported and/or unsafe
changes are tracked.

Run Details

93 of 111 new or added lines in 5 files covered. (83.78%)

1 existing line in 1 file now uncovered.

9532 of 10202 relevant lines covered (93.43%)

1.12 hits per line