skeema / skeema / 16208444803

93%

main: 92%

Build Type

push

github

Committed by evanelias

Commit Message

perf improvements: workspace cleanup, table size checks, inserts=seed

This commit implements performance improvements for several related code
paths:

* With workspace=temp-schema, if this Skeema process has created the
  temp-schema (as opposed to reusing and cleaning an existing schema), there
  is no need to confirm tables are empty prior to cleaning up the workspace.
  This change improves performance when network latency is high and the object
  count in the workspace is also high.

* With workspace=docker, during cleanup there is no need to drop tables
  individually prior to running DROP DATABASE, since the workspace is in an
  isolated Skeema-managed Docker container with no "real" query workload.
  Ordinarily we drop tables first to avoid having a single DROP DATABASE that
  can hold locks for an excessively long time, but that isn't necessary here.
  This is a small performance improvement when the table count is extremely
  high.

* In the rare situations where an existing workspace schema is re-used, table
  emptiness safety checks now use only ~1/4 as many connections and round-
  trips as before. Instead of using up to 15 concurrent connections each
  checking one table at a time, we now use a UNION query which checks 4 tables
  at once, times 4 concurrent connections.

* In Skeema Premium, whenever inserts=seeds checks table emptiness, the above
  optimization for using 1/4 as many conns and round-trips is also applied.

* When checking table sizes (safe-below-size, alter-wrapper-min-size, and the
  {SIZE} variable in alter-wrapper or ddl-wrapper), the number of round-trips
  has been cut in half. This is achieved by combining the queries for "does it
  have any rows" and "how large is it" into a single query.

Run Details

82 of 91 new or added lines in 4 files covered. (90.11%)

3 existing lines in 2 files now uncovered.

10631 of 11416 relevant lines covered (93.12%)

1.11 hits per line