wikimedia / restbase / 931

84%

Pull #220

travis-ci

Don't store a new render if the content did not change

This patch processes a no-cache update request for a parsoid content revision
by letting Parsoid parse the page as usual. However, instead of blindly
storing the new render, we now compare the content with the previous render's
content & don't store a new revision if nothing changed. The hope is that this
will reduce our storage requirements from template updates significantly,
assuming that many template-triggered re-renders don't actually change the
content. A statsd counter is set up to track the effectiveness of this
optimization.

Since Parsoid currently emits non-deterministic about attributes (see link to
task), we need to perform some slightly hacky HTML normalization to avoid
marking all HTML pages as differing. This hack should be removed once the
Parsoid bug is fixed.

I also used the opportunity to clean up a few things in the parsoid module:

- Move the method setup to the mod constructor. This makes those methods
  available to the module itself without going through routing. Also removed
  the factory function in favor of simple currying / bind.

- Create a new uuid directly in generateAndSave instead of passing it in from
  all users.

The tests have been updated to use a simpler and thus more reliable set of
pages to test changing vs. unchanging re-renders. A timestamp parser function
is used to force updates in one of the pages.

Bug: https://phabricator.wikimedia.org/T93779

54 of 54 new or added lines in 3 files covered. (100.0%)

786 of 910 relevant lines covered (86.37%)

124.0 hits per line