wikimedia / parsoid / 1804

85%

push

travis-ci

<a href="https://github.com/wikimedia/parsoid/commit/8966025e5">Bug fix in Async Token Transform Manager

* In some cases, the async ttm transformTokens loop was dropping
  result tokens on the floor. However, this required a bunch of
  rare events to come together:

  - a token transformer completes synchronously
  - the synchronous call returns more than 1 token
  - the synchronous call sets aborts further processing on the
    returned tokens by setting a rank higher than the aysnc phase&#39;s
    end rank.

* This was extremely rare because:
  - till recently, 2 in 3 tokens would go through the attribute
    transformer which would pretty much immediately bail into the
    async phase.

  - other token transformers in the async phase would also require
    some async work (ex: template, extension, image, dom-fragment
    handling)

  - those that completed synchronously (ex: urllinks, extlinks) and
    returned multiple tokens (ex: urllinks, extlinks) didn&#39;t set
    a high rank on the result tokens.

  - only one scenario (loop detection in templates) set the rank
    of result tokens to phaseEndRank (!)

* Fixes to the attribute transformer (coming in later patches because
  I rebased this above those patches) greatly reduced the # of tokens
  that go through the async phase. Since template expansion uses the
  same codepath, there is no longer a async-break opportunity for the
  template loop detection scenario. It completes synchronously and
  trips on the bug and was failing the &#39;Template infinite loop&#39; bug
  since the error message tokens were being dropped on the floor.

  That failure has now been fixed with this patch.

* Separately, it is worth examining our token tranformers if more
  code paths could set the phaseEndRank rank on their results to
  eliminate wasteful processing.

Change-Id: I</a><a class="double-link" href="https://github.com/wikimedia/parsoid/commit/f36d69a10ebac6aaef18aeeb3b5e... (continued)

8913 of 10952 branches covered (81.38%)

14220 of 16521 relevant lines covered (86.07%)

102879.83 hits per line