wikimedia / parsoid / 1884

85%

push

travis-ci

Fix newline handling in p-wrapper when # newlines > 2

wt -> html
* The old assumption was that newlines are handled in sets of 3.
  But, it looks like they are handled in pairs after discarding the
  first 2 newlines corresponding to the p-tags they trigger.

  But, the new logic (that emulates the PHP parser) cannot distinguish
  between some scenarios.

  "a\n\n\n=heading=" and "a\n\n\n\n=heading=".

  The normalized output is <p>a</p><p><br/></p><h1>heading</h1>
  in both cases. This limitation continues for every interval of 2
  newlines (5,6), (7,8), etc.

  This behavior is the same if you replace heading with lists,
  tables, block tags, and any other block tag that isn't a paragragh.

  Given the esoteric nature of this limitation, I am not going to
  do much more beyond documenting it.

html -> wt
* Updated the DOM handler for the <br> tag.
  - The tag is now content-transparent, i.e. it doesn't emit any
    content in the handler so that newline constraints accumulate
    from before to after and to following tags.

  - This new handler doesn't care for how many newlines there are
    in the HTML itself, it is more constraint-based compared to
    code before this.

* Added a "force" flag to newline handlers so that they can force
  the separator constraint handling code to preserve excess newlines
  across tags without losing them to the 'max' constraint of another
  handler that doesn't force it.

Tests
* Fixed couple of tests
* Fixed "Extra newlines followed by heading" to ensure it doesn't fail
  wt2wt because of the limitation mentioned above.
* The wt2wt failure for "Redirect followed by multiple newlines" is a
  result of the wt2html output that seems to lose one newline. The html
  before/after this patch is identical. The html2wt output generated
  in this patch for that html is better in this patch compared to master.
* I don't see anything obviously broken in selser blacklist -- I took
  a look at a couple of blacklisted changes.... (continued)

9045 of 11307 branches covered (79.99%)

14535 of 17095 relevant lines covered (85.02%)

99866.0 hits per line