wikimedia / parsoid / 2384

85%

push

travis-ci

<a href="https://github.com/wikimedia/parsoid/commit/31363b4ec">Tokenizer efficiency improvements

Miscellaneous changes which came out of reviewing pegTokenizer.pegjs in
the course of porting it to PHP.

* Don&#39;t use location(), ever. All it does is convert the string offsets
  in peg$savedPos and peg$currPos to line and column numbers, which we
  don&#39;t need. Maybe startOffset() and endOffset() callers should
  directly access these variables.
* In tplarg_or_template_guarded, lang_variant_text_no_semi, and
  lang_variant_text_no_semi_or_arrow, remove a wrapper which was rendered
  unnecessary by the rule parameter change.
* In almost every case where .join(&#39;&#39;) was used, use the text operator
  ($) instead. It is much faster to take an input substring than to
  construct an array and join it.
* In lang_variant_nowiki and lang_variant_text, remove unnecessary
  offset captures. The generated code captures the start and end
  positions by default.
* Also use the $ operator to improve efficiency in
  generic_attribute_name, table_attribute_name
* Introduce optional_spaces to replace space*, since there is an
  overhead associated with entering a rule.

Change-Id: I</a><a class="double-link" href="https://github.com/wikimedia/parsoid/commit/e16807dc0">e16807dc0</a>

9548 of 12008 branches covered (79.51%)

14858 of 17582 relevant lines covered (84.51%)

86053.33 hits per line