3
50%
master: 50%

Ran 07 Jul 2020 07:41PM UTC

Files 614

Run time 1min

Badge

Embed ▾

Committed 07 Jul 2020 07:35PM UTC coverage: 48.426% (+0.05%) from 48.375%

Job # CODACY_PROJECT_TOKEN=[secure]

Build Type

push

travis-ci

Committed by

web-flow

Commit Message

Use standard text normalization - fixes #2898 (#2900)

* Use standard text normalization - fixes #2898

Fixes #2898. Fixes #409. Refs #650

Replaces homegrown ISO Latin-1 only character subsitition
with standard Java Normalize to NFD, followed by diacritic
removal and a few custom character expansions/replacements.

* Fix Mac build

* Improve compatibility with previous code

One intentional change is folding O with stroke to
oe instead of o.

- Use more powerful NFKD instead of NFD
- strip punctuation after decomposition since it can generate
  new punctuation
- Add compatibility test for old asciify() method
- Add some graphically similar characters to substitution table

* Add oe character/ligature & more long S forms

* More tests for ligatures and Latin Extended

* Add Latin-1 Supplement tests

Coverage Stats

12196 of 25185 relevant lines covered (48.43%)

0.48 hits per line

OpenRefine / OpenRefine / 4153 / 3
50%
master: 50%

README BADGES
x

Markdown

Textile

RDoc

HTML

Rst

Source Files on job 4153.3 (CODACY_PROJECT_TOKEN=[secure])

OpenRefine / OpenRefine / 4153 / 3 50% master: 50%

README BADGES x

Markdown

Textile

RDoc

HTML

Rst

Source Files on job 4153.3 (CODACY_PROJECT_TOKEN=[secure])

OpenRefine / OpenRefine / 4153 / 3
50%
master: 50%

README BADGES
x