OpenRefine / OpenRefine / 23474365562

51%

push

github

feat(clustering): Replace Vicino with Apache Commons Text Levenshtein (#7721). Fixes #7726

* feat(clustering): Replace Vicino with Apache Commons Text Levenshtein

Switch the kNN clustering Levenshtein distance from the unmaintained
Vicino library to Apache Commons Text, which is already a transitive
dependency of the project.

The new ApacheLevenshteinDistance wrapper implements SimilarityDistance
and delegates to org.apache.commons.text.similarity.LevenshteinDistance.
Benchmarks show ~10x faster end-to-end kNN clustering compared to the 
MIT Simile Vicino implementation.

Changes:
- Add ApacheLevenshteinDistance wrapper class in modules/core
- register it as the default for the 'levenshtein' distance in controller.js
- tests to add knn Levenshtein tests (more tests needed for other knn stuff)
- fixed latent bug in existing tests
- Add commons-text dependency to modules/core/pom.xml
- Add JMH benchmarks: ApacheLevenshteinBenchmark and
  KNNLevenshteinClusteringBenchmark with benchmark README

5709 of 12792 branches covered (44.63%)

Branch coverage included in aggregate %.

14566 of 27153 relevant lines covered (53.64%)

2.7 hits per line