Kikobeats / html-urls / 40592996506

99%

push

github

perf: dedupe links with a Set in a single pass (#106)

* perf: dedupe links with a Set in a single pass

Every link went through `concat(acc, link)`, which copies the accumulator,
and with `removeDuplicates` also through `findIndex` over it and then a final
`uniqBy` per attribute: O(n^2) in the number of links. Links are now pushed
into one array, and a Set holds the dedupe keys of links already kept. The
order and content of the output do not change: the first occurrence of each
key, in attribute order and then document order, as `uniqBy` kept it.
`lodash.get` in `getLink` and the per-link whitelist `concat` leave the hot
loop too.

A/B, base vs this commit, 5 alternating runs per side, median (ms):

| case                            | base  | change | delta  |
|---------------------------------|-------|--------|--------|
| 1000 links removeDuplicates off | 71.6  | 59.2   | -17.3% |
| 1000 links removeDuplicates on  | 68.8  | 55.1   | -19.9% |
| 8000 links removeDuplicates off | 112.4 | 59.0   | -47.5% |
| 8000 links removeDuplicates on  | 93.3  | 58.8   | -37.0% |

node v24.16.0, arm64, load avg 5.7-6.2.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* test: keep empty url attributes out of the result

An empty href was not in the suite, so a null check would publish the
base URL. matcher does not mutate the pattern list, so the copy is gone.

Co-authored-by: Cursor <cursoragent@cursor.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Cursor <cursoragent@cursor.com>

38 of 39 branches covered (97.44%)

Branch coverage included in aggregate %.

21 of 21 new or added lines in 1 file covered. (100.0%)

81 of 81 relevant lines covered (100.0%)

72.16 hits per line