blevesearch / bleve / 30267049682

53%

master: 52%

Build Type

Pull #2385

github

Committed by maneuvertomars

Commit Message

collector: early-stop (bounded scan) for score=none + Size=k

With Score="none" and a bounded Size, the request means "return any Size+From
matching documents" — nothing about the result depends on the matches beyond that.
So the collector can stop pulling from the searcher once it has them, instead of
draining the entire match set to count it.

On a 500k-doc corpus this is the difference between scanning every match and
scanning ten: BenchmarkEarlyStopTermTier1ScoreNone goes 18.15ms -> 8.4us and
EarlyStopDisjMin2ScoreNone 41.79ms -> 22.6us on the full branch.

Because the scan really does stop counting, Total becomes a lower bound, and saying
otherwise would silently mislead callers. This adds SearchResult.TotalRelation
("eq" | "gte") to report that, plus the merge rule that any constituent reporting
"gte" makes a merged Total "gte" too.

Gated on the result being independent of unseen documents — no facets (every match
must be counted into the buckets), no KNN (separate hit set), no SearchAfter (the
cursor depends on the full ordering), no nested rollup, no reverse execution, and
sort-by-score only (which degrades to arrival order under score="none"; a field
sort could have its top-k anywhere in the match set).

Two deviations from perf-gar-mod-fix-2's version of this work, both because that
branch's WAND machinery is not present here:

  - TotalRelation is introduced here rather than in 0f565526, which bundled the
    public API with the WANDPruned collector plumbing and the disjunction-searcher
    changes that set it. On this branch the bounded scan is the only producer of a
    lower-bound Total, so the API arrives with its first user and nothing about it
    is dead.
  - index_impl derives TotalRelation from coll.EarlyStopped() alone, not
    "coll.WANDPruned() || coll.EarlyStopped()".

The upstream commit shipped no tests. Added earlystop_test.go, because the
preconditions above are the whole correctness argument and getting one wrong is a
sil... (continued)

Pull Request Pull Request #2385: perf: top-N collector optimizations (perf-gar v17-only, 1/3)

Coverage Stats

81 of 108 new or added lines in 5 files covered. (75.0%)

3 existing lines in 2 files now uncovered.

19977 of 38050 relevant lines covered (52.5%)

0.59 hits per line