rero / sonar / 32015241076

95%

master: 95%

push

github

fix(suggestions): suggest values, not records

The completion endpoint matched *records*, so a hit carried every value of
its field and Python had to re-extract and re-filter them. Worse, it only
looked at the first 50 scored documents.

A value can only be matched on its own if it is indexed on its own, which
only a dumper can do — `copy_to` copies values, not structure.

* Add `SuggestionsDumperExt`, which denormalizes the suggestable values
  into a `nested` field, one sub-document per value. It takes the field
  paths as argument, so any resource can reuse it: documents wire it into
  `IndexerDumper`, projects into their `SearchDumper` extensions.
* Read the values from a nested aggregation: no document cap, ordering by
  frequency, one request for all the requested fields, and `operator: and`
  applied per value.
* Align the analyzers on rero-ils: `autocomplete` / `search_autocomplete`
  pair, `edge_ngram` up to 30. The remaining `.suggest`
  sub-fields follow, which also lets the `$ref` pickers find `Zürich` with
  `zurich`.
* Query the real field paths again and drop the orphan `.suggest`
  sub-fields.

:warning: v1.13.0 migration will require a full `documents`, `projects`, `collections`,
`subdivisions` and `users` (6dfd183c) reindex!

Co-Authored-by: Pascal Repond <pascal.repond@rero.ch>
Co-Authored-by: Johnny Mariéthoz <johnny.mariethoz@rero.ch>

39 of 39 new or added lines in 6 files covered. (100.0%)

7626 of 8046 relevant lines covered (94.78%)

0.95 hits per line