HEPData / hepdata / 889

83%

push

travis-ci

Update to elasticsearch 7 (#194)

* First steps towards migration to elasticsearch 7

 * Updated elasticsearch and invenio-search pip modules
 * Updated mapping to work with ES7
 * Updated indexing to work with new mapping/ES7
 * Started to update queries (incomplete/untested)

Unit tests currently pass but e2e do not because parsing search results is not yet complete.
Should update search_tests to ensure queries are actually valid in ES.

* Further fixes for main search and processing results

 * Replaced the doc_type search with a filter (need to think about doing this more consistently across searches)
 * _primary is no longer an option for search, so instead use custom value 'primary' so queryless search is always on same shard
 * Updated results processing to reflect new ES hits structure
 * Fixed parent child query builder

Multiple searches and filters are not yet working.

* First stab at using elasticsearch_dsl for main search

Instead of rewriting query builder to work with ES7, it made sense to use elasticsearch DSL to make life easier.
Still needs a bit of refactoring - QueryBuilder probably has unused methods now.

* Updated author indexing/search

 * Removed doc type from authors index
 * Renamed config variable to reflect that it's now just an index, not a type
 * Updated search to use dsl query

* Reinstated fuzzy search

* Tidied up query builder, replacing more searches with dsl

Removed unused methods from QueryBuilder. Fixed search test (though it needs to have extra tests added).

* Further tidy-ups and tests

 * Removed more unused code from es_config
 * Removed the add_aggregations method from QueryBuilder as it was just a wrapper for the method in es_config
 * Added more search tests
 * Include es_config in test converage

* Added tests for search and search_authors

* Remove print statement from test

* Use default version of elasticsearch in Travis build

* Revert "Use default ... (continued)

243 of 243 new or added lines in 10 files covered. (100.0%)

2093 of 2687 relevant lines covered (77.89%)

0.78 hits per line