NaturalHistoryMuseum / splitgill / #106

95%

main: 95%

push

coveralls-python

feat: combine the parsed and data root fields into one structure in Elasticsearch

BREAKING CHANGE: combine the parsed and data root fields into one structure in Elasticsearch

This changes how the record's data is stored in elasticsearch so that the root fields data and parsed are combined into one structure. There are several advantages to this:

  - we can store the whole doc in ES, not just the data field, allowing highlight searches and enabling use of the reindex api if needed
  - the actual parsed value of each field is made available in the _source for use
  - probably reduced storage expense (to be tested!)
  - easier to understand doc model in ES

This is a breaking change because it is no longer possible to get the record's data out of elasticsearch without processing the doc you get back.
This is done through the added `splitgill.search.rebuild_data` function.

16 of 16 new or added lines in 3 files covered. (100.0%)

1183 of 1248 relevant lines covered (94.79%)

0.95 hits per line