DigitalTolk / ex / 40095295092

100%

push

github

search: raise ex_messages field limit; store: page ListMessages past 1 MB (#343)

* search: raise ex_messages total fields limit to 3000

Indices created before reactions were unmapped carry ~500 per-emoji
fields, leaving OpenSearch's default limit of 1000 with no headroom.
New and rebuilt ex_messages indices are created with a limit of 3000,
and EnsureIndices puts the same setting onto an existing index at boot,
so the limit holds however the index came to be.

* store: keep reading past DynamoDB's 1 MB page in ListMessages

A Query page ends at 1 MB as well as at its Limit. With large messages
the page came back short with a LastEvaluatedKey, hasMore read false,
and callers paging through a parent stopped early: the importer's
stats and search indexing saw only the newest few hundred messages of
channels with big bot posts. ListMessages now follows LastEvaluatedKey
until the page is full or the partition is exhausted.

11723 of 11723 branches covered (100.0%)

Branch coverage included in aggregate %.

25 of 25 new or added lines in 2 files covered. (100.0%)

50113 of 50114 relevant lines covered (100.0%)

1797.1 hits per line