airbnb / streamalert / 4519

95%

push

travis-ci

[apps] De-duplicate GSuite Events (contribution from @stoggi) (#804)

* [apps] Added optional context to the state parameter stored in ssm

* Created new key "context" stored in the state parameter in ssm
* Save the unique qualifiers from the most recent events in gsuite
* Added unit tests

* [apps] De-duplicate events using the app context stored in ssm

* Save at most 50 event ids in the context. Limit of 4096 characters in ssm.
* Correctly format dates with a decimal component.
* De-duplicate events based on last seen event ids.
* Update unit tests to output unique timestamps per generated log.
* Added unit tests.

* [apps] gsuite store one copy of last_event_ids in context

* Refactored to only store one copy of last_event_ids because _gather_logs is called many times per lambda execution.
* Increased the _MAX_EVENT_IDS since only one copy of last_event_ids is stored.
* Fixed typos.

30 of 30 new or added lines in 3 files covered. (100.0%)

4241 of 4378 relevant lines covered (96.87%)

0.97 hits per line