m-lab / etl / 7103

67%

main: 67%

push

travis-ci-com

Add Gardener v2 parser for switch datatype (#1044)

* Initial implementation of the new switch datatype parser.

* Fix error handling in parser/switch.go

* Fix error handling.

* Fill in summary field from samples

* Use reflection and add unit tests.

* Do not return an error on unknown metrics.

This means we just fill the fields we want in the summary, and ignore
any extra data.

* Update update-schema/update.go to refer to the new SwitchRow struct.

* Update switch datatype parser and schema to use int64

* Update dataset/table names in update.go

* Add ID to SwitchRow and Broadcast-related fields to the summary.

* The "a" field should be lowercase.

* Update partition field for the switch table.

* Use DATE instead of TIMESTAMP and add CollectionTime to the summary.

Also, it's easier to use machine and site if they are not hostnames.

* Update comments.

* Update regex to account for v1 M-Lab hostnames.

* Add accurate estimation of SwitchRow's size.

* Update go.mod/go.sum after running go mod tidy.

* Remove commented line

* Update comment.

* Update ID format and variable name.

* Update comments.

* Add condition to only remove octets.local.tx/rx when needed.

* Update switch parser tests.

* s/valField/deltaField/

* Update tests.

* Updated tests to verify switch discards.

* Update tests.

* Update comment.

* Sort output by timestamp in the switch parser

223 of 223 new or added lines in 3 files covered. (100.0%)

3920 of 6093 relevant lines covered (64.34%)

0.7 hits per line