mushorg / snare / 382

90%

push

travis-ci

Clone and serve headers (#241)

* Add server headers when cloning/serving

Cloner is now storing server headers in meta.json (except some ignored
ones). When serving a request, headers are overwritten if necessary.

* Remove default server header

Rely on the cloned one, if existing. If not, it can always be
overwritten by using the --server-header cmd line switch.

* Update list of ignored headers

The following are the default aiohttp headers:

Content-Type: text/plain; charset=utf-8
Content-Length: 12
Date: Wed, 26 Feb 2020 16:28:20 GMT
Server: Python/3.7 aiohttp/3.4.4

We don't want any overlap with those, except for "Server" and
"Content-Type". Also we should avoid messing with HTTP/1.1 "Connection",
as well as "Date" and a few others related to caching.

This list will most likely need updating in the future.

* Fix minor formatting/PEP-8 issues

* Remove content_type and use Content-Type header

* Update snare middleware to use new headers

* Fix line length issues

pycodestyle complaining...

* Change middleware to be backward-compatible

Old meta doesn't contain headers, so default to no headers.

* Change Tanner handler to be backward-compatible

Again, if no headers we should default to empty list.

* Improve handling of {'Content-Type': None}

* Fix check_meta_file to allow new headers

Co-authored-by: Evgeniia <vremendm@gmail.com>

53 of 53 new or added lines in 5 files covered. (100.0%)

364 of 567 relevant lines covered (64.2%)

1.28 hits per line