Unleash / unleash-client-python / 5198128389

95%

master: 97%

push

github

Count metrics for non-existing features (#256)

## Description

This PR is the follow-up to #253 and makes the client track metrics for features that don't exist.

It achieves this by modifying the Unleash client's `is_enabled` and `get_variant` methods:
Whenever they are called with a feature that doesn't exist, we create a new feature with that name, `enabled = False`, no strategies, and no variants. The feature gets added to the clients list of features, and is then queried for its state (to track metrics; we know it'll be false).

The tests ensure that metrics are sent for these nonexistent features, but does not care about the implementation, so we should be able to change that as much as we want later.

### Discussion point

I've been back and forth a couple of times regarding how best to track these features. Because the `send_metrics` function already takes care of everything related to sending and resetting metrics, it'd be nice to leave that as it is. As such, it'd be nice to track nonexistent features as part of the regular list of features. 

The current implementation uses a boolean field to indicate whether a feature exists only to track metrics or whether it should be evaluated properly. I considered using a subclass to tell them apart instead, but because the loader (`loader.py`) simply **updates** the objects instead of replacing them wholesale (probably to preserve metrics), then that caused some problems.

## Commits

* Test: add initial tests for new metrics behavior

* test: make tests run properly

* Feat: register metrics for features that don't exist

* Chore: run black

* Test: check that metrics are tracked before initialization too

* Test: remove test that didn't make sense

* feat: use a `only_for_metrics` field for metrics-only toggles

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

---------

Co-authored-by: pre-commit-ci[bot] ... (continued)

16 of 16 new or added lines in 3 files covered. (100.0%)

831 of 856 relevant lines covered (97.08%)

0.97 hits per line