pylipp / financeager / 89

100%

push

travis-ci

Simplify test framework

- remove test.suites and suite() functions in test modules
- fix/disable newly discovered tests

Explanation:
Previously the test framework worked in a slightly obscure way. Running
the tests was invoked by
    python setup.py test            # (I)
which ran 134 tests. Digging into the setuptools source code reveals
that this performs unittest-discovery, however it first looks for
non-test modules in the test/ directory. Here it finds the suites module
whose content is executed on the fly and which terminates the current
interpreter session by sys.exit without returning to the setup() frame.

Before knowing the activity of (I), it was expected that
    python -m unittest discover     # (II)
behaves identically. Yet it runs 140 tests of which 4 fail.

This is due to
a) (I) runs test.suites which calls the suite() function of the
individual test modules. This function should contain mappings of all
TestCase classes and their test methods of the current module. Since
this function has to be manually maintained, it was missed to add six
functions (which explains the observed difference in test numbers):
    - test_cli.CliFlaskTestCase.test_add_invalid_entry
    - test_cli.CliFlaskTestCase.test_invalid_request
    - test_cli.CliFlaskTestCase.test_parser_error
    - test_listing.PrettifyListingsTestCase.test_prettify_no_elements
    - test_period.TinyDbPeriodStandardEntryTestCase.test_add_entry_default_date
    - test_period.TinyDbPeriodRecurrentEntryTestCase.test_update_recurrent_entry_incorrectly

b) (II) runs unittest discovery which finds and runs every test method
of every test case in every test module. The order of executed tests is
alphabetical, and hence different from (I) which uses hardcoded lists of
test modules and methods. Since some test runs influence each other
(uh-oh), some now fail (they are fixed, see test_cli).

282 of 282 branches covered (100.0%)

Branch coverage included in aggregate %.

873 of 873 relevant lines covered (100.0%)

1.0 hits per line