PulpQE / pulp-smash / 41 / 4
63%
master: 63%

DEFAULT BRANCH: master
Ran
Files 1
Run time 0s
Badge
Embed ▾
README BADGES
x

If you need to use a raster PNG badge, change the '.svg' to '.png' in the link

Markdown

Textile

RDoc

HTML

Rst
coverage: 89.011%. Remained the same
41.4

push

travis-ci

Ichimonji10 
Split tests into sections

Fix #11:

> All tests currently reside in `pulp_smash/tests/`. This works, but it doesn't
> scale well. How can a user find the kind of test they're looking for?

To major options were discussed: splitting tests based primarily on the
transport mechanism or splitting based primarily on the plugin targeted. The
former leads to a layout like this:

    pulp_smash/tests/
    ├── api_v2
    │   ├── platform
    │   │   └── …
    │   ├── iso
    │   │   └── …
    │   └── rpm
    │       └── …
    └── cli
        ├── platform
        │   └── …
        ├── iso
        │   └── …
        └── rpm
            └── …

The latter leads to a layout like this:

    pulp_smash/tests/
    ├── platform
    │   ├── api_v2
    │   │   └── …
    │   └── cli
    │       └── …
    ├── iso
    │   ├── api_v2
    │   │   └── …
    │   └── cli
    │       └── …
    └── rpm
        ├── api_v2
        │   └── …
        └── cli
            └── …

Implement the latter layout for the following reasons:

* This layout bears a rough resemblance to Pulp Automation's layout, which is
  split in to `general_tests`, `consumer_agent_tests`, `regression_tests` and
  `nodes`. Similarity should ease the process of porting tests, and if the
  layout works over there, why change things?
* A user may not have a full set of test systems available, and in that case,
  the ability to easily run tests that target only specific plug-ins may be
  desirable. For example, if a user only has a Pulp system, they may wish to run
  only the "platform" tests.
* This approach may be the most useful to developers. For example, if a
  developer is fiddling with the Puppet plug-in, this approach makes it easy to
  run some or all of the tests for that plug-in.

There are some downsides to this approach:

* This layout may pose problems if, say, API v2 is phased out in favor of API v3
  and a user wishes to only run the v3 tests.
* Selecting tests based on their location in a filesystem is a highly limited
  approach. It assumes that all users can be served by selecting tests based on
  the transport mechanism used (API v1, API v2, CLI, etc), the plugin used (RPM,
  ISO, etc) or so on. A better approach would be to tag tests and then provide
  an easy way to select tests that match a specific criteria, such as with
  nose2's "attrib" plugin.

This commit clearly does not cater to all use cases, but it should be a
reasonable first stab at letting users select only certain tests. If this
approach proves insufficient, fixing #3 may be a good step in improving the
situation.

Test results:

    $ python -m unittest2 discover pulp_smash.tests
    ....F...
    …
    ----------------------------------------------------------------------
    Ran 8 tests in 4.954s

    FAILED (failures=1)

The one test failure is expected.

81 of 91 relevant lines covered (89.01%)

0.89 hits per line

Source Files on job 41.4