pantsbuild / pants / 8472
0%
main: 93%

LAST BUILD BRANCH: nodejs-tool-bundled-lockfiles
DEFAULT BRANCH: main
Ran
Jobs 8
Files 0
Run time –
Badge
Embed ▾
README BADGES
x

If you need to use a raster PNG badge, change the '.svg' to '.png' in the link

Markdown

Textile

RDoc

HTML

Rst
8472

push

travis-ci

stuhood 
Create a complete product graph for the experimentation engine, and use it to validate inputs

This change adds input product requirements to Planners, which allows us to inspect the product graph to determine (recursively) whether an output product can be produced for a subject.

This allows us to kill the `required` parameter on `def promise`, because the method will only ever be called when a product can definitely be produced. It also means that the `def plan` method of a Planner will only ever be called when all of its declared inputs are present... meaning that no input validation is necessary in Planners.

- Declare input product types on Planners as a series of input product requirements in DNF form.
    - (We can likely add a small DSL to make this less confusing to declare)
- Use the declared Product requirements to determine which Planners are capable of producing an output product for a subject (recursively).
    - Treat Configuration and Sources as products for the purposes of the validation, so that a planner can declare a product requirement on Configuration
    - Add `NoPlanner` to produce Plans that lift Sources and Configuration defined directly on a subject into the product namespace
- Make thrift configuration types unique per language in order to avoid needing to implement field matching for product requirements
- Remove the `required` flag, and use the product graph to determine when to request products. Planners must now always produce some Plan when requested

----

Finally, this also allows us to explode for the absence of config (see [#2525](https://github.com/pantsbuild/pants/issues/2525)). A new example target and test demonstrate this:

    $ ./pants run src/python/pants/engine/exp/examples:viz -- \
      tests/python/pants_test/engine/exp/examples/scheduler_inputs \
      compile \
      src/thrift/codegen/unconfigured
    ..
    pants.engine.exp.scheduler.PartiallyConsumedInputsError: While attempting to produce Classpath for Target(address=src/thrift/codegen/unconfigured), some products could not be consumed:
      To consume Sources(u'.thrift'):
        ScroogePlanner needed (ScroogeScalaConfiguration OR ScroogeJavaConfiguration)
        ApacheThriftPlanner needed (ApacheThriftJavaConfiguration)

The error message indicates that the `Sources.of('.thrift')` product was not successfully used by any Planner, and lists what additional inputs would be necessary in order to use it.

Testing Done:
https://travis-ci.org/pantsbuild/pants/builds/99310887

Bugs closed: 2525, 2756

Reviewed at https://rbcommons.com/s/twitter/r/3245/
Jobs
ID Job ID Ran Files Coverage
1 8472.1 (CI_FLAGS="-cjlpn 'Various pants self checks'") 0
Travis Job 8472.1
4 8472.4 (CI_FLAGS="-fkmsrcjlp 'Python contrib tests'") 0
Travis Job 8472.4
5 8472.5 (CI_FLAGS="-fkmsrjlpn -i 0/6 'Python integration tests for pants - shard 1'") 0
Travis Job 8472.5
6 8472.6 (CI_FLAGS="-fkmsrjlpn -i 1/6 'Python integration tests for pants - shard 2'") 0
Travis Job 8472.6
7 8472.7 (CI_FLAGS="-fkmsrjlpn -i 2/6 'Python integration tests for pants - shard 3'") 0
Travis Job 8472.7
8 8472.8 (CI_FLAGS="-fkmsrjlpn -i 3/6 'Python integration tests for pants - shard 4'") 0
Travis Job 8472.8
9 8472.9 (CI_FLAGS="-fkmsrjlpn -i 4/6 'Python integration tests for pants - shard 5'") 0
Travis Job 8472.9
10 8472.10 (CI_FLAGS="-fkmsrjlpn -i 5/6 'Python integration tests for pants - shard 6'") 0
Travis Job 8472.10
Source Files on build 8472
Detailed source file information is not available for this build.