pantsbuild / pants / 9352
0%
main: 93%

LAST BUILD BRANCH: dk-fix-straggler-logging
DEFAULT BRANCH: main
Ran
Jobs 7
Files 0
Run time –
Badge
Embed ▾
README BADGES
x

If you need to use a raster PNG badge, change the '.svg' to '.png' in the link

Markdown

Textile

RDoc

HTML

Rst
9352

push

travis-ci

kwlzn 
Initial round of pantsd + new engine + watchman integration.

- Implement a new SchedulerService for managing the online scheduler instance.
- Revive FSEventService and friends for Watchman integration.
- Avoid resetting Subsystem options to avoid uninitialized subsystem errors when attempting to use the WatchmanLauncher subsystem in pantsd - this gets stomped on run over run anyhow.
- Make use of FSEventService and SchedulerService optionable - defaulting to off.
- Implement a test target in exp to launch pantsd with a copy of the scheduler and listen for file events (currently all this does is log the events - graph invalidation coming soon!).
- Extend the `kill-pantsd` goal to also shutdown Watchman when so configured.

Testing Done:
CI is away @ https://travis-ci.org/pantsbuild/pants/builds/113015980

-------------------------------------------------------------------------------

while running this in a separate window:

    $ tail -F .pants.d/pantsd/pantsd.log .pants.d/watchman/watchman.log

spin up an unconnected pantsd instance using the test target w/ options to enable fs-event-detection:

    [illuminati pants (kwlzn/pantsd/engine_integration)]$ ./pants run src/python/pants/engine/exp/legacy:pantsd -q -- src/python/pants/pantsd:: -ldebug --pantsd-fs-event-detection
    *** pantsd launched ***
    DEBUG] acquiring lock: <OwnerPrintingPIDLockFile: u'/Users/kwilson/dev/pants/.pantsd.startup' -- u'/Users/kwilson/dev/pants/.pantsd.startup'>
    DEBUG] launching pantsd
    DEBUG] purging metadata directory: /Users/kwilson/dev/pants/.pids/pantsd
    DEBUG] released lock: <OwnerPrintingPIDLockFile: u'/Users/kwilson/dev/pants/.pantsd.startup' -- u'/Users/kwilson/dev/pants/.pantsd.startup'>
    DEBUG] pantsd is running at pid 78563

see log output in the `tail` window:

    D0301 17:45:57.109353 78563 pants_daemon.py:121] logging initialized
    I0301 17:45:57.109931 78563 pants_daemon.py:170] pantsd starting, log level is DEBUG
    I0301 17:45:57.111968 78563 pants_daemon.py:144] starting service <pants.pantsd.service.scheduler_service.SchedulerService object at 0x106902510>
    I0301 17:45:57.112404 78563 pants_daemon.py:144] starting service <pants.pantsd.service.pailgun_service.PailgunService object at 0x1069021d0>
    I0301 17:45:57.112715 78563 pailgun_service.py:51] starting pailgun server on port 56898
    I0301 17:45:57.112966 78563 pants_daemon.py:144] starting service <pants.pantsd.service.fs_event_service.FSEventService object at 0x106902450>
    I0301 17:45:57.116261 78563 watchman_launcher.py:56] watchman is running, pid=70857 socket=/Users/kwilson/dev/pants/.pants.d/watchman/watchman.sock
    D0301 17:46:02.122070 78563 watchman.py:147] watchman command_list is: [[u'subscribe', '/Users/kwilson/dev/pants', u'all_files', {'fields': [u'name'], 'expression': [u'allof', [u'type', u'f'], [u'not', [u'dirname', u'dist', [u'depth', u'eq', 0]]], [u'not', [u'match', u'.*', u'wholename']], [u'not', [u'match', u'*.pyc']]]}]]
    I0301 17:46:02.265883 78563 watchman.py:153] confirmed watchman subscription: {'subscribe': 'all_files', 'version': '3.1.0', 'clock': 'c:1456869416:70857:1:4433'}
    I0301 17:46:02.267790 78563 scheduler_service.py:44] enqueuing 4968 changes for subscription all_files
    D0301 17:46:02.292135 78563 scheduler_service.py:75] processing 4968 files for subscription all_files (first_event=True)
    D0301 17:46:03.268616 78563 fs_event_service.py:143] callback ID 1 for all_files succeeded

save various src files in my editor, which incur new log events:

    I0301 17:49:30.558384 78563 scheduler_service.py:44] enqueuing 1 changes for subscription all_files
    D0301 17:49:30.575396 78563 scheduler_service.py:75] processing 1 files for subscription all_files (first_event=False)
    D0301 17:49:30.575889 78563 scheduler_service.py:56] file src/python/pants/pantsd/service/fs_event_service.py changed!
    D0301 17:49:31.562772 78563 fs_event_service.py:143] callback ID 2 for all_files succeeded
    I0301 17:49:33.095844 78563 scheduler_service.py:44] enqueuing 1 changes for subscription all_files
    D0301 17:49:33.112721 78563 scheduler_service.py:75] processing 1 files for subscription all_files (first_event=False)
    D0301 17:49:33.113214 78563 scheduler_service.py:56] file src/python/pants/pantsd/service/pants_daemon_launcher.py changed!
    D0301 17:49:34.098309 78563 fs_event_service.py:143] callback ID 3 for all_files succeeded

shut it all down with:

    [illuminati pants (kwlzn/pantsd/engine_integration)]$ ./pants kill-pantsd -ldebug --pantsd-fs-event-detection
    18:03:39 00:00 [main]
                   (To run a reporting server: ./pants server)
    18:03:39 00:00   [setup]
    18:03:39 00:00     [parse]
                   Executing tasks in goals: kill-pantsd
    18:03:39 00:00   [kill-pantsd]
    18:03:39 00:00     [kill-pantsd]DEBUG] terminating pantsd
    DEBUG] sending signal 15 to pid 82783
    DEBUG] successfully terminated pid 82783
    DEBUG] purging metadata directory: /Users/kwilson/dev/pants/.pids/pantsd
    DEBUG] terminating watchman
    DEBUG] sending signal 15 to pid 82696
    DEBUG] successfully terminated pid 82696
    DEBUG] purging metadata directory: /Users/kwilson/dev/pants/.pids/watchman

    18:03:39 00:00   [complete]
                   SUCCESS

Bugs closed: 2989

Reviewed at https://rbcommons.com/s/twitter/r/3524/
Jobs
ID Job ID Ran Files Coverage
1 9352.1 (CI_FLAGS="-cjlpn 'Various pants self checks'") 0
Travis Job 9352.1
4 9352.4 (CI_FLAGS="-fkmsrcjlp 'Python contrib tests'") 0
Travis Job 9352.4
6 9352.6 (CI_FLAGS="-fkmsrjlpn -i 1/6 'Python integration tests for pants - shard 2'") 0
Travis Job 9352.6
7 9352.7 (CI_FLAGS="-fkmsrjlpn -i 2/6 'Python integration tests for pants - shard 3'") 0
Travis Job 9352.7
8 9352.8 (CI_FLAGS="-fkmsrjlpn -i 3/6 'Python integration tests for pants - shard 4'") 0
Travis Job 9352.8
9 9352.9 (CI_FLAGS="-fkmsrjlpn -i 4/6 'Python integration tests for pants - shard 5'") 0
Travis Job 9352.9
10 9352.10 (CI_FLAGS="-fkmsrjlpn -i 5/6 'Python integration tests for pants - shard 6'") 0
Travis Job 9352.10
Source Files on build 9352
Detailed source file information is not available for this build.