ICRAR / daliuge / 69
79%

DEFAULT BRANCH: master
Ran
Jobs 4
Files 60
Run time 1min
Badge
Embed ▾
README BADGES
x

If you need to use a raster PNG badge, change the '.svg' to '.png' in the link

Markdown

Textile

RDoc

HTML

Rst
69

push

travis-ci

rtobar 
Removing dependency on luigi

About 2 years ago we did some experiments with integrating luigi into
DALiuGE (called dfms back then...). This integration came in the form of
using the luigi scheduler to drive the execution of the graph inside a
session, instead of using the event-based driving. After those
experiments (which at least showed that it was possible) we didn't keep
using luigi at all, but we kept maintaining that bit of code, which not
only could be broken (it hasn't been thoroughly tested across multiple
nodes as far as I remember) but also relies on a *very* old version of
luigi. With so much time having gone by and nobody using this
integration (which could actually be broken, for all we know), the best
option is to then remove it.

One of the effects that this integration had was the incorporation of an
ExecutionMode enumeration which indicates how the execution of the graph
as a whole is being driven (is the drops themselves driving the
execution, or is there an external actor?). This mode is communicated to
each drop, which uses it to subscribe to events (or not) originating
from its inputs/producers. Supporting two execution modes might still be
useful in the future, so we are keeping it for now.

Two final nice effects that removing luigi has: 1) we finally don't need
to test/graphsRepository.py module anymore, and 2) the list of
dependencies that luigi itself imposed is now removed from daliuge
(including tornado, and some soft references we still had about it).

Signed-off-by: Rodrigo Tobar <rtobar@icrar.org>

5617 of 8418 relevant lines covered (66.73%)

2.67 hits per line

Jobs
ID Job ID Ran Files Coverage
1 69.1 60
66.68
 Travis Job 69.1
2 69.2 60
66.6
 Travis Job 69.2
3 69.3 60
66.7
 Travis Job 69.3
4 69.4 60
66.7
 Travis Job 69.4
Source Files on build 69
Coverage ∆ File Lines Relevant Covered Missed Hits/Line