aiidateam / aiida_core / 4156

74%

push

travis-ci

Make `update`, `submit` and `retrieve` transport tasks idem potent (#2111)

The transport tasks for job processes can be interrupted violently after having
completed the bulk of their task but before being able to create the state
transition and therewith creating a checkpoint. This means that when the daemon
is restarted the same task is executed. This could lead to various side effects.
Here we make sure that for the update, submit and retrieve transport tasks can
safely be executed twice. When the task finishes, if applicable, the calculation
state will be set. In the beginning of the task we check if the state is already
set, which indicates that the task was already ran, and we skip it.

This still leaves a small window for the task being executed twice, since there
is sometime between the task returning successfully and the state being set, but
this should minimize the time window.

Note that the submit task is not truely idem potent, as when it will be executed
a second time (after the first time successfully submitted a job, but didn't get
the chance to set the state), the job will be submitted once more to the scheduler.
However, there is really no way to guarantee that his happens as there will always
be a time window between the job being submitted to the scheduler and for us to
get the chance to reflect that in the node, in which the daemon process can be
abruptly killed.

22 of 22 new or added lines in 2 files covered. (100.0%)

22166 of 36609 relevant lines covered (60.55%)

1.71 hits per line