aiidateam / aiida_core / 6542
74%

DEFAULT BRANCH: develop
Ran
Jobs 4
Files 392
Run time 10min
Badge
Embed ▾
README BADGES
x

If you need to use a raster PNG badge, change the '.svg' to '.png' in the link

Markdown

Textile

RDoc

HTML

Rst
6542

push

travis-ci

web-flow 
Respect the minimum polling interval for scheduler updates (#3096)

The `aiida.engine.processes.calcjobs.manager.JobsList` container was
introduced for two purposes, related to managing running calculation
jobs in high-throughput mode:

 * It bundles the scheduler update calls for all active calculation jobs
   for given authentication info (the comination of a computer and user)
 * It ensures that consecutive scheduler update calls are separated at
   least by a time that is given by the `get_minimum_job_poll_interval`
   as defined by the computer of the authentication info.

However, the final requirement was not being respected. The problem was
twofold. The internal attribute `_last_updated` that records the
timestamp of the last update was not being updated after the scheduler
was queried for a status update.

The use of the `RefObjectStore` to create and delete these `JobsLists`
instances, however, was an even bigger problem. The reference store
would delete the `JobsLists` instance as soon as no-one held a reference
to it any more, since they are created through the
`request_job_info_request` context manager of the `JobManager` that each
runner has. As soon as no requests were active, the object was deleted
and with it the record of the last time the scheduler was queried. The
next time a request comes in a new `JobsList` would be created that
straight away call the scheduler as it has no recollection the last time
it was called for the given authinfo, if at all. This would result in
the minimum poll interval essentially never being respected.

Fundamentally, the problem lies in the fact that the data that needs to
be persistent, the "last updated" timestamp, was being stored in the
container `JobsList` that by implementation was made non-persistent.
The solution is to simply make the `JobsList` instances live for as long
as the python interpreter is alive. Each daemon runner creates a single
`JobManager` instanc... (continued)

23963 of 32375 relevant lines covered (74.02%)

2.74 hits per line

Jobs
ID Job ID Ran Files Coverage
5 6542.5 (TEST_AIIDA_BACKEND=django TEST_TYPE="tests") 0
67.85
 Travis Job 6542.5
6 6542.6 (TEST_AIIDA_BACKEND=django TEST_TYPE="tests") 0
67.8
 Travis Job 6542.6
7 6542.7 (TEST_AIIDA_BACKEND=sqlalchemy TEST_TYPE="tests") 0
69.42
 Travis Job 6542.7
8 6542.8 (TEST_AIIDA_BACKEND=sqlalchemy TEST_TYPE="tests") 0
69.37
 Travis Job 6542.8
Source Files on build 6542
Detailed source file information is not available for this build.