Early-Modern-OCR / emop-controller / 3
74%
master: 69%

LAST BUILD BRANCH: 3.1.0
DEFAULT BRANCH: master
Ran
Jobs 1
Files 37
Run time 1s
Badge
Embed ▾
README BADGES
x

If you need to use a raster PNG badge, change the '.svg' to '.png' in the link

Markdown

Textile

RDoc

HTML

Rst
3

push

travis-ci

treydock 
Add Globus Online suppport

The files needed to run jobs and produced by jobs are now staged in and out via Globus Online.  The interactions
with Globus Online are handled by 4 new subcommands added to emop.py.

* transfer status - Can be used to check status of a specific transfer or check status of ability to perform transfers
* transfer out - Transfer data from cluster to remote system
* transfer in - Transfer data to cluster from remote system
* transfer test - Run a simple test to verify transfers can be performed

The emop.py submit functionality has been updated to submit a transfer job that simply waits for necessary files to complete transfer.  The actual
OCR jobs will wait for the transfer job to finish via SLURM job dependencies.

When a OCR job completes, the files produced will be sent to Globus as a transfer before the results are uploaded to eMOP Dashboard.  This process is asynchronous.

The first time the Globus code is used, a prompt will ask for Globus Username/Password.  This is used to obtain GOAuth credentials which are good
for one year.

Config changes:

* The controller/input_path_prefix and controller/output_path_prefix now refer to location on cluster where files will be written for input and output
* Settings added for globus:
  * globus/cluster_endpoint - Globus endpoint of cluster
  * globus/remote_endpoint - Globus endpoint to pull data from and send data to when jobs finish
  * globus/username - optional - Username for Globus, will be prompted if not defined
  * globus/min_activation_time - optional - minimum time that must be left for endpoint activation
* Add scheduler/transfer_queue that defines the queue used for transfer jobs - which only wait for transfers to complete before allowing other jobs to continue
* Allow scheduler/min_job_runtime and scheduler/max_job_runtime to be defined as arithmetic expressions

Misc Changes:

* Add a few extra debug statements

EmopQuery:
* Renamed pending_pages to pending_pages_cou... (continued)

1230 of 1676 relevant lines covered (73.39%)

0.73 hits per line

Jobs
ID Job ID Ran Files Coverage
1 3.1 0
73.39
 Travis Job 3.1
Source Files on build 3
Detailed source file information is not available for this build.