• Home
  • Features
  • Pricing
  • Docs
  • Announcements
  • Sign In

cloudmarker / cloudmarker / 855
84%

Build:
DEFAULT BRANCH: master
Ran 21 May 2019 03:19AM UTC
Jobs 3
Files 41
Run time 20s
Badge
Embed ▾
README BADGES
x

If you need to use a raster PNG badge, change the '.svg' to '.png' in the link

Markdown

Textile

RDoc

HTML

Rst

pending completion
855

push

travis-ci-com

susam
Use processes and threads in Azure cloud plugins

Both `AzCloud` and `AzVM` now rely on multiprocessing and multithreading
to pull resources concurrently from Azure cloud in order to complete its
scan faster.

All of the multiprocessing and multithreading code involved is put in a
new `ioworkers` module. `AzCloud` and `AzVM` invoke `ioworkers.run()` to
launch multiple worker processes each of which launch multiple worker
threads. The callers, i.e., `AzCloud` and `AzVM`, need to provide this
`ioworkers.run()` function two callback functions:

  - An input function which when invoked yields tuples where each tuple
    defines a unit of work.
  - An output function which works on each tuple yielded by the input
    function, does something with it, and then yields outputs.

The output functions are run concurrently by `ioworkers` in multiple
worker threads within multiple worker processes. This two-level
approach, i.e., worker threads within worker processes, has been chosen
to maximize utilization of CPUs. This approaches provides a good balance
between the following two extremes:

  - Using a large number of processes only would consume too many IPC
    resources and increase the overhead of context switching for
    processes.
  - Using a large number of threads only does not scale well in Python
    due to global interpreter lock (GIL) which allows only one thread to
    execute at a time.

Lauching multiple workers sidesteps GIL and lets us utilize multiple
CPUs. Launching multiple threads within each process lets us scale even
further without incurring the overhead of more processes.

414 of 466 branches covered (88.84%)

Branch coverage included in aggregate %.

1946 of 2160 relevant lines covered (90.09%)

2.7 hits per line

Jobs
ID Job ID Ran Files Coverage
1 855.1 21 May 2019 03:19AM UTC 0
90.06
Travis Job 855.1
2 855.2 21 May 2019 03:19AM UTC 0
90.06
Travis Job 855.2
3 855.3 21 May 2019 03:19AM UTC 0
90.06
Travis Job 855.3
Source Files on build 855
Detailed source file information is not available for this build.
  • Back to Repo
  • Build #855
  • e6ebf046 on github
  • Prev Build on master (#845)
  • Next Build on master (#857)
STATUS · Troubleshooting · Open an Issue · Sales · Support · CAREERS · ENTERPRISE · START FREE · SCHEDULE DEMO
ANNOUNCEMENTS · TWITTER · TOS & SLA · Supported CI Services · What's a CI service? · Automated Testing

© 2025 Coveralls, Inc