MITLibraries / workflow

78%

master: 75%

LAST BUILD ON BRANCH worker-autoscale

branch: worker-autoscale

CHANGE BRANCH

x

Reset

• worker-autoscale
• HEAD
• alma-updates
• aspace_harvester_prod_prep
• dependabot/pip/apache-airflow-1.10.11
• dependabot/pip/apache-airflow-1.10.14
• deps
• disco-191-update-mario-commands
• ecr_login_fix
• fix-stage
• github-actions
• hoard
• ignore_tornado_warning
• main
• marcshot
• master
• networks
• plugin
• python3.8
• test-flow
• tests
• upgrade-airflow-1.10.11

Build # 36

Build Type

Pull #8

travis-ci-com

Committed by web-flow

Commit Message

Support worker autoscaling

Adding autoscaling to ECS for the worker service requires a few changes
to how the cluster is redeployed. Previously, everything was stopped by
setting the desiredCount to 0 and then started back up by setting the
desiredCount to 1. With ECS controlling the scaling for the worker
service it's no longer possible to rely on the desiredCount of the
worker service remaining at 0 until things are ready to be started
again.

The change here sets the desiredCount of the scheduler to 0 first, then
does a --force-new-deployment for the web and worker services, then
finally does a --force-new-deployment for the scheduler. While
redeploying becomes more complicated, there's no need anymore to track
the current worker count. We can just set it to 1 and let ECS scale as
needed.

I've also added stop-scheduler and start-scheduler commands which can be
used in conjunction with deploying a new service definition with
Terraform.

Pull Request Pull Request #8: Support worker autoscaling

Run Details

57 of 57 new or added lines in 2 files covered. (100.0%)

120 of 153 relevant lines covered (78.43%)

0.78 hits per line