oracle / karpenter-provider-oci / 29403216814

80%

Build Type

push

github

Committed by web-flow

Commit Message

fix: handle cases where OCI is out of capacity (#34)

When a NodePool runs out of host capacity, OCI Karpenter previously returned a
generic CloudProvider.CreateError from Create. Core Karpenter only deletes and
reschedules a NodeClaim on an InsufficientCapacityError, so on a CreateError the
NodeClaim stayed pinned to the exhausted NodePool/offering and retried in a tight
loop, producing retry storms and 429 throttling with no fallback to other
capacity types or NodePools.

This change makes host-capacity exhaustion behave correctly:

- Classify pool-wide exhaustion as insufficient capacity: when every launch
  attempt is exhausted and at least one failed with an out-of-host-capacity (or
  service-limit/quota) error, CloudProvider.Create returns
  cloudprovider.NewInsufficientCapacityError so core Karpenter deletes and
  reschedules the NodeClaim, enabling cross-NodePool fallback.
- Add a TTL-based unavailable-offerings cache
  (pkg/cache/unavailableofferings.go) keyed by shape, CPU/memory config, zone and
  capacity type. LaunchInstance marks offerings unavailable on capacity/limit
  errors and the instance-type provider gates offering availability, driving
  spot->on-demand and cross-NodePool fallback. TTL is configurable via
  --unavailable-offerings-ttl-seconds (default 180s).
- Treat an HTTP 500 "Out of host capacity" error as non-retryable at the SDK
  layer so the capacity-fallback logic handles it instead of the SDK amplifying
  the shortage.

Fixes #33 #51

Signed-off-by: Aniek Gul <13356402+aniekgul@users.noreply.github.com>

Coverage Stats

153 of 168 new or added lines in 8 files covered. (91.07%)

12 existing lines in 2 files now uncovered.

4908 of 6243 relevant lines covered (78.62%)

0.9 hits per line