kubeflow / trainer / 36759842357

72%

push

github

fix(framework): validate all ReplicatedJobs in Volcano priorityClassName check (#3918)

* fix(framework): validate all ReplicatedJobs in Volcano priorityClassName check

Volcano.Validate returned early when a ReplicatedJob used a reserved
priority class (system-cluster-critical or system-node-critical),
skipping priorityClassName validation for every ReplicatedJob listed
after it. Use continue instead of an early return so each
ReplicatedJob is validated independently.

Fixes #3917

Signed-off-by: Abhishek <abhikokadwar2@gmail.com>

* fix(framework): validate system priorityClassNames like any other name

system-cluster-critical and system-node-critical are PriorityClass objects
bootstrapped by kube-apiserver, which also refuses to delete them, so they
always resolve via the API. Skipping them was unnecessary and, because the
skip returned instead of continuing, it also stopped validation for every
remaining ReplicatedJob.

Drop the special case so each ReplicatedJob is validated uniformly.

Signed-off-by: Abhishek <abhikokadwar2@gmail.com>

* fix(manifests): grant the controller read access to priorityclasses

The Volcano plugin validates spec.priorityClassName by doing a client.Get
on the named PriorityClass, but the controller ClusterRole never granted
any access to scheduling.k8s.io/priorityclasses.

Because the plugin uses the cached controller-runtime client, the missing
permission makes the PriorityClass informer fail to list, so the cache
never syncs and the Get blocks until the webhook times out. Creating a
TrainJob whose runtime sets a priorityClassName then fails with an
InternalError instead of being admitted:

  Internal error occurred: failed calling webhook
  "validator.trainjob.trainer.kubeflow.org": context deadline exceeded

This affects any user-defined priority class today, and after the reserved
system names stopped being skipped it affects those as well.

Add the kubebuilder RBAC marker on the Volcano plugin, regenerate
manifests... (continued)

2 of 2 new or added lines in 1 file covered. (100.0%)

2781 of 3985 relevant lines covered (69.79%)

0.81 hits per line