openconfig / kne / 34531914240

44%

Build Type

push

github

Committed by web-flow

Commit Message

Add a K8s readiness check for device pods. (#747)

* Add a K8s readiness check for Cisco pods.

Cisco pods may have a running process, but fail to come to "Router up"
state. Prior to this change, KNE just hangs forever waiting for Router
up even for permanent failures.

This resolves this in two ways (belt and suspenders.) First, for
Cisco pods we add the check for "Router up" as a readiness check.
That way, a failure ends up reflected in K8s' reported pod status.
The pod will not transition to Ready state until that message is logged.
And if it doesn't show up within the threshold, the pod will report as
failed.

Second, KNE explicitly checks for failure in the logs and treats it
as such, rather than just waiting indefinitely. KNE may notice first,
since the failure message will likely arrive before the readiness check
exceeds its threshold.

* Check log close error

* Check the correct file for "Router up" as readiness

* Move to a more K8s native readiness check - gRPC port is open

The log check isn't working. This is a more typical approach for K8s.

* Do readiness for Cisco XRD also

* Add a readiness check for Juniper also

* Add a readiness check for Arista pods also

* Fix a readiness race also for Nokia, on certificate application

* Add a readiness check for Sonic

* Add readiness check for ciena

* Pick up service definition for readiness dynamically

* Design review comments

1. **Deterministic Probe Selection in node.go:234-275**:
    * Sorted pb.Services map keys before iterating in node.go:234-275 to prevent non-deterministic probe selection when multiple services
    match preferred names.
2. **Pod Readiness Condition Check in cisco.go:619-658**:
    * Updated cisco.go:619-658 to inspect p[0].Status.Conditions for corev1.PodReady == corev1.ConditionTrue before returning node.
    StatusRunning, ensuring XRD (and other Cisco models) do not bypass readiness probes.
3. **Single-Stream Log Fetching & Shadowing Fix in cisco.go:636-690**:
 ... (continued)

Coverage Stats

83 of 89 new or added lines in 7 files covered. (93.26%)

4 existing lines in 1 file now uncovered.

5885 of 13365 relevant lines covered (44.03%)

0.5 hits per line