• Home
  • Features
  • Pricing
  • Docs
  • Announcements
  • Sign In

karellen / kubernator / 24580658204

17 Apr 2026 06:29PM UTC coverage: 77.939% (+1.0%) from 76.951%
24580658204

push

github

web-flow
Refactor k8s apply: SSA test-ops, 409 retry, watch-based delete, imperative API (#102)

## Summary

- Harden the K8s apply path with SSA-derived JSON-patch `test` ops for
`/metadata/uid` and `/metadata/resourceVersion`, and wrap the SSA patch
branch in a 409-retry loop that recomputes the patch on conflict.
- Extend `K8SResource` with `list()`, `watch()`, `delete(wait=True)`
that uses watch + `get()` recheck, and add a `resource_generator()`
extension seam on `KubernetesPlugin`.
- Add `ktor.k8s.resource(manifest)` — a fully-wired `K8SResource`
factory for imperative CRUD from `.kubernator.py` scripts, bypassing the
declarative apply lifecycle.
- Thread the applied manifest as a 4th return from `_apply_resource` and
its inner patch/create/delete callables for external consumers.
- Move dump-file open/close from `app.py` into
`KubernetesPlugin.handle_apply`; fix dump-mode `patch_func` to seed
uid/RV on a copy of the local manifest by extracting them from the
patch's own test ops (via `jsonpointer`), so the simulated post-patch
manifest can be returned honestly.
- Rewrite `delete_create` and `resource_version_merge` integration tests
as single-phase using the new imperative API, replacing `kubectl`-based
phase-1 setup and the `TEST_PHASE` env re-invocation.
- Add unit test for 409 retry and integration tests for `watch()` and
`resource_generator`. Port `resource_version_merge` from minikube to
kind.

## Test plan

- [x] `pyb -vX run_unit_tests` (32 tests, all green — includes new
`k8s_apply_retry_tests`)
- [x] `pyb -vX run_integration_tests -P
integrationtest_file_glob=delete_create_tests.py` (single-phase, kind,
immutable-field delete+recreate path)
- [x] `pyb -vX run_integration_tests -P
integrationtest_file_glob=resource_version_merge_tests.py`
(single-phase, kind, dump mode patch verification)
- [ ] Full `pyb -vX run_integration_tests` suite via CI across the
Python 3.10–3.14 matrix

614 of 976 branches covered (62.91%)

Branch coverage included in aggregate %.

137 of 163 new or added lines in 4 files covered. (84.05%)

5 existing lines in 3 files now uncovered.

3039 of 3711 relevant lines covered (81.89%)

4.09 hits per line

Source File
Press 'n' to go to next uncovered line, 'b' for previous

81.02
/src/main/python/kubernator/plugins/k8s.py
1
# -*- coding: utf-8 -*-
2
#
3
#   Copyright 2020 Express Systems USA, Inc
4
#   Copyright 2021 Karellen, Inc.
5
#
6
#   Licensed under the Apache License, Version 2.0 (the "License");
7
#   you may not use this file except in compliance with the License.
8
#   You may obtain a copy of the License at
9
#
10
#       http://www.apache.org/licenses/LICENSE-2.0
11
#
12
#   Unless required by applicable law or agreed to in writing, software
13
#   distributed under the License is distributed on an "AS IS" BASIS,
14
#   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
15
#   See the License for the specific language governing permissions and
16
#   limitations under the License.
17
#
18

19

20
import json
5✔
21
import logging
5✔
22
import re
5✔
23
import sys
5✔
24
import types
5✔
25
from collections.abc import Mapping
5✔
26
from functools import partial
5✔
27
from importlib.metadata import version as pkg_version
5✔
28
from pathlib import Path
5✔
29
from typing import Iterable, Callable, Sequence, Optional
5✔
30

31
import jsonpatch
5✔
32
import yaml
5✔
33

34
from kubernator.api import (KubernatorPlugin,
5✔
35
                            Globs,
36
                            scan_dir,
37
                            load_file,
38
                            FileType,
39
                            load_remote_file,
40
                            StripNL,
41
                            install_python_k8s_client,
42
                            TemplateEngine,
43
                            calling_frame_source,
44
                            parse_yaml_docs)
45
from kubernator.merge import extract_merge_instructions, apply_merge_instructions
5✔
46
from kubernator.plugins.k8s_api import (K8SResourcePluginMixin,
5✔
47
                                        K8SResource,
48
                                        K8SResourcePatchType,
49
                                        K8SPropagationPolicy,
50
                                        api_exc_format_body)
51

52
logger = logging.getLogger("kubernator.k8s")
5✔
53
proc_logger = logger.getChild("proc")
5✔
54
stdout_logger = StripNL(proc_logger.info)
5✔
55
stderr_logger = StripNL(proc_logger.warning)
5✔
56

57
FIELD_VALIDATION_STRICT_MARKER = "strict decoding error: "
5✔
58
VALID_FIELD_VALIDATION = ("Ignore", "Warn", "Strict")
5✔
59

60

61
def final_resource_validator(resources: Sequence[K8SResource],
5✔
62
                             resource: K8SResource,
63
                             error: Callable[..., Exception]) -> Iterable[Exception]:
64
    final_key = resource.get_manifest_key(resource.manifest)
5✔
65
    if final_key != resource.key:
5!
66
        yield error("Illegal change of identifiers of the resource "
×
67
                    "%s from %s have been changed to %s",
68
                    resource.key, resource.source, final_key)
69

70
    if resource.rdef.namespaced and not resource.namespace:
5!
71
        yield error("Namespaced resource %s from %s is missing the required namespace",
×
72
                    resource, resource.source)
73

74

75
def normalize_pkg_version(v: str):
5✔
76
    v_split = v.split(".")
5✔
77
    rev = v_split[-1]
5✔
78
    if not rev.isdigit():
5✔
79
        new_rev = ""
5✔
80
        for c in rev:
5!
81
            if not c.isdigit():
5✔
82
                break
5✔
83
            new_rev += c
5✔
84
        v_split[-1] = new_rev
5✔
85
    return tuple(map(int, v_split))
5✔
86

87

88
class KubernetesPlugin(KubernatorPlugin, K8SResourcePluginMixin):
5✔
89
    logger = logger
5✔
90

91
    _name = "k8s"
5✔
92

93
    def __init__(self):
5✔
94
        super().__init__()
5✔
95
        self.context = None
5✔
96

97
        self.embedded_pkg_version = self._get_kubernetes_client_version()
5✔
98

99
        self._transformers = []
5✔
100
        self._validators = []
5✔
101
        self._manifest_patchers = []
5✔
102
        self._summary = 0, 0, 0
5✔
103
        self._template_engine = TemplateEngine(logger)
5✔
104

105
    def set_context(self, context):
5✔
106
        self.context = context
5✔
107

108
    def register(self,
5✔
109
                 field_validation="Warn",
110
                 field_validation_warn_fatal=True,
111
                 disable_client_patches=False):
112
        self.context.app.register_plugin("kubeconfig")
5✔
113

114
        if field_validation not in VALID_FIELD_VALIDATION:
5!
115
            raise ValueError("'field_validation' must be one of %s" % (", ".join(VALID_FIELD_VALIDATION)))
×
116

117
        context = self.context
5✔
118
        context.globals.k8s = dict(patch_field_excludes=("^/metadata/managedFields",
5✔
119
                                                         "^/metadata/generation",
120
                                                         "^/metadata/creationTimestamp",
121
                                                         "^/metadata/resourceVersion",
122
                                                         ),
123
                                   immutable_changes={("apps", "DaemonSet"): K8SPropagationPolicy.BACKGROUND,
124
                                                      ("apps", "StatefulSet"): K8SPropagationPolicy.ORPHAN,
125
                                                      ("apps", "Deployment"): K8SPropagationPolicy.ORPHAN,
126
                                                      ("storage.k8s.io", "StorageClass"): K8SPropagationPolicy.ORPHAN,
127
                                                      (None, "Pod"): K8SPropagationPolicy.BACKGROUND,
128
                                                      ("batch", "Job"): K8SPropagationPolicy.ORPHAN,
129
                                                      },
130
                                   default_includes=Globs(["*.yaml", "*.yml"], True),
131
                                   default_excludes=Globs([".*"], True),
132
                                   add_resources=self.add_resources,
133
                                   load_resources=self.api_load_resources,
134
                                   load_remote_resources=self.api_load_remote_resources,
135
                                   load_crds=self.api_load_crds,
136
                                   import_cluster_crds=self.api_import_cluster_crds,
137
                                   load_remote_crds=self.api_load_remote_crds,
138
                                   add_transformer=self.api_add_transformer,
139
                                   remove_transformer=self.api_remove_transformer,
140
                                   add_validator=self.api_remove_validator,
141
                                   add_manifest_patcher=self.api_add_manifest_patcher,
142
                                   get_api_versions=self.get_api_versions,
143
                                   create_resource=self.create_resource,
144
                                   disable_client_patches=disable_client_patches,
145
                                   field_validation=field_validation,
146
                                   field_validation_warn_fatal=field_validation_warn_fatal,
147
                                   field_validation_warnings=0,
148
                                   resource_generator=self.resource_generator,
149
                                   resource=self.resource,
150
                                   conflict_retry_delay=0.3,
151
                                   _k8s=self,
152
                                   )
153
        context.k8s = dict(default_includes=Globs(context.globals.k8s.default_includes),
5✔
154
                           default_excludes=Globs(context.globals.k8s.default_excludes)
155
                           )
156
        self.api_add_validator(final_resource_validator)
5✔
157

158
    def handle_init(self):
5✔
159
        pass
5✔
160

161
    def handle_start(self):
5✔
162
        self.context.kubeconfig.register_change_notifier(self._kubeconfig_changed)
5✔
163
        self.setup_client()
5✔
164

165
    def _kubeconfig_changed(self):
5✔
166
        self.setup_client()
×
167

168
    def _get_kubernetes_client_version(self):
5✔
169
        return pkg_version("kubernetes").split(".")
5✔
170

171
    def setup_client(self):
5✔
172
        k8s = self.context.k8s
5✔
173
        if "server_version" not in k8s:
5!
174
            self._setup_client()
5✔
175

176
        server_minor = k8s.server_version[1]
5✔
177

178
        logger.info("Using Kubernetes client version =~%s.0 for server version %s",
5✔
179
                    server_minor, ".".join(k8s.server_version))
180
        pkg_dir = install_python_k8s_client(self.context.app.run_passthrough_capturing, server_minor, logger,
5✔
181
                                            stdout_logger, stderr_logger, k8s.disable_client_patches)
182

183
        modules_to_delete = []
5✔
184
        for k, v in sys.modules.items():
5✔
185
            if k == "kubernetes" or k.startswith("kubernetes."):
5✔
186
                modules_to_delete.append(k)
5✔
187
        for k in modules_to_delete:
5✔
188
            del sys.modules[k]
5✔
189

190
        logger.info("Adding sys.path reference to %s", pkg_dir)
5✔
191
        sys.path.insert(0, str(pkg_dir))
5✔
192
        self.embedded_pkg_version = self._get_kubernetes_client_version()
5✔
193
        logger.info("Switching to Kubernetes client version %s", ".".join(self.embedded_pkg_version))
5✔
194
        self._setup_client()
5✔
195

196
        logger.debug("Reading Kubernetes OpenAPI spec for %s", k8s.server_git_version)
5✔
197

198
        k8s_def = load_remote_file(logger, f"https://raw.githubusercontent.com/kubernetes/kubernetes/"
5✔
199
                                           f"{k8s.server_git_version}/api/openapi-spec/swagger.json",
200
                                   FileType.JSON)
201
        self.resource_definitions_schema = k8s_def
5✔
202

203
        self._populate_resource_definitions()
5✔
204

205
    def _setup_client(self):
5✔
206
        from kubernetes import client
5✔
207

208
        context = self.context
5✔
209
        k8s = context.k8s
5✔
210

211
        k8s.client = self._setup_k8s_client()
5✔
212
        version = client.VersionApi(k8s.client).get_code()
5✔
213
        # Strip vendor-specific suffixes so OpenAPI lookups hit upstream tags.
214
        # EKS/GKE use a dash (e.g. v1.28.3-eks-..., v1.28.3-gke.100);
215
        # k3s uses a plus sign (e.g. v1.35.3+k3s1).
216
        git_version = version.git_version.split("-")[0].split("+")[0]
5✔
217

218
        k8s.server_version = git_version[1:].split(".")
5✔
219
        k8s.server_git_version = git_version
5✔
220

221
        logger.info("Found Kubernetes %s on %s", k8s.server_git_version, k8s.client.configuration.host)
5✔
222

223
        K8SResource._k8s_client_version = normalize_pkg_version(pkg_version("kubernetes"))
5✔
224
        K8SResource._k8s_field_validation = k8s.field_validation
5✔
225
        K8SResource._k8s_field_validation_patched = not k8s.disable_client_patches
5✔
226
        K8SResource._logger = self.logger
5✔
227
        K8SResource._api_warnings = self._api_warnings
5✔
228

229
    def _api_warnings(self, resource, warn):
5✔
230
        k8s = self.context.k8s
5✔
231
        self.context.globals.k8s.field_validation_warnings += 1
5✔
232

233
        log = self.logger.warning
5✔
234
        if k8s.field_validation_warn_fatal:
5✔
235
            log = self.logger.error
5✔
236

237
        log("FAILED FIELD VALIDATION on resource %s from %s: %s", resource, resource.source, warn)
5✔
238

239
    def handle_before_dir(self, cwd: Path):
5✔
240
        context = self.context
5✔
241
        context.k8s.default_includes = Globs(context.k8s.default_includes)
5✔
242
        context.k8s.default_excludes = Globs(context.k8s.default_excludes)
5✔
243
        context.k8s.includes = Globs(context.k8s.default_includes)
5✔
244
        context.k8s.excludes = Globs(context.k8s.default_excludes)
5✔
245

246
    def handle_after_dir(self, cwd: Path):
5✔
247
        context = self.context
5✔
248
        k8s = context.k8s
5✔
249

250
        for f in scan_dir(logger, cwd, lambda d: d.is_file(), k8s.excludes, k8s.includes):
5✔
251
            p = cwd / f.name
5✔
252
            display_p = context.app.display_path(p)
5✔
253
            logger.debug("Adding Kubernetes manifest from %s", display_p)
5✔
254

255
            manifests = load_file(logger, p, FileType.YAML, display_p,
5✔
256
                                  self._template_engine,
257
                                  {"ktor": context}
258
                                  )
259

260
            for manifest in manifests:
5✔
261
                if manifest:
5!
262
                    self.add_resource(manifest, display_p)
5✔
263

264
    def resource_generator(self):
5✔
265
        yield from self.resources.values()
5✔
266

267
    def resource(self, manifest, source=None):
5✔
268
        from kubernetes import client
5✔
269
        if not source:
5!
270
            source = calling_frame_source()
5✔
271
        if isinstance(manifest, str):
5!
272
            docs = [m for m in parse_yaml_docs(manifest, source) if m]
5✔
273
            if len(docs) != 1:
5!
NEW
274
                raise ValueError(f"ktor.k8s.resource() expects a single manifest document, got {len(docs)} from {source}")
×
275
            manifest = docs[0]
5✔
276
        res = self._create_resource(manifest, source)
5✔
277
        res.rdef.populate_api(client, self.context.k8s.client)
5✔
278
        return res
5✔
279

280
    def handle_apply(self):
5✔
281
        context = self.context
5✔
282
        k8s = context.k8s
5✔
283

284
        self._validate_resources()
5✔
285

286
        cmd = context.app.args.command
5✔
287
        file_name = context.app.args.file
5✔
288
        file_format = context.app.args.output_format
5✔
289
        dry_run = context.app.args.dry_run
5✔
290
        dump = cmd == "dump"
5✔
291

292
        status_msg = f"{' (dump only)' if dump else ' (dry run)' if dry_run else ''}"
5✔
293
        if dump:
5✔
294
            logger.info("Will dump the changes into a file %s in %s format", file_name or "<stdout>", file_format)
5✔
295

296
        patch_field_excludes = [re.compile(e) for e in context.globals.k8s.patch_field_excludes]
5✔
297
        dump_results = []
5✔
298
        total_created, total_patched, total_deleted = 0, 0, 0
5✔
299
        for resource in k8s.resource_generator():
5✔
300
            if dump:
5✔
301
                resource_id = {"apiVersion": resource.api_version,
5✔
302
                               "kind": resource.kind,
303
                               "name": resource.name
304
                               }
305

306
                def patch_func(patch):
5✔
307
                    if resource.rdef.namespaced:
5✔
308
                        resource_id["namespace"] = resource.namespace
5✔
309
                    method_descriptor = {"method": "patch",
5✔
310
                                         "resource": resource_id,
311
                                         "body": patch
312
                                         }
313
                    dump_results.append(method_descriptor)
5✔
314
                    return resource.manifest
5✔
315

316
                def create_func():
5✔
317
                    method_descriptor = {"method": "create",
5✔
318
                                         "body": resource.manifest}
319
                    dump_results.append(method_descriptor)
5✔
320
                    return resource.manifest
5✔
321

322
                def delete_func(*, propagation_policy):
5✔
323
                    method_descriptor = {"method": "delete",
×
324
                                         "resource": resource_id,
325
                                         "propagation_policy": propagation_policy.policy
326
                                         }
327
                    dump_results.append(method_descriptor)
×
NEW
328
                    return None
×
329
            else:
330
                patch_func = partial(resource.patch, patch_type=K8SResourcePatchType.JSON_PATCH, dry_run=dry_run)
5✔
331
                create_func = partial(resource.create, dry_run=dry_run)
5✔
332
                delete_func = partial(resource.delete, dry_run=dry_run)
5✔
333

334
            created, patched, deleted, result = self._apply_resource(dry_run,
5✔
335
                                                                     patch_field_excludes,
336
                                                                     resource,
337
                                                                     patch_func,
338
                                                                     create_func,
339
                                                                     delete_func,
340
                                                                     status_msg)
341

342
            total_created += created
5✔
343
            total_patched += patched
5✔
344
            total_deleted += deleted
5✔
345

346
        if ((dump or dry_run) and
5✔
347
                k8s.field_validation_warn_fatal and self.context.globals.k8s.field_validation_warnings):
348
            msg = ("There were %d field validation warnings and the warnings are fatal!" %
5✔
349
                   self.context.globals.k8s.field_validation_warnings)
350
            logger.fatal(msg)
5✔
351
            raise RuntimeError(msg)
5✔
352

353
        if dump:
5✔
354
            file = open(file_name, "w") if file_name else sys.stdout
5✔
355
            try:
5✔
356
                if file_format in ("json", "json-pretty"):
5✔
357
                    json.dump(dump_results, file, sort_keys=True,
5✔
358
                              indent=4 if file_format == "json-pretty" else None)
359
                else:
360
                    yaml.safe_dump(dump_results, file)
5✔
361
            finally:
362
                if file_name:
5✔
363
                    file.close()
5✔
364
        else:
365
            self._summary = total_created, total_patched, total_deleted
5✔
366

367
    def handle_summary(self):
5✔
368
        total_created, total_patched, total_deleted = self._summary
5✔
369
        logger.info("Created %d, patched %d, deleted %d resources", total_created, total_patched, total_deleted)
5✔
370

371
    def api_load_resources(self, path: Path, file_type: str):
5✔
372
        return self.add_local_resources(path, FileType[file_type.upper()])
×
373

374
    def api_load_remote_resources(self, url: str, file_type: str, file_category=None):
5✔
375
        return self.add_remote_resources(url, FileType[file_type.upper()], sub_category=file_category)
×
376

377
    def api_load_crds(self, path: Path, file_type: str):
5✔
378
        return self.add_local_crds(path, FileType[file_type.upper()])
5✔
379

380
    def api_load_remote_crds(self, url: str, file_type: str, file_category=None):
5✔
381
        return self.add_remote_crds(url, FileType[file_type.upper()], sub_category=file_category)
5✔
382

383
    def api_import_cluster_crds(self):
5✔
384
        context = self.context
×
385
        k8s = context.k8s
×
386
        client = k8s.client
×
387
        from kubernetes import client as client_module
×
388

389
        api = client_module.ApiextensionsV1Api(client)
×
390
        crds = api.list_custom_resource_definition(watch=False)
×
391
        for crd in crds.items:
×
392
            manifest = client.sanitize_for_serialization(crd)
×
393
            manifest["apiVersion"] = "apiextensions.k8s.io/v1"
×
394
            manifest["kind"] = "CustomResourceDefinition"
×
395
            self.add_crd(manifest)
×
396

397
    def api_add_transformer(self, transformer):
5✔
398
        if transformer not in self._transformers:
5!
399
            self._transformers.append(transformer)
5✔
400

401
    def api_add_validator(self, validator):
5✔
402
        if validator not in self._validators:
5!
403
            self._validators.append(validator)
5✔
404

405
    def api_add_manifest_patcher(self, patcher):
5✔
406
        if patcher not in self._manifest_patchers:
×
407
            self._manifest_patchers.append(patcher)
×
408

409
    def api_remove_transformer(self, transformer):
5✔
410
        if transformer in self._transformers:
5!
411
            self._transformers.remove(transformer)
5✔
412

413
    def api_remove_validator(self, validator):
5✔
414
        if validator not in self._validators:
×
415
            self._validators.remove(validator)
×
416

417
    def api_validation_error(self, msg, *args):
5✔
418
        frame = sys._getframe().f_back
×
419
        tb = None
×
420
        while True:
×
421
            if not frame:
×
422
                break
×
423
            tb = types.TracebackType(tb, frame, frame.f_lasti, frame.f_lineno)
×
424
            frame = frame.f_back
×
425
        return ValueError((msg % args) if args else msg).with_traceback(tb)
×
426

427
    def _patch_manifest(self,
5✔
428
                        manifest: dict,
429
                        resource_description: str):
430
        for patcher in reversed(self._manifest_patchers):
5!
431
            logger.debug("Applying patcher %s to %s",
×
432
                         getattr(patcher, "__name__", patcher),
433
                         resource_description)
434
            manifest = patcher(manifest, resource_description) or manifest
×
435

436
        return manifest
5✔
437

438
    def _transform_resource(self, resources: Sequence[K8SResource], resource: K8SResource) -> K8SResource:
5✔
439
        for transformer in reversed(self._transformers):
5✔
440
            logger.debug("Applying transformer %s to %s from %s",
5✔
441
                         getattr(transformer, "__name__", transformer),
442
                         resource, resource.source)
443
            resource = transformer(resources, resource) or resource
5✔
444

445
        return resource
5✔
446

447
    def _validate_resources(self):
5✔
448
        errors: list[Exception] = []
5✔
449
        for resource in self.resources.values():
5✔
450
            for validator in reversed(self._validators):
5✔
451
                logger.debug("Applying validator %s to %s from %s",
5✔
452
                             getattr(validator, "__name__", validator),
453
                             resource, resource.source)
454
                errors.extend(validator(self.resources, resource, self.api_validation_error))
5✔
455
        if errors:
5!
456
            for error in errors:
×
457
                logger.error("Validation error: %s", error)
×
458
            raise errors[0]
×
459

460
    def _apply_resource(self,
5✔
461
                        dry_run,
462
                        patch_field_excludes: Iterable[re.compile],
463
                        resource: K8SResource,
464
                        patch_func: Callable[[Iterable[dict]], Optional[dict]],
465
                        create_func: Callable[[], Optional[dict]],
466
                        delete_func: Callable[[K8SPropagationPolicy], None],
467
                        status_msg):
468
        from kubernetes import client
5✔
469
        from kubernetes.client.rest import ApiException
5✔
470

471
        rdef = resource.rdef
5✔
472
        rdef.populate_api(client, self.context.k8s.client)
5✔
473

474
        def handle_400_strict_validation_error(e: ApiException):
5✔
475
            if e.status == 400:
5!
476
                # Assumes the body has been parsed
477
                status = e.body
5✔
478
                if status["status"] == "Failure":
5!
479
                    if FIELD_VALIDATION_STRICT_MARKER in status["message"]:
5!
480
                        message = status["message"]
5✔
481
                        messages = message[message.find(FIELD_VALIDATION_STRICT_MARKER) +
5✔
482
                                           len(FIELD_VALIDATION_STRICT_MARKER):].split(",")
483
                        for m in messages:
5✔
484
                            self._api_warnings(resource, m.strip())
5✔
485

486
                        raise e from None
5✔
487
                    else:
488
                        logger.error("FAILED MODIFYING resource %s from %s: %s",
×
489
                                     resource, resource.source, status["message"])
490
                        raise e from None
×
491

492
        def create(exists_ok=False):
5✔
493
            logger.info("Creating resource %s%s%s", resource, status_msg,
5✔
494
                        " (ignoring existing)" if exists_ok else "")
495
            try:
5✔
496
                return create_func()
5✔
497
            except ApiException as __e:
5✔
498
                if exists_ok and __e.status == 409 and __e.body["reason"] == "AlreadyExists":
5!
NEW
499
                    return None
×
500
                raise
5✔
501

502
        merge_instrs, normalized_manifest = extract_merge_instructions(resource.manifest, resource)
5✔
503
        if merge_instrs:
5✔
504
            logger.trace("Normalized manifest (no merge instructions) for resource %s: %s", resource,
5✔
505
                         normalized_manifest)
506
        else:
507
            normalized_manifest = resource.manifest
5✔
508

509
        logger.debug("Applying resource %s%s", resource, status_msg)
5✔
510
        try:
5✔
511
            remote_resource = resource.get()
5✔
512
            logger.trace("Current resource %s: %s", resource, remote_resource)
5✔
513
        except ApiException as e:
5✔
514
            try:
5✔
515
                if e.status == 404:
5!
516
                    try:
5✔
517
                        return 1, 0, 0, create()
5✔
518
                    except ApiException as e:
5✔
519
                        if not handle_400_strict_validation_error(e):
5!
UNCOV
520
                            raise
×
521
                else:
522
                    raise
×
523
            except ApiException as _e:
5✔
524
                api_exc_format_body(_e)
5✔
525
                raise
5✔
526
        else:
527
            while True:
5✔
528
                logger.trace("Attempting to retrieve a normalized patch for resource %s: %s",
5✔
529
                             resource, normalized_manifest)
530
                try:
5✔
531
                    merged_resource = resource.patch(normalized_manifest,
5✔
532
                                                     patch_type=K8SResourcePatchType.SERVER_SIDE_PATCH,
533
                                                     dry_run=True,
534
                                                     force=True)
535
                except ApiException as e:
5✔
536
                    try:
5✔
537
                        if e.status == 422:
5!
538
                            status = e.body
5✔
539
                            # Assumes the body has been unmarshalled
540
                            details = status["details"]
5✔
541
                            immutable_key = details.get("group"), details["kind"]
5✔
542

543
                            try:
5✔
544
                                propagation_policy = self.context.k8s.immutable_changes[immutable_key]
5✔
NEW
545
                            except KeyError:
×
NEW
546
                                raise e from None
×
547
                            else:
548
                                for cause in details["causes"]:
5!
549
                                    if (
5!
550
                                            cause["reason"] == "FieldValueInvalid" and
551
                                            "field is immutable" in cause["message"]
552
                                            or
553
                                            cause["reason"] == "FieldValueForbidden" and
554
                                            ("Forbidden: updates to" in cause["message"]
555
                                             or
556
                                             "Forbidden: pod updates" in cause["message"])
557
                                    ):
558
                                        logger.info("Deleting resource %s (cascade %s)%s", resource,
5✔
559
                                                    propagation_policy.policy,
560
                                                    status_msg)
561
                                        delete_func(propagation_policy=propagation_policy)
5✔
562
                                        return 1, 0, 1, create(exists_ok=dry_run)
5✔
NEW
563
                                raise
×
564
                        else:
NEW
565
                            if not handle_400_strict_validation_error(e):
×
NEW
566
                                raise
×
NEW
567
                    except ApiException as _e:
×
NEW
568
                        api_exc_format_body(_e)
×
NEW
569
                        raise
×
570

571
                else:
572
                    logger.trace("Merged resource %s: %s", resource, merged_resource)
5✔
573
                    if merge_instrs:
5✔
574
                        apply_merge_instructions(merge_instrs, normalized_manifest, merged_resource, logger, resource)
5✔
575

576
                    patch = jsonpatch.make_patch(remote_resource, merged_resource)
5✔
577

578
                    resource_version = merged_resource["metadata"]["resourceVersion"]
5✔
579
                    resource_uid = merged_resource["metadata"]["uid"]
5✔
580
                    logger.trace("Resource %s adding resourceVersion %s and UID %s tests", resource, resource_version,
5✔
581
                                 resource_uid)
582
                    patch.patch.append({"op": "test", "path": "/metadata/uid", "value": resource_uid})
5✔
583
                    patch.patch.append({"op": "test", "path": "/metadata/resourceVersion", "value": resource_version})
5✔
584

585
                    logger.trace("Resource %s initial patches are: %s", resource, patch)
5✔
586
                    patch = self._filter_resource_patch(patch, patch_field_excludes)
5✔
587
                    logger.trace("Resource %s final patches are: %s", resource, patch)
5✔
588
                    if patch:
5!
589
                        logger.info("Patching resource %s%s", resource, status_msg)
5✔
590
                        try:
5✔
591
                            return 0, 1, 0, patch_func(patch)
5✔
592
                        except ApiException as e:
5✔
593
                            if e.status == 409:
5✔
594
                                logger.warning("Patching resource %s%s encountered a conflict - will retry: \n%s",
5✔
595
                                               resource, status_msg, yaml.dump(e.body))
596
                                continue
5✔
597
                            raise
5✔
598
                    else:
NEW
599
                        logger.info("Nothing to patch for resource %s", resource)
×
NEW
600
                        return 0, 0, 0, None
×
601

602
    def _filter_resource_patch(self, patch: Iterable[Mapping], excludes: Iterable[re.compile]):
5✔
603
        result = []
5✔
604
        for op in patch:
5✔
605
            if op["op"] != "test":
5✔
606
                path = op["path"]
5✔
607
                excluded = False
5✔
608
                for exclude in excludes:
5✔
609
                    if exclude.match(path):
5✔
610
                        logger.trace("Excluding %r from patch %s", op, patch)
5✔
611
                        excluded = True
5✔
612
                        break
5✔
613
                if excluded:
5✔
614
                    continue
5✔
615
            result.append(op)
5✔
616
        return result
5✔
617

618
    def _setup_k8s_client(self):
5✔
619
        from kubernetes import client
5✔
620
        from kubernetes.config import load_incluster_config, load_kube_config, ConfigException
5✔
621

622
        try:
5✔
623
            logger.debug("Trying K8S in-cluster configuration")
5✔
624
            load_incluster_config()
5✔
625
            logger.info("Running K8S with in-cluster configuration")
×
626
        except ConfigException as e:
5✔
627
            logger.trace("K8S in-cluster configuration failed", exc_info=e)
5✔
628
            logger.debug("Initializing K8S with kubeconfig configuration")
5✔
629
            load_kube_config(config_file=self.context.kubeconfig.kubeconfig)
5✔
630

631
        k8s_client = client.ApiClient()
5✔
632

633
        # Patch the header content type selector to allow json patch
634
        k8s_client._select_header_content_type = k8s_client.select_header_content_type
5✔
635
        k8s_client.select_header_content_type = self._select_header_content_type_patch
5✔
636

637
        return k8s_client
5✔
638

639
    def _select_header_content_type_patch(self, content_types):
5✔
640
        """Returns `Content-Type` based on an array of content_types provided.
641
        :param content_types: List of content-types.
642
        :return: Content-Type (e.g. application/json).
643
        """
644

645
        content_type = self.context.k8s.client._select_header_content_type(content_types)
×
646
        if content_type == "application/merge-patch+json":
×
647
            return "application/json-patch+json"
×
648
        return content_type
×
649

650
    def __repr__(self):
5✔
651
        return "Kubernetes Plugin"
5✔
STATUS · Troubleshooting · Open an Issue · Sales · Support · CAREERS · ENTERPRISE · START FREE · SCHEDULE DEMO
ANNOUNCEMENTS · TWITTER · TOS & SLA · Supported CI Services · What's a CI service? · Automated Testing

© 2026 Coveralls, Inc