[bug] k8s constraint ineffective

Description

When I add a ServiceTemplate which has a failing k8s constraint to a CD, I expect validation to fail but it does not.
The validation only fails if --enable-webhook=false but still successfully deploys the service.

Expectation

I expect the k8s validation to fail for both cases whether webhook is enabled or not.

When webhook enabled (default) case

Create a ClusterTemplate with .spec.k8sVersion=v1.32.8:

apiVersion: k0rdent.mirantis.com/v1beta1
kind: ClusterTemplate
metadata:
  name: aws-standalone-cp-1-0-20-dev
  annotations:
    helm.sh/resource-policy: keep
spec:
  k8sVersion: v1.32.8
  helm:
    chartSpec:
      chart: aws-standalone-cp
      version: 1.0.20
      interval: 10m0s
      sourceRef:
        kind: HelmRepository
        name: kcm-templates

Now create a CD with this ClusterTemplate and wait for it to successfully provision:

apiVersion: k0rdent.mirantis.com/v1beta1
kind: ClusterDeployment
metadata:
  name: wali-dev-1
  namespace: kcm-system
  labels:
    owner: dev-team
spec:
  template: aws-standalone-cp-1-0-20-dev
  config:
    clusterIdentity:
      name: aws-cluster-identity
      namespace: kcm-system
    controlPlane:
      amiID: ami-0eb9fdcf0d07bd5ef
      instanceProfile: control-plane.cluster-api-provider-aws.sigs.k8s.io
      instanceType: t3.small
    controlPlaneNumber: 1
    publicIP: true
    region: ca-central-1
    worker:
      amiID: ami-0eb9fdcf0d07bd5ef
      instanceProfile: nodes.cluster-api-provider-aws.sigs.k8s.io
      instanceType: t3.small
    workersNumber: 1
  credential: aws-cluster-identity-cred
  serviceSpec:
    provider:
      name: ksm-projectsveltos

Now create a ServiceTemplate with spec.k8sConstraint="<v1.30" so that it fails the ClusterTemplate's k8s version:

apiVersion: k0rdent.mirantis.com/v1beta1
kind: ServiceTemplate
metadata:
  annotations:
    helm.sh/resource-policy: keep
  name: ingress-nginx-4-13-0
  namespace: kcm-system
spec:
  k8sConstraint: "<v1.30"
  helm:
    chartSpec:
      chart: ingress-nginx
      sourceRef:
        kind: HelmRepository
        name: k0rdent-catalog
      version: 4.13.0

❌ Even though the k8s constraint should fail, the CD's status shows no validation failure and the service is successfully deployed on the target cluster:

➜  ~ kubectl -n kcm-system get clusterdeployment wali-dev-1 -o yaml
apiVersion: k0rdent.mirantis.com/v1beta1
kind: ClusterDeployment
metadata:
  annotations:
    kubectl.kubernetes.io/last-applied-configuration: |
      {"apiVersion":"k0rdent.mirantis.com/v1beta1","kind":"ClusterDeployment","metadata":{"annotations":{},"labels":{"owner":"dev-team"},"name":"wali-dev-1","namespace":"kcm-system"},"spec":{"config":{"clusterIdentity":{"name":"aws-cluster-identity","namespace":"kcm-system"},"controlPlane":{"amiID":"ami-0eb9fdcf0d07bd5ef","instanceProfile":"control-plane.cluster-api-provider-aws.sigs.k8s.io","instanceType":"t3.small"},"controlPlaneNumber":1,"publicIP":true,"region":"ca-central-1","worker":{"amiID":"ami-0eb9fdcf0d07bd5ef","instanceProfile":"nodes.cluster-api-provider-aws.sigs.k8s.io","instanceType":"t3.small"},"workersNumber":1},"credential":"aws-cluster-identity-cred","serviceSpec":{"provider":{"name":"ksm-projectsveltos"},"services":[{"name":"nginx","namespace":"nginx","template":"ingress-nginx-4-13-0"}]},"template":"aws-standalone-cp-1-0-20-dev"}}
  creationTimestamp: "2025-12-23T19:45:33Z"
  finalizers:
  - k0rdent.mirantis.com/cluster-deployment
  generation: 2
  labels:
    k0rdent.mirantis.com/component: kcm
    owner: dev-team
  name: wali-dev-1
  namespace: kcm-system
  resourceVersion: "17449"
  uid: ba1f3d0d-569d-4794-b10d-1bcaea204bb1
spec:
  config:
    clusterIdentity:
      name: aws-cluster-identity
      namespace: kcm-system
    controlPlane:
      amiID: ami-0eb9fdcf0d07bd5ef
      instanceProfile: control-plane.cluster-api-provider-aws.sigs.k8s.io
      instanceType: t3.small
    controlPlaneNumber: 1
    publicIP: true
    region: ca-central-1
    worker:
      amiID: ami-0eb9fdcf0d07bd5ef
      instanceProfile: nodes.cluster-api-provider-aws.sigs.k8s.io
      instanceType: t3.small
    workersNumber: 1
  credential: aws-cluster-identity-cred
  ipamClaim: {}
  propagateCredentials: true
  serviceSpec:
    continueOnError: false
    priority: 100
    provider:
      name: ksm-projectsveltos
    services:
    - name: nginx
      namespace: nginx
      template: ingress-nginx-4-13-0
    stopOnConflict: false
    syncMode: Continuous
  template: aws-standalone-cp-1-0-20-dev
status:
  conditions:
  - lastTransitionTime: "2025-12-23T19:45:33Z"
    message: ""
    observedGeneration: 2
    reason: Succeeded
    status: "True"
    type: CredentialReady
  - lastTransitionTime: "2025-12-23T19:45:35Z"
    message: Helm install succeeded for release kcm-system/wali-dev-1.v1 with chart
      aws-standalone-cp@1.0.20
    reason: InstallSucceeded
    status: "True"
    type: HelmReleaseReady
  - lastTransitionTime: "2025-12-23T19:45:33Z"
    message: ""
    observedGeneration: 2
    reason: Succeeded
    status: "True"
    type: HelmChartReady
  - lastTransitionTime: "2025-12-23T19:45:33Z"
    message: ""
    observedGeneration: 2
    reason: Succeeded
    status: "True"
    type: TemplateReady
  - lastTransitionTime: "2025-12-23T20:01:41Z"
    message: Object is ready
    reason: Succeeded
    status: "True"
    type: Ready
  - lastTransitionTime: "2025-12-23T19:45:33Z"
    message: ""
    observedGeneration: 2
    reason: Succeeded
    status: "True"
    type: ClusterDataSourceReady
  - lastTransitionTime: "2025-12-23T20:01:41Z"
    message: 1/1
    reason: Succeeded
    status: "True"
    type: ServicesInReadyState
  - lastTransitionTime: "2025-12-23T19:51:20Z"
    message: ""
    observedGeneration: 2
    reason: InfoReported
    status: "True"
    type: CAPIClusterSummary
  k8sVersion: v1.32.8
  observedGeneration: 2
  services:
  - lastStateTransitionTime: "2025-12-23T20:01:41Z"
    name: nginx
    namespace: nginx
    state: Deployed
    template: ingress-nginx-4-13-0
    type: Helm
    version: 4.13.0
  servicesUpgradePaths:
  - availableUpgrades:
    - versions:
      - name: ingress-nginx-4-13-0
        version: ingress-nginx-4-13-0
    name: nginx
    namespace: nginx
    template: ingress-nginx-4-13-0

When webhook is disabled

NOTE: This was observed with IsDisabledValidationWH set to True.

Create a ClusterTemplate with .spec.k8sVersion=v1.32.8
Now create a CD with this ClusterTemplate and wait for it to successfully provision.
Now create a ServiceTemplate with spec.k8sConstraint="<v1.30" so that it fails the ClusterTemplate's k8s version.
❌ Now add this service to the already created CD and the status will show that the validation failed but the service was still successfully deployed:

➜  ~ kubectl -n kcm-system get clusterdeployments.k0rdent.mirantis.com wali-dev-1        
NAME         READY   SERVICES   TEMPLATE                       MESSAGES                                                                                                                                                                                                                                                                                           AGE
wali-dev-1   False   1/1        aws-standalone-cp-1-0-20-dev   failed to validate ClusterTemplate K8s compatibility: k8s version v1.32.8 of the ClusterTemplate kcm-system/aws-standalone-cp-1-0-20-dev does not satisfy k8s constraint <v1.30 from the ServiceTemplate kcm-system/ingress-nginx-4-13-0 referred in the ClusterDeployment kcm-system/wali-dev-1   16m

The ServiceSet is still created:

➜  ~ kubectl -n kcm-system get serviceset
NAME         CLUSTER      MULTICLUSTERSERVER   PROVIDER             SELF-MANAGEMENT   AGE
wali-dev-1   wali-dev-1                        ksm-projectsveltos                     18m

The ServiceSet being created is fine. Perhaps we should fail the validation at ServiceSet level anyway because if we implement this feature for MCS then we will need to handle the k8s constraint validation in the ServiceSet.

❌ We can verify that the nginx service is successfully deployed on the target cluster despite the k8s constraint validation failure:

➜  ~ kubectl -n nginx get pod
NAME                                              READY   STATUS    RESTARTS   AGE
nginx-ingress-nginx-controller-66fc95fc97-rxjxk   1/1     Running   0          10m

@BROngineer

Proposal

Currently there is no validation webhook that does the k8s constraint validation anyway so let's make this validation part of the controller code which will always run irrespective of --enable-webhook.
Perform the k8s constraint validation in the ServiceSet controller so:
2a. ClusterDeployment controller creates a ServiceSet.
2b. The ServiceSet controller performs the k8s validation for each ServiceTemplate against the ClusterTemplate used by the CD.
2c. In case of self-management perform the validation based on the k8s version running on the management cluster. Self-management is KSM provider/adapter specific so there can also be another provider specific way to specify k8s version for the management cluster.
2c. The result of validation for each service is put in .status.services[].conditions of the ServiceSet, which is then fetched by the ClusterDeployment controller and put in the status for the CD.

With this approach the k8s constraint validation becomes the KSM adapter's responsibility (we may or may not want this) but the advantage is that the same process can be used for both CD and MCS. We also need to validate k8s constraint for self-management and self-management of the mothership cluster is a KSM adapter specific implementation already.

@BROngineer @zerospiel What do you think?

There is the validation (e.g., on the create event) for the clusterdeployment reßource

The problem with the code is that it had been implemented long before service providers/service sets, and the new ksm functionality has not been implemented in this regard. Thus, virtually no validation.

This particular function is being invoked from 2 places: from the webhook and if the webhook is disabled (that one piece of logic you've experienced).

	if len(cd.Spec.ServiceSpec.Services) == 0 || clusterTemplate.Status.KubernetesVersion == "" {
		return nil // nothing to do
	}

The cld object provided as an example has no services during the creation (though it has one afterwards, and I have to clue where it comes from). If something changes the spec of the cld then it is obvious why the validation does not work on creation (and as I can see from the given manifests, there were some changes to the cld object, generation field equals 2). The update event does not fail such a change because the webhook validation is being invoked only on the change of a referenced clustertemplate object.

The only thing I can suggest is to amend the already existing function in a way to support new resources/logic ksm had introduced.

With that being said, I see no reason the bug is related to kcm.

Navigation Menu

Repository navigation

[bug] k8s constraint ineffective #2304

Description

Expectation

When webhook enabled (default) case

When webhook is disabled

Proposal

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

[bug] k8s constraint ineffective #2304

Description

Description

Expectation

When webhook enabled (default) case

When webhook is disabled

Activity

wahabmk commented on Dec 24, 2025

Proposal

zerospiel commented on Dec 24, 2025

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions