There are times when in Kubernetes we want certain nodes to serve only pods from specific namespaces. Some use cases can be, let’s say when we have nodes with additional security features and/or layers (like firewall, or all-to-all access control list) and we want to run eligible pods on those nodes.

Isolating nodes on a namespace layer gives Kubernetes clusters administrators more power to control which namespaces can deploy on which nodes, while developers or namespace owners with fine-tuned RBAC will not have the ability to do that by themselves.

ValidatingAdmissionWebhook

For our needed feature to work, we can utilize Kubernetes ValidatingAdmissionWebhook. Basically, Validating Admission Webhook can run as a web server in a pod inside Kubernetes cluster and answer to HTTP calls from our Kubernetes API server either with HTTP status code 200 (OK - validation successful, pods can be deployed) or 403 (Forbidden - validation failed, pods will not be deployed).

Web server with the whole logic can be written in any language we want, for simplicity reasons, this will be demonstrated in NodeJS. Github repo for the whole code can be found here.

Label Kubernetes nodes

Assign critical node role:

kubectl label nodes $NODE_FQDN role=critical

Web server logic

In index.js we have three main parts. First one is where we defined allowed namespaces and restricted node roles:

var allowed_namespaces = ["kube-system"];
var restricted_node_roles = [ "critical", "master" ]

The other part is that when we’re getting JSON payload from Kubernetes API Server, we need to compare allowed_namespaces to restricted_nodes and assign our boolean value to namespace_allowed variable:

  for (var container of object.spec.containers) {
    if (typeof(object.spec.nodeSelector.role) !== "undefined") {
      if (restricted_node_roles.indexOf(object.spec.nodeSelector.role) != -1 && allowed_namespaces.indexOf(namespace) == -1) {
        admissionResponse.status = {
          code: 402
        };
        namespace_allowed = false;
      }
      else {
        namespace_allowed = true;
      }   
    };
  };
  if (namespace_allowed) {
    admissionResponse.allowed = true;
  }

And finally, we should provide a response to Kubernetes API Server with 200 or 403 HTTP status code:

  res.setHeader('Content-Type', 'application/json');
  res.send(JSON.stringify(admissionReview));
  res.status(200).end();

Running ValidatingAdmissionWebhook in Kubernetes

Next, we need to package our NodeJS app to Docker container:

FROM node:alpine

WORKDIR /usr/src/app

COPY package*.json index.js ./
RUN npm install
COPY . .

EXPOSE 8080
CMD [ "npm", "start" ]

Build it, run it and tell our Kubernetes API Server to get permissions from our ValidatingAdmissionWebhook.

namespace-validator.yaml (replace ${URL} either with your Kubernetes service URL of our deployed NodeJS web server or Ingress URL) :

---
apiVersion: admissionregistration.k8s.io/v1beta1
kind: ValidatingWebhookConfiguration
metadata:
  name: namespace-allowance-admission-webhook
webhooks:
  - name: namespace-allowance.example.io
    rules:
      - apiGroups:
          - ""
        apiVersions:
          - v1
        operations:
          - CREATE
        resources:
          - pods
    failurePolicy: Ignore
    clientConfig:
      url: ${URL}
      caBundle: ${CA_BUNDLE_BASE64}
kubectl apply -f namespace-validator.yaml

That’s it!