Kubernetes Security Project

This project is a demonstration of how to secure a Kubernetes cluster using Linkerd, Kyverno and Falco. It consists of a simple app making requests to two services. The services are secured using mTLS from Linkerd and the app is secured using Kyverno policies. The cluster is monitored using Falco.

Services are exposed using Contour Gateway Provisioner.

You can access the app on <LoadBalancer IP>/aggregator.

Requirements

• Kubernetes cluster with LoadBalancer (I use kind with cloud-provider-kind)
• kubectl
• Linkerd CLI
• helm

Install

Prepare the cluster

Note: If you are using kind, you can use the k8s/kind-config.yaml file to create the cluster.

# Install Linkerd
linkerd install --crds | kubectl apply -f -
linkerd install | kubectl apply -f -
linkerd check # Check for the install to be completed (can take up to a few minutes)
linkerd viz install | kubectl apply -f - # install the on-cluster metrics stack
linkerd viz check

# Install Contour, Kyverno and Falco
kubectl apply --server-side -k k8s/base

# Install Falco
helm repo add falcosecurity https://falcosecurity.github.io/charts
helm repo update
helm upgrade --install falco falcosecurity/falco \
  --namespace falco \
  --create-namespace \
  --set tty=true \
  --set falcosidekick.enabled=true \
  --set falcosidekick.webui.enabled=true

# Configure Gateway, Policies, etc.
kubectl apply -k k8s/config

Deploy the app

# Deploy the app
kubectl apply -k k8s/app

# Test the deployment
linkerd -n app check --proxy
curl $(kubectl get gateway -o jsonpath='{.items[0].status.addresses[0].value}')/aggregator

Policies

In order to enforce policies, Kyverno is used. The policies are defined in k8s/config/kyverno. The policies are:

• add-linkerd-mesh-injection: This policy adds the linkerd.io/inject: ingress annotation to all namespaces. This is required for mTLS to work. You can test this by deploying the app and running linkerd -n app check --proxy.

• add-default-security-context: This policy adds the default security context to all pods. You can test this by deploying the app. It should have the default security context.

• prevent-bare-pods: This policy prevents pods from being deployed without a deployment. You can test this by deploying the following pod: kubectl run test-pod --image=ghcr.io/noetarbouriech/k8s-security/aggregator:latest. It should be blocked.

• restrict-node-port: This policy prevents NodePort services from being deployed. You can test this by deploying the following service: kubectl create service nodeport test-service --tcp=80:80. It should be blocked.

• restrict-image-registries: This policy allows only images from the specific registries. You can test this by trying to deploy the following deployment: kubectl create deployment nginx --image=nginx. It should be blocked.

Runtime Security

The runtime security is provided by Falco. It checks for security violations in the cluster. The default set of rules is used. You can test the runtime security by running the following command:

# Do it before the installing the config or disable the policy that blockes this deployment
kubectl create deployment nginx --image=nginx
kubectl exec -it $(kubectl get pods --selector=app=nginx -o name) -- cat /etc/shadow

It should trigger a Falco alert. You can see the alerts in the Falco UI which is exposed on <LoadBalancer IP>/.

Service Mesh

The service mesh is provided by Linkerd. It provides mTLS, observability, etc.

In order to use the service mesh, you need to add the linkerd.io/inject: ingress annotation to the namespace. This is done by the add-linkerd-mesh-injection policy in Kyverno.

I also made a custom template for Contour Gateways, so that the Linkerd proxy is injected into the envoy pods that are created dynamically.

You can check that the service mesh is working by running the following command:

linkerd -n app check --proxy

And you can check the montoring dashboard by running:

linkerd viz dashboard &