Lab: Disallowing the latest Tag in Container Images Using Kyverno and CEL on Amazon EKS

Prerequisites

  • Amazon EKS Cluster: Already running and kubectl is configured to interact with it.
  • kubectl: Installed and configured to connect to your EKS cluster.
  • Helm: Installed on your local machine.

Part:1 Introduction to Common Expression Language (CEL)

  • What is CEL?

    • CEL is a simple, readable expression language used to write conditions and validations in code and configurations. In Kubernetes, it's used to define rules for resource validation.

  • CEL Playground

    • Practice writing CEL expressions using the online tool: CEL Playground.
Basic CEL Expressions

Here are four basic examples to illustrate how CEL works.

  • Example 1: Arithmetic Operations
// Expression
1 + 2 * 3 - 4 / 2

// Evaluates to
5.0
  • Example 2: String Operations
// Expression
"Hello, " + "World!"

// Evaluates to
"Hello, World!"
  • Example 3: Logical Operations
// Expression
true && !false

// Evaluates to
true
  • Example 4: Conditional Expressions
// Expression
size([1, 2, 3]) == 3 ? "Three elements" : "Not three"

// Evaluates to
"Three elements"
  • Applying CEL in Kubernetes Policies

Here's how CEL is used in a Kyverno policy to disallow the use of the latest tag in container images.

apiVersion: kyverno.io/v1
kind: ClusterPolicy
metadata:
  name: disallow-latest-tag
spec:
  validationFailureAction: Enforce
  rules:
  - name: disallow-latest-tag
    match:
      any:
      - resources:
          kinds:
          - Pod
    validate:
      cel:
        expressions:
          - expression: "object.spec.containers.all(c, !c.image.matches('.*:latest$'))"
            message: "Using the 'latest' tag is not allowed."
  • Explanation
    • object: The resource being evaluated (e.g., a Pod).
    • object.spec.containers: List of containers in the Pod.
    • all(c, condition): Checks that the condition is true for all containers c.
    • !c.image.matches('.*:latest$'): Ensures the image does not end with :latest.

Part:2 Hands on Lab

  • Navigate to the EKS Directory:
cd /workspaces/ecr_eks_security_masterclass_public/eks/
  • Ensure that kubectl can communicate with your EKS cluster.
# Verify cluster nodes
kubectl get nodes
  • Install Kyverno in the kyverno Namespace using Helm.
# Add Kyverno Helm repository
helm repo add kyverno https://kyverno.github.io/kyverno/

# Update Helm repositories
helm repo update

# Create Kyverno Namespace
kubectl create namespace kyverno

# Install Kyverno using Helm
helm install kyverno kyverno/kyverno -n kyverno
  • Check that the Kyverno pod is running.
# Check Kyverno pods
kubectl get pods -n kyverno
  • Understand CEL Basics in Kyverno

Common Expression Language (CEL) allows you to write expressions for custom validations in Kyverno policies.

  • Variables:

    • object: The resource being validated.
    • namespaceObject: The Namespace of the resource.

  • Expressions: Use CEL to define conditions that resources must meet.

  • Now, create a policy that blocks the creation of Pods using container images tagged with latest.

  • Use the cat << EOF > filename method to create the policy file.

# Create the policy file
cat << EOF > disallow-latest-tag.yaml
apiVersion: kyverno.io/v1
kind: ClusterPolicy
metadata:
  name: disallow-latest-tag
spec:
  validationFailureAction: Enforce
  background: false
  rules:
  - name: disallow-latest-tag
    match:
      any:
      - resources:
          kinds:
          - Pod
    validate:
      cel:
        expressions:
          - expression: "object.spec.containers.all(c, !c.image.matches('.*:latest$'))"
            message: "Using the 'latest' tag is not allowed."
EOF
  • Apply the policy using kubectl.
# Apply the policy
kubectl apply -f disallow-latest-tag.yaml
  • Now to validate, create a Pod definition file that uses the latest tag.
# Create the Pod definition file
cat << EOF > pod-with-latest.yaml
apiVersion: v1
kind: Pod
metadata:
  name: test-pod-latest
spec:
  containers:
  - name: nginx
    image: nginx:latest
EOF
  • Attempt to create the Pod.
# Apply the Pod definition
kubectl apply -f pod-with-latest.yaml
  • Expected Result will be error message indicating that the Pod creation is blocked by the Kyverno policy.
Error from server: error when creating "pod-with-latest.yaml": admission webhook "validate.kyverno.svc-fail" denied the request:

resource Pod/default/test-pod-latest was blocked due to the following policies

disallow-latest-tag:
  disallow-latest-tag: Using the 'latest' tag is not allowed.
  • Create a Pod definition file that uses a specific version tag.
# Create the Pod definition file
cat << EOF > pod-with-version.yaml
apiVersion: v1
kind: Pod
metadata:
  name: test-pod
spec:
  containers:
  - name: nginx
    image: nginx:1.19
EOF
  • Apply the Pod definition.
# Apply the Pod definition
kubectl apply -f pod-with-version.yaml
  • Expected Result will show that the Pod should be created successfully.
pod/test-pod created
  • Verify the Pod is running:
# Get the list of Pods
kubectl get pods

Clean Up

  • Delete the Pods and policy created during this lab.
# Delete the Kyverno policy
kubectl delete -f disallow-latest-tag.yaml

# Delete the Pods
kubectl delete -f pod-with-latest.yaml
kubectl delete -f pod-with-version.yaml


# (Optional) Delete the Kyverno Namespace and release
helm uninstall kyverno -n kyverno
kubectl delete namespace kyverno