AWS GuardDuty Hands-On Lab: Securing Your EKS Cluster

AWS GuardDuty is a threat detection service that continuously monitors for malicious activity and unauthorized behavior. In this lab, we'll focus on enhancing the security of your Amazon EKS (Elastic Kubernetes Service) cluster by enabling GuardDuty and simulating suspicious activities to see how GuardDuty detects and reports them.

Prerequisites

  • EKS Cluster: An existing Amazon EKS cluster (we assume it's already set up).
  • AWS CLI: Installed and configured with appropriate permissions.
  • kubectl: Installed and configured to interact with your EKS cluster.
  • Helm: Installed for deploying applications to Kubernetes.

Note: We'll skip cluster creation and tool installations to focus on GuardDuty.

Lab Overview

  1. Enable GuardDuty with EKS Runtime Monitoring.
  2. Deploy a Suspicious Pod to trigger GuardDuty findings.
  3. Verify GuardDuty Alerts in the AWS Console.
  4. Clean Up all resources.

Hands on Lab

  • Navigate to the EKS Directory & set the region for running the guardduty lab.

Warning: The script searches for clusters starting with peachycloudsecurity in us-east-1 and us-west-2. If clusters exist in both regions, the script may fail. In this case, manually set the region.

cd /workspaces/ecr_eks_security_masterclass_public/eks/

# Check if a cluster exists in us-east-1
cluster_in_us_east=$(aws eks list-clusters --region us-east-1 --query 'clusters[?starts_with(@, `peachycloudsecurity`)] | [0]' --output text)

# If no cluster is found in us-east-1, check us-west-2
if [ "$cluster_in_us_east" == "None" ]; then
    cluster_in_us_west=$(aws eks list-clusters --region us-west-2 --query 'clusters[?starts_with(@, `peachycloudsecurity`)] | [0]' --output text)
    
    # If a cluster is found in us-west-2, set region to us-west-2
    if [ "$cluster_in_us_west" != "None" ]; then
        export AWS_DEFAULT_REGION="us-west-2"
    else
        echo "No cluster found in either region."
    fi
else
    # If a cluster is found in us-east-1, set region to us-east-1
    export AWS_DEFAULT_REGION="us-east-1"
fi

# Show the selected region
echo $AWS_DEFAULT_REGION
  • Enable GuardDuty and its EKS-specific EKS Runtime Monitoring features using the AWS CLI.
# Create a GuardDuty detector
DETECTOR_ID=$(aws guardduty create-detector --enable --features '[{"Name" : "RUNTIME_MONITORING", "Status" : "ENABLED"}]' --query "DetectorId" --output text)

# Enable EKS Audit Logs and Runtime Monitoring
aws guardduty update-detector \
  --detector-id $DETECTOR_ID \
  --features '[{
    "Name": "EKS_AUDIT_LOGS", "Status": "ENABLED"},
    {
      "Name": "EKS_RUNTIME_MONITORING",
      "Status": "ENABLED",
      "AdditionalConfiguration": [{"Name": "EKS_ADDON_MANAGEMENT", "Status": "ENABLED"}]
    }
  ]'

Wait for about 10 minutes to allow GuardDuty to deploy the necessary agents to your EKS cluster.

  • Check if the GuardDuty agents are running in your cluster.

You should see pods like guardduty-agent-xxxx running.

kubectl get pods -n amazon-guardduty

kubectl wait --for=condition=Ready pod --all --namespace=amazon-guardduty --timeout=600s
  • Verify guardDuty coverage status.
# Check if GuardDuty coverage is healthy
DETECTOR_ID=$(aws guardduty list-detectors --query "DetectorIds[0]" --output text)

aws guardduty list-coverage --detector-id $DETECTOR_ID --query "Resources" --output json

Look for "CoverageStatus": "HEALTHY" in the output to confirm that GuardDuty is actively monitoring your EKS cluster.

  • Create the Suspicious Pod Manifest, use cat <<EOF to create the file.
cat <<EOF > suspicious-pod.yaml
apiVersion: v1
kind: Pod
metadata:
  name: suspicious-pod
spec:
  containers:
  - name: suspicious-container
    image: ubuntu
    command: ["/bin/sh", "-c", "sleep infinity"]
    securityContext:
      privileged: true
    volumeMounts:
    - mountPath: /host-root
      name: host-root
  volumes:
  - name: host-root
    hostPath:
      path: /root
  restartPolicy: Never
EOF
  • Deploy the Pod.

This pod is privileged and mounts the host's /root directory, which is a security risk.

kubectl apply -f suspicious-pod.yaml
sleep 15

Wait a 5 minutes for GuardDuty to detect the suspicious activity.

  • Check GuardDuty Findings using AWS CLI.
# Get the Detector ID
DETECTOR_ID=$(aws guardduty list-detectors --query 'DetectorIds[0]' --output text)

# List GuardDuty findings
aws guardduty list-findings --detector-id $DETECTOR_ID --query 'FindingIds' --output text
  • Get Details of the Findings:
# Get detailed information about the findings
FINDING_IDS=$(aws guardduty list-findings --detector-id $DETECTOR_ID --output text)

# Get detailed information about the findings
aws guardduty get-findings --detector-id $DETECTOR_ID --finding-ids $FINDING_IDS --query 'Findings[?Resource.EksClusterDetails.Name | starts_with(@, `peachycloudsecurity-`)]'

Verify from Console (Optional)

  1. Log in to the AWS GuardDuty Console.

  2. Navigate to Findings.

  3. Look for findings related to EKS, such as:

    • Runtime behavior alert observed in Amazon EKS cluster
    • Highly permissive security context detected

These findings indicate that GuardDuty has detected the suspicious pod.

Clean Up Resources

Delete the Pods:

kubectl delete pod suspicious-pod

Disable GuardDuty:

DETECTOR_ID=$(aws guardduty list-detectors --query "DetectorIds[0]" --output text)

aws guardduty delete-detector --detector-id $DETECTOR_ID

Additional Resources