Kubernetes cluster under Crypto Mining Threat

Microsoft has published a report today detailing a never-before-seen series of attacks against Kubeflow, a toolkit for running machine learning (ML) operations on top of Kubernetes clusters.

The attacks have been going on since April this year, and Microsoft says its end-goal has been to install a cryptocurrency miner on Kubernetes clusters running Kubeflow instances exposed to the internet.

But while the number of hijacked clusters is small in comparison to previous Kubernetes attacks, the profits for crooks and the financial losses to server owners are most likely much higher than other attacks seen before.

“Nodes that are used for ML tasks are often relatively powerful, and in some cases include GPUs,”.

“This fact makes Kubernetes clusters that are used for ML tasks a perfect target for crypto mining campaigns, which was the aim of this attack.”

As it learned more from its investigation into the early attacks, Microsoft now says it believes the most likely point of entry for the attacks are misconfigured Kubeflow instances.

Microsoft said that Kubeflow admins most likely changed the Kubeflow default settings and exposed the toolkit’s admin panel on the internet. By default, the Kubeflow management panel is exposed only internally and accessible from inside the Kubernetes cluster.

Kubernetes threat matrix

HOW TO DETECT HACKED KUBEFLOWS

In case server administrators may want to investigate their clusters for any hacked Kubeflow instances, Weizman provided the following steps.

• Verify that the malicious container is not deployed in the cluster. The following command can help you to check it:

kubectl get pods –all-namespaces -o jsonpath=”{.items[*].spec.containers[*].image}”  | grep -i ddsfdfsaadfs