Google recently received helpful information through the Vulnerability Rewards Program for Authorized Networks and Cloud Run/Functions on Google Kubernetes Engine (GKE). Based on that information, Google updated their product documentation and prioritized a plan to make engineering changes to GKE to restrict access to only GKE-related services. Those changes will roll out automatically to over 99% of their GKE customers by late August, and will proactively reach out to the remaining customers to work on migration issues together.
Google’s existing firewall rules allow the Kubernetes API server’s IP address to be reachable from the Cloud Run and Cloud Functions services. However, even with this access, calls to the API still need to be authenticated and authorized using either Google Identity and Access Management or GKE role-based access control. To further improve security, they will soon limit that access to GKE-related services and block access from Cloud Run and Cloud Functions.
Google plan to take the following steps:
- Migrate core GKE services that communicate with the API server onto a dedicated set of IP addresses.
- Notify customers that currently rely on being able to communicate from other cloud services to the Kubernetes API server that the access will be removed (approximately 1% of clusters). Google will provide instructions to migrate to a new solution and allow list existing customer usage to give them time to migrate.
- Remove the existing firewall rule and introduce a targeted rule allowing only the dedicated set of IP addresses belonging to the core GKE services.
Once these steps are complete, 99% of private clusters won’t be accessible from Cloud Run or Cloud Functions, with no action required from those customers. The remaining 1% will migrate on their own timeline as those customers need time to move their access to new solutions. Public clusters (where nodes have public IPs) will continue to be accessible from Google Cloud IPs as this is necessary for those nodes to communicate with the API server.
Google look forward to continuing to work with all their partners and customers, and the research community, to advance security for everyone.