Secure, Multitenant Kubernetes in Minutes: Pivotal Container Service Goes GA

February 12, 2018 Jared Ruckle

Pivotal Container Service (PKS) is now generally available. PKS aims to simplify how enterprises deploy, run and manage Kubernetes on any cloud. Download the release from PivNet. PKS is available as part of Pivotal Cloud Foundry, and as a stand-alone product.

Containers took the IT world by storm. Then, Kubernetes enjoyed a fast rise as a popular way to run and orchestrate containers.

Now every company wants to run containers in production. Many executives are asking: “how do I get Kubernetes to work in my data center? What other features do I need? What about security?”

PKS answers these questions with a feature set tailored to the requirements of enterprises. Here’s what makes PKS a complete solution for production Kubernetes workloads.

Use the PKS CLI to manage Kubernetes clusters.

Pure, Open Source Kubernetes With Constant Compatibility

PKS includes Kubernetes version 1.9.2. Developers use the standard Kubernetes commands. There’s no proprietary extensions to impede your container workflow. And PKS offers constant compatibility with the latest stable version of the Kubernetes project. That means your dev teams always have access to new features.

Most importantly, you can use the PKS API and command line interface (CLI) to create, scale and manage your Kubernetes clusters.

On-Demand Provisioning. PKS gives developers an enterprise-ready Kubernetes cluster in minutes. If your company is wrangling with a homegrown container project, speedy provisioning will be a welcome change.

 

Essential Security Features

Everyone loves containers. But before you go to production, you need security and governance in place. That’s where PKS shines. The product ships with controls like:

Rapid Fixes for CVEs. When CVEs hit, Pivotal posts a fix and proactively notifies customers. From there, your platform engineers can quickly apply the update to Kubernetes clusters without downtime. (Use your build pipeline or the included Operations Manager tool.) Update your systems in hours, not days!

Integration with NSX-T. PKS includes NSX-T network virtualization to programmatically manage and secure software-based virtual networks. NSX-T supports secure, multi-tenant clusters, as well as dynamic load balancing, network micro-segmentation, and policy management. That means InfoSec teams can easily apply and enforce corporate network policies and segment clusters according to their needs.

An Enterprise Container Registry. PKS comes with VMware Harbor, a registry server that stores and distributes Docker images. Large organizations like Harbor because it bolsters security for container images with vulnerability scanning and identity management.

Multitenancy. This is always a thorny issue. Without multitenancy, containers - and their respective network traffic - tend to get jumbled together. This increases the risk of a security breach. (Performance drops as well, due to “noisy neighbors.”) So how does PKS reliably implement multitenancy? How does the product ensure your containers stay isolated and compliant? Two ways: with single cluster and network segmentation, and via multiple clusters.

Let’s start with the single cluster approach. PKS uses NSX-T to secure different Kubernetes namespaces through network segmentation and advanced policies. Each tenant automatically gets its own namespace, isolated subnet, and logical switch. This way, the tenant’s traffic is completely isolated from its neighbors. So when you apply a network policy, NSX-T enforces it for both containers and the underlying VMs. The result: a comprehensive security model.

Second, PKS also allows the creation of multiple Kubernetes clusters directly from the PKS CLI or API. This method offers the highest possible isolation between tenants. Each tenant receives a completely separate cluster. Another benefit: Kubernetes clusters can all be managed using the same set of tools, the PKS CLI and API.

CredHub. This service manages credential generation, storage, and access for the Cloud Foundry ecosystem - including PKS. CredHub also implements comprehensive access and state-change logging, boosting security and audit compliance.

 

Fully Automated Operations

Automation is key to operational efficiency and developer productivity. To this end, PKS makes it easy to deploy, scale, patch, and upgrade the Kubernetes clusters in your system without downtime (thanks BOSH!).

 

Multiple Layers of High Availability (HA)

PKS keeps your containers online and serving traffic with three layers of high availability:

  1. The container layer. PKS relies on its Kubernetes core for re-scheduling failed pods and associated containers as needed.
  2. The process layer. PKS monitors all Kubernetes processes, including those running in Master nodes. It will restart any failed process automatically.
  3. The VM layer. Of course, PKS can run atop virtual machines. The product tracks the health of these, too. It will recover any VMs that fail automatically.

Now, let’s take a closer look at the product.

 

PKS: What’s Inside

Kubernetes 1.9.2. PKS always includes the latest stable version of Kubernetes. (Read the release notes for what’s new in 1.9.) Kubernetes 1.10 is due out in March, and we’ll be ready to support the new release shortly after its delivery. Another benefit: the universe of Kubernetes tooling and add-ons are available for use with PKS.

PKS API and CLI, the control plane for the service. After an operator has installed PKS, developers can use the PKS CLI (and API) to do the following:

  • create-cluster Creates a kubernetes cluster, requires cluster name and an external host name

  • delete-cluster Deletes a kubernetes cluster, requires cluster name

  • get-credentials Allows you to connect to a cluster and use kubectl

  • help Help about any command

  • clusters Show all clusters created with PKS

  • plans View the preconfigured plans available

  • resize Increases the number of worker nodes for a cluster

  • cluster View the details of the cluster

  • login Login to PKS

  • logout Logs user out of the PKS API

     

A container registry. VMware Harbor is an enterprise-class registry server that stores and distributes container images. Harbor allows you to store and manage images for use with Pivotal Container Service (PKS). Harbor provides vulnerability scanning and identity management for Docker images, and supports multiple registries. Deploying a registry alongside the PKS environment also improves image transfer efficiency.

NSX-T. Kubernetes flat networking on its own isn’t enough for production deployments. So we teamed up with VMware to add modern software-defined networking and dynamic load balancing. Administrators can instantly create networks, configure network microsegmentation, define network policies, segregate different tenants, and much more. NSX-T helps you expose your deployments too with dynamic load balancers.

Three VMs for the base installation. The footprint for PKS is very small. (That’s part of the reason why it’s so fast to setup.) There is one VM for the Pivotal Ops Manager; another VM is the PKS control plane, and the third is the BOSH Director. Once these VMs are in place, you can use the PKS control plane to provision Kubernetes clusters on-demand using BOSH. Clusters will have the familiar etcd, master, and worker components. Users can choose the size of each cluster during provisioning. Harbor and NSX-T each require one additional VM.

 

Cluster Lifecycle: Using PKS

Let’s take a look at how an enterprise team would use PKS to test and run Elasticsearch. The steps might look something like this.

  1. Claire, a platform operator, performs an initial install of PKS. (PKS is deployed atop NSX-T, simplifying network configuration).

  2. Reggie, an engineer, provisions a new Kubernetes cluster in PKS, for a test scenario. He sizes the cluster to accommodate the test.

  3. Reggie then pulls an Elasticsearch container image from Harbor (the container registry) to the new Kubernetes cluster. He also deploys standard Kubernetes network policies and a load balancer, exposing the container for external access. NSX-T notices the objects deployed in Kubernetes, and proceeds to dynamically deploy the necessary network elements.

  4. Kim, another engineer, deploys a product catalog microservice, binding it to the existing Elasticsearch container. She loads test data and runs integration tests.

  5. Reggie observes that load is increasing, and scales the Elasticsearch cluster due to increased load.

  6. Claire applies a patch release for a recent CVE to the Kubernetes cluster, with no downtime to the running workloads.

  7. Claire provisions a new, larger Kubernetes cluster in PKS, for production use.

  8. Reggie and Kim migrate the current workloads to the new production cluster. They can simply point kubectl to the new cluster endpoint.

  9. Claire scales down the test cluster.

Your engineers can focus on getting the job done. All the messy details of fiddling with the guts of Kubernetes are abstracted away; PKS does all of that for you.

 

PKS is Enterprise-Ready Kubernetes

As you evaluate your enterprise software portfolio, you’re going to have app platforms, containers, and functions. So of course you’re going to have plenty of Kubernetes clusters in the mix. Your job now is to give your team a rock-solid, secure Kubernetes “dial tone.” That’s Pivotal Container Service. And once you have that, the sky’s the limit for what you can build.

Ready to give your teams PKS? Download the bits here. Check out the documentation to learn how to get started. And be sure to read VMware's take on the GA of PKS.

About the Author

Jared Ruckle

Jared works in product at Pivotal.

Follow on Twitter More Content by Jared Ruckle
Previous
Using VMware’s Harbor with PKS (and Why Kubernetes Needs a Container Registry)
Using VMware’s Harbor with PKS (and Why Kubernetes Needs a Container Registry)

Container registries add important security features to Kubernetes. This post details how Harbor, part of P...

Next
How Cloud Foundry Container Runtime Tackles Both Day 1 & Day 2 Operations for Production Kubernetes
How Cloud Foundry Container Runtime Tackles Both Day 1 & Day 2 Operations for Production Kubernetes