A Pivotal CF Field Success Story: Installation, User-Provided Services, and Live Capacity Updates

I’ve had the privilege of visiting a few of Pivotal’s early access customers to help them install our Pivotal CF™ PaaS solution, which is powered by Cloud Foundry technology. I’d like to share a recent success story and highlight three benefits from Pivotal CF that were very well received.

This customer, like many we are working with, is in the process of planning a next generation application and data analytics platform focused on PaaS as a strategic centerpiece. We were engaged for a three-day proof of concept (POC) engagement, the focus of which was three-fold:

1. Install Pivotal CF in a newly created VMware vSphere environment.
2. Push an application which both ingested and consumed data to/from existing Hadoop and Solr environments. The application was to take advantage of Cloud Foundry service binding.
3. Demonstrate the live capacity updates and application instance scaling capabilities of the platform.

Let’s walk through each of these steps and see how Pivotal CF demonstrates the power of an end-to-end turnkey PaaS solution!

Installation

Pivotal CF is shipped as an OVA, or Open Virtual Appliance, which can be imported into a running VMware vSphere environment. After booting the resulting virtual machine, a web user interface guides you through the installation process. The first step was to assemble the following information:

1. The VMware vCenter IP address and a set of administrator credentials
2. The target vSphere Datacenter, Cluster, Network, and Datastore
3. An IP range in CIDR notation, along with any protected subranges, DNS, and Gateway addresses
4. A pool of NTP servers

Finally, we needed to select an IP address for the CF Router, and set up a wildcard DNS entry that would resolve to that IP address.

Step two? Enter this information into the wizard-like web forms.

Step three? Click Install. Approximately ninety minutes later, we were able to target a running installation of Pivotal CF and deploy test applications. Contrast this with the getting started experience of deploying CF with BOSH directly, which requires the assembly of the same information, but also requires the tedious and potentially error-prone construction of a deployment manifest, requiring several hundred lines of YAML.

It’s incredibly easy to make a mistake while authoring the manifest, and while much of the manifest is validated by BOSH, an extra layer of validation of all user inputs is baked in to the Pivotal CF installer, and the remainder of the manifest is tested as part of Pivotal CF’s continuous integration pipeline. We’ll discuss further value of this validation below.

Want to see it for yourself? Check out this video from VMworld San Francisco 2013:

User-Provided Services

One of the well advertised features of Pivotal CF is its ability to install, manage, and integrate with a host of data services, including:

1. Pivotal HD
2. RabbitMQ
3. MySQL
4. Pivotal Analytics

Developers building applications can use the familar means of cf create-service and cf bind-service commands to interact with any of these data services. What’s not as well known is the ability to target any other existing services in the environment using user-provided services, a feature added to Cloud Foundry this summer. User-provided services became the cornerstone of migrating the customer’s existing application from its existing environment to Pivotal CF.

This application used Apache HBase as one of its data services. In order to connect to HBase, it depended on the existence of a file, /etc/hbase/conf/hbase-site.xml, to obtain the proper connection details. It would then utilize this file in creating an instance of a customer-authored extension to org.apache.hadoop.conf.Configuration, which would in turn be used to create an instance of org.apache.hadoop.hbase.client.HTablePool.

This file will not be available on the filesystem presented by CF to the running application, so we needed to take the contents of this file and transfer them to a user-provided service, which would then be bound to the application dynamically as an environment variable at runtime. Here are the steps we followed:

1. Run cf create-service while targeting the desired organization and space.
2. Choose user-provided.
3. Assign a name (e.g. hbase-dev).
4. Assign each of the property names in hbase-site.xml as parameters (e.g. hbase.rootdir would become hbase_rootdir) in a comma-separated list.
5. When prompted for values for each parameter, specify the value found in the file (e.g. hbase_rootdir> hdfs://nameservice1/hbase).
6. Run cf bind-service to bind the application to hbase-dev.

The result, on application start, will be an additional service added to the JSON blob presented in the environment variable VCAP_SERVICES. Here’s an example of what that might look like:

{
   "name": "hdfs-dev",
   "label": "user-provided",
   "tags":[],
   "credentials": {
       "hbase_rootdir" : "hdfs://nameservice1/hbase",
       // ...
       "hbase_zookeeper_quorum" : "zk1.foo.com,zk2.foo.com,zk3.foo.com"
   }
}

The final step was to write a basic Spring Bean factory that would:

1. Retrieve the value of VCAP_SERVICES from the environment.
2. Parse the JSON.
3. Create the HTablePool bean.

That’s it. We rebuilt and repushed the application and, voilà, HBase access! The coding could have been made even simpler by writing basic extensions to the recently announced Spring Cloud library.

We are employing the same techniques to connect to all of the external services required by this application, including existing HDFS, Solr, and WAM services.

Live Capacity Updates

One of the great benefits of BOSH is the ability to update a running system without suffering any downtime. This is accomplished by employing canary-style deployments, where by a subset of each managed component is removed from service, updated, and if the update is successful, returned to service. If the update succeeds, another subset is updated as BOSH “rolls through” the system. If the update fails, the failed components are left in their failure state (for troubleshooting purposes) while the remaining healthy components continue to serve load.

This style of update allows a CF operator to easily add and remove application hosting capacity from a running system. When adding capacity, BOSH simply spins up additional Execution Agent VM’s, which then advertise themselves via NATS as being ready to serve application workloads. When removing (or updating) capacity, BOSH first triggers a “drain” event on the Execution Agent to be removed. This drain event triggers an “evacuation” of all application instances running on that Execution Agent to another Execution Agent. Once this evacuation is complete, the Execution Agent is either removed or updated. In either case, end user applications do not suffer downtime.

The Pivotal CF installer takes this to another level by providing a web form allowing an operator to scale components both vertically (CPU, RAM, Disk) and horizontally (number of instances). Simply update and save the form, and then click “Update” on the home page. This results in the installer application updating the deployment manifest followed by a “bosh deploy.” You can see this in action at the 3:50 mark in the embedded video. Here’s an updated screenshot showing the form as of our latest beta release:

Pivotal CF Runtime Resource Sizing

The ease with which platform capacity can be managed enables extreme agility in operations. One can imagine horizontally scaling a vSphere cluster by adding additional ESXi nodes and storage capacity and then allowing a running CF instance to expand across the newly added resources with a button click!

To put this agility into perspective, we decided “on the fly” during a demo session with the customer to demo scaling up the capacity. We visited the web form, increased the number of Execution Agent’s to 2, clicked “Save” and then “Update.” We were greeted with a “friendly” red error message:

Unable to allocate enough IPs to complete deployment

The Pivotal CF Installer was kind enough to validate that we had enough IP addresses left in the allocation we provided to it. An administrator in the room allocated another IP address for us. We updated the reserved ranges in our network configuration, and again clicked “Save” and then “Update.” A couple of minutes later we had a new Execution Agent running. Adding to the excitement in the room, one administrator happened to be watching the vCenter console on his laptop during the update and exclaimed “Look! There it is spinning up the VM!”

Summary

As you can see for yourself, Pivotal CF provides a turnkey experience of installing, managing, and using a private, on-premises instance of Cloud Foundry. Giving CF deployed applications access to all of your existing in-house data services is also quite simple, and allows developers to consume these services the same way they would those services which are managed by CF itself. And these are only a few of the features of Pivotal CF that are exciting to our early access customers! Want to learn more? Reach out to us at http://gopivotal.com/about-pivotal/contact-us.

Matt Stine @mstine

Community Engineer, Cloud Foundry

About the Author

Biography