How We Built AppFog Using Cloud Foundry

The value proposition of using the cloud is that “things will be easy this way.” Building and operating large scale clouds, however, is rarely easy. As Mark Lucovsky said in his recent interview, “Who said large scale distributed systems are simple?” While solutions like Cloud Foundry make building core cloud technology less daunting, it’s still not easy.

In this guest blog, Lucas Carlson, Founder and CEO of AppFog, illustrates the story of how AppFog was built using Cloud Foundry open PaaS technology and what they learned along the way.

Core Cloud Technology is Hard

Running a simple Cloud Foundry-based service is not very hard. Check out Micro Cloud Foundry^TM.

Building one that is suitable for production and enterprise workloads is fairly hard.

Building a Cloud Foundry-based service with one API end-point to run in 6+ different availability zones on 4+ independent infrastructure vendors and that is suitable for production and enterprise workloads, at scale and in general availability? Now that is really hard.

Hitting the Ground Running with Cloud Foundry

As you can imagine, ever since AppFog announced GA, people have been asking how we did it. How did we figure out how to scale up a public cloud offering for Cloud Foundry that delivers one interoperable interface across multiple clouds, multiple availability zones and multiple vendors?

First: The Cloud Foundry codebase is really very good. In choosing to build using Cloud Foundry we eliminated a number of the potential serious challenges and major time sinks likely with this sort of project.

Second: The leverage that we were able to gain from the additional tools, code and services from within the Cloud Foundry ecosystem dramatically accelerated our development.

Third: Having a non-Cloud Foundry-based public PaaS already in GA and already at scale was a massive boost for us. We had the operational experience in-house needed to pull off a great PaaS.

AppFog’s ops experience was invaluable for us. Although PaaS is closely tied to the NoOps movement, NoOps doesn’t mean no ops, it only means no ops for the developer. As anyone who has run Cloud Foundry knows, a lot of ops is necessary and essential to running PaaS.

And most importantly, we had a vibrant community of tens of thousands of developers (the users of PHP Fog) that we could consult to help us define, test, and evolve AppFog through private beta to Public Beta and now to GA.

We Listened and Delivered

It was this community that helped us figure out what AppFog at GA would need to be, how it would need to work, and what our pricing would be. This was what devs asked us for:

• The ability to easily scale apps across dozens or hundreds of load-balanced instances, even on the free plan
• Must run apps on the fastest servers available in the infrastructure (yes, we mean, for example, M2.4XL)
• Must be able to run apps across multiple infrastructures (Rackspace, HP, AWS, Azure–and with no price difference)
• Must simplify pricing. No more weird packages and bundles and names. Make it simple and affordable and most of all make it fair.
• True interoperability in the cloud, including single-click, zero-code migrations between cloud vendors, eliminating all traces of vendor lock-in
• Target of 30-second deployments to AWS, Rackspace, HP, Azure, etc.
• Support for any and all languages in the Cloud Foundry codebase
• Support for any and all relational and NoSQL data stores**

This is a daunting list. But it’s what devs wanted.

So let’s get into the nitty gritty… how did we actually do this?

How We Built a PaaS Around Cloud Foundry

This is a complex story. Fortunately, a few months ago, Jeremy Voorhis gave a talk at QCon in London describing how AppFog built a commercial system by using the Cloud Foundry OSS bits alongside our own set of custom extensions and enhancements. This is far and away the best place to start for understanding the Cloud Foundry-to-AppFog evolution. The slides for the talk can be found here and the video here.

We won’t go over everything in Jeremy’s hour-long talk here, but we’d like to cover some of the main points.

First, Jeremy clears up some of the confusion surrounding the signifier “Cloud Foundry.” On one hand, CloudFoundry.com is a hosted PaaS offering from VMware that runs on the vSphere virtualization platform based on the open source project found on CloudFoundry.org. VMware also has open sourced the tooling it uses to manage CloudFoundry.com–Cloud Foundry BOSH–and exposes the exact BOSH releases used live in production on their hosted service.

We do not run on the hosted VMware service but instead we are built from the same open source Cloud Foundry project. This open source distributed system enabling teams to manage and deploy apps and services over their lifecycle in the cloud is the core of the AppFog service.

AppFog is unique within the current Cloud Foundry ecosystem because we have built an infrastructure-agnostic Cloud Foundry service.

The beauty of Cloud Foundry is that it can, in principle, be implemented in conjunction with infrastructure offerings ranging from an OpenStack-based IaaS like Rackspace or HP to AWS, Joyent, Citrix, and beyond. This inherent multi-cloud portability is why the Cloud Foundry project was so inspiring to us.

For us, the ability to deploy and redeploy across infrastructures within the lifecycle of an application is the key to building a next-generation PaaS. It also, we think, fulfills the underlying mission of PaaS to: (a) sharpen product team focus in development; (b) produce a shorter feedback loop within app lifecycles, and; (c) provide the possibility of near-instant horizontal scalability.

Cloud Foundry’s Role in the Evolution of Cloud Computing

Even for those who are familiar with the space of cloud computing in general, it’s important to review historical context. For us at AppFog, Cloud Foundry has provided some of the crucial building blocks to assembling the first next-generation PaaS. But what do we mean by that?

First, it’s important to get some background. PaaS in its initial forms was enabled by a number of historical breakthroughs such as:

• The rise of data centers via companies like Rackspace in the 1990s
• The emergence of server virtualization via VMware in the early ‘00s
• The introduction of APIs to virtualized resources via AWS in the mid ‘00s
• The more recent rise of the DevOps paradigm with their associated and derived tools like Puppet and Chef.

But this trajectory really began coming to fruition in just the past two years. Tools like Cloud Foundry and OpenStack have emerged that enable full application lifecycle management. What this means is that the “platform” in “platform as a service” can now encompass all aspects of the application lifecycle outside of development. Jeremy refers to this as “NoOps” in the talk. What it means is emphatically not that no one does Ops work anymore. Instead, it means that responsibility for Ops work is passed on to the platform layer and remains there for most if not all of the app’s lifecycle.

Without a comprehensive tool like Cloud Foundry, we never would have been able to overcome the gap between (a) first-generation PaaS platforms, which we were essentially little more than gateways for (unwieldy) infrastructure management, and (b) next-generation PaaS like AppFog, which provide a far more comprehensive set of user benefits.

AppFog’s Modifications of Cloud Foundry

AppFog began as an effort to bring a great user interface and multi-cloud experience to the hosted Cloud Foundry ecosystem. We saw a number of stumbling blocks standing between the Cloud Foundry codebase and a full-fledged PaaS offering. The current Cloudfoundry.org project has CLI and various IDE integrations, but our vision included the full user experience, including fast sign-up, a web console, pre-built example applications that became what we call Jumpstarts, add-ons, and more.

While some Cloud Foundry users are indeed quite comfortable operating the system from the command line, Eclipse or the Spring Tool Suite (just as many AppFog users do), we also wanted to provide users with an elegant and intuitive console from which they could manage all of their apps. An example of the AppFog console is below: