PaaS Comparison: Cloud Foundry, Microsoft Azure, Google App Engine, Amazon, Heroku, and OpenShift

July 10, 2014 Adam Bloom

While the entire CF Summit rocked and included great talks by SAP, Monsanto, and many others, one of the most popular talks at CF Summit was based on the work done by Dr. Mike Maximilien from IBM Cloud Labs and James Bayer, Pivotal’s Cloud Foundry’s Director of Product Management.

The two shared research to date done to compare Cloud Foundry, Microsoft Azure, Google App Engine, Amazon, Heroku, and OpenShift.

Detail included formalized matrices comparing 50 features as well as qualitative feedback on what works well and not so well. The talk started off explaining how the research started with IBM in 2010—their intent was to help make purchase decisions as well as improvements to PaaS solutions.

The 30 minute video (and slide deck) ultimately covers background, limitations to research validity, methodology, results, insights, and potential next steps.

PaaS-Comparison_Cloud-Foundry_CF-Summit-2014_James-Bayer-Dr-Max-Milien

Dr. Mike Maximilien from IBM Cloud Labs and James Bayer, Pivotal’s Cloud Foundry’s Director of Product Management present the PaaS Comparison

PaaS Comparison—Validity and Methodology

Immediately, Dr. Maximilien underscored how important the limitations and methodology were to explain.

While the talk provided a very good baseline, he explained how it was not a scientific comparison. There are still challenges in terms of marketing or documentation claims versus day to day reality of use. As well, the market was very early and no standard measurement existed. Lastly, the popularity of PaaS created a low signal to noise ratio, making it difficult to compare. The two also recognized their bias with Cloud Foundry and hoped this framework could be used by others to do their own analysis or even improve the methodology. Bayer also explained—at the end of the day, the team wanted to measure the quality of experience (QoE) for developers.

The two cover the comparison in detail. Dr. Maximilien highlighted 50 features across eight areas—workloads, tooling, integration services, SLAs, data stores, programming models, management, and miscellaneous. Fundamentally, the key question was, “To what extent does each PaaS support a feature?” The answers included non, basic, strong, or leading.

PaaS Comparison—Quality of Experience

Bayer added that QoE portion was approached from a user experience perspective—they wanted to know what it subjectively felt like to experience a popular PaaS scenario. After some quantitative input, they chose a Java app on a relational database and inquired about the ability to push, scale, and update an app along with things like automatic healing.

The video and slide deck cover each PaaS (Cloud Foundry, Microsoft Azure, Google App Engine, Amazon, Heroku, and OpenShift) in terms of the 50 features. In addition, it covers a more qualitative view of what works well and not so well.

PaaS Comparison—Insights and Lessons Learned

At the end, they shared the following slide with insights and lessons learned:

PaaS-Comparison_Cloud-Foundry_CF-Summit-2014_Lessons-Learned

Insights and Lessons Learned from the Pivotal Cloud Foundry and IBM PaaS Comparison

Learn More:

About the Author

Biography

More Content by Adam Bloom
Previous
Pivotal CEO Paul Maritz @ Cloud Foundry Summit—Building a Lasting, Valuable Contribution for the World
Pivotal CEO Paul Maritz @ Cloud Foundry Summit—Building a Lasting, Valuable Contribution for the World

In this post, we highlight the video where Pivotal CEO Paul Maritz explains how the Cloud Foundry community...

Next
App Placement in Cloud Foundry Diego: A Classical Optimization Problem
App Placement in Cloud Foundry Diego: A Classical Optimization Problem

In a follow-up to the popular talk at CF Summit discussing how and why the Cloud Foundry team is rewriting ...

Enter curious. Exit smarter.

Register Now