Dell Technologies: Improving Operations and Developer Productivity with Pivotal Cloud Foundry

September 27, 2017

Dell Technologies is a global provider of information technology solutions and services made up of several distinct companies, including Dell, Dell EMC, Virtustream, VMware, Pivotal, RSA and SecureWorks. Founded in September 2016 with Dell’s acquisition of EMC, Dell Technologies employs over 138,000 people around the world serving everyone from consumers to large multi-national enterprises.

Dell EMC is one of seven companies that makes up Dell Technologies. Its mission is simple yet daunting: To help its enterprise customers accelerate digital transformation efforts while storing, managing, securing and analyze their most valuable assets - data and information. To this end, the company offers enterprise storage and infrastructure solutions along with commercial support services to help customers make the most of their investments.

Challenge

Just as its customers were eager to transform themselves for the digital era, so was Dell EMC. The company is well known for its enterprise support services, but by 2012 Dell EMC executives recognized the need to improve its application development capabilities. The goal was provide the company’s talented developers with an equally impressive platform that allowed them to spend more time building great software to support customers and less time provisioning hardware and dealing with other infrastructure concerns. The company also wanted to reduce the burden on its operations team and free it up to work on building new capabilities for the business.
Being a leading enterprise technology company, Dell EMC’s internal IT department decided to build the platform itself. The team set to work developing the homegrown platform in January 2012. It took a team of 12 engineers about 15 months before the platform, which Dell EMC referred to as its Enterprise Platform-as-a-Service (EPaaS), was up and running, according to Raj Markala, who then ran the company’s enterprise application services. EPaaS provided a number of benefits, including making it easier for operators to manage virtual machines and giving developers the ability to quickly spin up new application instances.

“It was really a great engineering feat by the folks here in IT," Markala said. The new platform was definitely a step up from the previous environment and architecture. “It was able to automate some things and it really was a true platform."

But challenges remained. “It had its own restrictions. It could only support certain kinds of architectures," Markala said. “It wasn’t a fully automated platform on top of the IaaS. It was actually more of a blueprint."

EPaaS didn’t actually automate tasks like selecting and provisioning web servers and application services, Markala said. The platform team still needed to use tools like Puppet to provision TomCat servers and RabbitMQ, for example. Developers also still needed to pick the messaging tools and databases they wanted to use, Markala said, and scaling applications up and down remained manual efforts. Nor could developers push applications into production themselves. That still required help from the operations team, Markala said.

The platform didn't make it easy to monitor health metrics and other performance data either, which resulted in Markala’s operations team “spending most of the day debugging problems," he said.

Ultimately, EPaaS didn’t enable Dell EMC developers to achieve the level of release velocity they were looking for and, as a result, adoption never took off. Maintaining the platform was another challenge.

“We ended up spending quite a few man hours both developing new features for the platform, maintaining and patching it, and implementing upgrades," Markala said. “It really required a lot of engineering resources and time to maintain. Again, it was a great platform, but considering the costs involved to keep it running smoothly we determined it didn’t make economic sense. That’s when we made the decision to look into Pivotal Cloud Foundry."

Solution

Markala and team began looking into Pivotal Cloud Foundry shortly after the platform's 1.0 release in 2014. Pivotal Cloud Foundry is a cloud-native platform purpose-built to support increased developer productivity and modern software development methodologies.

It automates the build/test/delivery pipeline and provides buildpacks that manage runtime dependencies, meaning developers can focus exclusively on writing great code. But what caught Markala’s attention was Ops Manager.

Ops Manager is a web application used to deploy and manage Pivotal Cloud Foundry. It automates many Day 2 operational tasks, such as scaling application instances and applying patches, allowing operations teams to manage a set of platform services rather than physical or virtual servers.

“The big value of Ops Manager is that it just makes it so much easier to install and manage the platform," Markala said. In addition to Ops Manager, Pivotal Cloud Foundry also includes PCF Metrics, which logs and displays application telemetry to help developers troubleshoot their applications throughout the complete software development lifecycle.

After a reviewing EPaaS's capabilities and potential benefits, Markala decided to shelve it and use Pivotal Cloud Foundry as Dell EMC’s platform of choice. With the help of Ops Manager, it took four months to standup Pivotal Cloud Foundry running on Dell EMC’s private cloud environment from proof-of-concept to full production grade environment. The new platform went live in January 2015.

At about the same time, Markala also decided to beef up his own team by adding a handful of new developers to his staff to serve as liaisons to Dell EMC developers embedded in the company’s various business units. Markala’s developers help other Dell EMC developers get started writing code on Pivotal Cloud Foundry and teach them the ropes of using PCF capabilities. They also teach microservices best practices and how to use other platform tools, and show them how to leverage messaging services like RabbitMQ and caching services like the Redis tile for Pivotal Cloud Foundry.

Business Benefits

With the introduction of Pivotal Cloud Foundry, Markala and his team were able to automate many if not all of the operational tasks that required manual intervention on EPaaS. This includes automated scaling of app instances and zero downtime deployment of patches and upgrades. “We also now have much better platform monitoring capabilities that allow us to understand the health of the application instances," Markala said. “All of that is being handled by the Pivotal Cloud Foundry. We don’t have to deal with any of the IaaS."

This makes it easier to manage the platform, giving Markala’s team more time to focus on higher value tasks, including developing new ways to get even more value from the platform. Pivotal Cloud Foundry also empowers Dell EMC’s developers, Markala said.
Developers now have a self-service environment where they can easily bind data services and other platform services to their applications and push code to production without needing to involve the operations team.

A follow-on benefit it is that Pivotal Cloud Foundry has reduced friction between developers and operators. “The developers are in control of making the decision of when and how to deploy and run their code," Markala said. “So there is no blame game anymore between developers and IT. It is left to the developers how to use the platform as long as they follow the defined standards."

One application running on PCF at Dell EMC is CloudIQ. CloudIQ is a SaaS application customers use to monitor and manage their Dell EMC storage environments. CloudIQ provides comprehensive monitoring, tailored insights, access to near real-time analytics, and the ability to manage storage systems from anywhere at any time, according to Ashutosh Nanekar, a principal software engineer at Dell EMC who works developing the CloudIQ application. It proactively identifies risk in storage environments to help prevent or at least mitigate issues. It also provides historical trending and predictions of future storage requirements for simpler and more intelligent planning and optimization.

CloudIQ is an important value-add application that helps set Dell EMC storage products apart from rivals. When Dell EMC made Pivotal Cloud Foundry available to developers, including the CloudIQ team, Nanekar was eager to move to the new platform in part to improve developer productivity. At the time, developers were following the Scaled Agile Framework, a scalable and modular approach of Agile methodology, but it still took up to six months for developers to move new features into production. That all changed when the team moved to Pivotal Cloud Foundry.

“Pivotal Cloud Foundry allows developers to focus on the software they write, which is what really matters," Nanekar said. “Today, we are developing using a microservices architecture on Pivotal Cloud Foundry, which enables us to push new capabilities faster than ever before." Microservices architecture involves breaking down monolithic applications into sets of discrete services that can be scaled and developed independently. The CloudIQ team is building microservices using Spring Boot and other components of the Spring ecosystem, which are ready-made for Pivotal Cloud Foundry.

“The team is now delivering on a twice-weekly basis," Nanekar said. “The pipeline allows deploying content anytime. The team developed the continuous delivery pipeline so that finished content can be deployed to production in a matter of a few minutes to a few hours after testing in the lower environments."

Nanekar also cited the platform's health metrics and monitoring capabilities as enabling his team’s productivity and improving the quality of its software. “One of the biggest benefits of Pivotal Cloud Foundry is the ability to do continuous delivery and continuous monitoring. Agile is all about iteration, getting feedback, and iterating again. And Pivotal Cloud Foundry gives us that level of detail about how users are using the application and lets us continuously improve it."

Conclusion

Markala and his team continue to evangelize Pivotal Cloud Foundry to developers throughout Dell EMC, he said. His team conducts internal webinars about the platform’s capabilities and publishes frequent documentation to help developers get up to speed quickly. As a result, adoption is steadily increasing, Markala said.

“We’re seeing more and more developers abandon some of the ad hoc environments they cobbled together on their own and were using previously," Markala said. “As soon as they see the power of Pivotal Cloud Foundry, they are eager to start developing their applications on it."

Or, as Nanekar puts it: “The developers are happy, which leads to better software, more features and happy customers. Nobody wants to build traditional applications anymore."

Previous
Allianz Presentation: Cloud Foundry Summit Europe 2017
Allianz Presentation: Cloud Foundry Summit Europe 2017

Next Presentation
Garmin, Comcast and Liberty Mutual – Cloud Foundry
Garmin, Comcast and Liberty Mutual – Cloud Foundry

1100 deploys a month. 50% improvement in time to market. Same day bug fixes with no downtime. 1500 producti...