Spinnaker is Netflix’s Cloud-Native, continuous delivery platform—built from the ground up to intelligently deploy software releases to cloud platforms. Spinnaker enables developers to ship code rapidly, with increased confidence and greater visibility. It is an open source project with additional cloud provider support contributed by Pivotal, Google, and Microsoft. The platform supports polyglot deployments and provides a cloud agnostic deployment model which supports AWS, Pivotal Cloud Foundry® (PCF), Google Compute Engine, and Microsoft Azure (forthcoming).
Spinnaker incorporates the deployment patterns and lessons learned at Netflix over the past six years of work with Asgard and achieves the following goals:
- Enables repeatable automated deployments captured as flexible pipelines and configurable pipeline stages
- Provides a global view across all the environments that an application passes through in a deployment pipeline
- Offers programmatic configuration and execution via a consistent and reliable API
- Is easy to configure, maintain, and extend
- Is operationally resilient
- Addresses a breadth of pipelines
Pivotal’s Involvement With Spinnaker
In addition to the goals above, Spinnaker planned to provide an open and pluggable system, enabling the open source community to easily extend the platform and support additional cloud platforms. Netflix is collaborating with industry leaders like Pivotal, Google, and Microsoft to ensure Spinnaker’s adoption and advance the technology. Pivotal has contributed Cloud Foundry support, ensuring our enterprise customers and community can easily take advantage of Spinnaker for their Cloud-Native deployments. Spinnaker itself can be deployed to PCF just like any other application. Additionally, a Cloud Foundry driver has been contributed. It maps Spinnaker’s deployment model to Cloud Foundry APIs, making it simple to deploy user applications.
Continuous Integration, Continuous Delivery And Cloud-Native
Delivering software into cloud environments involves continuous integration (CI) and continuous delivery (CD) software practices. Continuous integration often refers to the building and testing of code in a development environment. Continuous delivery takes CI to the next level by using discipline combined with quick and safe production deployments. CD pipelines also use production-like environments to verify the readiness of an application. This opens the door to a new level of confidence—deploying to production when the business is ready.
Spinnaker has a pluggable CI server integration that supports Jenkins out-of-the-box. It can be extended to support other build tools such as Concourse, Bamboo, and TeamCity, etc. While most CI servers enable deployment to any target platform and could be seen as part of a CD solution, Spinnaker leverages its deep integration with cloud provider metadata, providing more context for deployments. This cloud metadata enables more sophisticated use cases such as a conditional time-of-day deployment, manual verification steps, contextual notifications (email, SMS, etc.), and leveraging application and platform metrics as part of the deployment process. This is where Spinnaker complements traditional CI—by picking up where it leaves off—with complex, rule-driven, cloud provider integrated, high-volume deployments. Also, Spinnaker adds CD pipeline workflows to support more advanced rollout mechanisms like parallel deployments, time window restrictions, application canary deployments, and more. The following screenshot depicts Spinnaker’s visualization and status of a deployment pipeline successfully deployed to PCF.
Mapping Spinnaker’s Cloud Deployment Model To Cloud Foundry
Cloud Foundry’s Spinnaker modules have adopted Spinnaker terminology, where possible, to make the integration simpler. A few examples are given below to illustrate the mapping of Spinnaker concepts to Cloud Foundry concepts:
- Application. A complete, encapsulated application and all its parts. For example, the Sagan application, which hosts spring.io, is located in a single repository on GitHub with a list of commits.
- Cluster. An app that is located somewhere, and that “somewhere” maps onto a Cloud Foundry space. Again, the Sagan application is deployed in production, staging, and development spaces, some with more than one copy. Each of these is a cluster.
- Server Group. A physically running application. Server groups map 1-to-1 with Cloud Foundry applications. Server groups carry the concept of immutable deployment and have a version number embedded in their names. For example, if you just released sagan-prod-v001, and there is a bug, you don’t push a patch to that Cloud Foundry app. You create a new app at sagan-prod-v002 with the patch and retire the previous one.
- Instance. A scaled number of instances of an app actually running.
- Load Balancer. Spinnaker load balancers are where application traffic travels. For Cloud Foundry, this would be associated with a public-facing route assigned to one or more apps, i.e. server groups. Enabling and disabling server groups involves mapping and unmapping the Cloud Foundry route.
Spinnaker Processing Flow With Cloud Foundry
Spinnaker consists of a series of microservices, allowing the system to be scaled and managed independently in production. The primary components of the Spinnaker system are:
- Igor. A service that is responsible for reacting to Jenkins builds jobs and sending the notification to the Echo service
- Echo. A service that routes events to Orca, which launches the delivery pipeline
- Orca. A service that executes the delivery pipeline and sends a request to the configured Cloud Foundry Cloud Driver.
- Cloud Driver. A service that pulls the application deployment artifact from Jenkins and deploys to PCF.
- Deck. The user interface that displays the status and execution history of delivery pipelines
How do all these components work together? The simplest route to answering that is to take a simplified, stepwise view of the processing:
Step 1: A developer commits code, which kicks off a Jenkins build
Step 2: The Jenkins build completes and an application deployment artifact is created
Step 3: Igor reacts to the build’s completion and posts an event to Echo
Step 4: Echo routes the event to Orca, which and starts the delivery pipeline instance and sends a request to the Cloud Driver
Step 5: Cloud Driver pulls the artifact from Jenkins and deploys it to PCF
Step 6: The Orca pipeline completes when Cloud Driver detects a new server group up and running on PCF
Open Source Collaboration And Innovation
Pivotal and Netflix both share a commitment and passion for open source software, collaboration and innovation. Spinnaker is built on Pivotal’s Spring IO portfolio (Spring Boot, Spring MVC, Spring Batch, etc.), and it is a great example of the next generation applications that can be built with Spring. Also, Netflix is an active member of the Spring community and has contributed to numerous Spring projects, even sharing their broader experiences with the Spring community at SpringOne2GX.
Pivotal has also contributed to numerous Netflix OSS projects and combined Spring Cloud and Netflix OSS into a commercial offering called Spring Cloud Services for Pivotal Cloud Foundry. It provides enterprise development and operations teams with a new, production-grade scaffolding for Cloud-Native application architectures.
Spinnaker represents a major contribution to the open source world and packages up years of Netflix development efforts and lessons learned from the cloud trenches. Cloud adopters who have been forced to develop custom deployment workflows atop their continuous integration platforms, or within, will find much value in Spinnaker. This is a great example of how collaborative open source communities can influence and shape industries.
Learn More:
- Cloud-Native Workshop
- Spring Cloud Services for Pivotal Cloud Foundry
- Overview of the new PCF 1.6
- Download PCF
About the Author
