Spring Cloud Services for Pivotal Cloud Foundry (SCS), formed from Netflix and Pivotal technology, is now generally available.
SCS gives application development and operations teams new, production-grade scaffolding for Cloud Native application architectures. SCS builds on the existing Spring Cloud project, which itself wraps a subset of the open source components available from Netflix with the Spring programming model.
SCS 1.0 operationalizes components from Spring Cloud, making them available as managed services within the Pivotal Cloud FoundryⓇ marketplace, including a configuration server, service registry, and circuit breaker dashboard service. SCS fully automates the installation, configuration, deployment, and management of these infrastructure components.
This post will explain more about the new SCS services, the ways they help developers and operators, and why you will want to use them for Cloud Native architectures targeting the JVM.
Why Do These New Spring Cloud Services Matter?
To provide context, Cloud Native companies exhibit four primary characteristics—speed, safety, scale, and ubiquitous availability of services. Their architectures balance the ability to rapidly deliver software services with the need for stability, availability, and durability of those services. Their architectures also allow them to respond elastically to changes in demand, and their services are always on—available to anyone, anytime, anywhere. For additional depth, the O’Reilly book Migrating to Cloud-Native Application Architectures can be downloaded for free and is a quick read on the cultural, organizational, and technical changes implied by such architectures.
SCS helps teams build Cloud Native architectures—typically distributed systems composed from microservices. Chapter three of the book describes the nonfunctional requirements encountered when developing such systems. Spring Cloud Services provides the first and only secure, enterprise-ready distribution of core NetflixOSS and Spring Cloud components, specifically geared toward providing a firm foundation from which to address these non-functional requirements. Without them, developers are forced to build or identify their own components, and then they have to build and operationalize their own architectural scaffolding. The time spent on these efforts is subtracted from time spent building valuable, differentiating business components.
Diving Further Into Spring Cloud Services for Pivotal Cloud Foundry
Spring Cloud Services provides application development and operations teams with advanced tooling to address the key distributed systems challenges of configuration management, service discovery, and fault tolerance. These challenges are addressed by the following components:
- A Configuration Server, powered by Spring Cloud Config, and backed by a customer-provided Git or SVN repository. Customers also have the option of using the newly released GitLab Enterprise service for Pivotal Cloud Foundry.
- A Service Registry, powered by the battle-tested NetflixOSS Eureka server.
- A Circuit Breaker Dashboard, powered by the combination of NetflixOSS Turbine and Hystrix.
Each of these components is made available from the Pivotal Cloud FoundryⓇ services marketplace. The service catalog is presented to Pivotal Cloud Foundry by a service broker—behind the scenes, it manages each developer-created service instance as applications deployed to Pivotal Cloud Foundry! As such, they benefit from all of the operational capabilities provided by Pivotal Cloud Foundry—the platform that Mercedes-Benz just standardized on for their connected car platform.
The importance of this service catalog cannot be overstated. The delta between cloning the Git repositories provided by NetflixOSS and a turnkey service running in production is incredibly wide. Spring Cloud Services completely fills this gap, including:
- Zero-touch installation of the service broker via Pivotal Cloud Foundry Ops Manager
- Management of service instance state using a highly available, replicated MySQL cluster
- A messaging backbone provided by Pivotal RabbitMQ, supporting asynchronous service instance provisioning and circuit breaker metrics
- Service instances deployed and managed as applications on Pivotal Cloud Foundry, benefiting from its four levels of HA
- OAuth2-based single sign-on to all service instance management dashboards, governed by the Pivotal Cloud Foundry User Account and Authentication server (UAA)
- Force SSL for all HTTP traffic to service instance management dashboards and application to service instance communication
- A dedicated identity zone (utilizing the multi-tenant capabilities of the UAA) that governs unique OAuth2 clients, secrets, and scopes used for application to service instance communication and is secured via OAuth2 flows
- Dedicated Spring Cloud Connectors extensions that facilitate Spring Boot auto-configuration for each service simply by detecting the appropriate Pivotal Cloud Foundry service binding
- Dedicated Spring Boot starters that facilitate easy management of all Maven dependencies necessary for consuming each service in the catalog
Developers are able to consume this tooling by way of the Spring IO project ecosystem—which brings community, well-known frameworks, patterns for microservices, and much more. In many situations, all that remains is the application of the appropriate Java annotations to leverage these powerful distributed systems patterns.
Now let’s dive into each service in the catalog.
As we scale up to larger distributed systems, sometimes we want configuration capabilities that are difficult to achieve using bundled configuration files or even the simple OS-level environment variables recommended by the Twelve-Factor App. These may include:
- Changing logging levels of a running application in order to debug a production issue
- Changing the number of threads receiving messages from a message broker
- Report all configuration changes made to a production system to support regulatory audits
The Spring Cloud Services Configuration Server provides these capabilities by combining the versioning and auditing features of version control systems such as Git and Subversion with the flexible environment abstraction of the Spring Framework.
Configuration properties can be targeted to Spring applications and profiles simply by storing either Java properties or YAML files in a version control repository. Targeting a specific application and profile is as simple as naming a properties file “appname-profilename.properties” or embedding multiple profile-specific documents in a single “appname.yml” file. The Configuration Server translates these into a simple REST API that delivers Spring PropertySources as JSON documents.
Client Spring Boot applications consume these PropertySources simply by including the appropriate Spring Boot starter dependency, and then binding to the Configuration Server instance on Pivotal Cloud Foundry. The Spring Cloud Services connector then presents the application’s name and active profiles to the configuration service, obtains the resulting JSON PropertySources, and then coalesces those PropertySources with any local configuration. Configuration provided by the Configuration Server always takes precedence over any conflicting local configuration, ensuring centrally-managed consistency across a fleet of application instances.
All communication between the client application and the Configuration Server is conducted via SSL, and configuration will only be served to an application possessing a valid OAuth2 token containing a “read scope” for the targeted Configuration Server instance.
Cloud Native architectures are incredibly dynamic. Applications and services can randomly spike, go down, or have more instances added to the mix. In order to compose systems from this dynamic network of services, applications require a robust mechanism to discover the most current topology. The Spring Cloud Services’ Service Registry is this mechanism. Using the Service Registry, services publicize their availability, and clients find available services dynamically—no clients need built-in knowledge of the specific implementation and connection details for any of their dependencies.
At the heart of the SCS Service Registry is Eureka from NetflixOSS. Eureka is designed to be an eventually consistent, highly available store of service metadata. The Eureka server is kept up-to-date via regular “heartbeat” registrations from various Eureka clients. The Eureka client also caches all of the available registry metadata locally, and all service “lookups” are accomplished by accessing the cache. In this way, even if the Eureka server suffers downtime, clients can still discover services based on the last cache update. Should this happen, Pivotal Cloud Foundry’s health monitoring will restore the Eureka server, and all clients will re-register. After a configurable time period, clients will again refresh their respective caches and consistency will be restored.
Once again, client applications participate in service discovery by including the appropriate Spring Boot starter dependency, and then binding to the Service Registry instance on Pivotal Cloud Foundry. Applications can declaratively specify whether they would like to be discovered using their Pivotal Cloud Foundry route, which will cause client traffic to travel through the Pivotal Cloud Foundry load balancing/routing tier, or using their Cell IP/Port, which will enable direct point-to-point communication. Finally, the addition of an @EnableDiscoveryClient Java annotation completes the configuration.
The SCS Service Registry adds additional enterprise security features missing from Eureka. The Pivotal Cloud Foundry security team identified a vulnerability in Eureka by which a Eureka client could attempt to “man in the middle” (MITM) registered services by rewriting their metadata and routing traffic through itself. This vulnerability was addressed by enforcing the policy that only applications possessing a valid OAuth2 token containing a “read or write scope” for the targeted Service Registry instance can connect to that instance. Furthermore, only the originator or a service record in Eureka will be allowed to write to that record. In addition to addressing this vulnerability, all communication between the client application and the Service Registry is conducted via SSL.
Circuit Breaker Dashboard
The last of the new services is the Circuit Breaker Dashboard. Much like the breakers we find in our home electrical systems, software circuit breakers trip when a network dependency is determined to be overloaded or broken. Such fault tolerance is necessary in Cloud Native architectures composed from multiple microservices. As the number of nodes in any distributed system increases, the probability of any of those nodes being in an unhealthy state increases exponentially. Because of this, we must learn to construct reliable systems from unreliable components.
Spring Cloud-based applications can easily utilize Netflix’s Hystrix library to create circuit breakers using an annotation-based programming model. Remote calls to other services can be protected by annotating the enclosing Java methods with @HystrixCommand. A “fallback service” argument to the annotation can specify a backup or alternative service, return dummy data, or simply switch the feature off, when the primary target service is in a degraded state. For example, a broken personalized recommendation widget in the GUI could call up a widget with the top 10 sales instead of displaying an error.
Part of the effective use of software circuit breakers is visibility into their switching behavior. Hystrix smartly includes a metrics stream for each breaker, indicating the breaker’s state (open or closed), volume of requests, median and long-tail latency, and successful, failed, or short-circuited requests. These metrics streams are most effectively consumed through a graphical dashboard provided with Hystrix. Netflix OSS also provides Turbine to effectively aggregate multiple applications’ metrics streams into a single dashboard.
Spring Cloud Services provides turnkey Circuit Breaker Dashboard capabilities by provisioning a Pivotal RabbitMQ service and Turbine AMQP to aggregate event streams for any bound applications. The aggregated event stream is automatically piped to a Hystrix Dashboard accessible by clicking the “Manage” link for the service instance in Pivotal Cloud Foundry Apps Manager. As with the other services, client applications can stream circuit breaker metrics to the dashboard by including the appropriate Spring Boot starter dependency and binding to the Circuit Breaker Dashboard instance on Pivotal Cloud Foundry.
When the power of Netflix OSS, Spring, and Pivotal Cloud Foundry are combined, there is a really clear story about how businesses are better able to compete. With Spring Cloud Services for Cloud Foundry, developers get better tooling to address customer needs faster, perform experiments without risk, run services with less downtime, and reduce TCO.
- Download Spring Cloud Services for Pivotal Cloud Foundry
- SCS 1.0 product documentation for developers and operators
- Checkout the podcasts, “A Look at Spring Cloud Services for Pivotal Cloud Foundry” or “A Quick Look at Spring Cloud Data Flow“
- Find out how Netflix engineering uses Spring Cloud or see their presentation from SpringOne 2GX, “Spring Cloud at Netflix” along with all the other conference videos.
- SpringOne2GX 2015 session: 12 Factor, or Cloud Native Apps for Spring Developers
- SpringOne2GX 2015 session: Getting Started with Spring Cloud
- SpringOne2GX 2015 session: Developer Experience with Spring Cloud
- Read more on Pivotal Cloud Foundry 1.6
- Download an updated version of Spring Cloud Services for Pivotal Cloud Foundry
- Learn about the latest SCS for PCF 1.1 and 1.2 releases
- Get the SCS product documentation for developers and operators