Going All Microservices with Spring Cloud and Pivotal Cloud Foundry

August 12, 2015 Coté

sfeatured-podcastThis is a two part episode on Spring Boot and Spring Cloud. In the first part, we discussed Spring Boot and re-thinking about Java application in a process-centric way.

In the second part, we discuss Spring Cloud and what it does to handle the tedious parts of creating microservices. We also talk about how Pivotal Cloud Foundry fits in as the runtime.

PLAY EPISODE

SHOW NOTES

TRANSCRIPT

Coté:
Hello. This is part 2 of a conversation I had with Josh Long about Spring Boot and Spring Cloud. In order to understand this one in it’s full glory, you don’t really need to listen to the Spring Boot episode, but if you want to, you can go to pivotal.io/podcast and check it out. With that, here’s the show.

Let’s go to the second part of what we’re going to talk about, and that is we’ve got Spring Boot, as we’ve been discussing. Let me make you a nice … Let you, along with the initializr, basically create a micro-services ready, if you care about that self-contained thing.

Josh Long:
Yes.

Coté:
Right? And pair it down with that and let you start helping do more of a process-centric programming model with Java, which fits well with the people want to do cloud stuff now-a-days. Get you these benefits of … Whether you actually do need to scale, have lots of concurrent users and be performant, but it definitely helps you promote having smaller batches of code so you can release software more frequently.

Josh Long:
Right.

Coté:
Which all sounds great. Who doesn’t like gravy, right?

Josh Long:
Yeah. I love gravy. If you go to a restaurant where they just served gravy, I wouldn’t even need the potatoes.

Coté:
The gravy train.

Josh Long:
Yeah.

Coté:
There’s another project that’s been rising in interest in recent times that you’ve been working on quite a bit, called Spring Cloud. How does Spring Cloud fit into all of this?

Josh Long:
That’s where things get interesting. If Spring Boot is … Let’s be conservative. Let’s say that Spring Boot is the absolute best way to be productive on the JVM, and as far as this humble, albeit biased, person is concerned, it’s also the most productive way to build apps and services on any platform. Let’s just say that that’s … If it’s better than all the other options, maybe it’s just better by a few years. In theory, I could get there with Ruby & Rails, I could get there with the JSR Go. The goals are … Let’s say that Spring Boot is a great way to build a single app, or a single service. I want to build a web application, or I want to build a REST API.

Spring Cloud has no peer. There is nothing else like it. It’s a set of components, integrated through this Spring Boot auto-configuration mechanism, designed to support patterns and only become emergent in large distributed systems. When I say large, I mean more than a few services. More than a few moving pieces. Some of these patterns include, for example, centralized journaled and secured configuration, 12 factor app style configuration property files, for example. Environment and software release, and so on, specific configuration. That’s a problem that all these different large organizations have had to solve. As often as they solve it in different bespoke ways, and they write their own code and they pull things together, it’s just a problem. It’s starting to become a problem, that as more and more people are moving to the cloud, they have to start thinking about these same kind of things. These problems that were hitherto exclusively the domain of the Googles and the Ebays and the Netflixes and the Twitters, and the Alibabas and the Baidus.

Another problem that you get when you move to a dynamic cloud environment is service registration and discovery. I don’t want the ability for me to call a service to be coupled to where that service lives. Perhaps a naïve approach would be to use something like DNS, which of course suffers from a lot of drawbacks, not the least of which is that it’s cached. Time to live issues on DNS will frustrate your ability to have truly dynamic service resolution. You want to have that in a certain place.

Another benefit of a service registry, sort of like a phone book, a phone book for your services, a benefit of that is instead of using DNS and then routing to a load balancer, I can ask the registry for all of the available instances for a given service, and from among the available instances, make the decision on the client, which instance to route the call to. This is a very, very powerful feature. It means I can do things, for example, if I need more than just round robin load balancing, I can say, “let’s do availability’s own sensitive load balancing.” I just want to route to nodes that are near me, or perhaps I have some concept of multi-tenancy, where only this cluster of nodes, this configuration of nodes, should receive requests for these customer ID numbers.

Perhaps I have some concept of something sticky. Maybe I have a session concept. Maybe I’m streaming a video from one node, and I need to route calls to where the action is. Whatever that is, I can make those decisions and any manner of other decisions on the client now. Having a service registry is a very natural extension, but it’s a problem that all of these organizations have had to solve themselves. If you talk to Baidu, which is a search engine. In China they have something they call the Baidu name service. It’s their own registry. Of course, now, we have ample supply of really good open-source options like Eureka and Zookeeper and Consul, and SCD, and so on. It’s very clear that more and more people have these problems, but integrating with these technologies is kind of hard. It’s not something that people know how to slot into their existing code or existing services. Ideally, this should be transparent.

Another thing that people have is they want to have support for patterns that promote resilience. Graceful degradation in the face of failure. If you’re building a Cloud Native application, the goal isn’t to optimize for 100% up time. You’ll never get there for a given single node, anyway. That’s a unicorn, right? The goal instead, is to build services that rebound in the face of failure, that are tolerant to failure. The fail, already fail fast kind of mentality. The extension of that mindset is the whole anti-fragile movement. Build software that doesn’t crumble. You have these kinds of patterns.

Again, like the circuit breaker, for example, is something you can use to prevent failure from cascading from one system to another. Again, another very, very obvious use case, if you’re starting to build distributive systems. What about security? You have a fleet of services, you don’t want to force a client to log in to every micro-service that is in the chain of service. Instead, have a single sign-on, federated single sign-on, using something like OAUTH, OpenID, connect.

You have things like distributive tracing. This is a problem. If I have a … My portal or my webpage, or my web app, or whatever, my phone app, I have a screen, and I’m looking at that UI, but that UI may make calls to a service which then may branch out to 10 other services, and I may have an error. Something will bubble back to my client and show me an error. Where did that error happen? Discovering where that happened, tracings, through the invocation, it’s not our stack, I guess. The invocation queue, or the invocation field, is difficult if you can’t correlate which request passed through every service. These are things that you start to care about.

Spring Cloud is an attempt to address these kinds of concerns, and then more. We do it in the same, very convenient, minimalist, Spring Boot sort of way. You build a jar, you stand it up, and you have the right types in the class path, but you say oh look, I’d like to participate in service registration and discovery. There you go. Your service will register itself with the appropriate service registry. Spring Cloud is both an open-source project that you can consume at your own leisure, of course, and very usefully, it’s also available as a beta on PCF called Spring Cloud services.

These Spring Cloud services, this is a set of bundled service brokers that stand up the middleware that you need to be able to build this kind of system. Middleware like the configuration server. Middleware like the Eureka service registry, and so on. We’re like the dashboard that you can use to visualize all the circuits in your system. This is stuff that you could do with some code, using the straight OSS project, but it’s much, much more streamlined, much more convenient, and thus, you get more velocity when you have less to worry about. You can worry about the stuff that’s important and go faster.

Coté:
Right. To summarize a little bit. Spring Cloud, essentially collects together all the codified, and to make a pun, code-ified, I guess. Common services and best practices for … We discussed having all these independent little units of things, micro-services. Once you start having those things talk with each other, as you were enumerating, there’s a list of things that come up. Tracing and debugging and logging. If there’s an error between the two of them, you don’t want it to just totally mess everything up.

Josh Long:
Integration. I don’t want to have to configure every server.

Coté:
Right. It basically is the … Overlay is a charged word, but it’s the thing that allows all those little outputs of Spring Boot, so to speak, to collaborate with each other, and it unifies it and provides all of the … You brought up the phrase earlier, you have a distributive application. So much of what you do with cloud programming is a distributive application.

Josh Long:
Right.

Coté:
Which has a whole host of annoyances and delights when it comes to it. It seems like a lot of what Spring Cloud does is basically handle those annoyances for you, and gives you the … You’re not forced to use a circuit breaker, but it’s kind of a good idea.

Josh Long:
Right.

Coté:
You can gracefully fail, and operate correctly.

Josh Long:
Of course, and very much in Spring and Pivotal style, we’re not reinventing the wheel, where we don’t have to. Spring Cloud, just like Spring Boot, integrates with all these best of breeds Spring ecosystem projects as well as third party ecosystem projects like Hibernate and the Drop Wizard/Codahail Metrics Library, and so on. Just as Spring Boot integrates with all that, Spring Cloud integrates with really, really popular, tried and true, bulletproof, battle tested and proven components, like I said, Console, like Eureka from Netflix. The Hystrics and Feign also from Netflix.

These are not things that we’re writing from scratch. We’re pulling together from really, really powerful components. I just spent about 10 days talking to some of the biggest websites in the world that are out of China. They’re now moving to Spring Boot and now Spring Cloud based architectures. The real opportunity here is wouldn’t it be great to get these companies, they can throw away all the … They’ve already had to solve these problems, and they wrote frameworks, very often, by the way, on top of Spring. They already wrote frameworks that addressed these patterns. So did Netflix. Netflix also uses Spring. They’ve had to write this code, and it had to pull together and integrate it. Now there’s Spring Cloud, so they can throw away a lot of that stuff that was not present in Spring before, but is now there. We’ve commoditized this next-level layer of patterns that only the big web giants have had to think about previously.

Coté:
You’re mentioning pulling in as the best practices and technologies. Technologies is a great word, and frameworks are emerging out there. Spring Cloud is kind of an amalgamation and polishing up of all those. That begs the question of-

Josh Long:
Integration of, even.

Coté:
Yes, exactly. That’s what I meant by polishing up. It makes them smoothly work together. How do we product manage that? You’ve been around here for a while. I’m sure you’ve been observing the development of Spring Cloud. How do we, when some wacky fun new technology comes along, how do we consider including it or putting it in there?

Josh Long:
Spring Cloud is pretty new. 1.0 just came out. Thus far, the Spring Team we’re always very receptive to customer demand, but of course, one of the things Spring does is look ahead a little bit, we try and see where the puck will be instead of where it is, to borrow a Wayne Gretzky metaphor. We know from watching what Netflix is doing that this is an interesting pattern. We know from watching Twitter, that this other pattern is pretty interesting. We write support for it. We integrate it. Our distributive tracing support, for example, is based on Twitter Zipkin.

We support for the most popular service registries out there, from different companies and different technologies like Zookeeper. This is a no-brainer. We already know Zookeeper’s infinitely tested and very, very powerful. We know that Eureka is used at scale by Netflix. We know that it’s going to work. Where there are gaps, where these open-source technologies don’t provide the full suite, or they don’t work as well as they could, we try to fill the gaps. For example, our configuration server, which is sort of a proxy that manages a configuration repository full of configuration files, YAML property files, based on a Git repository. That configuration server is a net new thing, but it’s designed to meet all of the dimensions of configurability that we know that customers will want.

Again, we didn’t invent a net new journaled auditable repository. It turns out that Git is very good at that. We’re just using that. This proxy sits on top of a Git repository. It’s a big concern as well. Sorry, go on.

Coté:
It seems like it follows what’s often been the Spring ethos, if that’s the right word.

Josh Long:
Yeah.

Coté:
Which is we’re not going to replace best of breed tools. We’re just going to make using them better.

Josh Long:
Yeah, absolutely.

Coté:
And easier. Like you’re saying, there’s multiple registries that you could use, so as different good components come out, we do at least 2 things. One of them is, like I was saying, we make them … I always used to think of it as Spring Framework was very focused on usability. We try to make it easier to use these things. Then there’s also commonality and standardization that come to that.

Josh Long:
Right.

Coté:
We try to make them better to use. The point that you raised a few times, is also you integrate them better to work with other services. I may have many different types of registries, there may be multiple ways of doing a circuit breaker thing, and tracing logs might be another thing, and so forth and so on. There’s almost like if Spring Cloud has wrapped itself around giving it the warm springy embrace. You know that all these things can cooperate with each other.

Josh Long:
You bet.

Coté:
No matter which sort of combination you choose, which is much nicer than if you’re forced to be the counselor that has to make them cooperate with each other on your own.

Josh Long:
The pluggability is key. For example, Baidu and Alibaba, they have their own middleware, they have their own config server, for example. They’ll use that. That doesn’t mean that the rest of Spring Cloud is closed to them. They can plug it in. They have their own service registry. That’s fine. They have an SPI, so they just need to write the little bit of glue code to teach Spring Cloud how to talk to their service registry, and then the rest of the framework snaps in.

They get client side load balancing and they get the circuit breaker, they get the micro-proxy which knows about the service registry. All of that is generic. The nice thing about it, by the way, about having a common API is that you can plug that API into places where it’s not exactly the same, but it could be. For example, instead of using a service registry proper, if you’re running on Cloud Foundry, or Lattice, which is the extraction of the scheduler in the containerized support inside of a cloudfoundry.net or Diego. If you’re running on Lattice, Lattice and Cloud Foundry both already know, given a service ID, where that thing lives. You don’t have to tell it. It started the service in the first place. It can act as a service registry.

In this case, I can just tell Spring Cloud, go ahead and treat Cloud Foundry as my service registry. I don’t need to … In this case it’s just read-only. My services only ask questions. They don’t need to register themselves with the registry since Cloud Foundry, again, by definition, it already knows. That’s the nice thing. You get this common API. It’s one abstraction. You can now move around. You can move to a local system based on using SCD, for example, and if you want to move into a Cloud Foundry environment, you could deploy Eureka, no problem. Maybe just using the one that’s build into Cloud Foundry, which already has a bird’s eye view of the state of your system anyway, that works fine as well. Now you’ve got that latitude.

Coté:
Let’s go to the final circle, if you will, which is … We’ve gone over, we build our little, as you kept saying, Java jar. That’s fun. Once we have all of our independent little units, our atomic units that allow us to move fast and scale if we need to and all that delightful stuff, we have to have a way to have them all work with each other. They’re basically distributed applications. That’s not easy, unless you have help. You have a way of doing that in a fully integrated way. What’s the last part of everything that doing all of this with Pivotal Cloud Foundry does for you? You were mentioning you can use the registry that it has, but how does running all of this stuff in, as we like to call it, the Cloud platform sort of complete things, in your view?

Josh Long:
Again, going back to … I don’t know what you would say, my friend, Michael, but me, I would say Pivotal’s all about making agile real. I think that every level of the stack is … or maybe its continuous. I suppose that’s an even better way to think about it. It’s all about making that real. Spring is about doing the simplest thing that could possibly work, both in code and in terms of complexity. Your Spring Cloud is all about supporting the ability to do independent deployability and to build microservices. Cloud Foundry is about handling all these operational concerns. Things like logging, things like routing, things like self-healing that you’re going to run into when you’ve got a lot of moving processes.

Coté:
Right, and allocating all the computational resources you need, your storage, networking, and the other one, compute.

Josh Long:
Yeah, route.

Coté:
Yeah. Doing all of that. That exciting stuff that programmers love.

Josh Long:
Actually, I think if I could put a pretty little bow on all of this stuff, I would say that the really, really valuable part about all of this, yeah sure, Spring provides all the support for building apps, but the thing that I love about Spring Boot and Spring Cloud is the really, really good baked in support for non-functional requirements. Things that nobody gets promoted for. Nobody goes home and ignores their wife and children because they’re so excited to write a health check. Nobody goes home and stays up all night stressed out about a deadline. Not stressed out, but they’re not excited. Nobody goes home, gets excited to write code to build a better log aggregation system. Nobody cares. That’s not exciting code. Spring Cloud handles that. Spring Boot and Spring Cloud handles that.

How do I send my metrics and get a joined up view of the state of my system? It’s not exciting. I want to build my app. I don’t care about metrics. It’s stuff that you have to care about. All of this stuff, I think, plays very, very well together. Of course, that is, I think, the real win. We can provide this whole piece. Of course, we managed to go this entire time, and I don’t know how we managed to avoid it, but there’s the last piece, which is I’ve got services and I’ve managed them, they’re in production, they’re moving in scale, and of course, any service that doesn’t have any data is a pretty boring service, indeed. Where does that data live? Of course, that’s going to live somewhere. That is, I think, the bounce of the story, the Pivotal Big Data Suite. That’s the ability to collect, analyze, and react to data. That’s the very definition of the Netflix OODA loop. The observe, orient, decide, and act loop. All these things, taken together, gives you the ability to move quickly, safer.

Coté:
Yeah. The other thing, it’s always good to explain the purposeful or emergent principles that exist in a system like this. It seems like starting from the very beginning when you’re selecting these things you want in Boot, and then off to selecting the best practices that are integrated together in Spring Cloud, and then also, in Pivotal Cloud Foundry, here’s the way you would operate cloud infrastructure is also in there. At each level, there’s basically this curation that occurs, or to use an old term, it’s opinionated.

Josh Long:
Exactly. You used the o-word.

Coté:
That’s right. That curation, it also begins to be key because it’s almost like expertise injected into the system. Also to your point, it’s like we don’t really have an opinion about how you actually write your software. We have an opinion about how you run it, but what it actually does that’s all left up to your opinions. The less opinions you have about logging, the more opinions you can have about, I don’t know, emoticons and how they’re associated with a snarky comment, or whatever it is that you do.

Josh Long:
How to build a better cat picture website.

Coté:
That’s right.

Josh Long:
You’re right about that. I don’t know anybody, not anybody that I’d like to hang out with anyway, except people from the Cloud Foundry team, because they’re awesome. I take that back. I don’t know a lot of people that have opinions about logging. It’s just not something you should care about. Let us take care of it for you. There’s all this stuff that has nothing to do with your ability to win in the marketplace, whatever that marketplace is, whatever that theater is.

Coté:
Absolutely. Great. This has been exciting. We’ve gone over a lot.

Josh Long:
And yet, just the tip of the iceberg.

Coté:
If people are interested in following you around, where should they go? What could you throw out there for people to look into all of this stuff more and to get to know you better.

Josh Long:
I, Josh Long, I’m on the Twitters @starbuxman. I’m also a frequent nuisance on the Spring blog. Spring http://spring.io/blog. You’ll find all sorts of great stuff there. If you’re trying to learn more about … I tend to every now and then, I’ll moonlight on the Pivotal blog, when they let me and nobody’s looking. There are guides. We’re very keen on having people get going as quickly as possible. While you and I have just managed to paint this really glorious picture, I want to bring it back down to earth and connect people with the actual working code.

Start spring.io, and then check out spring.io/guides, and they’ll find a 15 to 30 minute long, no magic, no assumptions, every nauseating detail spelled out. How to do one thing, there’s a task focused guide. It should take 15 to 30 minutes for each one. How do I build a REST API with Spring? That’s it. You just follow that guide and you’ll get it. There’s no assumptions about even the build structure that you’re going to be expected to stand up. All that will be spelled out in nauseating detail. I want to encourage everybody and anybody that wants to get started, check out that start.spring.io, and spring.io/guides. That, I think, am I missing something I really obvious?

Coté:
You know, there’s something that doesn’t exist, that I’ve been thinking recently should exist.

Josh Long:
Uh oh.

Coté:
You should create a Tumblr account called pictures with Josh. For anyone who follows your Twitter account, it’s I’m sure days go by where this doesn’t happen, but to use a phrase, not a day goes by where you don’t have some sort of picture with someone. You travel around quite a bit. That would be a funny use of … I have one called blogging from the buffet. When you travel around, as I’m sure you know, there’s always … They’re not what I would call a lavish buffet, but there are abundant buffets that have food in abundance. It’s kind of funny, like an aggregate to look at what those are.

Josh Long:
That is a phase of my life.

Coté:
You need to launch a pictureswithjosh.tumblr.

Josh Long:
You know why that is, right? I have the luckiest job in the world. I serve as the Spring evangelist, and I have for 5 years. I’m the guy people give hugs when I bring the Spring. That’s the coolest thing ever. You go anywhere in the world and people love Spring, and they love what we’re doing. Especially in the last 3 years with the Pivotal adventure, thing is have gotten even more amazing. People take photos with me. People have my books, people have my books or they’ve seen my training videos, or they’ve seen the blogs, or they follow this week in Spring, which is a roundup of all of this great stuff we’re doing in the Spring and Pivotal ecosystem every Tuesday, which is today, Josh. Get on it, man. The community is amazing, and they love what we’re doing. I’m the guy that gets the hugs. Maybe I’ll give you one next time since you deserve more hugs and you’re largely to credit for a lot of this awesome stuff.

Coté:
More hugs. That’s what you should call it. I Hug Josh Long.

Josh Long:
Right. That’s what’s happening. People take photos. I’m the guy with the big grin and the Spring t-shirt on. They know what to do.

Coté:
That’s great. On that note, as always this has been Pivotal Conversations. If you go to pivotal.io/podcast, or if you’re feeling feisty, you can go to podcasts. It will all go to the same place. You can find … You’re probably already … Somehow found how to listen to this, but you can subscribe to it, and see show notes. We’ll put some links into stuff we’ve mentioned. It’s always nice to subscribe to it and recommend it to other people, if you think they would find it helpful. A lot of what we try to do is what we’ve been doing here. Explain what’s going on and give you a bigger pictures of things, so you can go find those guides, as it were. With that, thanks for joining us, Josh.

Josh Long:
Thank you for having me. This is a pleasure. Let’s do it again.

Coté:
Absolutely. We’ll see everyone next time.

About the Author

Biography

Previous
A Recipe For Smart Applications
A Recipe For Smart Applications

Big Data can seem complicated and overwhelming to many in the enterprise, especially to non-technical busin...

Next
The DAS vs NAS Debate For Apache Hadoop
The DAS vs NAS Debate For Apache Hadoop

If you want to spin up a Apache Hadoop cluster, you need to grapple with the question of how to attach your...