Many organisations are moving to a model of Continuous Deployment for their applications. This means that they want to deploy regular changes to production applications without affecting users. By deploying more changes more often, they can move more quickly in the competitive market space. By deploying smaller changes more frequently, they reduce deployment risk and “exercise their deployment muscle” more regularly—becoming better at it.
Getting new features into the hands of your customers sounds great, but it can be a little trickier than it looks. How do you ensure that a change does not “break” the system? How do you roll-back if something bad happens? How do you deploy a change so that no traffic is lost, packets are dropped or transactions missed?
Fortunately Pivotal CF provides built-in capability to handle this complexity for you. In this episode, we will dive more deeply into the concept of Blue/Green deployments, and the mechanism used by Pivotal CF to deploy changes into production with zero downtime and no packets dropped!
PLAY EPISODE #9
- Subscribe to this feed
- Find more episodes of All Things Pivotal podcast
- Feedback: firstname.lastname@example.org
- Links Referred to in the Show:
Welcome to the All Thing Pivotal podcast. A podcast at the intersection of agile, cloud and big data. Stay tuned for regular updates, technical deep dives, architecture discussions and interviews. Please share your feedback with us by e-mailing email@example.com.
Hello everyone and welcome back to the All Things Pivotal podcast. Fantastic to have you back. My name’s Simon Elisha, CTO and manager of field engineering here in beautiful Australia and New Zealand. Good to have you back for another week after some long conversations. We’re having a bit of a shorter session today.
Today’s episode is going to be about using pivotal CF to do blue green deployment. Let’s talk, firstly, about what is blue green deployment and why might we be interested in it? One of the big trends and big things that software organizations like to do now is to work to deliver software in the hands of their customers faster. This means more features, more often, more quickly and more easily hopefully. One of the approaches that really ties into this is a concept called continuous delivery. This is about getting changes into production. You may have heard of continuous integration which is all about integrating software together more frequently, doing builds more frequently, testing components together more frequently so that you have a potentially running piece of software available most of the time. This really goes a step further and says, not only do we have that potentially runable software but we actually want to put it into production.
As anyone knows if you’ve worked with software and production systems is that the migration into production is often a challenging, fraught and typically filled with downtime situation. What this is meant is that people tend to deploy less frequently and it becomes a bigger, more serious event. The challenge or the downside of this is it actually makes you less used to deploying into production. This means that you exercise that production deployment muscle less often. As with any muscle that doesn’t get used very often it atrophies and becomes weak. What we actually want to do is to deploy more frequently and that has a flow-on effect in as much as it allows you to deploy smaller changes more frequently, which actually means you reduce your deployment risk.
What about this little detail, Simon, that you’re not talking about which is the actual deployment of the software? This is often the challenge. People use different techniques like D&S, etc. to try and switch over, but often what happens is you either don’t have a system to test in once you’ve deployed into production or it is potential to lose traffic or lose packets, etc. to the new system here between the switch over. This is the problem that blue green deployment tries to fix. Essentially we have two environments. The current environment is really the one that’s called the blue environment and the green environment is the new environment. Now you can swap and choose those colors as you like, it’s just the sort of name that you use. If we have an existing piece of code running and that’s the blue environment, we introduce a new one called the green environment. That green environment can be completely deployed and built. We can actually run some test traffic to it and make sure it’s running as we think, do any last stage tests, etc. Then we want to flip over between the blue to the green. Once we’re flipped over the new system is taking over the traffic. We still can maintain the existing system temporarily because we may use that as a fall back.
Again, this is our risk mitigation. We can fall back to the previously known good system immediately because we don’t have to set anything up. Again, this is something that Cloud and Elastic Resources makes possible because you can essentially double the size of your environment for a short period of time then shrink down again. This switch over process becomes really, really important. Let’s work through this from a pivotal CF perspective. One of the benefits of using a platform as a service is that it rolls in these fundamental, yet challenging components that you need to have to make things like continuous deployment possible. Within Cloud Foundry we support blue green deployment completely natively. I’d like to walk you through an example. You can find this example in the documentation with the core features. I’m going to be painting the word pictures today. Essentially let’s think about our simple application, we’re going to call it demo-time. It’s a simple application, it produces a website, etc. We’ll just do a CF push and push that application out and that now becomes the blue application. We can map that to a particular URL, in this case it will be demo-time.example.com. We’ve just pushed an app, that’s fine. When we make a change to the application this is where things get interesting. We push an update to the application. Maybe we’ve changed some code, etc. and what we do is we push this using a different name. We give it the name green in this case. We use the CF push green and then we’ve passed the name of the application and we give it a different name. What we do is we call it demo-time-temp.example.com. This now becomes the green path. The Pivotal CF router will route existing traffic, everything for demo-time.example.com to blue because that was our existing application. As per normal, nothing’s changed. However there’s now another URL that you could use, demo-time-temp.example.com, which is mapping to the green instance.
What this means is that you can now use that URL to do any last minute testing, any verification, any validation that you like. Once you’re satisfied that things are running and are correct you can then switch the router across by re-mapping all the incoming requests from demo-time.example.com to the green version of the application. This is a single command, CF map route, it happens immediately, does not drop any packets. This means you can seamlessly transition between the two environments and start using it. Now, one thing you could do is if you do it this way you still continue to map traffic to the blue environment. What this is doing is allowing you to route traffic between the old and the new, which is in some cases a very suitable patent. You can run them in parallel, again, making sure that your new version is new and cool and as nice as you would like and you’re not degrading in any way.
This mapping will continue, or this load bouncing, I should say, will continue as you go. Then when you’re satisfied that you don’t want that blue environment anymore you can just un-map that route. Again, it just uses a CF un-map command and you just un-map it to that particular instance of your application and then all the data will be flowing through to the green environment. Also, at that time you can un-map that temporary one that we created, the dash-temp version, as well so we have a nice clean environment. Again, let’s walk that through really quickly. Push map location, it becomes blue. I map my URL to my blue application, push a new version of my application, that becomes green. I create a dedicated URL to that application I can test against. I can then map my existing URL to both blue and green or just to green, run my application, then un-map blue and dispose of blue and keep running on green. Green now becomes my blue environment and then I just rinse and repeat as I go forward.
Now you may say, ‘Ah ha, Simon that sounds fantastic’ but there is a nuance. What if I’m doing something that requires some sort of data conversion or a longer process? Yes, that does happen and there are specialized techniques around doing data schematic changes and database changes, etc. We have you covered for that. We can’t give you a zero downtime deployment for any of this stuff. There’s stuff you have to do offline, there’s stuff you have to do offline, but what you can do is adjust your route mapping pattern to display a static maintenance page during the maintenance window for anything that’s time consuming. Essentially, in that type of scenario the router switches all incoming requests from blue into the maintenance window and then at such time that you are satisfied that your maintenance is complete and your green environment is available you map to green. Again, you still have that very baked in process, we are not losing traffic. This is a way superior way to do it than to rely on D&S itself to do it. This is because often with cache time to lift values, or TTLs as you might’ve come across them, often clients will still be pointing to the old version of the system out of your control. This is because often ISPs and other providers do not honor the details that are provided and internet with stale references, etc.
Absolute nightmare when you’re trying to convert across applications. Blue green deployment automated by pivotal CF makes this a trivially easy thing to do, but super, super powerful because now you can roll production changes into production on large systems any time of the day that you want with a high degree of certainty and very, very low risk. One of the other nice things about Cloud Foundry is, of course, it’s extendability, the fact that it’s open source and that people are constantly building onto it. I just wanted to point out a Cloud Foundry plugin that an organization has created, it’s called autopilot and it basically wraps in some of that process I was talking about for blue green deployment into the actual manifest that the application uses. It does that mapping for you in a dynamic way so that you’re not necessarily doing those route changing steps yourself, it does it for you and it gets it sort of from a before state to an after state in a seamless way. I’ll put a link in to show notes to that because it’s an interesting little plugin. Again, a different variation on how you like to deploy.
What’s the message here? The message is to deploy good code you should be deploying often and you should be deploying at all times of the day and you should be deploying safely and you should feel confident that when you deploy code it will work and if it doesn’t deploy correctly that you can roll back seamlessly. The blue green testing deployment capability within pivotal CF makes this really, really easy. Again, this is one of those deceptively easy things. You sort of do it with a command line and do it a couple times and think that was easy, there’s nothing to it. If you go and talk to someone about how they would do it in their environment today you end up with a yak shaving experience extraordinaire that no one wants to go through. It really does make life a lot easier. I hope that was an interesting little tidbit for today.
Quick programming note, we’re going to take a bit of a break over the December period. It tends to be a very busy period for people. People tend to either want to catch up on old podcasts, etc. so there’s a whole lot of old podcasts that you can listen to in this series. We’ll be kicking off, probably, the first if not second week of January again with brand new episodes. I am recording some really cool interviews during December though with some very cool people to share with you in January so really looking forward to some of those episodes. Thank you all for listening to the podcast. Again, please do share it with others if you’re enjoying it. We’d like to get your feedback, firstname.lastname@example.org. It’s great to have you listening and great to have you on board. Have a great New Year and until then, keep on going.
Thanks for listening to the All Things Pivotal podcast. If you enjoyed it please share it with others. We love hearing your feedback so please send any comments or suggestions to email@example.com.
About the Author
Simon Elisha is CTO & Senior Manager of Field Engineering for Australia & New Zealand at Pivotal. With over 24 years industry experience in everything from Mainframes to the latest Cloud architectures - Simon brings a refreshing and insightful view of the business value of IT. Passionate about technology, he is a pragmatist who looks for the best solution to the task at hand. He has held roles at EDS, PricewaterhouseCoopers, VERITAS Software, Hitachi Data Systems, Cisco Systems and Amazon Web Services.More Content by Simon Elisha