Deploying to AWS Using Cloud Foundry BOSH

September 6, 2012 Martin Englund

featured-cf-genericCloud Foundry was designed and built to support distributed applications that can be moved between multiple clouds, including those running on different IaaS infrastructures such as vSphere, vCloud, OpenStack and Amazon Web Services. The key to supporting multiple clouds is BOSH, a cloud deployment and lifecycle management tool that was announced earlier this year.

This blog post will walk you through the steps to start using BOSH on AWS and then show you how to deploy a sample three-tier application.

BOSH

To start using BOSH, you will need to bootstrap your environment. The Cloud Foundry engineers created BOSH to deploy and manage distributed applications, such as Cloud Foundry, and we wanted to be able to start from scratch and create a working environment that is manageable and upgradable in a controlled manner. BOSH is a distributed application, so it’s only natural that we use BOSH to bootstrap and deploy BOSH itself. For this, we have a notion of a BOSH kernel, a single system running all the BOSH components called micro BOSH. While this is sufficient to manage simple applications, a full BOSH environment would be required for a production grade system. But for the sake of demonstrating BOSH on AWS, we will deploy a sample application directly from the micro BOSH instance.

Prerequisites

To get started with BOSH on AWS you need three things:

  1. an AWS account
  2. AWS credentials, i.e. access_key_id and secret_access_key
  3. a Mac or *NIX computer

Installing the BOSH Deployer

We assume you already have Ruby (1.9.2) and rubygems (1.8) installed. To install the BOSH deployer gem (which includes the BOSH cli):

gem install bosh_deployer

Micro BOSH AMIs

We have published micro BOSH AMIs in each AWS region. When you are ready to use the BOSH deployer, you will use the AMI that corresponds the EC2 region you are using from the table below.

Region AMI
ap-northeast-1 ami-7656eb77
ap-southeast-1 ami-64d59436
eu-west-1 ami-874c4af3
sa-east-1 ami-6280597f
us-east-1 ami-69dd6900
us-west-1 ami-4f3e1a0a
us-west-2 ami-7ac7494a

Deploying Micro BOSH

The first thing we need to do is deploy a micro BOSH instance on AWS. Since you can’t set the IP of an AWS instance, you need to create an elastic IP to use with micro BOSH, as the instances it will deploy need to communicate with the micro BOSH instance and the AWS public IP will change.

Creating an Elastic IP

  1. Go to the AWS Dashboard and look for the Elastic IP link in the upper right hand corner
  2. On the Allocate New Address pop-up window keep the EC2 default and click “Allocate”

  1. Now that you have one IP allocated, repeat the steps to create three more (one for micro BOSH and three for the WordPress demo application we are deploying).

Create the Directory Structure

The BOSH deployer will deploy applications based on files in expected directory locations:

mkdir ~/deployments
cd ~/deployments
mkdir aws

Create Micro BOSH Config File

Micro BOSH configurations are set in the micro_bosh.yml, which you need to create.

  1. Create ~/deployments/aws/micro_bosh.ymlusing the template that appears below
---
name: aws

logging:
  level: DEBUG

network:
  type: dynamic
  vip: x.x.x.x

resources:
  persistent_disk: 20000
  cloud_properties:
    instance_type: m1.small
    availability_zone: sa-east-1a

cloud:
  plugin: aws
  properties:
    aws:
      access_key_id: AKIAIYJWVDUP4KRWBESQ
      secret_access_key: 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
      default_key_name: bosh
      default_security_groups: ["bosh"]
      ec2_private_key: ~/.ssh/bosh
      ec2_endpoint: ec2.sa-east-1.amazonaws.com

apply_spec:
  agent:
    blobstore:
      address: x.x.x.x
    nats:
      address: x.x.x.x
  properties:
    aws_registry:
      address: x.x.x.x
  1. Update all instances of x.x.x.x with one of the elastic IPs you created and the region you are deploying to. Add your AWS credentials under the aws section.
  2. Update the availability zone and endpoint settings to match your availability zone. For instance, if you deploy to us-east-1 you should set the availability zone to either us-east-1a or us-east-1b.
  3. Save the file

Create a Keypair

If you already have a Key Pair for EC2 you can skip this section.

  1. Go to the AWS Dashboard and click Key Pair
  2. Click on the Create Key Pair button
  3. Name the Key Pair “bosh”
  4. Your browser should automatically download a private key named bosh.pem
  5. Rename the private key file to bosh and save it in your ~/.ssh directory

Create a Security Group

  1. In the AWS Dashboard, create a security group named bosh
  2. Add the following inbound rules from 0.0.0.0/0or more restrictive if you like:
    1. port 25555 (BOSH director REST API)
    2. port 6868 (BOSH agent HTTP interface)
    3. port 22 (ssh) for troubleshooting and poking around in the micro BOSH instance
  3. Click “Apply Rule Changes” to make the rules take effect

Deploying Micro BOSH

Micro BOSH can now be deployed from your deployments directory.

  1. Make sure you are in your deployments directory:
cd ~/deployments
  1. Select the deployment you created:
bosh micro deployment aws

Note: don’t be concerned by seemingly inaccurate message

``WARNING! Your target has been changed to `http://aws:25555``!

3. Start the deployment using the AMI from the BOSH AMIs section above:

bosh micro deploy ami-xxxxxxxx
  1. Within 20 minutes your instance of micro BOSH will be deployed. After the ‘Done’ message appears, you have a running micro BOSH instance.
Done             11/11 00:07:10
WARNING! Your target has been changed to `http://x.x.x.x.:25555'!
Deployment set to '/Users/bosh/deployments/aws/micro_bosh.yml'
Deployed `aws/micro_bosh.yml' to `http://aws:25555', took 00:07:10 to complete
  1. If your deployment failed for some reason use the following to clean up:
bosh micro delete
  1. Log in to the Micro BOSH:
bosh login
  1. Type the default account name is admin and the password is admin
  2. Change the account name and password using the command below. Don’t say we didn’t tell you if someone deletes your deployment!
bosh create user <username> <password>

Deploying an Application Using BOSH

We have created a sample three-tier application (Nginx, Apache + PHP with WordPress, and MySQL) to demonstrate how you can use BOSH, and the next step is to deploy it using your newly created micro BOSH instance.

Uploading the Sample Release

The sample release is on Github for your cloning convenience:

  1. First make a git clone of the sample application release repository:
cd ~
git clone git://github.com/cloudfoundry/bosh-sample-release.git
cd bosh-sample-release
  1. Upload the release to micro BOSH:
bosh upload release releases/wordpress-1.yml

Uploading the Latest Stem Cell

Now we download the latest stem cell to upload to our micro BOSH instance.

  1. Download the latest BOSH stem cell for AWS:
bosh download public stemcell bosh-stemcell-aws-0.6.4.tgz
  1. Upload it to your micro BOSH instance:
bosh upload stemcell bosh-stemcell-aws-0.6.4.tgz

Create a Deployment Manifest

  1. Create a security group called “open” which allows inter-security group communication, port 80 from anywhere so the web server can reached from the Internet, and port 22 from anywhere if you want to be able to log on using ssh.
  2. Get the director UUID using the following command:
bosh status
  1. Make a copy of the wordpress-aws.yml in the bosh-same-releasedirectory and update the sections in red.
---
name: wordpress
director_uuid: 718f5344-c8e4-4266-ab10-3a75e1638e25

release:
  name: wordpress
  version: latest

compilation:
  workers: 3 # only the required number are provisioned
  network: default
  reuse_compilation_vms: true
  cloud_properties:
    instance_type: m1.small

# this section describes how updates are handled
update:
  canaries: 1
  canary_watch_time: 3000-120000
  update_watch_time: 3000-120000
  max_in_flight: 1
  max_errors: 1

networks:
  - name: elastic
    type: vip
    cloud_properties: {}
  - name: default
    type: dynamic
    cloud_properties:
      security_groups:
      - open

resource_pools:
  - name: common
    network: default
    size: 4
    stemcell:
      name: bosh-stemcell
      version: 0.6.4
    cloud_properties:
      instance_type: m1.small
      availability_zone:
      key_name:

jobs:
  - name: mysql
    template: mysql
    instances: 1
    resource_pool: common
    networks:
      - name: default
        default: [dns, gateway]
      - name: elastic
        static_ips:
        - 107.21.118.59
  - name: nfs
    template: debian_nfs_server
    instances: 1
    resource_pool: common
    networks:
      - name: default
        default: [dns, gateway]
      - name: elastic
        static_ips:
        - 107.21.118.229
  - name: wordpress
    template: wordpress
    instances: 1
    resource_pool: common
    networks:
      - name: default
        default: [dns, gateway]
      - name: elastic
        static_ips:
        - 107.21.118.223
  - name: nginx
    template: nginx
    instances: 1
    resource_pool: common
    networks:
      - name: default
        default: [dns, gateway]
      - name: elastic
        static_ips:
        - 107.21.118.225

properties:
  nginx:
    workers: 1
  wordpress:
    admin: foo@bar.com
    port: 8008
    servers:
      - 107.21.118.223
    servername: 107.21.118.225
    db:
      name: wp
      user: wordpress
      pass: w0rdpr3ss
    auth_key: random key
    secure_auth_key: random key
    logged_in_key: random key
    nonce_key: random key
    auth_salt: random key
    secure_auth_salt: random key
    logged_in_salt: random key
    nonce_salt: random key
  mysql:
    address: 107.21.118.59
    port: 3306
    password: rootpass
  nfs_server:
    address: 107.21.118.229
    network: 107.21.118.223/255.255.255.255
  debian_nfs_server:
    no_root_squash: true

Deploy

  1. Select the deployment manifest you just created:
bosh deployment ~/wordpress-aws.yml
  1. Initiate the deployment:
bosh deploy
  1. Sit back and enjoy the show!

Conclusion

Congratulations. You just used BOSH to deploy an application to AWS! For now, a sample application is a good place to start. Soon we will post similar instructions for deploying Cloud Foundry. In the meantime, Dr Nic has written a number of getting started guides for BOSH which we highly recommend.

–Martin Englund, Cloud Foundry BOSH team

About the Author

Biography

Previous
The Healthy Gemfile
The Healthy Gemfile

Often when working on ruby projects that use Bundler, I see Gemfiles that look like this: gem 'rails', ...

Next
More trouble with the CDN
More trouble with the CDN

We're getting more reports this morning from customers who are unable to load Pivotal Tracker images/css. W...

×

Subscribe to our Newsletter

!
Thank you!
Error - something went wrong!