Deploying BOSH Lite in a Subnet-Accessible Manner

March 16, 2015 Brian Cunnie

BOSH is a tool that (among other things) deploys VMs. BOSH Lite is a user-friendly means of installing BOSH using Vagrant.

A shortcoming of BOSH Lite is that the resulting BOSH VM can only be accessed from the host which is running the VM (i.e. bosh target from another workstation does not succeed). [1]

This blog post describes three techniques to make BOSH Lite accessible from workstations other than the one running the VM. This would be useful in cases where a development team would need a single BOSH and find it convenient to access it from any development workstation.

We’ll describe three scenarios. If you’re not sure which one is right for you, choose the first scenario (port-forwarding):

  1. enable port-forwarding to the BOSH VM
  2. configure the BOSH VM to use DHCP
  3. assign the BOSH VM a static IP address

Make sure to secure BOSH after we have made it subnet accessible; the default accounts and passwords are very weak.

We limit our discussion to deploying to VirtualBox (i.e. we won’t discuss deploying BOSH Lite to Amazon AWS or VMware Fusion).

Procedure

Scenario 1: Enable Port-Forwarding to BOSH VM

The procedure follows the BOSH Lite README, stopping at the section, Using the Virtualbox Provider (i.e. we have already cloned the BOSH Lite repository to ~/workspace/bosh-lite)

cd ~/workspace/bosh-lite

Edit the Vagrantfile to enable port-forwarding:

vim Vagrantfile

Add the following line (this is what the changes look from git‘s viewpoint):

  override.vm.network "forwarded_port", guest: 25555, host: 25555

We now continue with the remainder of the BOSH Lite’s README’s instructions (e.g. vagrant up).

We are now able to bosh target from other machines. For example, let us say we installed BOSH Lite on the workstation tara.nono.com, and we want to access the BOSH from the laptop hope.nono.com. On hope, we type the command bosh target tara.nono.com, and we can log in with the user admin and password admin.

Caveats

Although you can access the BOSH from other workstations via the BOSH CLI, you will not be able to access other services (e.g. you will not be able to SSH into the BOSH VM from a workstation other than the host workstation).

IPv6: For dual-stack systems, don’t target the IPv6 address; target the IPv4 address. [2]

Scenario 2: Configure the BOSH VM To Use DHCP

The procedure follows the BOSH Lite README, stopping at the section, Using the Virtualbox Provider (i.e. we have already cloned the BOSH Lite repository to ~/workspace/bosh-lite)

cd ~/workspace/bosh-lite

Edit the Vagrantfile to enable DHCP:

vim Vagrantfile

Add the following line (this is what the changes look from git‘s viewpoint):

  config.vm.network :public_network, bridge: 'en0: Ethernet 1', mac: '0200424f5348', use_dhcp_assigned_default_route: true

Note:

  • Do not choose the same MAC address we did (‘0200424f5348’ [3] ). To choose a mac address, set the first byte to “02” and the remaining 5 bytes with random values (i.e. dd if=/dev/random bs=1 count=5 | od -t x1)

We need to create a DNS hostname and assign an IP address for the BOSH VM. In this example we’ve assigned the hostname bosh.nono.com and assigned the IP address 10.9.9.130

We need to create an entry in our DHCP configuration file to reflect the MAC address and IP address; this is a snippet from our ISC DHCP server’s configuration file, dhcpd.conf:

  host bosh { hardware ethernet 02:00:42:4f:53:48; fixed-address 10.9.9.130; }

ISC’s DHCP daemon requires a restart for the new configuration to take effect.

We now continue with the remainder of the BOSH Lite’s README’s instructions (e.g. vagrant up).

We are now able to bosh target from other machines. For example, let us say we installed BOSH Lite on the workstation tara.nono.com, and we want to access the BOSH from the laptop hope.nono.com. On hope, we type the command bosh target bosh.nono.com, and we can log in with the user admin and password admin.

Caveats

If the network is hard-wired and uses VMPS, you may need to register the MAC address with IT [4].

Scenario 3: Assign the BOSH VM a Static IP Address

The procedure follows the BOSH Lite README, stopping at the section, Using the Virtualbox Provider (i.e. we have already cloned the BOSH Lite repository to ~/workspace/bosh-lite)

In this example, we set the BOSH with the following parameters:

  • IP: 10.9.9.130
  • Subnet: 255.255.255.0
cd ~/workspace/bosh-lite

Edit the Vagrantfile to enable port-forwarding:

vim Vagrantfile

Add the following line (this is what the changes look from git‘s viewpoint):

  override.vm.network :public_network, bridge: 'en0: Ethernet 1', ip: '10.9.9.130', subnet: '255.255.255.0'

We now continue with the remainder of the BOSH Lite’s README’s instructions (e.g. vagrant up).

We are now able to bosh target from other machines. For example, let us say we installed BOSH Lite on the workstation tara.nono.com, and we want to access the BOSH from the laptop hope.nono.com. On hope, we type the command bosh target 10.9.9.130, and we can log in with the user admin and password admin.

Caveats

Only machines on the local subnet are able to target the BOSH VM (e.g. in this case, machines whose IP address starts with “10.9.9”). (The BOSH VM’s default route is the host VM, which will in turn NAT the traffic, which will break any TCP connection outside the local subnet to 10.9.9.130). This scenario (static IP) is not a good solution for geographically distributed teams, e.g. if the host machine is in San Francisco but some team members are located in Toronto.

If the network is hard-wired and uses VMPS, you may need to register the MAC address with IT [4].

Securing BOSH

After making BOSH available on the local subnet, we need to secure it (the default BOSH Lite credentials of admin/admin are not terribly secure).

1. Secure the BOSH user

We create a new account director and delete the admin account:

# we target our BOSH
#   scenario 1: target the host workstation (e.g. tara.nono.com)
#   scenarios 2 & 3: target the BOSH VM directly (e.g. bosh.nono.com)
# account is 'admin', password is 'admin'
bosh target tara.nono.com
  Target set to `Bosh Lite Director'
  Your username: admin
  Enter password:
  Logged in as `admin'
# create new account 'director' with hard-to-guess password
bosh create user director
  Enter new password: **************
  Verify new password: **************
  User `director' has been created
# log in as newly-created account 'director'
bosh login director
  Enter password:
  Logged in as `director'
# delete the admin account
bosh -n delete user admin
  User `admin' has been deleted

2. Secure the UNIX user

This step only applies to scenarios 2 & 3 (i.e. to scenarios where users on the local subnet can ssh to the BOSH VM). We need to change the password to the vcap account (default password is c1oudc0w) and to the vagrant account (default password is vagrant):

 # we ssh to the BOSH VM (e.g. bosh.nono.com) as the 'vcap' user
ssh vcap@bosh.nono.com
  Warning: Permanently added 'bosh.nono.com,10.9.9.130' (RSA) to the list of known hosts.
vcap@bosh.nono.com's password: c1oudc0w
  Welcome to Ubuntu 14.04 LTS (GNU/Linux 3.13.0-44-generic x86_64)
  ...
vcap@bosh-lite:~$ passwd
  Changing password for vcap.
  (current) UNIX password: c1oudc0w
  Enter new UNIX password:
  Retype new UNIX password:
  passwd: password updated successfully
 # we change the vagrant user's password, too
sudo passwd vagrant
  Enter new UNIX password:
  Retype new UNIX password:
  passwd: password updated successfully

3. Secure other Services (e.g. PostgreSQl)

This step only applies to scenarios 2 & 3.

We will not discuss securing postgres or other services, but be aware that those services are running and are only loosely secured. BOSH Lite should not be deployed in a hostile environment (i.e. on the Internet).


Footnotes

1 We acknowledge that there are technical workarounds: the teams working on other workstations can ssh into the workstation that is running the BOSH and execute all BOSH-related commands there. Another example is ssh port-forwarding (port 25555). We find these workarounds clunky.

2 BOSH doesn’t listen on IPv6 interfaces. For example, the host tara.nono.com has both IPv4 (10.9.9.30) and IPv6 addresses, so when targeting tara.nono.com, explicitly use the IPv4 address (e.g. bosh target 10.9.9.30). If you mistakenly target the IPv6 address, you will see the message [WARNING] cannot access director, trying 4 more times...

This type (dual-stack) problem can also be troubleshot by attempting to connect directly to the BOSH port (note that telnet falls back to IPv4 when IPv6 fails):

telnet tara.nono.com 25555
Trying 2601:9:8480:3:23e:e1ff:fec2:e1a...
telnet: connect to address 2601:9:8480:3:23e:e1ff:fec2:e1a: Connection refused
Trying 10.9.9.30...
Connected to tara.nono.com.
Escape character is '^]'.

3 We chose a MAC address (‘0200424f5348’) by first setting the locally-administered bit (the second least-significant bit of the most significant byte), i.e. the leading “02”.

We chose “00” as the next byte.

We chose the last four bytes by using the hex representation of the ASCII code for “BOSH” (i.e. echo -n BOSH | od -t x1, i.e. ’42:4f:53:48′)

4 When VMPS has been deployed, all MAC addresses must be registered with the local IT organization to avoid knocking the host workstation off the network. For example, Pivotal in San Francisco uses VMPS, and when the ethernet switch detects unknown MAC address on one of its ports, it will configure that port’s VLAN to be that of the guest network’s; however, this will also affect the workstation that is hosting the BOSH VM. In practical terms, the workstation’s ethernet connection will ping-pong (switching approximately every 30 seconds from one VLAN to the other), effectively rendering the BOSH VM and the host workstation unusable.

About the Author

Biography

More Content by Brian Cunnie
Previous
Using Data Science to Predict TV Viewer Behavior and Formulate a Hit TV Show
Using Data Science to Predict TV Viewer Behavior and Formulate a Hit TV Show

In this post, Pivotal data scientists outline how machine learning, topic modeling, elastic net regression,...

Next
IntelliJ F6-based Refactoring Keyboard Shortcuts on OS X
IntelliJ F6-based Refactoring Keyboard Shortcuts on OS X

Use powerful refactoring commands to safely make small and large-scale changes. Undo any refactoring with c...

Enter curious. Exit smarter.

Register Now