The Cloud Foundry team has released new features to improve management of Ruby gems in apps running on CloudFoundry.com. These features include support for using Git URLs in Gemfiles, handling of the BUNDLE_WITHOUT environment variable, and platform specification to control the gem installation process. With these improvements, it is now easier than ever to get your existing Ruby projects up and running on CloudFoundry.com.
History
In 2003, RubyGems was launched as Ruby’s package manager. Six years later, Rubyists began using Bundler–a means for managing and installing gem dependencies in the context of an application. The combination of these two technologies has given developers the ability to run Ruby applications without having to worry about the specific gem version, gem source, or the platform that is available on the server. In this blog, we will review the changes in the Cloud Foundry
4 that provide better support for gems. We will discuss using Git URLs as a gem source, how you can use BUNDLE_WITHOUT to manage gem groups, and how Cloud Foundry installs only the platform specific gems that make sense.
Using Git URLs as a Gems Source
Most of the time, developers use default “rubygems” source to fetch gems from the official Ruby Gems repository. Alternatively, Bundler supports Git source URLs in order to associate a gem name and version with a certain Git repository. In this latter scenario, Bundler will automatically clone the latest version of a gem and install it. As of today, CloudFoundry.com fully supports using these Git URLs.
How does it work?
In the same way that Bundler installs gems from a Git source URL, resolving Git branches and references, Cloud Foundry locates Git dependencies in the Gemfile.lock, fetches the source code, and checks out the specified revision. Next, Cloud Foundry will find all of the gemspecs, build the gems and inject them into the application exactly where Bundler expects to see them. When the application gets started via “bundle exec,” Bundler picks up all installed dependencies as usual. To optimize the staging process, CloudFoundry.com also caches fetched Git sources. For example, if you reference the
Rails Git URL, CloudFoundry.com clones the repository and caches it in the local filesystem, so the next request for Rails will use the cache. If a requested revision is missing from the cache, there is no need to clone from scratch because only the missing objects will get downloaded from the original repo.
Example
Let’s take a look at the application
padrino shortener-demo. This demo is using the latest version of Padrino (a Sinatra-based web framework). As we can see in application Gemfile, gem padrino is requested to be provided from GitHub:
... gem 'padrino', :git => 'git://github.com/padrino/padrino-framework.git' ...
The padrino gem Git revision was locked in Gemfile.lock.
... GIT remote: git://github.com/padrino/padrino-framework.git revision: 17c748f8173185e57f9254829f53ee34327fa90d specs: padrino (0.10.1) ...
As we push this application, we provide the MongoDB service.
$ vmc push shortener Would you like to deploy from the current directory? [Yn]: Detected a Rack Application, is this correct? [Yn]: Application Deployed URL [shortener.cloudfoundry.com]: Memory reservation (128M, 256M, 512M, 1G, 2G) [128M]: How many instances? [1]: Bind existing services to 'shortener'? [yN]: Create services to bind to 'shortener'? [yN]: y 1: mongodb 2: mysql 3: postgresql 4: rabbitmq 5: redis What kind of service?: 1 Specify the name of the service [mongodb-9e641]: mongodb-shortener Create another? [yN]: Would you like to save this configuration? [yN]: Creating Application: OK Creating Service [mongodb-shortener]: OK Binding Service [mongodb-shortener]: OK Uploading Application: Checking for available resources: OK Processing resources: OK Packing application: OK Uploading (38K): OK Push Status: OK Staging Application 'shortener': OK Starting Application 'shortener': OK
Now, if we check application logs, we can see that padrino was provided to the application among other gems:
$ vmc logs shortener ==== /logs/staging.log 3.2.3 ==== ... Need to fetch mongo-1.3.1.gem from RubyGems Adding mongo-1.3.1.gem to app... Need to fetch bson-1.3.1.gem from RubyGems Adding bson-1.3.1.gem to app... Need to fetch tzinfo-0.3.29.gem from RubyGems Adding tzinfo-0.3.29.gem to app... Need to fetch padrino-0.10.1.gem from Git source Adding padrino-0.10.1.gem to app... Need to fetch http_router-0.10.2.gem from RubyGems Adding http_router-0.10.2.gem to app... Need to fetch rack-1.3.2.gem from RubyGems Adding rack-1.3.2.gem to app... ...
And now we can generate a shortened URL and track its visitors.
There may be situations where using the official published gem versions is not enough, such as using the current HEAD of the project, or specific branch, tag or fork. Cloud Foundry supports Git URLs in Gemfile, so it’s easy to point to a Git repo with the exact version of the library you need, and it will be downloaded and packaged as a part of your Cloud Foundry app.
Using BUNDLE_WITHOUT to Manage Gem Groups
Gemfiles support the grouping of gems so that a test server, for example, can get a different group of gems than a production one. The second feature we are announcing is that Cloud Foundry allows developers to take advantage of these groups by using the BUNDLE_WITHOUT environment variable, just as you would locally. Setting BUNDLE_WITHOUT causes Cloud Foundry to skip installation of gems in excluded groups.
Example
BUNDLE_WITHOUT is particularly useful for Rails applications, where there are typically “assets” and “development” gem groups containing gems that are not needed when the app runs in production. Let’s take a look at an example.
Spacely is a Rails 3.2 application that provides image upload via drag and drop. The Gemfile contains several gems in a group called “assets.”
... group :assets do gem 'sass-rails', '~> 3.2.3' gem 'coffee-rails', '~> 3.2.1' # See https://github.com/sstephenson/execjs#readme for more supported runtimes # gem 'therubyracer', :platform => :ruby gem 'uglifier', '>= 1.0.3' end ...
Let’s push the application to CloudFoundry.com without the gems in the “assets” group. We need to run “bundle install” first to generate a Gemfile.lock, which Cloud Foundry requires. Spacely includes a VMC manifest.yml file, so we can easily push the app without the full interaction. Notice that we push the app with the “–no-start” flag, so we can set the BUNDLE_WITHOUT environment variable before starting the application. We will make this step easier in the new version of VMC by enhancing the manifest support.
$ bundle install $ vmc push --no-start Would you like to deploy from the current directory? [Yn]: Pushing application 'spacely'... Creating Application: OK Creating Service [spaceltdb]: OK Binding Service [spaceltdb]: OK Uploading Application: Checking for available resources: OK Processing resources: OK Packing application: OK Uploading (6M): OK Push Status: OK
Now let’s set BUNDLE_WITHOUT and start the application:
$ vmc env-add spacely BUNDLE_WITHOUT=assets Adding Environment Variable [BUNDLE_WITHOUT=assets]: OK $ vmc start spacely Staging Application 'spacely’: OK Starting Application 'spacely': OK
Now, if we check application logs, we can see that gems such as “sass-rails” are not installed.
$ vmc logs spacely ==== /logs/staging.log ;==== ... Adding carrierwave-0.6.1.gem to app... Adding activesupport-3.2.6.gem to app... Adding i18n-0.6.0.gem to app... Adding multi_json-1.3.6.gem to app... Adding activemodel-3.2.6.gem to app... Adding builder-3.0.0.gem to app... Adding fog-1.5.0.gem to app... ...
Cloud Foundry also supports the exclusion of multiple groups. For example, if Spacely included a “test” group, we could have excluded gems in both assets and tests with “vmc env-add spacely BUNDLE_WITHOUT=assets:tests”.
Excluding Gems by Platform
Bundler allows developers to use the “platforms” method in Gemfiles to specify that a gem be used on particular platforms. This is the final piece of polish for gems that we are adding today. Cloud Foundry will skip the installation of gems on irrelevant platforms, as it should. The following Gemfile specifies that the rb-inotify gem should be used in non-Windows environments, while three other gems are for Windows only. When this app is deployed to CloudFoundry.com, only the rb-inotify gem will be installed.
# Unix Rubies (OSX, Linux) platform :ruby do gem 'rb-inotify' end # Windows Rubies (RubyInstaller) platforms :mswin, :mingw do gem 'eventmachine-win32' gem 'win32-changenotify' gem 'win32-event' end
The “platforms” designation can also be used to selectively install gems based on Ruby versions. For example, certain gems can be excluded when switching between Ruby 1.8 and Ruby 1.9.
Support for Windows Gemfiles
When a Gemfile.lock is generated on a Windows machine, it often contains gems with Windows-specific versions. This results in versions of gems such as mysql2, thin, and pg containing “-x86-mingw32.” For example, a Gemfile that contains the following:
gem ‘sinatra’ gem ‘mysql2’ gem ‘json’
When you run “bundle install” with the above Gemfile on a Windows machine, it would result in the following Gemfile.lock:
GEM remote: http://rubygems.org/ specs: json (1.7.3) mysql2 (0.3.11-x86-mingw32) rack (1.4.1) rack-protection (1.2.0) rack sinatra (1.3.2) rack (~> 1.3, >= 1.3.6) rack-protection (~> 1.2) tilt (~> 1.3, >= 1.3.3) tilt (1.3.3) PLATFORMS x86-mingw32 DEPENDENCIES json mysql2 sinatra
Notice the “-x86-mingw32” in mysql2’s version number. Previously this would cause a failure on deployment, as Cloud Foundry would attempt to install the Windows-specific gem. However, now Cloud Foundry will install the Ubuntu-friendly version of these gems without requiring any modification to Gemfile.lock. Developers can seamlessly migrate their app between local Windows machines and CloudFoundry.com.
Conclusion
The ability to use Git URLs as a dependency source and specify which gem groups should be installed depending on the platform provides greater flexibility for Ruby developers. The work we have done to enhance gem and bundler support is part of our commitment to providing a Platform as a Service that meets real needs of Rubyists building and deploying applications in the cloud. Follow us on Twitter at
@cloudfoundry and let us know how these new features are working for you! – Jennifer Hickey and Maria Shaldibina The Cloud Foundry Team
About the Author
