net/http alternatives

December 30, 2008 Pivotal Labs

net/http is slow. (and so are libraries that depend on it, like open-uri)

Performance Disclaimer: this ought to matter in your app, measurably, before you do anything about it. If you profile and ruby-prof is showing a bunch of classes like BufferedRead and Timeout at the top of the list, your app qualifies. And in addition if you know that your app is dependent on data transfer over http (let’s say you’re interacting with a Solr server, and you’re storing sizable documents in Solr), you should be aware of the problem.

Otherwise net/http or open-uri might be just fine for you.

The problems with net/http, and benchmarks of ruby http client lbraries are nicely written about in An analysis of Ruby 1.8.x HTTP client performance.

Some good alternatives:

Our findings matched the article referenced above – the alternatives have pros and cons but each was at least 10x faster than net/http for transfers of 50-300k response bodies.

The fastest solution we found was curb, reusing the Curl::Easy object:

require "curb"

curl = Curl::Easy.new

2.times do
    curl.url = "http://www.pivotaltracker.com"
    curl.perform
    puts curl.body_str
end

About the Author

Biography

More Content by Pivotal Labs
Previous
Best Buy & Pivotal Labs
Best Buy & Pivotal Labs

Kevin Matheny, Senior E-Biz Architect at Best Buy, has an excellent article today on BusinessWeek.com about...

Next
Keeping your errors in line
Keeping your errors in line

How many times has this happened to you? You get a cool design for your website, and you spend a bunch of ...

Enter curious. Exit smarter.

Register Now