Sanitizing POST params with custom Rack middleware

June 11, 2009 Pivotal Labs

The problem: Improperly escaped post data

I recently worked on an app that processed xml files. Once a week, a legacy system posted a large xml document to the app. For almost a year the app worked perfectly, and then we updated to rails 2.3.2 and the posts started failing spectacularly. Looking at the log files, I noticed that the params were incorrect:

<code>{"message"=>"hello", "xml"=>"<xml>Foo &amp", "Bar</xml>"=>nil, "action"=>"not_scrubbed", "controller"=>"examples"}</code>

After looking into it further, I realized that the data that was being posted contained semi-colons:

<code>xml=<xml>Foo %26amp; Bar</xml>&message=hello</code>

It turns out that rails used to only split params on ampersands, but that rack splits on both ampersands and semi-colons. We couldn’t change the legacy system, so we had to remove the semi-colons before the post params got to rails.

The solution: Rack middleware

Using Rack middleware it’s was easy to insert code before rails params parsing code executed. To start, build a class that conforms to the signature of a rack middleware layer, like so:

<code>
# lib/scrubber.rb
class Scrubber
  def initialize(app, options)
    @app = app
    @routes = options[:routes]
  end

  def call(env)
    scrub(env)
    @app.call(env)
  end

  private
    def scrub(env)
      return unless @routes.include?(env["PATH_INFO"])
      rack_input = env["rack.input"].read
      params = Rack::Utils.parse_query(rack_input, "&")
      params["xml"] = Rack::Utils.unescape(params["xml"])
      env["rack.input"] = StringIO.new(Rack::Utils.build_query(params))
    rescue
    ensure
      env["rack.input"].rewind
    end
end
</code>

Then register the middleware from environment.rb:

<code>
  config.middleware.insert_before ActionController::ParamsParser,
                                  "Scrubber",
                                  :routes => [ "/examples/scrubbed" ]
</code>

To verify that this works, use curl to send the request, like so:

<code>curl -d 'xml=<xml>Foo %26amp; Bar</xml>&message=hello' http://localhost:3000/examples/scrubbed</code>

I’ve put together a sample app on github that gives a working example of the code above which you can find at http://github.com/zilkey/params-scrubber/tree/master.

About the Author

Biography

More Content by Pivotal Labs
Previous
First New York Tracker Users Group meetup on Jun 30
First New York Tracker Users Group meetup on Jun 30

We're starting a Tracker Users Group in New York, and the first meetup is on Jun 30, at 6:30pm, at the Pivo...

Next
Announcing Tweed for the Palm Pre and Palm webOS
Announcing Tweed for the Palm Pre and Palm webOS

We are happy to announce Tweed, a twitter client for the Palm® Pre™ and Palm webOS™ Check out http://tweed...

Enter curious. Exit smarter.

Register Now