Black Hole Proxy for Selenium Testing

at July 29th, 2013

Even if you are new to automated testing of websites, you have probably already discovered Selenium. It’s a great tool for mimicking real user interactions with web applications. Selenium will help you cut down on the tedium of testing every single form.

There is a price to pay because all of Selenium’s testing is done in a way that closely mimics a real human being. Tests are often slow because each page has to be loaded with all the external images and external services. This not a typical issue to humans, because when the page is 99% a human knows that interaction can begin. Robots execute exactly as they are told; so, “wait for page to load” means they will wait for that 1% to finish even though page is technically loaded.

At one point we would have 10 or more builds failing, because the Facebook Like button (or similar 3rd-party asset) was not loading fast enough. This became increasingly frustrating, because the tests, which were failing, were not testing the external service integration.

The solution is obvious. Why not block out any and all 3rd party services if we are not explicitly testing them? Tests run faster and they are less flakey. Our solution is the “Black hole Proxy.” The idea being all requests made via the proxy disappear and cannot escape the gravity of how great an idea this was! (Yes, I can hear you groaning.)

The proxy takes simply returns a status code of 200 with an empty body. Images from services will not be loaded on the screen. However, if you are not explicitly testing those images or those buttons, this should not be an issue at all.

Let’s look at some code!

Step 1: Take over “em-proxy” ruby gem

Now, you don’t have to write this is ruby, the idea is simple to do in any language of choice

Here is some sample code:

 
require 'rubygems'  
require 'em-proxy'  
require 'http_parser.rb'  
require 'uuid'  
require 'uri'

Proxy.start(:host => "127.0.0.1", :port => 9001) do |conn|

  @p = Http::Parser.new  
  @p.on_headers_complete = proc do |h|  
    session = UUID.generate
    @request_host, request_port = h['Host'].split(':')
    File.open("blocked_external_urls.log", 'a') {|f| f.write("#{@request_host}n")}
    @request_host = "127.0.0.1"
    conn.server session, :host => @request_host, :port => (request_port || 80)  
    conn.relay_to_servers @buffer
    @buffer = ''  
  end

  @buffer = ''

  conn.on_connect do |data,b|
  end

  conn.on_data do |data|  
    @buffer << data  
    @p << data  
    data  
  end

  conn.on_response do |backend, resp|  
    resp  
  end

  conn.on_finish do |backend, name|
  end

end

The only interesting part of this is the @p.on_headers_complete method. We decided to capture all of the external URL requests into “blocked_external_urls.log” file. After a couple of test runs, you may be surprised how full that file gets after a single test run.

Step 2: Tell your browser to use the Black Hole Proxy

#Setup Firefox profile to use proxy  
profile = Selenium::WebDriver::Firefox::Profile.new  
profile["network.proxy.type"] = 1  
profile["network.proxy.http"] = "127.0.0.1"  
profile["network.proxy.http_port"] = 9001

#Add urls to be exluded from Black Hole proxy  
profile["network.proxy.no_proxies_on"] = "localhost, 127.0.0.1"

#Add profile to options  
options[:profile] = profile

#Create browser  
browser = Selenium::WebDriver.for :firefox, options

Things to look at here are the “network.proxy” configurations that come from Firefox. Basically we are pointing all of the browser’s requests to go through the proxy.

Next is the whitelist “network.proxy.no_proxies_on” where we tell Firefox to ignore proxy settings for some hosts. Currently we cover localhost and 127.0.0.1, but you can add external pages like www.google.com in you rely heavily on them. This is not ideal, because you are cancelling the benefit of the proxy in the first place.

What you should do is group any test that explicitly needs Facebook, or Google, and let them start their own Firefox profile that does not use the proxy at all. This way you isolate all of the tests, which depend on external services away from your local stable tests.

Step 3: Profit

After adding this proxy to our Jenkins instance, the failure rate of selenium tests dropped significantly. Tests, which were failing 1 out of 30 runs due to external timeouts, stopped failing all together.

No Tags


3 thoughts on “Black Hole Proxy for Selenium Testing

  1. Great idea Dima! Thanks for sharing it. I had the problem with 3rd party services in my Selenium tests too, with this update, tests work much bette. One type of timeout less to worry about :)

    by Igor Balos on September 24, 2013 at 10:51 am
  2. Great idea, but I was wondering, Is there a way to add a timeout to the current proxy connection ? Because if it's a heavy load server, sometimes you may want to use a mirror for proxy.

    by Alex on May 24, 2014 at 7:49 pm
  3. @Alex: After re visiting this topic in writing a book about selenium stability, I noticed that an actual proxy is not nessery for Firefox unless you wanna keep records of the external urls. So a non existing proxy to www.foo-bar-something.com port 8888888 will work just as well to get rid of 3rd party dependence

    by Dima on May 29, 2014 at 7:50 am

Leave a Reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <s> <strike> <strong>