I-Tier: Testium

at January 9th, 2014

We recently shared the story of our transition to Node.js at Node Summit. As part of that transition, we developed an integration testing tool called testium.

We’d like to announce that testium is now open source!

What is testium?

With testium you can write integration tests that:

  • use a familiar BDD syntax
  • are written in CoffeeScript/JavaScript
  • have a synchronous API
  • can leverage Selenium

Why Create a New Tool?

Since the beginning of our transition to Node.js, we have been looking for a great way to include integration testing in our Node.js applications. In Ruby on Rails, we had a Cucumber setup that could run tests in real browsers via Selenium. The relationship with testing tools was not strongly defined for Node.js (the platform) or even Express (the underlying framework).

That lead us to look for other existing tools we could adopt. We tried a couple of tools, but they just didn’t fit our needs for different reasons.

Exploring WebDriver

We wanted to write tests in JavaScript and Mocha with a synchronous API that takes advantage of our existing Selenium infrastructure. WD.js came close, but the challenge for full adoption came with the syntax, which is callback-based or promise-based. These are typical patterns in JavaScript projects, but for an integration testing tool, we only want to execute serial actions. A synchronous API made the most sense.

A spike seemed to be in order. We read up on the WebDriver spec and started implementing Node.js bindings that used http-sync to get synchronous http. The spike used Mocha to run the tests and provided an API that executed WebDriver calls against a local Selenium Standalone Server. This approach seemed to be so beneficial for us that we decided to move forward with it.


testium is the result of that effort. Here are a few examples.

Navigation is easy:

DOM Interaction can be done in a couple of ways:

Cookies can be set before making requests:

Screenshots can be taken and (experimentally) diffed:

Failures automatically take screenshots for you:

You can see that a screenshot has been saved “[TESTIUM] Saved screenshot {app}/test/integration_log/screenshots/on_failure.png”.

Testing Stack Consistency

One great benefit of this setup is that our test suites are now very similar:

Server Unit Tests: Mocha + Bond + Assertive
Client Unit Tests: Mocha + Bond + Assertive
Integration Tests: testium (Mocha) + Assertive


The philosophy of WebDriver is that it should allow you to programmatically act like a user, but it does have limitations we have to work around. As a result there are some features that the maintainers will not implement because a user would not be able to exercise that feature autonomously.

Further, there are existing technical limitations due to WebDriver being a common standard across all browsers. Some aspects can’t be implemented in a specific browser and therefore these features are not implemented at all.

There are four specific issues with WebDriver we wanted to resolve:

  • response status codes unavailable
  • response headers unavailable
  • request headers not modifiable
  • page must be loaded to set cookies

Details on our workarounds will come in a follow-up post.

Going Forward

testium is already serving our needs, but we have some features planned for the future. Development will continue on the GitHub repository. Check out the Roadmap and let us know what you’d like to see by creating issues. Pull requests are always welcome!

No Tags

7 thoughts on “I-Tier: Testium

  1. [...] via Hacker News https://engineering.groupon.com/2014/misc/i-tier-testium/ [...]

    pingback by Testium, Groupon’s node.js tool for browser integration testing now open source | Enjoying The Moment on January 10, 2014 at 11:38 am
  2. Very nice work, I look forward to giving testium a try.

    by Josh McVey on January 10, 2014 at 4:13 pm
  3. We use similar technique - fibers to make async JS synch to eliminate callbacks, here's some slides http://petrush.in/blog/2013/client-side-qa We also found that creating DSL / helpers and scopes may be greatly beneficial, because it makes tests looks more like human language and less like CSS selectors.

    by Alexey Petrushin on January 10, 2014 at 10:59 pm
  4. I 'm s student in the U of Utah,SaltLakeCity. As a research project , I am working on Node Js with Express Framework to develop a a Single Page Application. I 'm interested in summer internship in Groupon. Do you have opportunities.

    by Venkata Vineel on January 18, 2014 at 9:52 pm
  5. the first three problems will be solved with browsermobproxy

    by Artem on January 29, 2014 at 4:30 am
  6. Hello, Groupon is accepting candidates for summer internships! Please access the "University" link and follow the the appropriate directions to view all available opportunities. Link: https://jobs.groupon.com/university/ Thanks!

    by Allison on February 21, 2014 at 11:59 am
  7. This is amazing! Thank you so much for creating this (and for webdriver-http-sync, too)!

    by Jason Huggins on February 22, 2014 at 9:22 am

Leave a Reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <s> <strike> <strong>