JR Richardson, Software Apprentice Graduate

at April 10th, 2012
Joe Banks and JR Richardson

Joe Banks and JR Richardson

Last fall JR Richardson moved from Iowa to join Groupon’s Chicago engineering team as a Software Apprentice. This week, he has graduated from the intensive six month on-the-job training program.

Congratulations, JR!

During his time in this paid software apprenticeship program, JR worked closely with his Groupon mentor, Joe Banks, a Groupon Tech Lead. Joe has been mentoring apprentices for several years, first as a Senior Consultant at Obtiva, and now at Groupon. Joe’s responsibilities as a mentor in the program were threefold: 1) introduce JR to Groupon engineering’s teams, technologies, and practices; 2) guide JR’s pet project; and, 3) expose JR to frequent learning opportunities. These opportunities included subjects such as Ruby on Rails, JavaScript,  software engineering theory, test-driven development, iterative project management and product management practices such as storyboarding (just to name a few).

During JR’s apprenticeship, he had three primary mandates:

  • Develop production software day-in and day-out as a member of several different Groupon engineering teams;
  • Develop a greenfield “pet” project that would demonstrate his ability to fully lead a non-trivial software project; and,
  • Demonstrate to the Groupon engineering team, through three equally-spaced milestone presentations, his ability to rapidly learn new technologies and techniques.

The pet project that JR decided to build over the six months was bugwatch, which he has been made available via github. bugwatch is a tool that helps integrate flog scores (how tortured your code is) to git commits. It helps release engineers and developers understand what commits are most at risk of wrecking havoc on the existing codebase.

You can learn more about Groupon’s Software Apprentice position by visiting the job listing.


Case Study: Poorly Written Cucumber Tests

at April 5th, 2012

Case Study: Poorly written Cucumber tests.

Today I had the pleasure of debugging a Selenium test written with Cucumber. The test would fail every 10 or so runs, and because of that it was disabled for several months. When I tried to fix it, it became pretty obvious why it was not a happy test, and why it got disabled in the first place.

Let’s do a quick run through of things NOT to do with cucumber.

1) If your feature file is 240 lines but only describes 8 features, you are not using cucumber as intended. Your feature file is probably filled with a lot of steps that look like this “And I fill in ‘Hello’ into ‘#this_is_some_id’”

The idea behind Cucumber is to describe the features of your application, and not the implementation of them. Leave the nitty-gritty implementation of the test for a step definition file. The rule of the thumb is, if you cannot understand what the test is supposed to check for in 30 seconds or less, your test needs a re-write.

2) Don’t Repeat Yourself (or DRY) principle applies to cucumber features as in any other programming situation. You can call any step definition from within any other step definition in Cucumber. Make small, one or two line step definitions and compile them into larger ones to complete the task at hand.

3) Feature descriptions and scenario names have to be very clear and educational. This is especially true if your feature is 84 lines long. Just like any comment for a complicated function, make the scenario name very descriptive. When a person has given up trying to understand the steps that you have written down, they will always jump up to scenario name to understand what is going on.

As for the Feature description, think of this area as an open text field. You do not have to limit yourself to one line description. Write several sentences; describe what is going on in the feature file. It’s far too common to see only 1-word feature descriptions.

4) Tags are a great way to group your tests. However, some teams use them for more purposes than just grouping. For example, adding @javascript or @selenium tag to a feature will force the feature to run in a web browser via selenium instead of a much faster headless browser.

Furthermore, some tags are used to start up a service that the feature is using. Some of these services take a long time to start up, and add a significant overhead to overall test run. Lesson being, do not just copy paste the tags from other features, or put a tag at the top of the feature file. Instead, start writing a feature without any tags, and slowly add the tags that apply.

5) Many times I have seen a feature file that is 300+ lines. My personal record has been a feature file with 1,200 lines. This is actually really bad for test performance and speed. For example, test parallelization gem “parallel_tests” reads in all of them feature files that you have, and segments them into groups. Often this means that one of the segments will contain the 800 line feature file, which means that after all the other segments have finished running we are still waiting for the big feature file to complete.

Try to keep your feature file down to 10 or less features, and find better way to group the feature files into smaller groups.


The Urbanairship Gem: Sending Push Notifications with Ruby

at February 6th, 2012

If you’re building an app that sends push notifications, Urban Airship is a service that can save you a lot of headache. They provide a common API for sending notifications to iOS, Android and Blackberry devices, as well as some useful features like notification batching, scheduling, and the ability to tag devices to make sending a notification to a large group of users more manageable.

At Groupon we wrote the urbanairship gem to wrap these API interactions, which you can install with gem install urbanairship.

Registering a device token

Before sending notifications to a device, you have to register it with UA. The simplest way to do this is:

Urbanairship.register_device('DEVICE-TOKEN')

You can also give it an alias and a set of tags.

Urbanairship.register_device('DEVICE-TOKEN',
  :alias => 'user-1234',
  :tags => ['chicago-users']
)

The registration call is idempotent. If you want to change a device’s tags or aliases later, you can resend the registration request. Just note that any attributes that are missing from the request will be removed, so be sure to include every tag and alias that you want to associate with that device token each time you make the registration call.

You can also set a ‘quiet time’ and timezone for each device. Check out the Urban Airship API docs for more options.

Sending a notification

Once your device tokens have been registered, sending simple notifications is easy.

Urbanairship.push({
  :device_tokens => ['DEVICE-TOKEN'],
  :aps => {:alert => 'You have a new message!', :badge => 1}
})

You can also specify tags, aliases, and even scheduled delivery times for your notifications. This code sends a push notification, delayed by one hour, to all the devices you’ve tagged with ‘chicago-users’.

Urbanairship.push({
  :tags => ['chicago-users'],
  :schedule_for => [1.hour.from_now],
  :aps => {:alert => 'Hello Chicago!', :badge => 1}
})

The Urban Airship API docs detail even more options that you can specify.

Batched and broadcast notifications

If your back-end system uses some sort of batch process for generating and sending push notifications to multiple users, you can use Urban Airship’s batch push method to cut down on the number of API requests you need to make.

Urbanairship.batch_push(
  {
    :device_tokens => ['DEVICE-TOKEN'],
    :aps => {:alert => 'Message one', :badge => 1}
  },
  {
    :device_tokens => ['DEVICE-TOKEN-TWO'],
    :aps => {:alert => 'Message two', :badge => 1}
  }
)

You can also send a message to ALL of your app’s registered device tokens with the broadcast push method.

Urbanairship.broadcast_push({
  :aps => {:alert => 'Hello EVERYBODY!', :badge => 1}
})

The feedback API

Sometimes a user will opt-in for push notifications on their device and then later disable them or uninstall your app. In these cases, if you try to send a push notification to that device, it will fail and Apple will register the failure in their feedback API. If you send too many repeat notifications to devices that don’t want to receive them, Apple will send you a warning or even revoke your ability to send push notifications.

Fortunately, Urban Airship steps in to help once again. If they notice you trying to send a notification to a device that can’t receive it, UA will refrain from forwarding that notification along to Apple, sparing you from their wrath. But this is really just a safety measure. They also offer a feedback API which you can use to find tokens that have opted-out. You should poll this API periodically and, for each token that comes back, delete or disable it on your end.

Urbanairship.feedback(24.hours.ago) # =>
# [
#   {
#     "marked_inactive_on"=>"2011-06-03 22:53:23",
#     "alias"=>nil,
#     "device_token"=>"DEVICE-TOKEN-ONE"
#   },
#   {
#     "marked_inactive_on"=>"2011-06-03 22:53:23",
#     "alias"=>nil,
#     "device_token"=>"DEVICE-TOKEN-TWO"
#   }
# ]

This retrieves all devices which rejected a notification in the last 24 hours. We’d then go through our database and mark those tokens as inactive or just get rid of them.

That’s it! With Urban Airship and a few lines of Ruby you can send cross-platform push notifications.\

(Source: The Urbanairship Gem: Sending Push Notifications with Ruby)


Lessons Learned From Teaching Rails

at February 6th, 2012

For the past six months, I’ve been teaching a four-day course for new Groupon Engineering hires that covers Ruby, Rails, RSpec, Cucumber, Git, and some Groupon-specific workflow, modeling, and scale issues. In general, new technical hires go through the course even if they aren’t immediately starting work on the web application, which means that the attendees tend to have a wide range of background and technical experience.

That’s a lot of ground to cover, everybody is tired by the end.
We keep tweaking the course in response to student feedback, my own impressions each time through, and random whim.

Here are some things I think I’ve learned about teaching Ruby and Rails and stuff.

The order matters, but it matters differently

The two biggest questions about teaching a course like this one are a) what order to approach the material, and b) how to handle hands-on work.

Broadly speaking, the order question is whether you go top down, starting with a high-level web page and working down to Ruby details, or bottom-up, starting with Ruby details and building up to a Rails page.

Both approaches have problems. If you start with Rails, you are doing a lot “we’ll explain that later” kind of handwaving, and the danger is that the student never gets a deep understanding of the material. That said, you do tend to focus on issues relevant to creating Rails apps.

If you start with Ruby, then you are doing a lot of “trust me, this is important” handwaving. The danger is that the early material in Ruby is deeply without context and meaningless.

I’ve come to the conclusion that there’s no one path through this material that works best for everybody. In our course, we mostly go bottom up, starting with a tour of Ruby before looking at a Rails app. In part, this is because of who the students are – most of them have some experience in web programming and generally get what the context is. Another reason is that as an instructor, I’m more comfortable saying “you’ll see why this is important soon” than “you’ll see how this works soon”.

Is it the best way to go? Not sure – one weird thing about doing these courses is that it’s hard to generalize from one group to the rest. I do worry that it’s taking longer and longer to get to Rails. However, the way we do it now does let us teach BDD/RSpec outside of a Rails context, which I think is valuable.

Hands On

The next big question is how much hands-on work to have in a course like this and how that should be structured. In general, people retain information better when they actually do something more active than listen to me talk. But it’s a slow process, and if it’s not structured correctly it can be very frustrating, especially if a student is struggling with some detail that isn’t necessarily the focus of the course. Which is not uncommon, since there are a lot of moving parts in any problem complicated enough to be interesting.

When we first started the course, we had the students start a Rails application all the way back from rails .. And I still think there’s some value in that, in seeing what the framework does for you. We don’t do that any more, we start with a preexisting, but small application that the students add features to. There are a few reasons for the switch:

  • Engineering management gently suggested that it would be rare for a new employee to actually start an application from scratch as part of their job.
  • Starting from scratch is somewhat limiting in the sense that the students have to build up all the infrastructure needed to explore more interesting areas. As a result, it’s hard to keep people from getting tangled in the weeds.
  • The desire to have the sample app be consistent with technology choices in the Groupon main app led to more and more setup, until it became easier to hand the application to the students with all the setup in place.

The last point was really what carried it. In particular, adding Haml to a Rails 2.x application, and converting the scaffolding to Haml wound up being a 20-minute march through arcane command line territory. That’s not a fun territory to be taking novices who, by definition, aren’t sure what’s important to remember. So we provide a skeleton with Haml, RSpec, Cucumber, authentication, jQuery, and some reasonable CSS defaults all set up. Probably a couple of other things too. Setup takes a lot less time, and since there’s a little bit the app already, the students can see some samples of say, a Cucumber feature, as examples.

I’m kind of perpetually dissatisfied with this part, though. Right now we do model stuff before view, which means it’s a very long time before the students make a significant visual change. Switching that order might be the next tweak. I also think there’s a perennial problem with making the problems sequence well, it always seems like there’s a big complexity gap between the initial interactions with the Rails app and the first – “here add a feature” problem.

It’s endlessly fascinating how reluctant students in this course are to pair on the exercises. I get it, I’d be reluctant to pair too – often the students don’t know each other, they don’t want to struggle in front of another person or they don’t want to be constrained by another person. Or maybe they just think there’s a fixed amount of learning in each exercise and pairing means they’ll only get half the learning. I need to insist more – you get much more out of a learning exercise if you are trying to explain what you are doing to another person. As an added bonus, it’s easier for the instructor to get a sense of the room when people are talking then when people are silently seething at their text editor. It’s easier for me to see where people might be stuck if they are already describing their situation out loud to the other pair.

Teaching Rails

There’s been a lot of discussion back and forth about how complicated Rails has become and what makes Rails complicated to teach. REST is a commonly noted culprit, and I think I’d say that the issue is not so much that REST a-la-Rails is that complicated. The issue is that it’s nearly impossible for a novice web developer to get exactly what problem REST is trying to solve or why it’s that much better than controller/action. (That assumes that there is a general problem being solved, I guess…)

I think Rails 3 is an improvement here because both the routing language and the Arel/Active Record syntax are easier to manage than the Rails 2.x counterparts. Although, I have yet to meet anybody that I’m sure understands the Rails 3.1 asset pipeline, let alone is ready to teach it.

Beyond that, the biggest issues I see in teaching Rails:

  • The sheer size of the framework and the way it bleeds into the standard Ruby library. There’s so many potential things to talk about, and there’s always confusion as to what Rails does, what Ruby does, and what’s actually part of the application.
  • The ecosystem, which is awesome, but intimidating. Even in this four-day course we cover at least four distinct syntaxes (Ruby, Cucumber, Haml, Sass), plus RSpec, plus an authentication API. And that’s just to get off the ground.

Teaching Testing

Testing is a big part of what we try to get across in the class. We’ve made a lot of progress in the structure of the class to encourage BDD, and the last few times most of the exercises have been done using RSpec and Cucumber.

My experience is that it is much easier to make the value proposition for BDD/RSpec to a skeptical student then it is to make the value proposition for Cucumber. I’ve had surprisingly little resistance to test-first in general, though there is a frequent undercurrent of students who don’t take to RSpec’s syntax. (To overgeneralize, this is often people from a more Enterprisey Java background who don’t like the showy metaprogramming.)

I’d caution against making broad generalizations from that – part of the issue is definitely that I’m more confident presenting the value proposition for RSpec. Another part of the reason is clearly the amount of overhead needed to walk through your first Cuke. Actually, I think the now-deprecated Cucumber web_steps.rb file gets in the way even for beginners – walking somebody through the web_steps.rb and paths.rb file to resolve something like “the users page” to a URL quickly takes on the appearance of a practical joke.

Quick hits:

  • It continually surprises me how hard it is to explain Ruby symbols. A lot of languages don’t really have analogous structures. (Although a lot of languages don’t have Ranges either, and usually people get those pretty quickly). But I’ve struggled to get the concept clear a couple of times when students really struggled with how to think about them.
  • In contrast, I generally don’t have troubles explaining blocks – even developers who have never worked in a language that has a similar construct can generally approach them as analogous to functions. (That said, nobody ever understands why Ruby has blocks, Procs, and lambdas. But that’s okay, I never quite got it either.)
  • Personally, my favorite part of the class is always when we talk about the Ruby object model and metaprogramming. First off, I get to say things like “The instance representing the class Document is different from an instance of the class Document” and “class methods of the class are really instance methods of the instance representing the class”. And I show off the source for StringInquirer, which on one occasion caused an entire class to burst into laughter when they all figured it out more or less simultaneously.
  • I think I’ve finally gotten to the point where I can confidently explain git rebase vs. merge. It has only taken four years.

Emptying the Cup. Forgetting Everything I Know About Java and Embracing Ruby

By
at February 6th, 2012

The Warrior, abandoned by his old masters searches the land and finds a  new one.  The Warrior asks to be of service. The master notes his dress and stance. “It seems that you are skilled and ready, but your masters taught you skills that would be of no service to us here.”

The Warrior is a quick study, and this does not cause dismay. “I am very disciplined and loyal. You will teach me new skills, and I will serve you well.”

The Master agrees, and invites the Warrior in for tea. They sit and the Master places a cup before him and fills it with tea.  He allows it some time to cool, but before he takes his first sip, the Master pours again and watches as it flows over the sides and pools on the table.

Puzzled, the Warrior moves to clean it up.  The Master stays his hand.  “This is the first lesson you must learn from me. Your cup is already full, making it impossible for me to fill you with any more knowledge or skill. How can you be of service to me?”

The Warrior ponders the flow of the pool before him, and lifts his eyes to the Master.  Slowly, he brings the cup to his lips and drains it. “Then I must empty the cup”

That’s a familiar parable if you’ve ever watched Kung Fu movies or pretended to be a Buddhist. I wouldn’t put it past a Yoga instructor to tell it either – but in some cases, this holds true.  I just left 10 years of working in Java shops for an awesome opportunity. The only problem, is that they are strictly a Ruby on Rails team – and even worse, they have all been doing it since Rails inception.  They are talented – I am too, but Ruby looks like one of those overly concise sysops languages (*cough* Perl *cough*). It looks like half the language is missing, and when a developer gets really good at it – it becomes mind-blowingly unreadable.

At least it does right now. I have a full cup of Java. It was the core language in college and I’ve used it and all the associated Web frameworks for 80 % of every piece of code I’ve ever used.   I compile. I use return statements. I like my variables typed!  This is alien, but let’s see what I can do with it.

Join me as I empty my cup, and see how far and fast I can fill it back up again.


The Rails Engine That Could – In Motion

at February 6th, 2012

I recently gave a talk at Groupon’s weekly GeekFest event titled “The Rails Engine That Could – In Motion”. It was an introduction to Rails Engines that highlighted a successful use of them while consulting for a client through Obtiva.

I will include here a summary of my talk as well as the slide deck.

In a nutshell, the problem that Rails Engine aims to solve is code reuse that cuts across models, views, and controllers. This usually arises whenever there is a need to reuse screens of information across multiple applications while slightly customizing the look and feel and data displayed per app.

Here is an example of such functionality:

College Search Map

 

Developers often reuse the data models involved by possibly offloading the model logic into gems or web services that can be reused from the different applications, but they still duplicate the controller and view logic, especially when there is a need to make adjustments.

Rails Engines solve that problem by allowing developers to reuse models, views, and controllers while supporting customizations across multiple applications.

I actually stumbled upon several patterns in my use of Rails Engines that ensure their implementation is application-agnostic (low coupling to applications) while still allowing for high flexibility for different needs. The patterns scale up in complexity to allow developers to employ them in an Agile fashion, starting with simpler solutions and scaling them up as applications demand more customizability from engines.

Here is a summary of the Rails Engine patterns discovered:

  • Common Domain: allow multiple apps to share a common domain
  • Expose Helper: customize view presentation logic in different apps
  • Expose Partial: customize a part of the view content in different apps
  • Extension Partial: allow different apps to contribute content for a specific part of the view
  • Extension Point: allow different apps to contribute content in different parts of the view
  • Configurable Features: allow different apps to request features from an engine in different combinations or adjust an engine’s configuration settings

The key benefit for relying on Rails Engines is eliminating duplication and improving productivity in the long run via code reuse cutting across the MVC layers. Refinery CMS is one public example of a Rails Engine that can be dropped in Rails applications to instantly add content management capabilities, including screens and routes. It would be interesting to see how they will shape future Rails projects as more developers find uses for them in the Rails community.

Here are the slides of the talk that I gave at Groupon’s GeekFest:

Hopefully, this will motivate other developers to avoid duplication and rely on Rails Engines whenever they need to reuse functionality cutting across models, views, and controllers.


GrouHackPon

By
at November 28th, 2011

Hey All,

Today was our inaugural GrouHackPon, one of many more to come.  This one was open to internal Groupon folks, however the next will be open to the public.  We ate food, we were merry, and we hacked on some awesome code.  We left the format pretty loose, so we have several break out projects.

There was a group working on a Node.js project that Dan Nawara started, which is a planning poker app which hooks into Pivotal Tracker and makes estimating stories more interactive, did I mention that it was in Node.js?  There were other teams improving performance of the site, speeding up specs and tacking an expensive DB call that we have.

Overall, it was a great first start, looking forward to our next one, end of November, which will be in tandom with the Node.js meetup.