Bugwatch: Visualization for Release Engineers

at May 8th, 2012

Visualizing the changes in health across your codebase and collecting metrics can be a meaningful strategy to catching issues early before they can cause destruction in your production environment. With this goal in mind, I developed a metrics dashboard for Ruby projects called Bugwatch. Bugwatch provides a convenient dashboard that enables you to track and display certain metrics based on commits as soon as the commit makes its way into the remote repository.

Bugwatch complexity chart

As a developer, you might already know some of the painful classes and methods in your codebase. You’ve felt the pain of running slow tests and you’ve experienced the growth in complexity of your business domain. Add a large team, and technical debt is bound to accumulate. You are encouraged to identify and trim technical debt as a part of your development process, but deadline pressure and the growing corpus of code you aren’t familiar with complicates your effort to unravel the mess.

Bugwatch augments the development process by creating a tight feedback loop as soon as code is pushed. Once a commit has made its way into Bugwatch and been analyzed, you receive an email notification if there are outstanding changes.

What is an outstanding change?

Bugwatch uses a strategy from Predicting Faults from Cached History to help identify potentially harmful hot spots in your code. It analyzes the complete git history of your project for the first time and identifies hot spots by looking for certain keywords in the commit message. During new commit analysis, Bugwatch will inspect the modified code and cross reference it with a cache of prioritized hot spots. An alert will be generated for each hot spot touched, and the developer is encouraged to review their code and make sure they are not causing a regression. Read about FixCache and how Google uses it here.

Notifications are useful to developers when they’re hacking on code, but in aggregate they form a picture of the overall health of the codebase and the difficulty a release engineer might encounter on the next deploy.

As a release engineer, it is your duty to assure that new code can be successfully deployed to production. Sometimes a deploy does not go well, and a decision needs to be made to either fix the code right away or revert the code to a prior state. Bugwatch helps improve this process by giving you the ability to inspect the commits going out in a deploy at a higher level of abstraction. The advantages of this approach are visualizations of the amount of code going out in a deploy and which changes might have a higher impact by the amount of complexity they accumulate.

Bugwatch tag diff

Future plans

Bugwatch started as a rudimentary project, but has grown into a configurable commit analysis platform. Tracking commit metrics is just the beginning, Bugwatch has identified and helped tighten the feedback loop and alert the developer of potential issues with their code. It complements the development process and provides release engineers with a high level visualization of deploys. My next target feedback loop is in the production environment, where exceptions are wild and need to be fixed.

Stay tuned…

No Tags

No comments yet

Leave a Reply

Your email address will not be published. Required fields are marked *