Clover Test Optimization

The new Clover 'Test Optimization' feature seems pretty appealing.  Using coverage data, Clover can determine which tests need to be run to cover the changes you've made, and then run these (or run these first), such that you can get your testing done faster and/or fail faster. 

Ruby on Rails on Bamboo: Unit Tests

In my previous post about Continuously Integrating Ruby on Rails with Bamboo, I walked through a simple setup for a Ruby on Rails project in Bamboo. That setup is a good starting point, but it's missing useful elements, like understanding the unit tests within the project, and test coverage:


Unit Testing Ruby on Rails with Bamboo
Let's start with unit tests. Bamboo understands the JUnit XML format, which Ruby doesn't normally emit (what with not using JUnit and all). Fortunately, someone's written a Rails plugin called ci_reporter that emits these files for us.

$ gem install ci_reporter
Bulk updating Gem source index for: http://gems.rubyforge.org
Successfully installed ci_reporter-1.3.3
Installing ri documentation for ci_reporter-1.3.3...
Installing RDoc documentation for ci_reporter-1.3.3...
$ script/plugin install http://svn.caldersphere.net/svn/main/plugins/ci_reporter
+ ./ci_reporter/tasks/ci_reporter.rake
$ svn add vendor/plugins/ci_reporter/
A         vendor/plugins/ci_reporter
A         vendor/plugins/ci_reporter/tasks
A         vendor/plugins/ci_reporter/tasks/ci_reporter.rake
$ svn ci -m "Added CI reporter to project."
Adding         vendor/plugins/ci_reporter
Adding         vendor/plugins/ci_reporter/tasks
Adding         vendor/plugins/ci_reporter/tasks/ci_reporter.rake
Transmitting file data .
Committed revision 411.

You'll probably have to ensure the ci_reporter gem is installed on your instance of Bamboo as well. Once this is complete, you'll want to modify your Plan:



And build the project:


See? Tests! Bamboo can find the XML files generated in test/reports, and use them to understand the testing of your Ruby on Rails application. Now, in a Java/JUnit project, Bamboo does a fine job of turning the test methods into readable names, but for now that's not true with Test::Unit test names, but they're working on it.


Next step: Coverage. But that'll be another day.

Ruby on Rails continuously integrated with Bamboo

Since I tripped over a few things setting up Bamboo for Rails, I thought it would be good to share my approach. After my last post, someone from Atlassian thought that would be a good idea as well, so, what the heck.

Creating a Simple Rails Project

In order to demonstrate configuring a project on Bamboo, it's easiest for me just to create a simple rails project, with a model and scaffold. This'll come with tests, albeit useless ones. But if you wanted a tutorial on how to use Rails, you've come to the wrong place.

geoffrey@paraietta:~/projects$ rails bamboo-rails
create
create  app/controllers
create  app/helpers
create  app/models
...
create  log/production.log
create  log/development.log
create  log/test.log
geoffrey@paraietta:~/projects$ cd bamboo-rails/
geoffrey@paraietta:~/projects/bamboo-rails$ mysql -u root
Welcome to the MySQL monitor.  Commands end with ; or \g.
Your MySQL connection id is 8
Server version: 5.0.38-Ubuntu_0ubuntu1-log Ubuntu 7.04 distribution

Type 'help;' or '\h' for help. Type '\c' to clear the buffer.

mysql> create database `bamboo-rails_development`;
Query OK, 1 row affected (0.01 sec)

mysql> create database `bamboo-rails_test`;
Query OK, 1 row affected (0.00 sec)

mysql> quit
Bye
geoffrey@paraietta:~/projects/bamboo-rails$ script/generate model foo bar:string baz:string
exists  app/models/
exists  test/unit/
exists  test/fixtures/
create  app/models/foo.rb
create  test/unit/foo_test.rb
create  test/fixtures/foos.yml
create  db/migrate
create  db/migrate/001_create_foos.rb
geoffrey@paraietta:~/projects/bamboo-rails$ rake db:migrate
(in /mnt/home/geoffrey/projects/bamboo-rails)
== CreateFoos: migrating ======================================================
-- create_table(:foos)
-> 0.0040s
== CreateFoos: migrated (0.0041s) =============================================

geoffrey@paraietta:~/projects/bamboo-rails$ script/generate scaffold foo
exists  app/controllers/
exists  app/helpers/
create  app/views/foos
...
create  app/views/layouts/foos.rhtml
create  public/stylesheets/scaffold.css

Add the Project to Source Control

Once the project's created, it has to be somewhere that Bamboo can reach it. The usual practice is a source control system. In this case, I'm using Subversion.

geoffrey@paraietta:~/projects$ svn import bamboo-rails http://messis/svn/spikes/bamboo-rails -m "A simple Rails/Bamboo integration project."
Adding         bamboo-rails/test
Adding         bamboo-rails/test/unit
Adding         bamboo-rails/test/unit/foo_test.rb
...
Adding         bamboo-rails/public/stylesheets
Adding         bamboo-rails/public/stylesheets/scaffold.css
Adding         bamboo-rails/public/favicon.ico

Committed revision 190.

Configure Bamboo for Ruby

In order to build anything in Bamboo/Ruby, you're going to want to configure a builder in Bamboo. There may be a way to use one of the existing builders, but this is what worked for me. Just add a builder for Ruby and point it to your ruby executable.

Create a Plan for your Ruby Project

You're going to want to create a Project and Plan for your Ruby project. Bamboo projects are umbrellas for related Plans:

Plans tell Bamboo what you're trying to build, where to find the source code:

They also tell Bamboo how to build what it finds:

They also tell Bamboo what to do with the results, and whom to notify, but those steps aren't unique to Rails in any way, so I'm going to leave them out. At this point, you have a project and plan defined in Bamboo, so you can check to see if it worked:
It didn't. I didn't create the database on the build server. Once I've done that:

Et Voila

You've got a basic Ruby on Rails project building in Bamboo. This is the bare minimum setup. There's a lot of things you can add from here. In my next post, I'll cover getting Bamboo to recognize your test results and calculate coverage.

Bamboo, Ruby on Rails, Test Reports and RCov

Over the last day, I've had an opportunity to integrate Atlassian's bamboo with a Ruby on Rails project, which has been fruitful. It was actually pretty easy. Create a builder that runs Ruby (rather than Bash or Maven), point it at 'rake' with the appropriate target, and it builds.

Tests
If you want your test reports to be visible as test reports, you might want to look up the ci_reporter plugin and gem. This simply ensures that when your tests run, they generate JUnit-style XML files that many CI servers can pick up, including Bamboo.

Coverage
If you add the rails_rcov plugin to your project, you can run the test:test:coverage goal in order to generate a coverage report. All you need to do then is tell Bamboo to treat it as an artifact, scoop up all the contents of coverage/test. Then, from your build results, you can go to the artifact tab, and pick up the resulting coverage report, nicely viewable in your web browser.

See Also
If you want to find out more about integrating Atlassian's Bamboo with Ruby on Rails, I recommend you not search for 'bamboo rails' or 'bamboo rake', both of which are likely to take you elsewhere. ;)

Are any of you using Bamboo and Rails together, and if so, do you have more interesting tips to add?

Ambient Orb: Build Status

A few people updated this morning after a published build failure, and are waiting for the fixes to come down before they can really proceed.

Now, because the build failure was published, I only have so much sympathy, but it got me thinking about possible build statuses. Coming back to the Ambient Orb approach, it occurs to me that I'd really like three colors to be used for build status:

• Green: Last build was successful. No new changes (or, at least, not aware of new changes). Safe to update.
  
• Yellow/Amber: Last build was successful. Aware of new changes that could theoretically destabilize the build (commits to the project, or projects on which this project depends). Status of build unknown, but tentatively safe.
  
• Red: Last build was not successful. There may or may not be changes, but at this point, it's probably safer not to update, until the orb goes back to green or amber.

If you used an orb this way, you'd expect it to be in 'yellow' state a fair amount, but it seems useful to be able to tell the difference between yellow and green. In order to ensure that the orb goes yellow as soon as possible, I'd consider commit-hooks rather than repository-polling.