Clover Test Optimization

The new Clover 'Test Optimization' feature seems pretty appealing.  Using coverage data, Clover can determine which tests need to be run to cover the changes you've made, and then run these (or run these first), such that you can get your testing done faster and/or fail faster. 

SubEtha SMTP, Maven and a Community Repository

We'd been using Dumbster to run some email integration tests. I've used it before, and it gets the job done. However, when running the overnight timed build (to drive out problems, we run the build repeatedly overnight), the email integration tests would occasionally lock up, never exit, and cause the build to run forever. A few logging statements drove out the source of the problem -- the Dumbster.start() method was going away and not coming back.

I'd previously seen the suggestion that SubEtha's Wiser was as capable and better code (less flaky, fewer problems) so I quickly swapped out Dumbster for SubEtha. Or, so I hoped. The swap was pretty easy, the hard part was getting a Maven-ready version of Dumbster with:

• A versioned JAR: Grab the JAR from the download, upload it to our local repository, give it a group id and a version number (org/subethamail/subethasmtp/2.0.1/subethasmtp-2.0.1.jar).
  
• A versioned JAR of sources: ZIP the source directory starting with the top-level package, rename it to a JAR, upload it to the local repository with the same group and artifact id as well as a version number (org/subethamail/subethasmtp/2.0.1/subethasmtp-2.0.1-sources.jar).
  
• A POM listing the transitive dependencies: Create a POM, check all the dependencies in the lib directory, look up each to find out the appropriate artifact and group identifier, spend a little time evaluating the smallest set of dependencies that are necessary (if one includes another, such as was true for parts of Mina), then upload that as the POM (org/subethamail/subethasmtp/2.0.1/subethasmtp-2.0.1.pom).

This isn't a vast amount of work, and it could be simplified a little by tooling (that last step in particular), but it's a pain, and it's entirely preventable, if someone, ideally the project itself, provides the JARs and POM for you.

Since I'd already done the work, I posted my work to the subetha issue tracker in the hopes that they'd adopt it. No such luck. Their response:

That's great and all, but I don't really care about Maven, nor do I really want to add or maintain a pom.xml in the project. Sorry.

I admit, if I weren't using Maven on my project, the idea of keeping a POM up to date and uploading it to a remote repository would seem like a pain, and I've seen that attitude before. That said, if you want your project to be adopted by people, this is one way to ease the transition for some group of users, so it's unfortunate that projects take this stance.

A Community Repository
With that in mind, it'd be great if there were a public community repository for Maven which was like the public repository, but held artifacts for projects that were unwilling to supply them, but willing for someone else to maven-ize their project. Members of the community who'd created a Maven-enabled version for their own use could post their version for others to pick up.

This could be as simple as a second public repository which could be added, or perhaps even simpler, a top-level folder in the Maven public repo called unofficial where unofficial artifacts could be posted. The latter would ensure that Maven users would be fairly clear up front that they were pulling in an unofficial artifact, by virtue of the group id.

The primary danger in this approach is that people would fail to respect the license terms of software -- by packaging a piece of software for Maven, you may be considering redistributing it, and you'd have to be careful, as some software projects might not wish to have this happen.

As long as it were possible to have one's project removed from the repository and if the community tried to respect the desires of those projects, this still seems like it could be a very helpful addition.

The Pain of Java after Ruby

Having done several months of Ruby in the past year, I occasionally find myself looking at a page of Java code and thinking to myself, "Wow, that's Java alright."

For instance, within the past few weeks, I needed a URI for testing. I made it a constant (private static final). Since it can throw an exception, I had to put the constant in a static initializer. Since the exception might be thrown, the variable might not be initialized, and so on and so forth. Eventually, you end up with this:

   private static final URI SOURCE;

   static {
       URI uri = null;
       try {
           uri = new URI("/");
       } catch (URISyntaxException e) {
           Assert.fail("Couldn't create a URI for testing; please fix the test.  This shouldn't happen.");
       }
       SOURCE = uri;
   }

Although I understand the rationale behind each of the steps that gets you here, it's a bit of a boiled frog syndrome. By solving one problem with slightly more complex code, and repeating that pattern a few times, you eventually end up with pretty horrific code. Thanks, Java!

Now, to be fair, there are other ways around this code. For instance, I could easily create this as an instance variable in the test setup code, which in many ways would be a more normal approach, and it'd take all the above code away. That's probably the most obvious resolution for the problem at hand, but there are certainly others.

In any case -- because there are alternate solutions, this isn't really a problem, but having painted yourself into this particular corner it's easy to think to yourself, "Wow, this code would be much nicer in Ruby."

Qcon in Review: November 7th

Summarizing November 7th, for those not following my tumbl'd thoughts.

Kent Beck's opening keynote was well-presented, but I didn't take much away from it other than the reason behind his shaved head.

A panel discussion on Architecting for Performance and Scalability was good, but it's hard to cover that kind of topic in an hour, particularly with a panel. Of particular note was Ari Zilka arguing that scaleout for relatively small scale should focus on load-balancing rather than partitioning.

After lunch, I attended Designing for Testability, with Cedric Beust and Alexandru Popescu. This was perhaps a mistake for me, having done a lot of pervasive testing and having used TestNG, I didn't really learn very much. This was the second time I felt that way, so I realized I was going to have to watch which topics I attend -- the ones I'm most interested in are often the ones I know the most about, and am least likely to learn something from an hour-long primer. TestNG contributed a great deal to the world of Java testing, but I feel like JUnit has been revitalized by that competition and the choice between the two is less clear than it was between TestNG and JUnit 3.8.1. Cedric also showed his usual bias against agile and TDD: which is fine, as far as I'm concerned, you use the process that works for you.

Next up was Eric Evans on Strategic Design. This was a good talk, although he's a measured speaker rather than overflowing with energy. Still, he's a clear communicator, and there were some interesting points. I particularly liked the metaphor of maps as models. His anecdote about pudding, which relies on the language context (american pudding: goey custard-like dessert; british pudding: any dessert) was also entertaining. Some of the detailed elements seemed vague, but it was still enjoyable.

I followed that up with Cameron Purdy's Java Scalability and Reliability. I'd seen this presentation (or a heavily related one) before, but he's such an entertaining speaker, I didn't mind seeing it again. He's also not afraid to stir up a little humorous controversy: taking a shot at Bob Lee and calling FTP the precursor to SOAP.

Finally, a panel discussion on What will the future of Java be? was really great. There was a lot of hunger in the room to see Java evolve in some fashion, whether or not it's part of its current platform, or in the shape of a new platform on the JVM. The panel was well-comprised with different viewpoints, and there were some pithy comments and good debates.

Joshua Bloch opined that Java is mature, and shouldn't go around in a pink miniskirt and pierced navel, but rather continue to evolve in a very measured fashion, making room for a new platform on the JVM, one that solves some of the existing fundamental problems. This was an interesting perspective, but I wish I could see evidence that there was serious effort being put into this concept (sorry, Charles Nutter, I don't think JRuby's that solution, as interesting as it is). A fair amount of time was spent talking about static/dynamic languages and backwards compatibility and the size of the Java download. Erik Meijer concluded with an interesting statement about DSLs: a disaster waiting to happen. This was probably my favorite session so far, both the topic and the speakers were great. Well worth watching when InfoQ puts it up.

Hiding a Swing UI Test Run

We've recently been doing a little work in Swing, and being the pervasive testers that we are, this means doing some Swing UI tests.

We played with a few Swing UI testing frameworks briefly, and settled on Fest, for now. (Incidentally, Swing UI testing frameworks are typically inactive projects with inactive communities and not nearly as mature as web UI testing frameworks or non-UI java testing frameworks).

In order to get these tests running on our build / continuous integration server, we needed a solution to run Swing tests on a headless server. One of my colleagues had previously used VNC for this purpose, so we quickly adapted that approach to the build.

I've only just realized that this same approach can be used to hide the UI test run from the main X Windows session, so that you can continue to use your windowing environment without having windows popping up, appearing and disappearing, and otherwise making it difficult to work.

Accordingly, I've done the following:

sudo apt-get install vncserver
vncserver :2
DISPLAY=:2 mvn clean test

This is an immense improvement. Now we can run our UI tests without interrupting the rest of our workflow.

jMock 2 (RC1)

The ever-so-infrequent releases of jMock lead to great excitement when they do release a new version. jMock 2 RC1 has been coming for some time, so it's great to see it happen. With new syntax and better support for automated refactoring, I'm expecting this to be an enjoyable upgrade.

If you haven't tried jMock, I'd urge you to give it a look. Although its authors sometimes take their design approach to the point of near-zealotry (which I do find occasionally frustrating), it's still the best 'Mock Objects' tool I've used, precisely because it allows you to indicate what you want to test and what you don't care about in a way that no other mock framework that I've used really addresses.

Merging Sessions with EclEmma

A subtle feature of EclEmma that I didn't notice yesterday is the ability to merge sessions.

While this certainly has application in areas where multiple test runs, suites or executions are responsible for some indication of 'total coverage', that aspect wasn't particularly exciting to me.

What I didn't realize is that this lets you merge in changes in coverage with a previous high-level coverage run. For instance. A test suite that I ran yesterday out of curiosity took 35 minutes to run through EclEmma.

Since that time, I improved the coverage of a few classes just to see the results. I can and did run a coverage test against those few individual tests just to verify that the coverage was as expected, but only this morning did I realize that I could merge those individual test runs back into the overall suite.

This allows me to get a decent 'update' of the overall coverage without running the whole suite again. Useful.

Eclipse Code Coverage with EclEmma

It's really great to see (via the Eclipse Awards) that someone has taken the time to integrate a free coverage tool with Eclipse: EclEmma.

I've used Emma before, but hadn't followed the recent development that sees it well-integrated with the IDE. Frankly, this was the one thing that really made the commercial offerings (e.g. Clover) attractive still, so now that Emma has taken this step, there's less reason to look elsewhere.

This, to me, feels like a great model for addressing test coverage -- right in the tool you use to write the code and the unit tests. Gives you a great capability to write tests, check coverage, and write a bit more.

Even more impressive, it seems as if EclEmma can also support loading coverage results from an external launch. So, for instance, if your nightly build ran a long project-wide coverage with Emma, it seems as if you'd be able to take the results of that coverage run, fire up EclEmma and see the results.

I'll be testing this out soon.