Google Reader's Lesson on Privacy in Software Design

There's been a big kerfuffle over the last week (or two) about Google Reader's revised sharing feature. In the past, your Google Reader shared items were only available to people to whom you'd given a relatively obscure URL. While I quickly published that shared URL to a few places, other people were clearly taking advantage of its obscurity.

Recently, Google Reader added a 'friends' section such that the shared items for all of your GMail contacts (as well as anyone you add explicitly) will be easily accessible in the menu. Unfortunately for Google, not everyone is happy with this.

Some people have shared items that they've chosen to expose to a particular subset of friends / contacts, but don't want to expose to others. Imagine, for instance, that you share your love of bare-chested firemen with a few close contacts but you may not want your colleagues to know.

It's an interesting case, because on the surface, to anyone who was already treating the Shared Items as relatively public, it seems like a useful feature. I was happy to see the feature arrive, because I'm looking forward to using it and I was already treating my shared items as public. On the other hand, I can understand that this raises a privacy concern for some people, and I can respect that and see that it could be quite damaging.

It just goes to show that privacy can be a tricky thing in software design, and that sharing of information of any kind should be thought through in detail to avoid the kind of negative backlsh that the Google Reader team is facing.

It seems to me like there are some relatively simple solutions to the Google Reader Privacy Crisis (TM). I'm inclined to believe they should roll back the change until they're able to put some access controls in place. The simplest solution is to add a preference that allows people to make their shared items visible to their GMail contacts. Even if that preference defaults to 'on', it gives people the simplest control over the publication of their shared items. After that, it might be nice if there were options to share certain tags with certain groups of people, for instance -- so if I wanted to tag some feeds as private, or "for special friends" and share those as distinct from my publically shared feeds, that seems like a useful, if more complex, form of sharing that could add value.

Anyway -- I'm not sure if Google Reader is going to cave in to customer demand or stick to their guns, but I'm pretty sure there's an interesting lesson on software design and privacy here for those who care to learn it.

Ruby 1.9.0 - Released

Ruby 1.9 has been released, a little later than estimated. There are a lot of changes for Rubyists to absorb, and if you're using Ruby on Rails, Ruby 1.9 with Rails 2.0 is a pretty novel, but powerful, combination. We'll be seeing the impact of those changes in 2008.

Will YARV give Ruby the performance it needs, or will JRuby storm the enterprise? Time will tell. Maybe 2008 will be the rise of Erlang, JavaScript/EcmaScript 4.X, or some other new and powerful language.

In the meantime, if you haven't given Ruby or Rails a chance, maybe this is your chance.

The Windows download is slightly harder to find than most, so I'll link to it here.

Aren't Packaging Systems Supposed to Make Life Easier?

So, I'd like to conduct a little experiment with CXF. Specifically, the version of CXF that the current version of ServiceMix uses.

ServiceMix uses a complex multiproject Maven build, which is pretty normal these days. Due to the parenting structure, it took me a little while just to figure out which version of CXF ServiceMix uses. I eventually found it simpler to download the source and run a grep. It uses 2.0.2-incubator.

Equipped with this knowledge, I added the version to my own Maven POM for the project, only to discover that CXF 2.0.2-incubator uses jaxb-impl 2.0.5, which isn't in the central Maven repo. A little digging with Nabble shows me that it's in the java.net repo, but that the java.net repo uses the Maven 1 repository structure.

I did a quick dig on the java.net repo to ensure that there isn't a Maven 2 structure as well. It seems like there is, but that jaxb-impl doesn't publish new versions there (although there seems to be an old version there, for whatever reason).

Because it's not always possible to find a Maven repository that holds the dependency you need, and because they're not always reliable, we're using Artifactory as a mirror/proxy for Maven repositories. This means that in order to take on the java.net repo, we need to add the repo to Artifactory rather than to the project POM.

Unfortunately, it turns out that Artifactory, unlike the Maven POM, doesn't support a configuration option for legacy maven repositories. So now I have to download the dependency and upload it to artifactory.

So, due to lots of leaking abstractions, i've now spent about two hours setting up something that, five years ago, would ahve taken ten minutes - download ServiceMix, grab its CXF jar, drop it in a lib directory, and reference it from Ant.

Three cheers for packaging systems. maven/artifactory and leaking abstractions.

New Google Analytics API

The new capabilities in the Google Analytics API look impressive; could replace the way a lot of people do internal tracking of custom events. However, despite the fact that their front end is quite impressive in and of itself, I'd still like the option of retrieving the data so I could present it in my own interface.

Mocking Methods that Don't Exist with RSpec

I've done some tests/specs using RSpec in Ruby with Mock Objects, but until I read this message on the RSpec List it hadn't occurred to me that you could mock methods (messages) that aren't present in the class itself.

I guess you could see that as a natural consequence of the kind of duck-typing system that dynamic languages offer, but it's a little awkward. You can end up, as the poster suggests, with a view that's out of sync with your model, and tests that don't pick it up.

As a result, not only do dynamic languages make testing more important, they make integration testing more important. Interesting.

I'd be interested to see a large codebase written entirely in Ruby and get a sense for the comprehensiveness and duration of a test run -- within three months of writing tests and specs on a Ruby/Rest project, we'd already racked up a five-minute test run (but, hey, we were Ruby newbies, so I'd be surprised to discover that people who've been at it longer have good strategies for maximizing test comprehensiveness and minimizing test-run time.

Performance: Groovy/Grails vs. Ruby/Rails?

So, in the comment thread that followed: The Pain of Java after Ruby, I off-handedly commented that I'd been given the impression that Groovy or Grails' performance was worse than that of Ruby or Rails. I was inevitably challenged on that point.

Let me back up a little and cover some of the ground I covered in that comment: I haven't attempted to benchmark Groovy and/or Grails at all; Grails didn't have great support for REST services when I last looked at it, so I didn't include it in some of my performance explorations at the time. We were able to get 600 requests per second from Restlet, and were able to get something on the order of 40 requests per second on the same hardware using Ruby on Rails. I'm sure both of those results could be improved upon. (At the time, JRuby was scoring about 28 requests per second; I gather performance has improved since then).

So, it begs the question: Does Groovy and/or Grails outperform Ruby and/or Rails? You'd certainly like to think so, given the Java VM underneath it, but what have people said?

Well, the Grails folks would probably argue that their solution performs better although it's still clearly a tight race. There's a little grumbling from some people with more Rails expertise, but there doesn't seem to be any outright disagreements.

That said, if you compare either of those solutions back to a Java equivalent, the results are usually pretty striking. There's a few examples of that out there.

So for now, I'll say that it seems like Groovy may have the edge on performance compared to Ruby, although it's still very much a two-horse race, and both have many further opportunities to gain ground.

However, it's also clear that both of these two solutions are, in the overall spectrum of development platforms, relatively slow. Depending on what you're building, that may not be a big deal, but it can certainly be a concern.

The Pain of Java after Ruby

Having done several months of Ruby in the past year, I occasionally find myself looking at a page of Java code and thinking to myself, "Wow, that's Java alright."

For instance, within the past few weeks, I needed a URI for testing. I made it a constant (private static final). Since it can throw an exception, I had to put the constant in a static initializer. Since the exception might be thrown, the variable might not be initialized, and so on and so forth. Eventually, you end up with this:

   private static final URI SOURCE;

   static {
       URI uri = null;
       try {
           uri = new URI("/");
       } catch (URISyntaxException e) {
           Assert.fail("Couldn't create a URI for testing; please fix the test.  This shouldn't happen.");
       }
       SOURCE = uri;
   }

Although I understand the rationale behind each of the steps that gets you here, it's a bit of a boiled frog syndrome. By solving one problem with slightly more complex code, and repeating that pattern a few times, you eventually end up with pretty horrific code. Thanks, Java!

Now, to be fair, there are other ways around this code. For instance, I could easily create this as an instance variable in the test setup code, which in many ways would be a more normal approach, and it'd take all the above code away. That's probably the most obvious resolution for the problem at hand, but there are certainly others.

In any case -- because there are alternate solutions, this isn't really a problem, but having painted yourself into this particular corner it's easy to think to yourself, "Wow, this code would be much nicer in Ruby."

Individual and Collective Responsibility and Standup Meetings

Although I believe that stand-up meetings can be a powerful tool, I think they often go astray: they turn into status meetings. Others have covered the patterns and anti-patterns of standup meetings before, but I'd like to delve into this one point in more detail.

When standup meetings turn into status meetings, I'm inclined to think the problem often traces back to a sense of where the responsibility lies: with the individual or with the team.

Individual Responsiblity
Does each member of the team think that he or she is responsible for a task and must let the rest of the team (or a subset of the team) know where he or she is at? That's a status meeting, and it stems from a sense of individual responsibility.

With this mindset, individual team members may feel as if he or she is personally late to bring a task to completion mid-iteration, that the tasks others are on is interesting, but basically unrelated to the task at hand, and that a standup meeting is simply a way of keeping in touch with the rest of the team, and more importantly, making sure that some "important" members of the team know what's going on. The danger here is that each individual focuses on their task to the exclusion of the team's goals.

Collective Responsibility
What you should be striving for is a sense of collective responsibility. The team, as a whole, should understand where they're trying to go, what's important and what isn't, whose tasks may be on the critical path, and what those tasks mean. Without this collective responsibility, a team will often not act like a team, but rather like a collection of individuals.

When a team member on an important task is stymied, a team with a sense of collective responsibility will react naturally to that, attempting to route around damage. Rather than worrying about their own task they'll react to the needs of the team, trying to accomplish the goals set for the team, rather than worrying about a task that may or may not have been assigned to (or, better, selected by) them.

It's from this collective responsibility that "self-organizing" arises, and without it, an agile team is simply a set of very small teams managed by a central command structure.

Lazy Web
Have you experienced both mindsets? Have you managed to move a team from individual to collective responsibility? Does this make sense to some of you?

Mis-Attributed

Once upon a time, I was in University spent a fair amount of time on Usenet, and in some writing groups (I put together the proposal for rec.arts.sf.written, and maintained the FAQ for a while). One of the taglines I used once in a signature, I cribbed from an essay written by, I believe, Ursula K. LeGuin. My memory fades, but it was a good essay:

On the whole, audiences prefer that art be not a mirror held up to life, but a Disneyland of the soul, containing Romanceland, Spyland, Pornoland and all the other escape lands which are so much more agreeable than the complex truth."

I guess I failed to attribute it as well as I should, because the Internet has decided, in all its vast wisdom, to attribute that quote to me.

I'd love to take credit. It's a nice quote. I wish I had written it. But I didn't. And I'm pretty sure, although I don't have the essay at hand, that Ursula K. LeGuin did. So if you run a quote site, aim the attribution at her. It may not be accurate, but it's closer than me. Better still, either track it down or remove it entirely.

Eclipse 3.4M3

I was a little distracted by QCon when Eclipse 3.4M3 hit the street. There are a few interesting bits in the New and Notables:

• Eclipse will sense when you've verified that a class is an instance of something more specific, and offer code completions for that more-specific class with automatic casting. I don't need to do this a lot, but when I do, I find the process a little irritating, so it's nice to have a little quick support for that.
• Personally, I think with the Save Actions in 3.3, projects should bite the bullet, define a project-specific formatter, and set Eclipse to format on every save. That said, for people not willing to go that route, the new ability to format only edited lines offers a possibility to reduce the chaos on check-in (at the potential cost of having a file that is partially formatted).
• The ability to detect unnecessary @SuppressWarnings continues to improve the job Eclipse can do to clean up after the mess we make for ourselves.
• The improvements to Call Hierarchy are welcome; I've often wanted the ability to do a call hierarchy from a member field, not just a member method. If you haven't used call hierarchy, think about hitting Ctrl-Alt-H any time you might otherwise have reached for Ctrl-Shift-G. By offering a multi-level trace on what code access a method or field, you can often find more, faster than a find usages.

This is in addition to some of the highlights from M2 and M1:

• Differentiating between read and write access on Find Usages output.
• Quick assists to create getters/setters and extract methods.
• Replace all with preview diff looks really nice.
• The extract class refactoring to group method parameters into an object: often, when a given set of parameters is passed around to more than one method, this is a sign that there's really an object at work that just hasn't been created.

Eclipse 3.4 seems to be part of the Ganymede simultaneous release. There's a list of projects in that simultaneous release, but very few of them seem to have released much information of what's to come. Looks like Eclipse 3.4 will be followed by 4.0, according to the draft plan.

QCon: LinkedIn Architecture

Notes from the Linked-In: Lessons learned and growth and scalability session at QCon with Jean-Luc Vaillant.

Their architecture includes:

• Java (trying out some Ruby, adding some C++, as little as possible)
• Oracle 10g and MySQL
• Spring
• ActiveMQ (tried OracleMQ, doesn't recommend it)
• Tomcat & Jetty
• Lucene

Graph computations don't perform very well in a relationship database: with large numbers of members, and large numbers of connections, the combinatorics can be staggering. Add to this that simple approaches to storing this information would require extensive joining. Best way to get performance was to run the algorithms on the graph in RAM.

That raises the connection of how to keep the RAM database in sync at all times. One option is to update the database and inform other engines of changes through direct RPC, reliable multicast, JMS. This has the typical problems of two-phase commit.

An alternate approach that LinkedIn has used is to log changes in a transaction log which can be pulled from each graph engine into RAM as necessary. The approach is currently Oracle-specific, but it is applicable to just about any database.

Once that's in place, the in-memory techniques for traversing the graph are far less painful. Breadth-first traversal to get connections of various degrees. Using symmetry to find connections from both sides.

Having run into issues with Read-Write Lock, he prefers Copy On Write.

QCon: eBay Architecture

Notes taken during Randy Shoup's session on eBay's architecture at QCon

Partition Everything
Functional and horizontal segmentation. Partitioning data based on modulo of key, ranges, etc, depending on the data. Load balance pools of application servers, divided functionally. Search index separate from read-write listings.

Avoid database transactions - seek consistency through other approaches: BASE.

Absolutely no session state - transient state maintained/referenced by:

• URL Rewriting for Small Data
• Larger data in Cookies (up to 4k)
• Largest state in a Scratch Database (e.g. Multi-page flows, like listing an item)
  

Asynchronous Everywhere
By pushing dependencies off into asynchronous calls, it decouples availability, performance. Can retry. Improved user experience of latency although the data/execution latency might actually drop. Can allocate more time to procssing than user would tolerate.

More interestingly, this allows you to spread the cost of load over time. Spikes are less important, because much of the processing can queue up, then catch up in off-peak cycles.

Message dispatch with pub-sub cycles - when listing something, triggers an ITEM.NEW event which can be consumed by the summary update, user metrics, image processing, etc. They have over 100 logical consumers consuming ~300 events. As described the other day, they use at-least-once delivery and any order, rather than trying for only-once and ordered.

Event consumers go back to the primary source of the data rather than relying on the data in the event.

eBay also uses periodic batch processing for infrequent, periodic or scheduled processing, and for problems that are difficult to partition (e.g. full table scan), such as: generating recommendations, importing third-party data, computing sales ranks, archiving and purging deleted items. Often drives further downstream processing through message dispatch events.

Automate Everything
Machines are cheaper, and scale better and more cheaply than humans. They also adapt to a changing environment.

One approach here is adaptive configuration. Define SLAs for logical event consumers (e.g. 99% of events processed in 15 seconds), and then allow the consumer to dynamically adjust to meet defined SLA by tweaking event polling size, polling frequency, number of threads, and minimize cost by adjusting to changing environment (e.g. add more instances to consumer pool: can ramp down polling frequency).

Also employing machine learning to collect user behavior, aggreate that and make decisions, redeploy the metadata results of the learning. Can use this to choose pages/modules/inventory that provide best experience for user and context. Need to perturb the system to try alternatives in order to avoid getting stuck on local maximums.

Remember that Everything Fails
In order to be as available as possible: Assume everything can fail, all resources will become unavailable. Detect failure and recover from failure as rapidly as possible. Do as much as possible even when failure detected.

eBay logs all activity (request, exceptions, app-generated information), especially database and resources, logs on a messaging bus (1.5TB of log messages per day!). Listeners automate failure detection and notification. Compare scenarios with data-warehoused possibilities for root-cause detection (did we roll out new code? which database partitions are affected? etc.)

Make sure that all changes to the site can be rolled back - in every two week period, eBay rolls out 100,000 lines of code. Many changes involve dependencies between pools, so rollout plans contain explicit transitive set of dependencies. Automated tools execute a staged rollout with checkpoints and immediate rollback if necessary. Automated tool also enables rollback, including full rollback of dependent pools.

Audience question: "How do you test this?"
Randy: "We test it every two weeks, with our blood, sweat, effort."
Dan: "When the rollout plan is risky, there will be an explicit test of the rollout, not just the features."

Also, all features have on/off state driven by central configuration, so features can be turned off for operational or business reasons. This decouples code deployment from feature deployment. Applications can check for the availability of features in the same way that they check for the availability of resources.

Often don't go back and remove the on-off capability, although this does eventually result in configuration and code bloat. Allocate "a pretty decent percentage" of time to head-room ("We're going to bump our head up against that wall unless we fix this", such as developer productivity and refactoring) in which some of that 'clean up' can be addressed.

In failure detection, it's much easier to detect 'failed' than 'slow'. Once a server/resource is considered failed, it is "marked down" by sending alerts and no longer sending requests to the resource. If the resource isn't critical, that functionality is suspended. If it is critical, functionality is retried (alternate resource) or deferred (guaranteed asynchronous message). Explicit mark up allows resource to be restored and brought online in a controlled way - system can be marked up for different parts of the infrastructure at a time, very important when the failure was a load-induced failure. If the entire system were told the resource was back online, it might result in immediate return to failure state.

Qcon in Review: November 7th

Summarizing November 7th, for those not following my tumbl'd thoughts.

Kent Beck's opening keynote was well-presented, but I didn't take much away from it other than the reason behind his shaved head.

A panel discussion on Architecting for Performance and Scalability was good, but it's hard to cover that kind of topic in an hour, particularly with a panel. Of particular note was Ari Zilka arguing that scaleout for relatively small scale should focus on load-balancing rather than partitioning.

After lunch, I attended Designing for Testability, with Cedric Beust and Alexandru Popescu. This was perhaps a mistake for me, having done a lot of pervasive testing and having used TestNG, I didn't really learn very much. This was the second time I felt that way, so I realized I was going to have to watch which topics I attend -- the ones I'm most interested in are often the ones I know the most about, and am least likely to learn something from an hour-long primer. TestNG contributed a great deal to the world of Java testing, but I feel like JUnit has been revitalized by that competition and the choice between the two is less clear than it was between TestNG and JUnit 3.8.1. Cedric also showed his usual bias against agile and TDD: which is fine, as far as I'm concerned, you use the process that works for you.

Next up was Eric Evans on Strategic Design. This was a good talk, although he's a measured speaker rather than overflowing with energy. Still, he's a clear communicator, and there were some interesting points. I particularly liked the metaphor of maps as models. His anecdote about pudding, which relies on the language context (american pudding: goey custard-like dessert; british pudding: any dessert) was also entertaining. Some of the detailed elements seemed vague, but it was still enjoyable.

I followed that up with Cameron Purdy's Java Scalability and Reliability. I'd seen this presentation (or a heavily related one) before, but he's such an entertaining speaker, I didn't mind seeing it again. He's also not afraid to stir up a little humorous controversy: taking a shot at Bob Lee and calling FTP the precursor to SOAP.

Finally, a panel discussion on What will the future of Java be? was really great. There was a lot of hunger in the room to see Java evolve in some fashion, whether or not it's part of its current platform, or in the shape of a new platform on the JVM. The panel was well-comprised with different viewpoints, and there were some pithy comments and good debates.

Joshua Bloch opined that Java is mature, and shouldn't go around in a pink miniskirt and pierced navel, but rather continue to evolve in a very measured fashion, making room for a new platform on the JVM, one that solves some of the existing fundamental problems. This was an interesting perspective, but I wish I could see evidence that there was serious effort being put into this concept (sorry, Charles Nutter, I don't think JRuby's that solution, as interesting as it is). A fair amount of time was spent talking about static/dynamic languages and backwards compatibility and the size of the Java download. Erik Meijer concluded with an interesting statement about DSLs: a disaster waiting to happen. This was probably my favorite session so far, both the topic and the speakers were great. Well worth watching when InfoQ puts it up.

QCon November 6th: DSL Tutorial

For those not following my more stream-of-consciousness reporting on QCon via Tumblr, I thought I'd summarize on a daily basis here.

On November 6th, I attended the Domain Specific Languages tutorial put on by Martin Fowler and Kent Beck. They started the presentation by letting us know they were only prepared for a half-day tutorial and had only recently realized they were scheduled for a full day. They then proceeded to get through the first 'hour' of material by lunch, and took the afternoon for the rest, so all in all it seems like the content was well-suited to the timeframe.

They did a good job of describing the kinds of domain specific languages (they used the terms internal and external, rather than the 'embedded' term that's been used elsewhere), showed some good examples of DSLs in Java and Ruby as well as completely custom DSLs, the tiny languages that the Pragmatic Programmers talk about. They also spent some time describing the boundaries of DSLs: What's a DSL vs. an API; a DSL is usually a thin veneer over a framework; the hardest part in writing a DSL is writing the framework beneath it.

The last portion focused on those external DSLs, using Antlr as an example on how you might build and use a parser for a domain-specific language. This was the most interesting to me, because although I've read a little on the subject, including some of Martin Fowler's exploration of Antlr already, it's the area that I knew the least about (having used jMock's Java DSL for years, and having done a few months of Ruby in anger).

There were a few interesting moments:

• Charlie Poole indicated that some domain specific languages make it harder for non-english speakers to use the API, because they lack the common use of the idioms of the language. For instance, is "2.days.from.tomorrow" easier than Calendar.getInstance().add(Calendar.DAY,2).getTime() for a german speaker? (I dunno, I'm not German).
• Neal Ford, describing the muddy boundary between an API and a DSL, said, "DSLs are like pornography: Hard to define, but I know it when I see it."
• Martin Fowler and Neal Ford have slightly different takes on open classes, the ability to add methods and behavior to classes you don't control, easy to do in Ruby. Martin argued that it was reasonable within the context of an application you control, but riskier in a library that will be integrated with someone else's application (and other libraries). Neal Ford argued that experienced Rubyists find this conflict arises very rarely, and that the benefits outweigh the risks.
  

In the end, I don't know that I actually learned that much, but that's to be expected when you already know a fair bit about the area. I expect that when they release a book on the subject, it'll be a good one, with the kind of detail you can't stuff into a day-long presentation, and particularly valuable to those of you who are interested in, but have little experience with, DSLs.

I ended the day by having some Mexican food from Mijita Cocina Mexicana, possibly the food highlight of my trip (so far).

Tumbling QCon

Blogger isn't ideal for short snippets; I'd been considering tumblr for a while, and since they've recently released a new version, and I'm going to want to put out snippets at QCon, this seems like the ideal time to give it a whirl.

So, in the meantime, I'm thinking of using this blog for longer, article (or articella) pieces, and tumblr for links, quotes, and other very simple posts.

Hiding a Swing UI Test Run

We've recently been doing a little work in Swing, and being the pervasive testers that we are, this means doing some Swing UI tests.

We played with a few Swing UI testing frameworks briefly, and settled on Fest, for now. (Incidentally, Swing UI testing frameworks are typically inactive projects with inactive communities and not nearly as mature as web UI testing frameworks or non-UI java testing frameworks).

In order to get these tests running on our build / continuous integration server, we needed a solution to run Swing tests on a headless server. One of my colleagues had previously used VNC for this purpose, so we quickly adapted that approach to the build.

I've only just realized that this same approach can be used to hide the UI test run from the main X Windows session, so that you can continue to use your windowing environment without having windows popping up, appearing and disappearing, and otherwise making it difficult to work.

Accordingly, I've done the following:

sudo apt-get install vncserver
vncserver :2
DISPLAY=:2 mvn clean test

This is an immense improvement. Now we can run our UI tests without interrupting the rest of our workflow.

Toronto: High-Turnout for Tech Events?

I was interested to learn that both Joel Spolsky, on the FogBugz World Tour and Mike Chambers at Adobe's On AIR Bus Tour said that their Toronto stops had one of the highest (if not, in fact, the highest) turnouts.

To be fair, Mike Chambers did check the numbers and said that Toronto wasn't the highest turnout, just close.

Still -- this is good news. It means that Toronto's a good place to have technology events, conferences, and the like. If we're willing to turn out en masse to hear vendors pitch their products, imagine how many people would show up for something actually interesting. Although we've got Agile 2007 headed our way next year, I'd like to see more conferences make a showing here.

And, no, NFJS, out by the airport doesn't really count.

Software Developer Pattern: Worker Bee

Second in a series of patterns describing software developers. The original treatise included the Enthusiast pattern.

Worker Bee (Drone; Cog)
Worker bees are willing (more or less) and able (more or less) to implement the code assigned to them in the manner in which they've been told (more or less). They usually do not exhibit a strong passion for software development, a keen design sense or a drive for process improvement.

Motivation
Worker bees are really just hoping to get through the day so that they can go home and do whatever it is that really interests them: spend time with the family, watch TV, roller-disco, whatever. Their motivation for showing up at work every day to develop software is usually something basic like the need to support their family and hobbies, or "I dunno, the pay's good", or something equally gripping.

Applicability
Worker bees are best suited to an environments that other developers might find stifling. For instance, if the development process and procedures is onerous and ineffective, this will not bother the worker bee, who, frankly doesn't care that much as long as you continue to sign the cheques. They are relatively reliable employees who are less likely to move around as more interesting and lucrative opportunities open up elsewhere.

Because they tend to follow the path of least resistance, Worker Bees have a tendency to deliver code of average quality or sometimes lower-than-average. This can be mitigated by regular contact, supervision, code review or simply accepted: not every project and task requires a stellar codebase.

Worker bees are also good maintenance programmers, because, again, they're not looking for fulfillment from their job, so won't mind so much being stuck fixing bugs in other people's lousy designs.

Some organizations believe that development teams should be like factory lines, with a large number of relatively low-skill workers supervised carefully by one or more foreman-types. These organizations are looking for worker bees, and if you hire someone else to fill that role, they'll probably just get frustrated and move on.

Collaboration
Some worker bees are willing to engage and be engaged; these benefit greatly from an Enthusiast, whose enthusiasm and energy may rub off over time. A sheepdog is a perfect fit for worker bees, as long as you don't mind mixing a metaphor. Sheepdogs are happiest when they have a team to shepherd through daily obstacles, and worker bees can thrive in that nurturing environment.

Pairing worker bees with architecture astronauts can be disasterous. Worker bees will happily implement the most wildly outrageous ideas that the astronauts generate without once stopping to question the sanity or applicability of the idea. On the other hand, if you have a highly-placed architecture astronaut whose energy you're trying to divert, sacrificing a worker bee to the task such that the remainder of the team can focus on doing the real work can be effective.

Consequences
A well-channeled worker bee can regularly turn in reasonable (if unexceptional) code in an environment that would drive most developers to distraction. A misdirected worker bee can plant unintentional timebombs through your codebase, or waste the time of other team members.

Sample Conversations

• "You'd like an abstraction layer over the database so that we could swap it out for a cache, a filesystem, a content repository or a filing cabinet? Sure, I'll get started on that right away."
• "No, that's ok, I don't need to know how this feature will be used."
• "The last development book/website I read? Well, there was this one book in college that I liked..."
  

Well-Known Examples
You've worked with these people before, I know you have. Worker bees, as a general rule, don't make it to Well-Known.

Related Patterns

• Cargo Cultist: Although enthusiastic, cargo cultists remain just as oblivious to the rationale behind the practices they follow.
• Consultant: Same basic motivation, but better at self-promotion, sometimes better at coding, and radically more expensive.

Referral: The Most Honest Feedback

When I recommend someone to work at my company, it's because I've worked with that person before and I actively want to do so again. That's probably the single-most clear, honest and positive feedback I can give to any previous colleague.

Because, let's face it, everyone has some good qualities and some bad ones. If I say something nice about you, I probably mean it, but it doesn't mean that you don't have bad qualities that counterbalance and overwhelm the positive. Whereas if I refer you to a company at which I work, I've weighed all those factors one against the other, and I've decided that, in the balance, I'd still like to work with you.

If a potential employer asks me about someone with whom I've worked, I'll try and give balanced feedback that includes the things I like about that person's work and things I disliked, or found to be challenges. Most of the time, I'll emphasize the positive, but I'll still try and give something from both sides. I don't want to bad-mouth that person, but I do want the employer to get a sense of what the tradeoff is. That said, if that potential employer isn't my employer, I'm not going to push too hard. Even if I don't want to work with that person again, it doesn't mean that I don't want them to find employment anywhere. In fact, I'd prefer them to find employment elsewhere, because that guarantees they won't seek employment with me.

Most people want to be nice, to be perceived as nice by others and by themselves. And as long as being nice doesn't have a direct cost, most people are willing to do so. So if I've worked with someone, and I don't want to work with that person again, doesn't mean I'm not willing to comment on your good sides (and bad) to a potential employer, and let them make up their own mind. If the employer is using different criteria than I am (or isn't listening very closely to what I say), they may end up wanting to hire that person. So much the better; if there's a fit, that's good for everyone.

But if my employer is asking, I have to ask myself, "Do I want to work with this person?" If the answer is no, then I'm going to do my best to make sure that my employer understands that, even if I use the nicest possible terms to get the job done, "Joe is a great guy; friendly, works hard. I'm not sure he's ever going to be a great programmer. He can get stuff done, but he doesn't seem to really have that design sense that you need to be really good, and I just didn't see any sign that he was getting any better as time went on. I'd probably pass." It's honest, it's direct, and most employers will make the right call if you're being that direct. (I've never worked with a developer named Joe; he's just an example).

If another company were asking about Joe, I might be a little less direct. "Joe is a great guy, friendly, works hard. He wasn't in a senior role when I worked with him; I'm not really sure I can picture him in that role, but I haven't worked with him for a few years, so he may have taken on new responsibilities and skills since I worked with him." This is much more non-committal; a really astute observer might ask some probing questions that I'd answer more or less honestly and get to the heart of the matter, but most people would stop there, and draw their own conclusions.

This is probably why, in part, referred employees tend to be better than the ones that come through regular employment channels. Because the referrers have pre-filtered their past colleagues through the "Do I want to work with that person again?" question, and anyone they refer has passed the test. And despite this fact (or, rather, because of it), it's probably a good idea not to make referral bonuses too high: you don't want to get to the point where the incentive to refer outstrips the incentive to avoid referring people with whom you don't want to work.

So, if I've ever referred you to a company at which I've worked, and they've hired you, take heart, that means I'm willing to work with you again, and said good things about you, and that puts you in the upper tier of people I've worked with. It's probably the most unambiguous feedback you could get from me. And if any of you choose to refer me in the future, I think I'll take that as a high compliment.

Rails: Resource Links in Resource List

Navigating a REST application should not require a deep understanding of the namespace of the application's addressable resources. As much as possible, one REST query should lead to another.

The most obvious case of this is when one has a resource that is a list or collection (e.g. /customers) of the available resources (e.g. /customers/1), e.g.:

<customers>
<customer>
  <name>The Customer</name>
  <a href="myserver/customers/1.xml" />
</customer>
</customers>

And yet, it's surprisingly difficult to put these kind of links into Rails resources, particularly if you use the approach that resource_scaffold suggests.

Resource scaffold generates the XML representation of the collection as by calling the to_xml method that Rails adds to Array, which in turn calls to_xml on the model classes, something like this:

@customers = Customers.find(:all)
render :xml => @customers.to_xml

In order to add links to the XML representation, you'd need two pieces of information: the URL structure for resource and the id of the resource. The ActionController has the most understanding of the URL structure (as it can use
url_for and named resource helpers like customer_path), but the model class knows its id and is responsible for generating the XML.

This awkward split in the available information makes adding these links to the XML somewhat painful; perhaps no more painful than building the XML by hand would normally be, but it's a departure from the usual ease of accomplishing common tasks in Rails.

After trying a number of alternatives, we settled on approach. In the controller, we do:

# GET /customers.xml
def index
  @customers = Customer.find(:all)
  respond_to do |format|
    format.xml do
     render :xml => @customers.to_xml( :format => :summary, :base=>url_for(:controller=>'customers') )
    end
  end
end

# GET /customers/1.xml
def show
  @customer = Customer.find(params[:id])
  respond_to do |format|
    format.xml  { render :xml => @customer.to_xml( :format => :full ) }
  end
rescue ActiveRecord::RecordNotFound
  head 404
end

This relies on some model code, like this:

def to_xml( options = {} )
 case options[:format]
 when :summary
  resource_url = proc { |options| options[:builder].a(:href=>"#{options[:base]}/#{id}.xml") }
  super( options.merge!( :only => [:id, :name], :procs => [resource_url] ) )
 when :full
  super( options.merge!( :include => [ :main_contact_info, :billing_contact_info ],
    :except => [ :main_contact_id, :billing_contact_id ] ) )
 else
  super( options )
 end
end

It's still a little awkward, and I'm hopeful that we'll find a better alternative, but it does the job and is less ugly than some of the other alternatives we considered.

The only other option we looked at that seems reasonable is to generate all the XML within the controller, which means reproducing the work done by Array.to_xml, but is otherwise relatively sensible.

Rails Plugins: Globalize

Imagine that you've built yourself a small application in Ruby on Rails. Here's the good news: there are all sorts of ways to simply add significant functionality to your Rails applications through plugins. Let me introduce you to a few.

I've been investigating the Globalize and Acts As Audited (acts_as_audited) and Acts As Versioned (acts_as_versioned) plugins for Ruby on Rails over the past few days. I've got a lot to share, so I'll break it up into a few posts.

Let's start by imagining that you've built yourself a really, really simple model:
You've cut a customer model class with a unique identifier and a name. Now, let's add to that.

Globalize
If your application needs to operate in a multilingual or global environment, you'll be pleased to discover that not only is Globalize a general-purpose internationalization and localization (i18n/l10n) mechanism for Ruby on Rails, but it can even translate your model classes.

If you're going to translate fields within your model, you should be aware that it has two storage models for the translated information. Historically, it did this by storing the translated text in a separate table, globalize_translations. I'll call this the external storage model.

Using our model from above, the name field corresponds to the name in the base language (whatever you decide that is; en-US is a common choice in this part of the world), and all other translations are stored in the globalize_translations table as follows:

When you load or save your model class within the context of a particular Locale, Globalize does its magic behind the scenes to make sure that the text is loaded from or saved to the appropriate spot -- base language text goes to the primary table, and text from any other language goes to the translated-text table.

As often seems to be true, this magic interferes with some of ActiveRecord's default mechanisms, and :select and :include clauses stop working as advertised. There are workarounds, apparently.

However, in recent versions of Globalize, you have another choice - you can store the translations directly in your model, as follows:

This interferes less with ActiveRecord, although it's better-suited to applications that need to work with a known number of languages up front, rather than an unknown and changing set of languages, as each language addition requires new columns for every translatable field. It has a few caveats of its own.

Globalize 'Internal Storage', First Look

I took a moment this morning to examine the new internal storage model for Globalize, wherein translations are kept within the original table. It seems to have a few more caveats:

• Rails doesn't instinctively understand that the translated column names are not otherwise normal attributes; that means that any "iterate over all fields" code may have the translated columns show up as well. (e.g. scaffold show html).
• If you haven't inserted the translation column for a locale to which the user can set, rather than using the base language, the user will not see any value. So, for instance, I turned on internal-storage on an example class with a single translated 'name' field. I created an example record (e.g. name='Hello') and switched to an 'fr' locale; instead of getting the contents of the name field ('Hello'), I got an empty string ('').
  

It's an interesting choice, mostly because it breaks less of Rails than Globalize's other model. I prefer the 'external storage' approach in many cases, particularly if you're dealing with a product or SaaS application where the number of supported languages is not clearly known up-front.

Software Developer Patterns I: Enthusiast

In keeping with Christopher Alexander's seminal tome: A Pattern Language: Architects, Designers, Construction Workers, I have decided to document the pattern language of Architects, Developers, QA.

Although I cannot hope to document every pattern you see in your co-workers, I can, at least, start by codifying those patterns familiar to all. I do not claim to have invented these patterns, only to have begun the process of collecting them together into a common language to aid in communication.

Instead of grinding your teeth at your next design review, you can instead retort, "You were being a bit Enthusiast there, weren't you?" Your co-workers and the intended target, having read these patterns as well, will immediately understand what you're implying, and, chastened, acquiesce to your superior intellect.

Although developers are geeks, and therefore one-dimensional, maybe two-dimensional at best, you'll find that some of your co-workers exhibit not just one, but possibly two, or even three of these patterns. Some might even defy classification: occasionally, you may find that rather than treating your co-workers as the avatar of one of these patterns, you may have to treat him or her as an individual, with complex motivations and rationale. This is an edge case which can be resolved through good QA, or possibly an exception-handling routine.

Structure
The structure for each pattern will be as follows:

• Name: The name of the pattern
• Overview: An overview or description of said pattern, how it might be recognized.
  
• Motivation: The primary motivation of the pattern-exhibitor.
• Applicability: Where this pattern is useful, where it isn't.
• Collaboration: Who this pattern can collaborate with in order to form a team.
• Consequences: The likely outcome of employing this pattern.
• Sample Conversation: An example of a conversation between an arbitrary observer and a pattern-exhibitor.
• Known Examples: Well-known members of the community who exhibit this pattern regularly.
• Related Patterns: Similar patterns; ways in which these are similar and different.

In keeping with the broadcast television model, I'm going to stretch these patterns out over a series of intermittent posts. If I thought you'd stick around in between, I'd play advertising, but I figure most of you have Tivos or other PVRs by now, and wouldn't watch them anyway. That said, I'm told that it makes economic sense to give a 'free sample' to get you hooked, so without further ado:

Enthusiast (Fan; Zealot; Cultist)
Enthusiasts feel strongly about their technology choices. They identify with them on a personal level. When push comes to shove, they will defend these technology decisions with the zeal of a cornered badger using all the techniques in their arsenal: ad hominem, straw man, yo mamma, whatever approach prevents them from having to face the deeper issues.

Motivation
Enthusiasts often relate closely with technology because they have trouble relating to other humans and/or animals. Unable to form friendships and relationships, or even to own a pet, they transfer this emotional bond to objects with which they feel the most intimacy: a programming language, a hardware manufacturer, or something of that nature.

Applicability
Enthusiasts make good bloggers, evangelists and marketers. Their enthusiasm is so strong it can be infectious, particularly when they're preaching to the already-converted. They are not well-suited to fielding criticism, performing quality-assurance or to lend an objective eye. They cannot be trusted to make high-level decisions that touch upon their areas of enthusiasm. A Lua enthusiast asked to choose a programming language for his or her project will stack the deck in favour of Lua.

Enthusiasts are a good choice to help a team succeed with a technology to which the team is already committed. If you're well down the path of implementing your system in Erlang, it's great to have an Erlang enthusiast around who can tell you about Erlang Conf East, or what's happening at the Erlang user group, or what libraries the Erlang community is talking about.

Collaboration
Enthusiasts pair well with Worker Bees, who may gain energy from the enthusiasm, and learn from the Enthusiast's constant stream of well-meaning knowledge. They can have productive discussions with Builders, but should probably not be kept in close quarters for too long. Rarely do an enthusiasts and Dependency Minimalists see eye-to-eye. They get along well with Architecture Astronauts, but nothing good can come from this union.

Consequences
If an enthusiast is channeled well, they will act as a font of enthusiastic energy and useful knowledge to the team. A misdirected enthusiast with too much power will make arbitrary decisions that hurt the company and dismay the team.

Sample Conversations
"ASP.NET allows you to drag-and-drop UIs!"
"The iPhone API is composed of HTML, Javascript and Ajax."
"Macs aren't really that much more expensive, when you consider what you're getting."
"Smoking cuts ten years off your life."
"'Well you know. Smoking takes ten years off your life.' Well it's the ten worst years, isn't it folks? It's the ones at the end! It's the wheelchair kidney dialysis fucking years. You can have those years! We don't want 'em, alright!?"
"I'm thirty times more productive in Ruby on Rails than I ever was in Java."

Well-Known Examples
Apple customers. Ruby zealots. Bruce Tate.

Related Patterns

• Early Adopter: Like an enthusiast, but the enthusiasm is applied broadly to 'all things new' and tails off as the pattern-exhibitor grows bored of each item.

• Hawthorne Effectors: Not necessarily enthusiastic about technologies, but exhibit improvements in work when confronted with change.

• Cargo Cultist: Adopting enthusiasm for practices or technologies without understanding the reasons behind the enthusiasm.

CodeGear: Red Diamond Non-Disclosable

Well, in order to download a Beta for CodeGear's Ruby IDE (based on DLTK, code-named Red Diamond), I have to agree not to disclose information about it, including its existence and features. So instead of telling you about it, and how much it adds on top of DLTK, I will, instead, pretend it doesn't exist. If you ask me to suggest an IDE for Ruby code, I will not mention this one, even if it's better. That's what they want.

Eclipse: Go to Implementation

I've been pairing with one of my colleagues at The FeedRoom today (hi, Ivan!), and he passed on a plugin I've failed to notice before: Implementors.

Basically, it allows you to jump to the implementation of a method; really handy when you've got Interface/Implementation splits in your code and you know a particular method falls into that bucket, and you want to go to its implementation rather than the interface your code points to.

If there's more than one implementation, it brings up a context-menu of all the implementations it can find. If the interface in question is, say, java.util.List, this can take a while, so be patient.

Really handy; I imagine I'll be using that a lot.

Ruby on Rails on Bamboo: Unit Tests

In my previous post about Continuously Integrating Ruby on Rails with Bamboo, I walked through a simple setup for a Ruby on Rails project in Bamboo. That setup is a good starting point, but it's missing useful elements, like understanding the unit tests within the project, and test coverage:


Unit Testing Ruby on Rails with Bamboo
Let's start with unit tests. Bamboo understands the JUnit XML format, which Ruby doesn't normally emit (what with not using JUnit and all). Fortunately, someone's written a Rails plugin called ci_reporter that emits these files for us.

$ gem install ci_reporter
Bulk updating Gem source index for: http://gems.rubyforge.org
Successfully installed ci_reporter-1.3.3
Installing ri documentation for ci_reporter-1.3.3...
Installing RDoc documentation for ci_reporter-1.3.3...
$ script/plugin install http://svn.caldersphere.net/svn/main/plugins/ci_reporter
+ ./ci_reporter/tasks/ci_reporter.rake
$ svn add vendor/plugins/ci_reporter/
A         vendor/plugins/ci_reporter
A         vendor/plugins/ci_reporter/tasks
A         vendor/plugins/ci_reporter/tasks/ci_reporter.rake
$ svn ci -m "Added CI reporter to project."
Adding         vendor/plugins/ci_reporter
Adding         vendor/plugins/ci_reporter/tasks
Adding         vendor/plugins/ci_reporter/tasks/ci_reporter.rake
Transmitting file data .
Committed revision 411.

You'll probably have to ensure the ci_reporter gem is installed on your instance of Bamboo as well. Once this is complete, you'll want to modify your Plan:



And build the project:


See? Tests! Bamboo can find the XML files generated in test/reports, and use them to understand the testing of your Ruby on Rails application. Now, in a Java/JUnit project, Bamboo does a fine job of turning the test methods into readable names, but for now that's not true with Test::Unit test names, but they're working on it.


Next step: Coverage. But that'll be another day.

Hpricot Limitations

Well, after fighting with Cygwin and Hpricot for a few days, I've given in and dropped Cygwin from my stack for the Ruby/Windows testing for now.

Within five minutes of doing that, I was able to get something done with Hpricot, and bump into its limitations within a minute or so after that.

For example:

irb(main):001:0> require 'rubygems'
=> false

irb(main):002:0> require 'hpricot'
=> true

irb(main):003:0> require 'open-uri'
=> true

irb(main):004:0> doc = Hpricot(open("http://code.whytheluckystiff.net/hpricot/wi
ki/HpricotBasics"))
=> #<Hpricot::Doc {doctype "<!DOCTYPE html\n" "    PUBLIC \"-//W3C//DTD XHTML 1.

...

"> "http://trac.edgewall.com/" </a>} "\n " </p>} "\n" </div>} "\n\n\n\n "} </bod
y>} "\n" </html>} "\n\n">

irb(main):005:0> (doc/"//img")
=> #<Hpricot::Elements[{emptyelem <img src="/hpricot/chrome/site/images/hpricot-
small.png" alt="hpricot">}, {emptyelem <img src="/hpricot/chrome/common/trac_log
o_mini.png" height="30" alt="Trac Powered" width="107">}]>

irb(main):006:0> (doc/"//img[@alt='hpricot'")
=> #<Hpricot::Elements[{emptyelem <img src="/hpricot/chrome/site/images/hpricot-
small.png" alt="hpricot">}, {emptyelem <img src="/hpricot/chrome/common/trac_log
o_mini.png" height="30" alt="Trac Powered" width="107">}]>

irb(main):007:0> (doc/"//img[@alt='hpricot']")
=> #<Hpricot::Elements[{emptyelem <img src="/hpricot/chrome/site/images/hpricot-
small.png" alt="hpricot">}]>

irb(main):008:0> (doc/"//a/img[@alt='hpricot']")
=> #<Hpricot::Elements[{emptyelem <img src="/hpricot/chrome/site/images/hpricot-
small.png" alt="hpricot">}]>

irb(main):009:0> (doc/"//a[img/@alt='hpricot']")
=> #<Hpricot::Elements[]>

There's nothing wrong with the second query; it's a valid XPath expression, it's just not supported by jQuery, which means it isn't supported by Hpricot. Too bad; I guess i'll have to work around the syntax limitations.

JRuby Goldspike in Context Root

If you're using JRuby's Goldspike plugin to create a WAR for deployment, you might discover that rake war:standalone:run puts the application into a named context (e.g. http://localhost:8080/myapplication). Since most Ruby applications run at the root, this might disrupt your application if it wasn't built to handle a changing path (or your remote tests, which might have the same problem).

Fortunately, it's very easy to modify Goldspike to use the root:

1. Open vendor/plugins/goldspike/lib/run.rb
2. Find the WebAppContext definition
3. Adjust the second argument from
   <Arg>/<%= config.name %></Arg>
   to:
   <Arg>/</Arg>

You're done. When Jetty boots up, you'll see something like "NO JSP Support for /, did not ...". The "/" is the context, so you're in good shape.

Ruby on Rails continuously integrated with Bamboo

Since I tripped over a few things setting up Bamboo for Rails, I thought it would be good to share my approach. After my last post, someone from Atlassian thought that would be a good idea as well, so, what the heck.

Creating a Simple Rails Project

In order to demonstrate configuring a project on Bamboo, it's easiest for me just to create a simple rails project, with a model and scaffold. This'll come with tests, albeit useless ones. But if you wanted a tutorial on how to use Rails, you've come to the wrong place.

geoffrey@paraietta:~/projects$ rails bamboo-rails
create
create  app/controllers
create  app/helpers
create  app/models
...
create  log/production.log
create  log/development.log
create  log/test.log
geoffrey@paraietta:~/projects$ cd bamboo-rails/
geoffrey@paraietta:~/projects/bamboo-rails$ mysql -u root
Welcome to the MySQL monitor.  Commands end with ; or \g.
Your MySQL connection id is 8
Server version: 5.0.38-Ubuntu_0ubuntu1-log Ubuntu 7.04 distribution

Type 'help;' or '\h' for help. Type '\c' to clear the buffer.

mysql> create database `bamboo-rails_development`;
Query OK, 1 row affected (0.01 sec)

mysql> create database `bamboo-rails_test`;
Query OK, 1 row affected (0.00 sec)

mysql> quit
Bye
geoffrey@paraietta:~/projects/bamboo-rails$ script/generate model foo bar:string baz:string
exists  app/models/
exists  test/unit/
exists  test/fixtures/
create  app/models/foo.rb
create  test/unit/foo_test.rb
create  test/fixtures/foos.yml
create  db/migrate
create  db/migrate/001_create_foos.rb
geoffrey@paraietta:~/projects/bamboo-rails$ rake db:migrate
(in /mnt/home/geoffrey/projects/bamboo-rails)
== CreateFoos: migrating ======================================================
-- create_table(:foos)
-> 0.0040s
== CreateFoos: migrated (0.0041s) =============================================

geoffrey@paraietta:~/projects/bamboo-rails$ script/generate scaffold foo
exists  app/controllers/
exists  app/helpers/
create  app/views/foos
...
create  app/views/layouts/foos.rhtml
create  public/stylesheets/scaffold.css

Add the Project to Source Control

Once the project's created, it has to be somewhere that Bamboo can reach it. The usual practice is a source control system. In this case, I'm using Subversion.

geoffrey@paraietta:~/projects$ svn import bamboo-rails http://messis/svn/spikes/bamboo-rails -m "A simple Rails/Bamboo integration project."
Adding         bamboo-rails/test
Adding         bamboo-rails/test/unit
Adding         bamboo-rails/test/unit/foo_test.rb
...
Adding         bamboo-rails/public/stylesheets
Adding         bamboo-rails/public/stylesheets/scaffold.css
Adding         bamboo-rails/public/favicon.ico

Committed revision 190.

Configure Bamboo for Ruby

In order to build anything in Bamboo/Ruby, you're going to want to configure a builder in Bamboo. There may be a way to use one of the existing builders, but this is what worked for me. Just add a builder for Ruby and point it to your ruby executable.

Create a Plan for your Ruby Project

You're going to want to create a Project and Plan for your Ruby project. Bamboo projects are umbrellas for related Plans:

Plans tell Bamboo what you're trying to build, where to find the source code:

They also tell Bamboo how to build what it finds:

They also tell Bamboo what to do with the results, and whom to notify, but those steps aren't unique to Rails in any way, so I'm going to leave them out. At this point, you have a project and plan defined in Bamboo, so you can check to see if it worked:
It didn't. I didn't create the database on the build server. Once I've done that:

Et Voila

You've got a basic Ruby on Rails project building in Bamboo. This is the bare minimum setup. There's a lot of things you can add from here. In my next post, I'll cover getting Bamboo to recognize your test results and calculate coverage.

Bamboo, Ruby on Rails, Test Reports and RCov

Over the last day, I've had an opportunity to integrate Atlassian's bamboo with a Ruby on Rails project, which has been fruitful. It was actually pretty easy. Create a builder that runs Ruby (rather than Bash or Maven), point it at 'rake' with the appropriate target, and it builds.

Tests
If you want your test reports to be visible as test reports, you might want to look up the ci_reporter plugin and gem. This simply ensures that when your tests run, they generate JUnit-style XML files that many CI servers can pick up, including Bamboo.

Coverage
If you add the rails_rcov plugin to your project, you can run the test:test:coverage goal in order to generate a coverage report. All you need to do then is tell Bamboo to treat it as an artifact, scoop up all the contents of coverage/test. Then, from your build results, you can go to the artifact tab, and pick up the resulting coverage report, nicely viewable in your web browser.

See Also
If you want to find out more about integrating Atlassian's Bamboo with Ruby on Rails, I recommend you not search for 'bamboo rails' or 'bamboo rake', both of which are likely to take you elsewhere. ;)

Are any of you using Bamboo and Rails together, and if so, do you have more interesting tips to add?

Mark Occurrences in Aptana/Ruby

I've been using Aptana's Ruby/Rails editors in Eclipse 3.3. It has a 'mark ocurrences' feature that isn't working too well for me. It highlights things well, but the highlights don't go away, and when you save, they often move around, end up highlighting mid-word, including whitespace, and otherwise being incredibly confusing.

Happily, I've learned that they can be turned off here:

Preferences > Ruby > Editor > Mark Occurrences

Maybe this will save some of you the same pain.

At Work

At Work
Originally uploaded by diathesis

I knew I was going to really like this photo when I took it; I'm glad that I still like it after selection and processing.

This was taken at Doors Open in Toronto, on an upper floor of a building in the Distillery district.

In Review: Late June, Early July

• This Isle Lounge looks like it would be fun, if you had a nice large outdoor space [NOTCOT].
• Like Miis, you can make your own Simpsons avatar [NOTCOT].
• A really nice 'wine purse', for carrying wine about. There are so many of these, and yet, so few reasons to actually employ them, unless you have a large group of oenophile friends [NOTCOT].
• Stephen O'Grady on Distributed SCM, and why this trend hasn't permeated beyond open source projects [redmonk].
• Although this was announced some time ago, it seems that RedHat has actually released the Excadel plugins as open source; it'll be interesting to see if this increases their adoption, or the adoption of JBoss frameworks like Seam [InfoQ].
• The Wheel Clock is really quite nice [NOTCOT]
• Steve Yegge's Rhino on Rails. Tuesday was a pretty crazy day, as after reading this, I got into a series of discussions with the InfoQ editorial team and Steve Yegge about whether or not to cover this, then wrote the article and had it ready for the morning. It can be a surprising amount of work [InfoQ, Steve Yegge, John Lam]. This effort also turned up TrimPath.
  
• ThinkGeek's "I'm In UR Blank...", for lolcat fans everywhere. [NOTCOT]
• Building Ubuntu backports; I've been using Feisty Fawn at the Feedroom.
• Some initial discusson how to use the 'iPhone SDK' by designing for Safari on the iPhone.
• The periodic table of storing produce sounds handy, although I've yet to print it.
• The vegetarian directory for toronto has already come in handy: it let me know that Jean's Vegetarian Kitchen had opened in our neighbourhood. Yummy! I've been once, and will definitely be returning.
• Rails tips on adding your own config, flexible application configuration, validating dates, and using ActiveRecord::Validations on non-persistent classes.
• A tail utility for Windows may come in handy.
• Green tea should ideally be brewed in water before boiling. Yet, very few kettles come with this kind of temperature control. I always wondered what asian cultures used, but couldn't find much. Turns out, i should have been looking for electric dispensing pots, not kettles.
• Lessig's next ten years.
  

Eclipse Java EE: What happened?

On Friday, the download page listed a number of distributions including "for Java EE".

When I went back on Saturday, the 'Java EE' edition was gone. I checked again on Sunday and today, but it's still gone.

So ... what happened? Why was it removed? If I downloaded it, should I avoid using it?

Eclipse 3.3 is Out ... Sorta

Well, the download page has been updated. Looks like they're splitting it into 'distributions':

• Eclipse IDE for Java Developers: The essential tools for any Java developer, including a Java IDE, a CVS client, XML Editor and Mylyn.


• Eclipse IDE for Java EE Developers: Tools for Java developers creating JEE and Web applications, including a Java IDE, tools for JEE and JSF, Mylyn and others.
  
• Eclipse IDE for C/C++ Developers: An IDE for C/C++ developers.
  
• Eclipse for RCP/Plug-in Developers: A complete set of tools for developers who want to create Eclipse plug-ins or Rich Client Applications. It includes a complete SDK, developer tools and source code.
  
• Eclipse Classic: The classic Eclipse SDK: the Eclipse Platform, Java Development Tools, and Plug-in Development Environment, including source and both user and programmer documentation.
  

The follow-on pages don't yet work.

Update: The 'more info' pages now work, the downloads don't.

Update: The downloads work.

The Secret to Life, Career and Happiness

Well, a secret, anyway, maybe not the secret.

Nobody else knows what they're doing either.

Really. That's something I think you need to know. I hope this isn't a surprise most of you, but for those of you whom it is, I'm glad you're here, reading this.

Assuming you're not psychopathically confident, there are probably times in your life or your career when you've felt a little lost, like you weren't sure what to do. This could be when you were trying to ask that cute girl out at the school dance, and you weren't sure how to approach her, or when you're trying to decide between one job and another, or when you're confused about an architectural decision your company has just made.

I get this feeling all the time, I'm not ashamed to tell you. It's probably a daily occurrence. When I was younger, this used to bother me, made me feel like I was clueless where everyone else knew how to navigate their way through life, and I was hapless and insecure.

Over time, I've come to the realization that it's not true. Many, if not most of you, feel confused or lost exactly the same way, probably almost as much as I do. It's part of the human condition, we're all just muddling along.

So, why is this A Secret to Life, Career and Happiness
Well, if you were labouring under the false impression that it was just you, and that the rest of us always know what you're doing, I hope this takes a burden off your shoulders. We don't. In fact, much of the time, we're making it up as we go along.

Interestingly, once you realize this and start to behave accordingly, you'll find that people respond well to you admitting that you don't know what you're doing. In fact, you'll find that admitting you're lost and confused is basically a leadership trait. That may sound odd to some of you, but I'm largely convinced that it's true.

Admit You're Lost at Work
Assume, for instance, that you're working on a project, and the enterprise architect for your enterprisey company has decided that this project Must Be SOA (TM). You don't see any likelihood of other programs interacting with yours, nor is there a technology boundary or multiple user interfaces, so you think to yourself, "Why SOA? It's going to cost us time, but what does it buy us?"

Chances are, some of your coworkers are feeling the same way, but unfortunately, many of them are feeling as I used to: that the reason they're confused is that they don't know what they're talking about, and that the architect and their coworkers are all happy with the decision because they do know what they're talking about.

So if you step up, and say what you're thinking, "Do we really need a service-oriented architecture here?" you'll often find two things are true:

• The people who were feeling the same way are relieved. It becomes clear to them that they're not alone, confused about the architectural choice. They're not alone in their confusion. This is a good feeling, so they're happy.
• Secondly, if your team and enterprisey company are any good, they'll recognize that you're perceptive and analytical, that you ask the right questions and aren't afraid to look stupid in order to advance your understanding and the team's.

These are all good things, and I've found that asking these kinds of questions almost always helps. Usually, either I come away with the understanding I was looking for, or the question helps to drive change. Either way, it's a win.

Don't Swallow that Foot
Don't take this too far; having a question doesn't mean you should be obnoxious. If your company is pursuing a second product line when the first is having some troubles, you could ask, "Our first product line is already failing, so why are we wasting time trying to build another product line that is already failing?" Depending on where you work, this could get you a laugh, a stern conversation or a pink slip.

If you're treading on politically sensitive ground in a politically sensitive company, you could try using yourself as the scapegoat for the question. "I'm sorry, I'm having a little trouble understanding, maybe I missed something. I feel like we're having trouble proving the success of our first product line and now we're going to end up splitting our effort on a new product line. Would it be better to focus all our energies on the first product for the time being?" Most people have a hard time criticizing a question when you start it by implying clearly that you could just be confused or mistaken.

So, be free. Admit that you're lost. And when you're done, come back and tell me how it went.

The Feedroom

For those of you who haven't had a chance to come visit me at the new office yet ...



Welcome to The Feedroom Toronto, courtesy of my cellphone.

MySQL - No Millisecond-Precision Date/Time

I guess I've been spoiled not doing anything particularly serious in MySQL thus far; I'm only just now discovering that it doesn't support millisecond-precision date-time datatypes directly. Fascinating.


Not a big deal most of the time, but for some precise date-comparison routines, we've used this capability before, in Oracle, and in other databases, so I'm just a little surprised.

VirtualOS

People are very excited about the benefits of virtualization. I don't disagree with them; there's some interesting ideas there.

Virtual workstations, for things like testing and QA of software on different environments is very useful, and something that existing virtualization tools do pretty well.

Virtual server environments, while successful, seem to be a bit of a leaking abstraction. If I have an application in ASP.NET that I want to deploy on a virtual host, do I really care if my Virtual Windows has Internet Explorer and DirectX? I wouldn't think so -- not unless my application relies on the presence of either.

So I'm interested in seeing the operating system for virtualization get pared down. This would probably start by paring down existing operating systems, but could eventually result in an operating system that sees no use other than as a thin layer between virtualization technology and an application development/deployment stack.

For those of us building in Java, a virtual os that binds the virtulization technology to a JVM and offers those relevant services to the JVM might also result in improved performance for virtual servers that are hosting Java applications.

Week in Review: June 10-16

Another busy week. I started at The Feedroom, which has been interesting so far. We'll see how I feel in a few months, but so far, so good. I'll be more comfortable when I have a better understanding of the domain in which we operate, and a stronger sense of the needs and requirements of the software we'll be building.

Looking through my Google Reader shared links, del.ico.us and a few others, I've saved the following:

• W3C's (ancient) description of lost-update came up in the context of Atom Publishing Protocol (APP); if you haven't looked at this before, and you do web applications and web services, you probably should.
  
• The Easy Glider looks like it'd have a lot of the potential value of the Segway, but with a radically lower cost that would make it a lot more attainable. Sure, the technology behind it is simpler, less impressive, but is that really the point? But then, isn't that where the bicycle excels?
• The Trail-Gator looks like a nice way to keep your kids 'in line' when you need to and let them explore when you don't. At least, if you and your children both cycle.
  
• The Capsule Turntable Room is stylish outdoor furniture which would look really nice in the right setting. S'probably expensive.
• This belt is really interesting. It's price is not. I could get a half-decent lens for that price.
• Acabion GTBO is yet another of those car/motorcycle hybrids that get a lot of interesting press but nobody actually buys.
• The Humanscale Daybed looks like a nice place to relax and use the laptop.
• Marc Andressen lists Killer OS X applications for 2007 and an extensive list of cheatsheets: handy.
  
• Every year or two, I feel like I want a smoker. So then I look them up, read about the Weber Bullet, determine where to buy supplies, and ... then do nothing for another two years.
• Archy looks like an interesting way to list and select Maven archetypes.
  
• I'd almost forgotten xiao xiao's stick figure combat; I was cleaning off the server downstairs and found one of these videos.
• The Whirlpool Factory Clearance Centre in Mississauga might have the blender ring we seem to have lost.
  

The Feedroom

I guess now the news is out 'internally', I can let the cat out of the bag here: I'm moving on from Exchange Solutions to The Feedroom. Excited and nervous, both, but looking forward to a change.

It's always hard for me to move on; I try not to take jobs I can't get invested in, and having invested of myself in a project and a company, it's a hard thing to separate. So, I'm leaving behind a bit of an extended family for another one. Fingers crossed.

The Danger of Selling Development Tools

If you have a platform—like Microsoft Windows or Mac OS X operating systems, Java and .NET virtual machines, the iPhone, etc.—and you make development tools for it, I believe you're better off delivering said tools for free.

Now, you can charge for the platform, for the tools, for both, or for neither, but I'd argue that you want the development tools to be as cheap as possible, and, if possible, free. You want to do this because you want to encourage as much development
as you can, and any barrier to development, including the cost of tools, reduces the likelyhood of outside developers building software for your platform. And an active development culture helps to make your platform compelling.

You can't build everything yourself. You can build the premiere applications for your platform, and establish a look and feel, a set of standards that everyone strives for -- Apple does this well, but if you try and lock out the outside developer, you will eventually lose ground to someone else, because an active development community is vibrant. They'll turn out crap, and they'll turn out some weird stuff, and they'll turn out some really great things, and you'll miss out on all of that. I'm biased, because I'm a developer, but even as a consumer, I've shied away from platforms that felt as if they were a development monoculture.

For instance, I've tended to prefer PCs because the Mac development community has often been less vibrant, IMO. In recent years, this has seemed less true.

You also want to make your tools extensible, for the very same reason. If you attempt to build all the tools yourself, you'll lose ground to someone whose tools can be extended by the community, because, again, you can't build all the tools yourself, and if you help developers to help themselves, they'll build an array of supporting tools that make your tool that much more powerful. This leads to more developers using your tools, developing for your platform.

This is, I believe, one of the areas where Microsoft has gone wrong with Visual Studio. They charge for most of the editions of Visual Studio, which encourages some developers to look elsewhere. They offer a free edition, Express, which is a good move, but the free edition cannot be extended (see the TestDriven.NET story for that; I won't attempt to argue who's in the right there, but I do believe it is in Microsoft's interests to allow developers to extend Visual Studio Express).

Now, because Microsoft Windows is one of the most successful platforms on the market, the tool strategy may not hurt them as overtly as it would hurt others, but I do believe that it hampers the adoption of .NET, and reduces the number of potential Windows/ASP.NET programmers.