Unreliable Java -version:XXX Command-Line

In order to help the operations team administer a web application, we'd built them some command-line tools in Java.  To launch those tools, we included a helpful shell script.  Since the application required Java 1.6+, it seemed prudent to use the Java command-line option that exists for that purpose.  Accordingly, we put "-version:1.6+" in the command-line options for Java.

In testing that script, we discovered that while it seemed to work reasonably well on Windows machines, it didn't work on any of the Linux installations we tried, from Red Hat to CentOS to Unbutu, some with manually-installed Java, and others with the package install direct from the operating-system repository.

At this point, I'd have to recommend that you treat the -version:XXX option in the Java command-line as unreliable.  If you control the environment in which it's being used throughly enough to know that -version:XXX will work, then you can probably also control the version of Java being used, thus making it moot.

What does Java 6 have against image/png?

In mapping from filenames to mime types, I've typically used Java Activation Framework's MimetypeFileTypeMap.  Leaving aside the fact that the code for this class (and possibly most of JAF) is pretty old and ugly, it mostly works.  Today, however, we discovered that it was giving us application/octet-stream for a ".png" file, and I decided to dig a little deeper.

I started by reading the docs, and checking the mimetypes.default included in activation-1.1.jar/META-INF, which seems to include png and PNG.  I turned on just enough logging for JAF to spit out what files it was loading and what information it got from each:

    @Test
    public void testPng() {
        System.setProperty( "javax.activation.debug", "true" );
        Logger logger = Logger.getLogger( "javax.activation" );
        logger.setLevel( Level.FINEST );
        logger.addHandler( new ConsoleHandler() );
        MimetypesFileTypeMap map = new MimetypesFileTypeMap();
        Assert.assertEquals( "image/png", map.getContentType( "myImage.png" ) );
    }

That told me that JAF could only find the mimetypes.default, but that it believed png wasn't in the file:

MimetypesFileTypeMap: load HOME
MimetypesFileTypeMap: load SYS
MimetypesFileTypeMap: load JAR
MimetypesFileTypeMap: !anyLoaded
MimetypesFileTypeMap: not loading mime types file: /META-INF/mime.types
MimetypesFileTypeMap: load DEF
Added: MIMETypeEntry: text/html, html
Added: MIMETypeEntry: text/html, htm
Added: MIMETypeEntry: text/html, HTML
Added: MIMETypeEntry: text/html, HTM
Added: MIMETypeEntry: text/plain, txt
Added: MIMETypeEntry: text/plain, text
Added: MIMETypeEntry: text/plain, TXT
Added: MIMETypeEntry: text/plain, TEXT
Added: MIMETypeEntry: image/gif, gif
Added: MIMETypeEntry: image/gif, GIF
Added: MIMETypeEntry: image/ief, ief
Added: MIMETypeEntry: image/jpeg, jpeg
Added: MIMETypeEntry: image/jpeg, jpg
Added: MIMETypeEntry: image/jpeg, jpe
Added: MIMETypeEntry: image/jpeg, JPG
Added: MIMETypeEntry: image/tiff, tiff
Added: MIMETypeEntry: image/tiff, tif
Added: MIMETypeEntry: image/x-xwindowdump, xwd
Added: MIMETypeEntry: application/postscript, ai
Added: MIMETypeEntry: application/postscript, eps
Added: MIMETypeEntry: application/postscript, ps
Added: MIMETypeEntry: application/rtf, rtf
Added: MIMETypeEntry: application/x-tex, tex
Added: MIMETypeEntry: application/x-texinfo, texinfo
Added: MIMETypeEntry: application/x-texinfo, texi
Added: MIMETypeEntry: application/x-troff, t
Added: MIMETypeEntry: application/x-troff, tr
Added: MIMETypeEntry: application/x-troff, roff
Added: MIMETypeEntry: audio/basic, au
Added: MIMETypeEntry: audio/midi, midi
Added: MIMETypeEntry: audio/midi, mid
Added: MIMETypeEntry: audio/x-aifc, aifc
Added: MIMETypeEntry: audio/x-aiff, aif
Added: MIMETypeEntry: audio/x-aiff, aiff
Added: MIMETypeEntry: audio/x-mpeg, mpeg
Added: MIMETypeEntry: audio/x-mpeg, mpg
Added: MIMETypeEntry: audio/x-wav, wav
Added: MIMETypeEntry: video/mpeg, mpeg
Added: MIMETypeEntry: video/mpeg, mpg
Added: MIMETypeEntry: video/mpeg, mpe
Added: MIMETypeEntry: video/quicktime, qt
Added: MIMETypeEntry: video/quicktime, mov
Added: MIMETypeEntry: video/x-msvideo, avi
MimetypesFileTypeMap: successfully loaded mime types file: /META-INF/mimetypes.default

That implied to me that there was another mimetypes.default available, which a quick classloader test shows to be true:

    @Test
    public void testDefaultResource() throws IOException {
        System.out.println( "Displaying default resources: " );
        int index = 1;
        Enumeration resources = Thread.currentThread().getContextClassLoader().getResources( "META-INF/mimetypes.default" );
        while( resources.hasMoreElements() ) {
            URL item = resources.nextElement();
            System.out.printf( "\t%d: %s\n", index++, item );
        }
        System.out.println( "Done printing resources." );
    }

Turns out, the other mimetypes.default is coming from Java 6 (update 4) resources.jar, which inexplicably doesn't include an image/png content type:

Displaying default resources: 
 1: jar:file:.../jre1.6.0_04/lib/resources.jar!/META-INF/mimetypes.default
 2: jar:file:.../m2_repo/javax/activation/activation/1.1/activation-1.1.jar!/META-INF/mimetypes.default
Done printing resources.

Since activation 1.1's been out for a long time, I can't think of a good reason why image/png would be missing from a relatively modern JRE.

As always, in order to hedge against running into this again and help anyone else who might hit the same issue, thought it was worth writing down.

Story Granularity, Complexity and Planning

Obie Fernandez wrote about some problems he's been running into with respect to story granularity, and Bill de hÓra picked up the thread.  This is a subject I've struggled with a number of times, so I'm going to give myself a little free therapy and try and work through some of it here.

Story Granularity

Whenever possible, it's good to break down the desired features of an agile project into stories that are pretty granular.  What "pretty granular" means to your team and company varies somewhat, but something in the range from a day to a week would be relatively common in many of the projects I've worked on.  

Once the story gets significantly beyond that threshold, most teams will do their best to try and break it down into smaller pieces.  But there are times where that's difficult, or possibly even undesirable.

Minimal Marketable Feature

One of the ways to look at story granularity is that you want to assemble the minimal marketable feature.  You want the story to represent exactly the minimum amount of work that you could do and still "market" or "sell" the feature to the desired userbase.  If anything were removed from the story, the story could no longer be sold, it won't be credible or cohesive.  I find this perspective is often useful to help me "right-size" stories.  Stories that are larger than the minimal marketable feature are gold-plated, and stories that are smaller than the minimal marketable feature don't deliver meaningful user value.

Unfortunately, it often seems to be true that the minimal marketable feature is larger than your desired story size, and that creates a subtle tension between project and product management in an agile environment.

There are, of course, ways to try and work around this problem, and de hÓra talks about some of these.  Fundamentally, what tends to happen is that you subdivide the minimal marketable feature.  When you do this, it tends to create some other interesting problems.

How can you subdivide the minimal marketable feature?

Sometimes, this isn't very hard.  There are natural lines in the feature that allow you to implement it in several phases, each of which makes a cohesive piece of functionality, just not functionality that is individually marketable.

Often, however, the functionality cannot be broken into cohesive chunks.  If you end up having to deliver it in pieces that might individually break parts of the system, it's difficult to find the right division lines.  It gets even more complicated when in order to subdivide the feature and estimate the pieces, you need to pick an implementation approach.  Selecting a detailed implementation approach is a decision that many agile teams would like to defer past the initial planning stage, but if you can't easily finish your release and iteration planning without dividing the story and you can't divide the story without talking about implementation, then you're somewhat stuck.  

In these cases, you could take a 'theme' or an 'epic' past your planning stage, knowing that it's a risk and an unknown, but preferring that to talking about implementation concerns too soon.  Alternately, you could buckle down and do some up-front design and hope that whatever plan you come up with still makes sense when it's time to do the detailed implementation.  There are tradeoffs to be made here, and I can't make them for you.

What happens when the minimal marketable feature spans several iterations or releases?
If the subdivision is necessary for tracking progress, but all the work will be done within an iteration, then you've dodged another bullet.  If, on the other hand, the pieces that make up the minimal marketable feature will be split across two or more iterations, you're facing a new problem.  Even if those iterations are within the same release right now you may be reducing your ability to release, to adapt, to change.  

This is only a small problem if the subdivisions each deliver cohesive functionality that are simply too small to be individually valuable.  It's a much bigger problem if the elements aren't individually cohesive, where the result of a particular story might leave your project in an inconsistent state.  In these cases, you may need approaches that allow you to deliver code without linking it directly to the application, or the ability to deliver features that are disabled.  If, like some organizations, these approaches are already necessary (e.g. to work with multi-tenant SaaS solutions), this may not be incredibly painful.

Are large stories a cause of problems or a symptom?

Sometimes, the presence of large stories isn't the real source of the problems you're facing.  The problem is that the story shouldn't really be as large as it is, and has become so for a few reasons.

First, you want to be sure that the minimal marketable feature really is minimal.  This can happen when the product owner / manager is relatively new to agile planning and hasn't yet experienced the joy of discovering that your users can get by with a lot less than they think they need.  It can also happen when clients are unreasonable, when features are designed by committees, or filtered through a decision-making structure where each layer needs to 'innovate'.  It can happen when someone is trying to avoid past mistakes by ensuring that new features are robust and flexible, a sort of anti-YAGNI tendency.  If this is happening, you need to understand why it's happening so that you can try and find a solution that doesn't involve building extra-large stories where a medium-sized story will do.

You also need to come to terms with overall system complexity.  I've often seen systems that grow and evolve, add special cases and new features that interact with each other to create a surprising amount of complexity.  When this happens, adding even simple features can require a great deal of work.  "Sure, we can add that 'customer profile' box on the home page, but we'll need to create a customer profile display that is aware of the multi-tenant additional fields and meets the role-based security constraints.  We'll need to add several new service methods to customer web service API, and a new component to the user interface layer.  Unfortunately, three of those fields aren't available to this system, so we're going to need to make a JMS call to the CMS in order to get those pieces. We'll need to be able to turn the component on and off on a per-client and per-interface basis, and possibly on a per-role basis in order to work with existing clients, not all of whom are going to want to display this kind of information, and we may also  need to add field-specific display configurations."  

I might be exaggerating a little, but as systems grow, they tend to accumulate complexity, and many organizations don't do a strong job of keeping complexity down, either because they don't understand the heavy price they're paying for the complexity, or because they simply feel that the complexity is necessary.

In Summary

It's easy to say stories should be small.  It's not always easy to make it happen.  The devil's in the details.