blog community

The word "agile" is used more and more, and everything around software development are supposed to be agile these days. Both Microsoft and IBM claim their new tools are "Agile", but didn't the agile manifesto say "people and interactions over process and tools"?

The notion of expensive, feature-rich tooling vs the agile principles may seem contradicting. Kent Beck wrote up an article explaining that you of course still want to use tools in an agile project, but that the agile approach puts different demands and emphasis on the use of the tools. Anyway, go download and read the article from the Microsoft website.

This week Richard and myself will be visiting the IBM Rational Software Developer Conference (or RSDC for short). Like the name suggests, the conference is for people developing with IBM Rational related stuff. It's a pretty big conference on a nice location in Orlando. With over 3500 visitors and over 270 sessions, the schedule is packed. The conference is a lot less crowded than other conferences I visited, which is a good thing. A lot of the visitors are IBM people and product specialists as well, there is plenty of opportunities to talk about your favorite/hated Rational product X features and bugs.

With the first day nearing to an end, a lot of interesting stuff already passed by. Jazz is getting a lot of attention and several new products and upgrades are positioned around this new ALM platform. Team Concert, the basic Jazz powered project tooling, is shipping Real Soon Now and it will be interesting to see how the new capabilities and extension possibilities will be used. To me, Jazz seems likes the right move to further align the many, many different tools IBM is offering right now, and make them work more seamless. The other thing what's new about Jazz is that it is much more suitable for smaller teams. Most current Rational products are very clunky and big, and really need a big team to justify the added complexity of the tooling. Jazz brings a modular system that starts simple and enables stacking up tooling as a project becomes more complex.

With such a product-oriented conference, it is sometimes hard to cut through the featuritis and marketing to get some good info. Fortunately, there also some techy sessions and shows to keep geeks like me happy as well. The weather is great too. :) All in all I am having a great time and look forward to the rest of this week!

Last wednessday was the NL-JUG J-Spring 2008 conference, which is probably the biggest Java event in Holland by now. The event was packed with over 1100 visitors, but organized very well and everything went smooth. NL-JUG co-lead and collegue Bert Ertman was awarded the very prestigious Java Champion title which makes him the one of two Dutch champions at this time! The conference pavilion was packed with all kinds of companies doing stuff with Java. The Info Support blondes strolled around and took pictures for a bluetooth virtual keyboard contest. See the photo gallery for a huge amount of pictures, personally, I don't think most people really cared about winning a keyboard. ;)

I started the conference with a session on Gridgain. The presenters had this funny idea to create a grid by using laptops in the audience. Unfortunately, this meant that a lot of time went to seeing someone waiting on console output and messing around with configuring stuff, instead of telling something about Gridgain. Gridgain seemed like an interesting product, though the lack of dashboards and decent admin tools will make it difficult to run a more complex grid.

Next I went to a session on iBatis. On my current project we are considering adopting this data access framework. A very nice presentation which showed hands-on examples of most iBatis concepts. iBatis is a simple and flexible framework that will keep you in control of the query creation, while automating the boring data mapping stuff. Alternatives like Hibernate/JPA automate more, but on our data model this would be horrible, as these are complex, inter-related and large data sets.

After the break I visited a session on Kanban. An entertaining speaker told about the Kanban concepts using some examples. I guess people ran out of English buzzwords, so now we are getting Japanese buzzwords! The speaker did not have time to tell a lot about the underlying "Lean" development theories (if you want to know more about lean, I can highly recommend watching this Mary Poppendieck talk). Without understanding the theories on systems planning and just-in-time flow, Kanban can be perceived as quite stupid, so a part of the crowd immediately switched to asking all kinds of defensive questions. For me, Kanban seemed a simple and practical way to introduce some lean concepts, but I still don't get why I should drop iteration-based planning for it.

The session on annotation processing was informative, although a more practical example might have helped. Java 6 certainly made annotation processing a lot cleaner and easier to do. Still, you can create all kinds of evil bugs doing annotation processing, so be careful.

The session on code reviews was stimulating. The speaker was very original, doing some car jokes and handing out code examples to be reviewed by the audience, but he did had some messy slides and complex definitions. I completely agree that code reviews are an effective learning tool, and I will follow up on some book references on the slides on how to make this efficient. Interesting stuff.

Finally I went to a MDA session using all kinds of Eclipse modeling projects. Although the session was a great overview on all the available frameworks I kind of missed the practical examples and in-depth explanations of the different aspects of doing MDA. When answering questions he went a bit more in-depth on how they implemented MDA which was much more interesting. Basically the story was: don't try to model every detail, but keep hooks in to code very specific details by hand, and don't expect 10x productivity improvements promised by some vendor.

 All in all, the conference was full of interesting sessions and a great way to meet up with fellow Java developers I worked with in the past. See you at J-Fall 2008!

Eclipse Monkey is an amazingly useful tool if you want to do some quick hacking in a big workspace. On the project I am currently working on, the workspace consists of 180+ Eclipse projects. Scripting changes to configuration settings and check if settings are conforming is a lifesaver.

Three examples of scripts I wrote to do some project analysis. You only need to install Monkey for these scripts, no additional bells & whistles are needed. The examples are prefixed and suffixed with some wacky lines, these are used by Eclipse Monkey for copy-paste publishing support.

This example finds projects which are in a differently named directory (this happens often because renaming a project does not automatically rename its containing directory).

--- Came wiffling through the eclipsey wood ---
/*
 * Menu: Info Support > Find project name mismatch
 * Kudos: Peter Hendriks
 * License: EPL 1.0
 * DOM: http://localhost/org.eclipse.eclipsemonkey.lang.javascript
 */
function main(){
    loadBundle("org.eclipse.core.resources");
    
    var ResourcesPlugin = Packages.org.eclipse.core.resources.ResourcesPlugin;
    
    var projects = ResourcesPlugin.getWorkspace().getRoot().getProjects();
    var totalMismatch = 0;    
    for (var i = 0; i < projects.length; i++) {
    	var project = projects[i];
    	var projectDirName = project.getLocation().makeAbsolute().lastSegment();
    	var projectName = project.getName();    	
    	if (!projectName.equals(projectDirName)) {
    		out.println(projectDirName + " - " + projectName);
    		totalMismatch++;
    	}
    }
    out.println("Total projects: " + projects.length + ", total mismatch: " + totalMismatch);
}

--- And burbled as it ran! ---

This will find Java projects that have no Checkstyle config defined:

--- Came wiffling through the eclipsey wood ---
/*
 * Menu: Info Support > Find invalid Checkstyle config
 * Kudos: Peter Hendriks
 * License: EPL 1.0
 * DOM: http://localhost/org.eclipse.eclipsemonkey.lang.javascript
 */
function main() {
	loadBundle("org.eclipse.core.resources");
	    
    var ResourcesPlugin = Packages.org.eclipse.core.resources.ResourcesPlugin;
    
    var projects = ResourcesPlugin.getWorkspace().getRoot().getProjects();
    var totalMismatch = 0;    
    for (var i = 0; i < projects.length; i++) {
    	var project = projects[i];
    	if (project.isOpen() && project.hasNature("org.eclipse.jdt.core.javanature") && !isProperCheckstyle(project)) {
    		out.println(project.getName());
    		totalMismatch++;
    	}
    }
    out.println("Total projects: " + projects.length + ", total mismatch: " + totalMismatch);
}

function isProperCheckstyle(project) {
	var InputStreamReader = Packages.java.io.InputStreamReader;
    var BufferedReader = Packages.java.io.BufferedReader;
    
    var checkstyleFile = project.getFile(".checkstyle");    
    if (!checkstyleFile.exists()) {
    	return false;
    }
    var reader = new BufferedReader(new InputStreamReader(checkstyleFile.getContents()));
    try {
        var line = reader.readLine();
        var re = /\.*check-config-name=\"Endeavour_CS\".*/;
        while (line) {
        	var result = re.exec(line);
        	if (result) {
        		return true;
        	}
            line = reader.readLine();
        }
        return false;
    } finally {
    	reader.close();
    }
}

--- And burbled as it ran! ---

This will find comment patterns that are probably commented code and tag tasks for them (Code is sometimes commented out to try alternative code, but never removed). For extra browny points, this is done using a progress bar, because this operation takes a while on our workspace:

--- Came wiffling through the eclipsey wood ---
/*
 * Menu: Info Support > Find commented code
 * Kudos: Peter Hendriks
 * License: EPL 1.0
 * DOM: http://download.eclipse.org/technology/dash/update/org.eclipse.eclipsemonkey.lang.javascript
 */
function main() {
	loadBundle("org.eclipse.core.jobs");
	
	var ProgressMonitorDialog = Packages.org.eclipse.jface.dialogs.ProgressMonitorDialog;
	var IRunnableWithProgress = Packages.org.eclipse.jface.operation.IRunnableWithProgress;

  	var runnableWithProgress = new IRunnableWithProgress({ run: runCommentSearch });
  	new ProgressMonitorDialog(window.getShell()).run(true, true, runnableWithProgress);
  	
  	window.getActivePage().showView("org.eclipse.ui.views.TaskList");
}

function runCommentSearch(monitor) {
	var files = resources.filesMatching(".*\\.java");
	monitor.beginTask("Searching for commented code...", files.length);		
	try {
	  	var match;   
	  	for each( file in files ) {
	  		monitor.subTask(file.getEclipseObject().getName());
	    	file.removeMyTasks();
	    	var previousLineCodeComment = false;
	    	for each( line in file.lines ) {
		    	if (monitor.isCanceled()) {
		  			return;
		  		}
	      		if (match = line.string.match(/^.*\/\/.*[;{}]\s*$/)) {
	      			if (!previousLineCodeComment) {
	        			line.addMyTask("Commented code: " + match[0]);
	        		}
	        		previousLineCodeComment = true;
	      		} else {
	      			previousLineCodeComment = false;
	      		}
	    	}
	    	monitor.worked(1);
	  	}
  	} finally {
  		monitor.done();
  	}
}

--- And burbled as it ran! ---

Hacking Monkey scripts is pretty easy if you know the Eclipse API and where to look for documentation. However, the docs are limited and if you are not really into Eclipse development, documentation and help is pretty rough. You can always google around and try to find some useful scripts on someone's blog though. ;)

Kent "father of JUnit" Beck is a great story teller and not afraid to pioneer on controversial topics, especially when a new book he wrote is available. ;) In this talk (2004) he brings up the notion of "software health" as opposed to just measuring quality.

The quality of software can be measured in many ways, but it is usually a static snapshot of certain point in time. Healthy software on the other hand will be able maintain a certain quality over time, even when unforeseen events happen, such a sudden changes or deadline shifts. He compares this with people. If two people are sitting in a chair reading a book, measuring the heart rate will not get much difference. However, when they suddenly have to run around, the person that is out of shape will quickly be panting and the difference will be obvious.

Kent argues that health is more important than quality. I fully agree with him. This is a business where changes happen, requirements are known to be unknown and technology evolves faster than an Apple iPod. It's more important to evolve and maintain practices that provide a certain quality baseline opposed to just meeting a certain quality metric on a certain date. I am happy that these ideas and concepts are getting more and more accepted and that we as a company try to put this into practice.

In another talk on the same conference writing developer tests is compared to "going to the gym". You are reluctant to do it because it seems like extra effort and a waste of time, but when you are done you feel really good. I like this analogy as well. Although writing tests does not provide the endorphins of a workout, it does shift the focus of the mind and looking at the same problem from the other direction. A refreshing and sometimes confronting change of perception that is a very valuable byproduct of testing.

Periodic reflection, improving and evolving the software seems like a good investment for software projects. Like going to the gym however, it is often perceived as extra work that does not fit in the schedule. We try to make up with fruity drinks and skipping some sweets, but staying healthy takes more than that. When was the last time you went to the gym?

Finding code that can be thrown away is a highlight of my day. Deleting code means it no longer has to be compiled, maintained or tested. No one has to see the code and wonder "hmm, what's this doing here?".

Especially on big projects with shifting insights in features or architecture and design, old code tends to be left behind. To determine if code is really unused is hard and may take a lot of manual inspection. Fortunately, the Eclipse Core Tools contains a utility called "Find unused members". It tries to determine if code is no longer used.

Download and install the Core Tools plug-in. As an example, consider this code:

package com.infosupport.demo.unused;

import java.io.Serializable;

public class SimpleTalker implements Talker, Serializable {

    private static final long serialVersionUID = 1L;

    public String sayHello(String message) {
        return "Hello " + message;
    }

    public String sayGoodBye(String message) {
        return "Goodbye " + message;
    }

    public String say(String message) {
        return message;
    }

    public static void main(String[] args) {
        System.out.println(new SimpleTalker().sayHello(args[0]));
    }
}

package com.infosupport.demo.unused;

public interface Talker {

    public String say(String message);
}

(Note that you can copy paste this code to Eclipse and it will generate an example project for you). If you right click on the Java Project you will get a new option:

When you run it, you get the following results:

Note that not all the unused elements are actually unused. The main method and serialVersionUID are special in Java and may be invoked by the VM. Code that is invoked through reflection is not detected. If code that calls this code is not in your workspace (or does not use a project reference to link to the code), it is not detected. Conversely, if you call code only from your unit tests, this tool will not report it because it is still used in the tests. Remember after removing code to re-run the tool, as now more code may have become unused because it was only called from unused code.

Still a lot of gotchas, but nevertheless a great first scanning tool to get rid of that old junk!
 

Everybody agrees improving software development speed is a good thing. It's a simple math game: the benefit starts when the result is done. Also, software development resource costs are 99% people, and people are payed by the hour. So if you are done sooner, you get profit sooner and have to pay less hours.

However, there is often no real metric defined for software speed. Sure, there are the SLOCs or function points, but these are volume metrics. Any metric that measures speed should also include start and finish time measurements.

Also, speed often gets sacrificed at the altar of perceived efficiency. For instance, individual developers are slowed down by poor networks, outdated computers or overly constraining policies. By keeping a slow network or computers running longer no investment is necessary and this is perceived as a cost saving. However, the slow network will mean everyone is waiting instead of working. Outdated computers prevent speed improvement and feature upgrades, and will break more often, causing more strain on your support staff.

How long does it take to replace your broken computer? In many organizations, this can take days, because someone from support has to pick up a box and install it with a company image. The support person is often paid less and is efficient at this, so this is seen as a cost saving. But, every second the computer is broken is a waste for me. Even when the support guy or girl would respond the second I called, I am just waiting for the replacement. Instead of two people involved (one waiting, one doing work), it is much cheaper to just give me the replacement with some very simple instructions to install it. And it is faster too. (I know this is possible because it's kind of the way it works at Info Support).

The above examples are pretty low level, but in the grand scheme of things, speed is often inhibited by policies and practices without very much thought. Mary "lean grandma" Poppendieck has a wonderful story on this called "Competing on the basis of Speed". First of all Poppendieck provides a concrete speed metric:

The time it takes from the first instance a user tells "I need this / this is not working right" to a real working solution available to the user (in production).

This seems like a good metric: from a user perspective, this is how long the software development organization takes to do any work.

However, from an organizational standpoint, this is huge. The entire chain impacts your development speed: resource allocation (money, people), having to do bids, gearing up a project team, doing the actual implementation, get it into production, everything more and in between. Measuring speed using specific volume metrics like LOCs or function points no longer seems to make much sense now. Also, delaying policies like "we only do something in production once a year" will have a huge impact on the overall speed. If you want to optimize you need to look at the entire chain: what is needed, where is something delayed, and why? Poppendieck has some interesting pointers for doing this.

Poppendieck also has another message: speed is a competitive advantage. If you are able to move faster, you will outperform your competitors. If you do not focus on speed and your competition does, you will be left behind. Software plays a crucial role in most organizations these days, software development speed is very important. It's not just about being faster is being cheaper. It's also about getting the big bucks (if you are faster, you are first), or become extinct.