Tuesday 27 November 2007

Java 7 - Extension methods

A recent document revealed some possible language changes that are being proposed. One of these is is extension methods.

Extension methods

Extension methods allow a user to 'add' a method to an interface or class that you don't control. The original document is linked with BGGA closures. And was followed up by Peter Ahe (broken link).

The classic example of the proposal is as follows (known as use-site extension methods):

// in the application
import static java.util.Collections.sort;
List list = ...

Thus, the sort method appears to be a method on the list, even though we haven't actually changed the List interface. When it is compiled, the extension method is removed, and replaced with the static method call.

Peter Ahe pointed out some flaws with this design. He also proposes declaration-site extension methods:

// in the JDK
public interface List {
  void sort() import static java.util.Collections.sort;

// in the application

In this case, it is easier for the user to find the implementation of the sort method, as it genuinely is a method on the interface, just one implemented elsewhere. Unfortunately, the downside is that now only the JDK authors can add methods to the List interface in this way.

My proposal is that the freedom given to the user by the use-site approach is far more desirable, but the possible side-effects are nasty. My preference would be to add a visible marker to show that this isn't a regular method:

// in the application
import static java.util.Collections.sort;
List list = ...

Note the '.do.'. This provides the user with the clear readability to show that something 'out of the ordinary' is going on.

In addition, I would argue that the static method should be marked to indicate that it can be used as an extension method, probably using an annotation:

// in the JDK
public class Collections {
  public void sort(List list) { ...}

This provides the final piece of the puzzle, preventing innapropriate methods from being used as extension methods. While it does suffer from the same issue of pushing control back to the library author (the JDK), it avoids the issue of which methods get added to the List interface.

The point is that as many methods as appropriate can be tagged with @ExtensionMethod. There is no conceptual overhead in doing so, and it doesn't expand the conceptual weight or complexity of the ListInterface itself.


I've outlined an alternative proposal for extension methods, that tries to focus on freedom for users, without compromising readability or allowing confusing options.

Opinions welcome, as always :-)

Sunday 11 November 2007

Kijaro - for Java language changes

I'd like to announce the creation of a new project - kijaro. Kijaro is designed as a place where ideas for changes to the Java language can be implemented.


The kijaro project has been setup following various discussions on blogs, mailing lists and email. Its aim is to provide a very open place for those interested in implementing a change to javac to gather and code.

Kijaro is similar in its scope to the Kitchen Sink Language. The KSL project has been open for quite a while, but has yet to see any new features. KSL also aims to have code reviews and experienced compiler writers involved, which can be seen as quite formal.

Kijaro aims to be lightweight in rules:

  • Documentation. Each new language feature must have some form of associated document, even if its just a blog. It doesn't have to be much, but should have an outline of why the feature is needed and the syntax implications.
  • Backwards compatibility. On svn TRUNK all existing Java code must compile.
  • Comments. Each change must have a comment so we can find it later, such as 'FCM-MREF'.

So far, the proponents of three language enhancements have expressed an interest in working at kijaro:

In fact, the FCM and Properties code is already checked in.

So, do you have a favourite language change that you want to see implemented in Java 7? Or, would you like to download and try out one of these changes? Then, please join us at kijaro! The more ideas we get implemented the better!

After all, real working prototypes tend to produce good feedback and really encourage the decision makers that any change is practical.

Tuesday 6 November 2007

Implementing FCM - part 1

Its been a long coding weekend, but after much head scratching, I might finally be getting the hang of changing the java compiler. My goal? to implement First class methods (FCM).

Implementing FCM

Currently, on my home laptop I have an altered version of javac, originally sourced from the KSL project which supports Method literals (section 2.1 of the spec). What this means in practice is that you can write:

  Method m = String#substring(int,int);

This will now compile and return the correct Method object. The left hand side can be a reference to an inner class such as Map.Entry without problem. It will also validate the standard access rules, preventing you from referencing a private/package/protected method when you shouldn't.

In fact, the implementation is currently more generous than this, as the left hand side can also be a variable. This is needed for later phases of FCM, but isn't really appropriate for a method literal.

Making a language change like this is definitely intimidating though, and mostly because the compiler feels like such a big scary beast. The first two parts - Scanner and Parser are actually pretty easy. Once those are done you have an AST (Abstract Syntax Tree) representing your code. In this case that involves a new node, currently called 'MethodReference'.

After that it gets more confusing. The main piece of work should be in Lower, which is where syntax sugar is converted to 'normal' Java. For example the new foreach loop is converted to the old one here. Since method literals are just syntax sugar, Lower is where the change goes.

Unfortunately, to get to Lower, you also have to fight with Attr which is where the types get allocated, checked and validated. That was a real head scratcher, and I'm sure a compiler expert would laugh at my final code. It does work though, and after all thats the main thing.

The next step will be to convert 'MethodReference' to 'MemberReference' and enable Field and Constructor literals. After that, the real FCM work begins!


So, by now you probably want to have a download link. But I'm not going to provide one. As the code is GPL, I can't just publish a binary, the source has to come too. And personally, I'd rather actually have it in a SVN repo somewhere before I let it escape at all (because that can probably count as the GPL source publication). So, I'm afraid you'll have to be patient :-)

This probably should go in the KSL, but right now I don't know whether it will. The trouble with the KSL is that it still has a few guards and protections around it. And to date, no-one other than Sun has committed there, even in a branch. I'm feeling more comfortable in creating a separate, 'fewer rules', project at sourceforge. In fact, it would be great if other javac hackers like Remi wanted to join too ;-)

One final thing... if anyone wants to help out with this work I'd much appreciate it - scolebourne|joda|org. Compiler hacking isn't easy, but its a great feeling when you get it working!