More detail on Closures in JDK 7

This blog goes into a little more detail about the closures announcement at Devoxx and subsequent information that has become apparent.

Closures in JDK 7

At Devoxx 2009, Mark Reinhold from Sun announced that it was time for closures in Java. This was a big surprise to everyone, and there was a bit of a vacuum as to what was announced. More information is now available.

Firstly, Sun, via Mark, have chosen to accept the basic case for including closures in Java. By doing so, the debate now changes from whether to go ahead, to how to proceed. This is an important step.

Secondly, what did Mark consider to be in and what out? Well, he indicated that non-local returns and the control-invocation statement were out of scope. There was also some indication that access to non-final variables may be out of scope (this is mainly because it raises nasty multi-threading Java Memory Model issues with local variables).

In terms of what was included, Mark indicated that extension methods would be considered. This would be necessary to provide meaningful closure style APIs since the existing Java collection APIs cannot be altered. The result of what was in was being called "simple closures".

Finally, Mark offered up a possible syntax. The syntax he showed was very close to FCM:

  // function expressions
  #(int i, String s) {
    System.println.out(s);
    return i + str.length();
  }

  // function expressions
  #(int i, String s) (i + str.length())
  
  // function types
  #int(int, String)

As such, its easy to say that Sun has "chosen the FCM proposal". However, with all language changes, we have to look at the semantics, not the syntax!

The other session at Devoxx was the late night BOF session. I didn't attend, however according to Reinier Zwitserloot, Mark indicated that Exception transparancy might not be essential to the final proposal.

According to Reinier, Mark also said "It is not an endorsement of FCM. He was very specific about that." I'd like to consider that in a little more detail (below).

The final twist was when a new proposal by Neal Gafter was launched which I'm referring to as the CFJ proposal. After some initial confusion, it became clear that this was written 2 weeks before Devoxx, and that Neal had discussed it primarily with James Gosling on the basis of it being a "compromise proposal". Neal has also stated that he didn't speak to Mark directly before Devoxx (and nor did I).

There are a number of elements that have come together in the various proposals:

	CICE	BGGA 0.5	FCM 0.5	CFJ 0.6a	Mark's announcement (Devoxx summary)
Literals for reflection	-	-	Yes	-	No (No info)
Method references	-	-	Yes	Yes	Worth investigating (No info)
Closures assignable to single method interface	Yes	Yes	Yes	Yes	Yes (No info)
Closures independent of single method interface	-	Yes	Yes	Yes	Yes
Access non-final local variables	Yes	Yes	Yes	Yes	No (Maybe not)
Keyword 'this' binds to enclosing class	-	Yes	Yes	Yes	No info (Probably yes)
Local 'return' (binds to closure)	Yes	-	Yes	Yes	Yes
Non-local 'return' (binds to enclosing method)	-	Yes	-	-	No
Alternative 'return' binding (Last-line-no-semicolon)	-	Yes	-	-	No
Exception transparancy	-	Yes	Yes	Yes	No info (Maybe not)
Function types	-	Yes	Yes	Yes	Yes
Library based Control Structure	-	Yes	-	-	No
Proposed closure syntax	Name(args) {block}	{args => block;expr}	#(args) {block}	#(args) {block} #(args) expr	#(args) {block} #(args) (expr)

A table never captures the full detail of any proposal. However, I hope that I've demonstrated some key points.

Firstly, the table shows how the CFJ is worthy of a new name (from BGGA) as it treats the problem space differently by avoiding non-local returns and control invocation. Neal Gafter is also the only author of the CFJ proposal, unlike BGGA.

Secondly, it should be clear that at the high level, the CFJ and FCM proposals are similar. But in many respects, this is also a result of what would naturally occur when the non-local returns and control invocation is removed from BGGA.

Finally, it should be clear that it is not certain that the CFJ proposal meets the aims that Mark announced at Devoxx ("simple closures"). There simply isn't enough hard information to make that judgement.

One question I've seen on a few tweets and blog posts is whether Mark and Sun picked the BGGA or FCM proposal. Well, given what Mark said in the BOF (via Reinier), the answer is that neither was "picked" and that he'd like to see a new form of "simple closures" created. However, it should also be clear that the choices announced at Devoxx were certainly closer to the FCM proposal than any other proposal widely circulated at the time of the announcement.

At this point, it is critical to note that Neal Gafter adds a huge amount of technical detail to each proposal he is involved with - both BGGA and CFJ. The FCM proposal, while considerably more detailed than CICE, was never at the level necessary for full usage in the JDK. As such, I welcome the CFJ proposal as taking key points from FCM and applying the rigour from BGGA.

As of now, there has been relatively little debate on extension methods.

Summary

The key point now is to focus on how to implement closures within the overall scope laid out. This should allow the discussion to move forward considerably, and hopefully with less acrimony.

Closures in JDK 7

So, on Wednesday, Mark Reinhold from Sun announced that it was time for closures in Java. This came as a surprise to everyone, but what was announced?

Closures in JDK 7

Mark took the audience through the new features of JDK 7 in his presentation at Devoxx. As part of this he showed the Fork Join framework.

Part of this framework features a set of interfaces to support functional style predicates and transforms. But, in order to implement this in Java required 80 interfaces, all generified, and that only covered 4 of the primitive types! Basically, Mark, and others, had clearly come to the conclusion that this code was important to Java, but that the implementation with single-method interfaces, generics and inner classes was too horrible to stomach for the JDK.

Thus, 'its time to add closures to Java'.

Mark announced that Sun would investigate, and intended to commit, closures to OpenJDK for JDK 7. While there will be no JSR yet, it isn't unreasonable to expect that there will be an open aspect to discussions.

Mark noted that the work in the closures discussion, and particularly the detail in the BGGA proposal, had allowed the options to be properly explored. He emphasised how this had resulted in the decision to reject key aspects of BGGA.

JDK 7 closures will not have control-invocation statements as a goal, nor will it have non-local returns. He also indicated that access to non-final variables was unlikely. Beyond this, there wasn't much detail on semantics, nor do I believe that there has been much consideration of semantics yet.

On syntax, Mark wrote up some strawman syntaxes. These follow the FCM syntax style:

  // function expressions
  #(int i, String s) {
    System.println.out(s);
    return i + s.length();
  }

  // function expressions
  #(int i, String s) (i + s.length())
  
  // function types
  #int(int, String)

Mark also argued that a form of extension methods would be added to allow closures to be used with existing libraries like the collections API.

Although I must emphasise that everything announced was a proposal and NOT based on any specific proposal (BGGA, FCM or CICE). However, the syntax and semantics do follow the FCM proposal quite closely.

In addition, Neal Gafter has produced an initial formal writeup of what was announced derived from BGGA. Based on my initial reading, I believe that Neal's document represents a good place to start from. I also hereby propose that we refer to Neal's document as the CFJ proposal (Closures for Java), as BGGA rather implies control invocation.

So, lets hope the process for closures provides for feedback, and we get the best closures in the style most suitable for Java in JDK 7.