Closures - comparing options impact on language features

At JavaOne, closures were still a hot topic of debate. But comparing the different proposals can be tricky, especially as JavaOne only had BGGA based sessions.

Once again, I will be comparing the three key proposals in the 'closure' space - BGGA from Neal Gafter et al, CICE from Josh Bloch et al, and FCM from Stefan and myself, with the optional extension JCA extension. In addition I'm including the BGGA restricted variant, which is intended to safeguard developers when the closure is to be run asynchronously, and the ARM proposal for resource management.

In this comparison, I want to compare the behaviour of key program elements in each of the main proposals with regards the lexical scope. With lexical scoping here I am referring to what features of the surrounding method and class are available to code within the 'closure'.

Neal Gafter analysed the lexical scoped semantic language constructs in Java. The concept is that 'full closures' should not impact on the lexical scope. Thus any block of code can be surrounded by a closure without changing its meaning. The proposals differ on how far to follow this rule, as I hope to show in the table (without showing every last detail):

Is the language construct lexically scoped when surrounded by a 'closure'?
	Inner class	CICE	ARM	FCM	JCA	BGGA Restricted	BGGA
variable names	Part (1)	Part (1)	Yes	Yes	Yes	Part (7)	Yes
methods	Part (2)	Part (2)	Yes	Yes	Yes	Yes	Yes
type (class) names	Part (3)	Part (3)	Yes	Yes	Yes	Yes	Yes
checked exceptions	-	-	Yes	Yes (4)	Yes (4)	Yes (4)	Yes (4)
this	-	-	Yes	Yes	Yes	Yes	Yes
labels	-	-	Yes	- (5)	Yes	- (5)	Yes
break	-	-	Yes	- (5)	Yes	- (5)	Yes
continue	-	-	Yes	- (5)	Yes	- (5)	Yes
return	-	-	Yes	- (6)	Yes	- (8)	Yes

(1) Variable names from the inner class hide those from the lexical scope
(2) Method names from the inner class hide those from the lexical scope
(3) Type names from the inner class hide those from the lexical scope
(4) Assumes exception transparency
(5) Labels, break and continue are constrained within the 'closure'
(6) The return keyword returns to the invoker of the inner method
(7) The RestrictedClosure interface forces local variables to be declared final
(8) The RestrictedClosure interface prevents return from compiling within the 'closure'
BTW, if this is inaccurate in any way, I'll happily adjust.

So more 'Yes' markers means the proposal is better right? Well not necessarily. In fact, I really don't want you to get that impression from this blog.

What I'm trying to get across is the kind of problems that are being tackled. Beyond that you need to look at the proposals in more detail with examples in order to understand the impact of the Yes/No and the comments.

Let me know if this was helpful (or innaccurate), or if you'd like any other comparisons.

JavaOne JSR-310 Date and Time API BOF tonight!

If you're at JavaOne and hate Date and Calendar then come along to our BOF tonight! Its at 21:55 in MC105 - BOF-2794.

We'll be having a short talk about what's wrong with Date and Calendar, what current alternatives are out there and outline the initial thoughts of the JSR group. There will also be a mini-puzzler with mini-chocolate reward!!!

The main aim of the BOF is to have a discussion however - what calendar systems should we support, what kind of API, should we address SQL access? We'd love to hear your input, so come along if you can!

First time at JavaOne

So, its my first time at JavaOne. I'm about to head off to the warm up CommunityOne event.

My main task for the week is the BOF for JSR-310. This is on Wednesday evening at 21:55. I'd love to see you there if you're around.

Apart from that, I'll be following all the debates on Java language changes and generally wandering around. If you see me (and recognise me) then say hi.

Closures options - FCM+JCA is my choice

In my last post I drew a diagram showing how I viewed the main proposals in terms of complexity. It has since been pointed out to me that I didn't clearly identify whether the JCA part of FCM was included or not. So I'll redraw the diagram and express my opinon.

The three key proposals in the 'closure' space are - BGGA from Neal Gafter et al, CICE from Josh Bloch et al, and FCM from Stefan and myself. FCM has an optional extension called JCA which adds control abstraction [1].

Control abstraction [1] is that part of the closures language change which allows a user to write an API method which acts, via a static import, as though it is a built in keyword. This is an incredibly powerful facility in a programming language. However, reading many forums, this is a facility that many Java developers also find concerning.

Given this, I'll redraw the diagram from yesterday:

 --------------------------------------------------------------------------
  Simpler                                                     More complex
 --------------------------------------------------------------------------
   No change   Better inner   Method-based   Method-based   Full-featured
                 classes        closures     closures and   closures with
                                             control abst.   control abst.
 --------------------------------------------------------------------------
 |        No control abstraction [1]      |    Control abstraction [1]   |
 --------------------------------------------------------------------------
   No change      CICE*           FCM           FCM+JCA         BGGA
 --------------------------------------------------------------------------

* It should also be noted that Josh Bloch has written the ARM proposal for resource management that tackles a specific control abstraction [1] use case.

So, what is my preference?

Firstly, I believe that Java should be enhanced with FCM-style closures (where the 'return' keyword returns to the invoker of the closure).

Secondly, I believe that control abstraction [1] would be a very powerful and expressive language enhancement. Personally, I would like to see Java include control abstraction, but I am very aware that it is a real concern for others in the community. I am also not yet fully confident of the risks involved by rushing into control abstraction [1].

Thus, my preferred option is FCM+JCA for Java 7. However I would also accept a potentially lower risk option of including just the FCM-style solution in Java 7 and adding JCA-style control abstraction [1] in Java 8.

Feel free to add your preference to the comments.

[1] Update: Neal Gafter has indicated that I am using conflicting terminology wrt 'control abstraction'. Neal uses the term 'control abstraction' to refer to any closure which can abstract over an arbitrary block of code including interacting with break, continue and return. Whereas, in this blog post I have been referring to control abstraction as adding API based keywords (described in paragraph 3), which Neal refers to as the 'control invocation syntax'. Apologies for any confusion

Cautious welcome on closure JSR - too soon for consensus

Neal Gafter announced a draft closures JSR a few days ago. I'd like to give the JSR a cautious welcome, but also wish to emphasise that the 'consensus' term could be misleading.

At present, there are three key proposals in the 'closure' space - BGGA from Neal Gafter et al, CICE from Josh Bloch et al, and FCM from Stefan and myself. Conceptually, these proposals meet differing visions of how far Java should be changed, and what is an acceptable change.

 ----------------------------------------------------------------
  Simpler                                           More complex
 ----------------------------------------------------------------
   No change    Better inner     Method-based     Full-featured
                  classes          closures         closures
 ----------------------------------------------------------------
   No change       CICE              FCM              BGGA
 ----------------------------------------------------------------

In deciding to support this JSR, a number of factors came into play for me:

• I want to see closures in Java 7 - unfortunately it may already be too late for that
• I'm not a language designer, and don't have the knowledge to run a JSR myself on this topic
• The text of the draft JSR is clearly phrased as a derivation of the BGGA proposal
• There has been positive feedback on Java forums about each of the three proposals - the Java community does not have a settled opinion yet
• Many Java developers feel uncomfortable with any change in this area
• Groovy, a Java derived language, uses an approach similar to FCM
• This is the only JSR proposal currently available to support
• Private emails

In the end, taking all of these factors into account, I took the judgement that this JSR was currently the only game in town and it would be better for me to participate rather than stand outside. And that is why I give it a cautious welcome.

Unfortunately, the extensive list of pre-selected Expert Group members (not including FCM) and Expert Group questionnaire came as rather a surprise, especially given the 'consensus' tag.

On the point of consensus, some commentators have assumed that this means a compromise proposal has been created, or agreement reached. This isn't the case.

Looking forward, my preference would be for Sun to finally play their hand on this topic. As a neutral player, they are well placed to encourage all parties to take a step back for a minute and put the focus firmly on what is best for Java going forward. Let the Sun shine :-)

ASF open letter to Sun

Despite a great deal of behind the scenes effort, the Apache Software Foundation (ASF) has been forced to write an open letter to Sun Microsystems. This letter is regarding the inability of the ASF to obtain a version of the testing compatibility kit for Java SE, as needed by Apache Harmony, under a suitable license.

As the open letter details, Sun is attempting to avoid its contractual obligations from the JSPA, which is the legal agreement of the Java Community Process (JCP). This seriously places at risk the ability of the JCP to act as a truly independent open standards organisation.

Considerable efforts (over 7 months) have gone into trying to avoid reaching this point. It is hoped that now the issue is public, the Java community can bring to bear pressure on Sun to meet its contractual obligations quickly and amicably.

Further questions may be answered in the FAQ.

On this issue, Geir Magnussen Jr acts as the official ASF representative. I am speaking here simply to provide information.

Java Control Abstraction for First-Class Methods (Closures)

Stefan Schulz, Ricky Clarkson and I are pleased to announce the release of Java Control Abstraction (JCA). This is a position paper explaining how we envisage the First-Class Methods (FCM) closures proposal being extended to cover control abstraction.

Java Control Abstraction

So, what is control abstraction? And how does it relate to FCM? Well its all about being able to add methods in an API that can appear as though they are part of the language. The classic example is iteration over a map. Here is the code we write today:

  Map<Long,Person> map = ...;
  for (Map.Entry<Long, Person> entry : map) {
    Long id = entry.getKey();
    Person p = entry.getValue();
    // some operations
  }

and here is what the code looks like with control abstraction:

  Map<Long,Person> map = ...;
  for eachEntry(Long id, Person p : map) {
    // some operations
  }

The identfier eachEntry is a special method implemented elsewhere (and statically imported):

  public static <K, V> void eachEntry(for #(void(K, V)) block : Map<K, V> map) {
    for (Map.Entry<K, V> entry : map.entrySet()) {
      block.invoke(entry.getKey(), entry.getValue());
    }
  }

As can be seen, the API method above has a few unique features. Firstly, it has two halves separated by a colon. The first part consists of a method type which represents the code to be executed (the closure). The second part consists of any other parameters. As shown, the closure block is invoked in the same way as FCM.

The allowed syntax that the developer may enter in the block is not governed by the rules of FCM. Instead, the developer may use return, continue, break and exceptions and they will 'just work'. Thus in JCA, return will return from the enclosing method, not back into the closure. This is the opposite to FCM. This behaviour is required as the JCA block has to act like a built-in keyword.

One downside of the approach is that things can go wrong because the API writer has access to a variable that represents the closure. The API writer could store this in a variable and invoke it at a later time after the enclosing method is complete. However, if this occurred, then any return/continue/break statements would no longer operate correctly as the original enclosing method would no longer be on the call stack and a weird and unexpected exception will be thrown.

The semantics of a pure FCM method invocation are always safe, and there is no way to get one of these unexpected exceptions. But, for JCA control abstraction we could find no viable way to stop the weird exceptions. Instead, we have chosen to specifically separate the syntax of FCM from the syntax of control abstraction in JCA.

Our approach is to accompany the integration of control abstraction into Java by a strong set of messages. Developers will be encouraged to use both FCM callbacks and JCA control abstractions. However, developers would only be encouraged to write FCM style APIs, and not JCA.

Writing the API part of any control abstraction (including JCA) is difficult to get right (or more accurately easy to get wrong). As a result, some coding shops may choose to ban the writing of control abstraction APIs, but by having a separate syntax this will be easy to do for the tools. It is expected, of course, that the majority of the key control abstractions will be provided by the JDK, where experts will ensure that the control abstraction APIs work correctly.

Summary

This document has taken a while to produce, especially by comparison with FCM. In the end this indicated to us that writing a control abstraction is probably going to be a little tricky irrespective of what choices the language designer makes. By separating the syntax and semantics from FCM we have clearly identified the control abstraction issue in isolation, which can only be a good thing.

Feedback always welcome!