Thursday, 19 November 2009

Closures in JDK 7

So, on Wednesday, Mark Reinhold from Sun announced that it was time for closures in Java. This came as a surprise to everyone, but what was announced?

Closures in JDK 7

Mark took the audience through the new features of JDK 7 in his presentation at Devoxx. As part of this he showed the Fork Join framework.

Part of this framework features a set of interfaces to support functional style predicates and transforms. But, in order to implement this in Java required 80 interfaces, all generified, and that only covered 4 of the primitive types! Basically, Mark, and others, had clearly come to the conclusion that this code was important to Java, but that the implementation with single-method interfaces, generics and inner classes was too horrible to stomach for the JDK.

Thus, 'its time to add closures to Java'.

Mark announced that Sun would investigate, and intended to commit, closures to OpenJDK for JDK 7. While there will be no JSR yet, it isn't unreasonable to expect that there will be an open aspect to discussions.

Mark noted that the work in the closures discussion, and particularly the detail in the BGGA proposal, had allowed the options to be properly explored. He emphasised how this had resulted in the decision to reject key aspects of BGGA.

JDK 7 closures will not have control-invocation statements as a goal, nor will it have non-local returns. He also indicated that access to non-final variables was unlikely. Beyond this, there wasn't much detail on semantics, nor do I believe that there has been much consideration of semantics yet.

On syntax, Mark wrote up some strawman syntaxes. These follow the FCM syntax style:

  // function expressions
  #(int i, String s) {
    System.println.out(s);
    return i + s.length();
  }

  // function expressions
  #(int i, String s) (i + s.length())
  
  // function types
  #int(int, String)

Mark also argued that a form of extension methods would be added to allow closures to be used with existing libraries like the collections API.

Although I must emphasise that everything announced was a proposal and NOT based on any specific proposal (BGGA, FCM or CICE). However, the syntax and semantics do follow the FCM proposal quite closely.

In addition, Neal Gafter has produced an initial formal writeup of what was announced derived from BGGA. Based on my initial reading, I believe that Neal's document represents a good place to start from. I also hereby propose that we refer to Neal's document as the CFJ proposal (Closures for Java), as BGGA rather implies control invocation.

So, lets hope the process for closures provides for feedback, and we get the best closures in the style most suitable for Java in JDK 7.

17 comments:

  1. The news was a really good surprise. And the choice of syntax and semantic seems very well done, too. *crossingmyfingers* :)

    ReplyDelete
  2. What a nice suprise!

    ReplyDelete
  3. Any idea if there is a syntax sugar for calling invoke() on a function type?

    i.e. my understanding of function types goes along the simplified lines of...

    instead of writing this (note:simplified from the full docs above, just to get the point across)



    interface Foo {

    int invoke(String s);

    }

    Foo parseInteger;





    you can write this new shorthand variable declaration, it is equivalent...



    #int(String) parseInteger;





    you could then provide an implementation of the interface above like this...



    parseInteger = #(String s) String.parseInt(s);





    and I assume you'd be able to call it yourself like this...



    int birthYear = parseInteger.invoke("1972");



    ...

    so my question is, can I call something like

    parseInteger("1972")

    as a shortcut to

    parseInteger.invoke("1972")

    Cheers

    Jez

    ReplyDelete
  4. Hi, Stephen,

    I understand that you are one of the author for closure proposal "First-class methods: Java-style closures"
    http://docs.google.com/View?docid=ddhp95vd_6hg3qhc

    I have extracted an example from the proposal of FCM and will like to understand how closure works in the following scenarios.


    Extracted portion ===================================================================
    The proposal mentions that
    "The Comparator interface is the standard mechanism in Java to provide the code needed to sort lists. Here is a Comparator that sorts by length of String:

    List list = ...
    Collections.sort(list, new Comparator() {
    public int compare(String str1, String str2) {
    return str1.length() - str2.length();
    }
    });

    With the changes in this proposal, the code could be written as follows:

    List list = ...
    Collections.sort(list, #(String str1, String str2) {
    return str1.length() - str2.length();
    });

    This syntax is termed an anonymous inner method, and is a logical extension to the anonymous inner class concept. The conversion to a Comparator is automatic. "

    ========================================================================

    and now assuming the following scenario, how will Java determine which Comparator to use ?

    For example:

    Let assume that Collections has overloaded sort method to accept Comparator and Comparator2.


    Collections.sort(list, new Comparator() {
    public int compare(String str1, String str2) {
    return str1.length() - str2.length();
    }
    });

    Collections.sort(list, new Comparator2() {
    public int compare(String str1, String str2) {
    return str1.length() - str2.length();
    }
    });


    with overloaded sort method

    Collections.sort(list, Comparator a)

    Collections.sort(list, Comparator2 b)


    and if by using closure equivalent,

    Collections.sort(list, #(String str1, String str2) {
    return str1.length() - str2.length();
    });


    Question 1:
    Which type of Comparator is closure delegated to ? Comparator or Comparator2 ?


    Question 2:
    will there be a AS operator like Groovy ?

    Collections.sort(list, #(String str1, String str2) {
    return str1.length() - str2.length();
    } as Comparator2 );

    Question 3:
    What if there is multiple methods in Comparator that use similar method signatures ?

    public int compare(String str1, String str2)
    public int compareOne(String str1, String str2)
    public int compareTwo(String str1, String str2)


    thx

    ReplyDelete
  5. Stephen Colebourne19 November 2009 at 09:17

    @Jeremy, Yes the function invocation syntax you describe was mentioned by Mark. But he only presented a strawman proposal. See CFJ and FCM for more details on real proposals.

    @GeekyCoder, Q1, ambiguous compile error. Q2, no As operator. Q3, only single method interfaces are supported.

    ReplyDelete
  6. Stephen,
    thx for the reply.

    To add on,
    So it means that no change is needed for existing Java API (only the Java compiler need change) to take advantage of closure since if the case in Q1 and Q3 should occurs, anonymous inner class should be used instead ?

    ReplyDelete
  7. With the new extended timeline of Java 7, is there any chance that the new date and time API (JSR-310) could squeeze in?

    ReplyDelete
  8. Syntax Error: #(int i, String s) (i + str.length())
    str is undefined! =)
    The same applies to return i + str.length();

    ReplyDelete
  9. "He also indicated that access to final variables was unlikely." - did you mean "access to non-final"? I really hope that's not true. I'm okay with dropping the most advanced parts of BGGA - the new proposal from Neal is quite good - but dropping access to non-final vars, including the ability to update such vars, would be terrible, we couldn't even call the result real closures.

    ReplyDelete
  10. As dangeregg, I am also curious if the schedule change means that JSR-310 could sneak in. That would be a very welcome addition.

    Best,
    Ismael

    ReplyDelete
  11. I think giving closures access to non-final variables would be a recipe for subtle bugs. Restricting them to final vars (like anonymous inner classes are) would be the right choice. I can't remember when I last used a mutable local variable.

    ReplyDelete
  12. See also Cay Horstmann's blog: http://weblogs.java.net/blog/cayhorstmann/archive/2009/11/18/closures-java-7 summarizing the changes to BGGA. From that summary, it looks like the answer to the "access to non-final variables" issue is: "yes, you can do that, but you have to use a @Shared annotation to mark the variable".

    ReplyDelete
  13. Does anyone have a link to a decent example that shows this "tons of interfaces needed unless closures are used? Either the actual 80 interfaces that Mark R. was considering or something similar would be fine. Thanks.

    ReplyDelete
  14. Now as I've read the FCM proposal and the BGGA proposal I miss one thing in FCM:
    A return that returns from the enclosing method of the closure, like BGGA proposes, but I do think your reasons to just return from the closure are very good.

    Alas it comes with inflexibility.

    So why not take the best of both worlds and do it like this:

    "return" would return from the closure, as you propose.

    "this.return" would return from the enclosing method, like BGGA proposes.

    As "this" in FCM references the enclosing class, it would also seem quite reasoable for me that this return targets the method of the enclosing class instead if the Closure itself.

    This would also be somewhat similar to the construct "Class.this":

    MyClass{
    //..
    void bar(){
    Runnable r = new Runnable(){
    public void run(){
    MyClass.this.foo();
    foo();
    }
    void foo(){...} /last called
    }
    void foo(){...} //first called
    }

    Any thoughts on this?

    ReplyDelete
  15. Will we still be able to use the old syntax?

    ReplyDelete
  16. Stephen Colebourne20 November 2009 at 08:41

    @Andy, the CFJ document (v0.6a) is also a proposal. Its not Sun's approved position, just a really good starting point.

    The closest link to the 80 interfaces I could find quickly is http://www.javac.info/jsr166z/jsr166z/forkjoin/package-summary.html

    @Gary, No old syntax will be removed

    ReplyDelete