Thursday, 28 February 2008

FCM closures - options within

The FCM closures proposal, with the JCA extension, consists of multiple parts. This blog outlines how those parts fit together.

FCM+JCA spec parts

The FCM+JCA spec, contains the following elements:

  • Method literals, also constructor and field literals
  • Method references, also constructor references
  • Inner methods
  • Method types, aka function types
  • Control invocation, in the JCA extension

These five parts all fulfil different roles in the proposal. But what is often not understood is how feasible it would be to implement less than the whole specification.

Method references and literals

Reviewing response to the entire debate, it is clear to me that method references and/or method literals have generally widespread appeal. The ability to reference a method, constructor or field in a compile-time safe and refactorable way is a huge gain for a statically typed language like Java.

It should be noted that although Method References appear only in the FCM spec, they could be added to the BGGA or CICE spec without difficulty. Also, they are included in the closures JSR proposal.

There are three areas of contention with method references and literals.

Firstly, should both references and literals be supported, or just references. Or, looking at the question differently, should literals have a different syntax.

The problem here is that if a method reference and literal have the same syntax then it is unclear as to what the type of the expression is. The FCM prototype demonstrates, I believe, that this can be solved using the same syntax. The approach taken is to say that the default type is a method literal, but that it can be converted (boxed) to a method reference at construction time. Any ambiguity is an error.

 Method m = Integer#valueOf(int);
 IntToInteger i = Integer#valueOf(int);

In this example, IntToInteger is a single method interface. Because the expression Integer#valueOf(int) is assigned to a single method interface, the conversion occurs (generating a wrapping inner class).

The second issue is what a method reference can be boxed into. This is essentially a question of whether function types should be supported, and I'll cover that below.

The third issue is syntax, specifically the use of the #. Personally, I find this syntax natural and obvious to read, but I know others differ. I think it is important to get the syntax right, but final decisions on that can come later.

So, are method literals and reference required when implementing FCM? I would say 'yes'. These are simple, popular, constructs that naturally extend Java in a non-threatening way. Some have suggested omitting the literals as reflection is not type-safe, however this misses the point of the large number of existing frameworks and APIs that accept reflection Method as an input parameter.

Inner methods / Closures

 ActionListener lnr = #(ActionEvent ev) {
   ...
 };

This is where the key difference with BGGA lies, notably over the meaning of return and the value, and safety, of non-local returns.

Opinions on this appear to me to be impacted by the generics implementation, where the decision was made to do what feels like 'half a job'. As a result, there is a meme that runs 'we must implement closures fully or not at all'. This meme is extremely unfortunate, as it is not allowing a rational analysis of the semantics of the proposals. Anyone supporting BGGA really needs to consider the mistakes that developers will make again and again with the non-local return/last-line-no-semicolon approach.

So, are inner methods required when implementing FCM? I would say 'effectively, yes'. Although you could just implement method literals and references alone, there are even bigger gains to be had from adding inner methods. They greatly simplify the declaration of single method inner classes, and allow much of the impact of closures in the style of Java.

Function types / Method types

 #(int(String) throws IOException)

These allow a new powerful form of programming where common pieces of code can be easily abstracted. They simply act as types, but they have two different properties from other types.

Firstly, they have no name. This means an absence of Javadoc, including any semantic requirements of the API, such as thread-safety or null/not-null.

Secondly, they only describe the input and output types. This is a higher abstraction than Java has previously used, and will require a mindset shift for those using them.

So, are function types required when implementing FCM? I would say 'no, not required'. It is perfectly possibly and reasonable to implement FCM without method types. In fact, that is what the prototype does. In practice, this just means that all conversions from method references and inner methods must be to single method interfaces rather than method types.

Omitting method types greatly simplifies the conceptual weight of the change. The downside is that true higher order functional programming becomes near impossible. That may be no bad thing. Java is not, and never has been, a functional programming language. It seems very odd to try and push it in that direction at this point in its life.

A better alternative would be to pursue supporting primitive types in generics. This would greatly reduce the overhead of single method interfaces required by something like the fork-join framework.

Similarly, making single method interfaces easier to write (lightweight interfaces) would be a direction to take in the absence of method types.

Control invocation

 withLock(lock) {
   ...
 }
 public void withLock(#(void()) block : Lock lock) {
   ...
 }

Control invocation forms are perhaps the only way forward in Java longer term because they allow us to escape from many of these language change debates. They allow anyone to write methods that can be used in the style of control statements. It is vital to remember that they are just methods however.

BGGA appears to build much of its spec around control invocation, and the non-local returns make perfect sense in this area.

The JCA spec defines that the calling code should be identical to BGGA, but the method invoked should be written differently. The aim of JCA is to provide an element of discouragement from using control invocation. This is because of the additional complexities in getting the code right (exception transparancy, completion transparancy, non-local returns etc). A different, special, syntax encourages this feature to be restricted to senior developers, or heavily code reviewed.

So, is control invocation required when implementing FCM+JCA? I would say 'no'. It is perfectly possibly and reasonable to implement FCM+JCA without control invocation (although of course that means it would just be FCM!).

The inclusion or omission of method types is also linked to control invocation, as method types are a pre-requisite for control invocation in the JCA spec.

Summary of possible implementations

Thus, here are the possible FCM+JCA implementation combinations that make sense to me:

  1. Literals and References
  2. Literals, References and Inner methods
  3. Literals, References, Inner methods and Method types
  4. Literals, References, Inner methods, Method types and Control invocation

My preferred options are number 2 and number 4.

Why? Because, I believe inner methods are too useful to omit, and I believe method types are generally too complex unless you really need them. (Also Java isn't a functional programming language.)

The key point of this blog is to emphasise that FCM is not a take it or leave it proposal. There are different options and levels within it that could be adopted.

This extends to versions of Java. For example, it would be feasible to implement option 1, literals and references in Java 7, whilst adding inner methods and maybe more in Java 8.

Summary

I've shown how FCM has parts which can be considered separately to a degree. I've also indicated which combinations make sense to me.

Which combinations make sense to you?

10 comments:

  1. Thanks for the insights, Stephen.
    I would like to add on the JCA topic a bit. Although, the current proposal bases on method types, in my opinion, it would be easy to detach JCA from closures completely. As a JCA only can take one void-return block, introducing a separate definition syntax and an implicit block invocation syntax would provide the same abilities. Simple idea:

    public static void forEach(K, V : Map map) {
    ´ ´ for (Entry entry : map.entrySet()) {
    ´ ´ ´ ´ do(entry.getKey(), entry.getValue());
    ´ ´ }
    }

    Apart from how to implement control abstraction, I just had a discussion yesterday on an argument that I did not think of before, even historically being a Smalltalk guy: seeing that Java has about 50 keywords currently (plus operators and so on), it seems interesting to see that Smalltalk code in most cases is much more readable/elegant than Java, only having about a handfull of keywords and -symbols. The main reason is that Smalltalk does not have control structures for computations but actually allows control abstractions by applying messages (methods) and blocks (closures).

    ReplyDelete
  2. Stephen,

    If I've understood the proposal and your examples correctly, once the prototype deals with generics better the following code will compile and output 'false' when run - is that correct? I expect that would result in a great deal of confusion if so...

    Mark


    import java.util.*;

    public class Foo {

    public static void main(String[] args) {

    List methods = new ArrayList();
    methods.add(Foo#bar());
    System.out.println(methods.contains(Foo#bar()));
    }

    public static void bar() { }
    }

    ReplyDelete
  3. Stephen Colebourne28 February 2008 at 23:23

    Stefan: Agreed that there ought to be ways to make control invocation independent of method types.

    Mark: Your example would return false at present, however that is a question of whether method references should define equals and hashcode. I suspect that the final spec will need to. Certainly, in your example it makes sense for the underlying generated inner class to be a singleton.

    ReplyDelete
  4. Stephen,

    Actually it seems to return true at the moment - the expression Foo#bar() is translated to an instance of java.lang.reflect.Method in both the call to add() and the call to contains().

    However, once generics are handled properly, I'd expect Foo#bar() to be translated to a Runnable in the first case - since we're calling add(Runnable) - but still to
    a java.lang.reflect.Method in the second case - calling contains(Object) - as such, I think the result would be false.

    Mark

    ReplyDelete
  5. What do you think about this syntax?

    Method m = instanceof Integer.valueOf(int);

    No new keywords or tokens.

    ReplyDelete
  6. 1. References

    Good idea - there is also a long standing RFE on the Sun database for this

    2. Literals

    Probably not worth the trouble - you can use an inner method just as easily

    3. Inner methods

    I have suggested the method keyword:

    http://www.artima.com/weblogs/viewpost.jsp?thread=182412

    Your example using a method keyword is:

    ActionListener lnr = method( ActionEvent ev ) {
    __...
    };

    You could also add some type inference to this:

    ActionListener lnr = method( ev ) {
    __...
    };

    or:

    declare lnr = ActionListener.method( ev ) {
    __...
    };

    For right to left inference. Type inference could be added to any of the proposals.

    Also I have suggested that the behaviour of inner methods should be as inner classes. In particular an unqualified this in the case of ambiguity refers to the inherited class not the enclosing class. I think different behaviour from inner classes will be confusing and leads to problems. The closures require boxing for example in BGGA to play with existing APIs. An OO language should have objects, full stop. Lets not introduce any more autoboxing.

    For non-local returns I have suggested named returns, see above URL or:

    http://www.artima.com/weblogs/viewpost.jsp?thread=220920


    4. Method types

    Not worth the trouble, just add generic interfaces to java.lang:

    interface Method0 { R call(); }
    interface Method1 { R call( A1 a1 ); }
    ...


    5. Control invocation

    The simplest option is to make the () optional when calling a method (provided that it isn't ambiguous), e.g.:

    withLock lock, method{
    __...
    };

    ReplyDelete
  7. Method references / literals are nice. But I won't buy any closures proposal without full "control invocation" features. Don't call your proposal a closures proposal, if this feature is not included. Perhaps it would be best not to compete with BGGA and specify a new proposal just for method references / literals.

    ReplyDelete
  8. Stephen Colebourne2 March 2008 at 11:14

    Mark: Yes, your example is a pain. But its really more an example of the poor implementation of generics (if the list retained its type then the contains method would also convert to a Runnable)

    Paul: I understand the desire to avoid a new symbol, but this doesn't work for me. instanceof is all about classes and types, not references.

    Faith: I don't believe that the computer science feature 'closures' requires control invocation - I may be wrong of course. Part of the purpose of this post is to empahsise that the reference/literal parts could be taken separately, and that means adding it to another proposal if desired.

    ReplyDelete
  9. Stephen Colebourne2 March 2008 at 11:24

    Howard: I think you may have references and literals the wrong way around. The use of a 'method' keyword instead of # is a valid syntax alternative we should consider. I don't think 'boxing' is the right term for BGGA or FCMs approach, as the conversion is solely from the literal form on construction. I don't agree with optional parameters in Java.

    On 'this', I believe that if the calling code includes the class name to be implemented then it is OK for 'this' to refer to that class. However, if the class name is not mentioned (as with #), then 'this' should refer to the lexical class.

    ReplyDelete
  10. I do think inner methods that behave differently than inner classes are like boxing and function types even more like boxing, e.g. in BGGA:

    pool.execute( { => System.out.println( "Hello" ); } );

    is OK, but:

    { => void } printer = { => System.out.println( "Hello" ); }
    pool.execute( printer );

    Isn't because the closure got boxed into a synthetic function interface on the first line and the second line is looking for a Runnable. In the first version the closure is boxed into a Runnable.

    This boxing behaviour is why you need to have the RestrictedFunction interface, to turn off the normal behaviour. I think a better alternative to have closures that are objects - just like everything else. The changed behaviour simply removes power, you can't inherit, and causes problems with interoperation between existing APIs and closures.

    Remember with the behaviour of this we are only talking about what happens in the case when there is an ambiguity. These ambiguities are rare, this is a double reason to go with the existing practice of having to qualify enclosing references. Imagine someone debugging some code and they get one behaviour from an inner class and a different behaviour from an inner method and that the bug is rare, so they are not familiar with the two behaviours. This is going to be really tough to debug and really tough to teach.

    ReplyDelete