Tuesday, 18 December 2007

Closures - Comparing the core of BGGA, CICE and FCM

In this blog I'm going to compare the core of the three principle 'closure' proposals. This is particularly apt following the recent surge in interest after Josh Bloch's Javapolis talk.

Comparing 'closure' proposals

The three principle closure proposals are:

  • BGGA - full closures for Java
  • CICE - simplified inner classes (with the related ARM proposal)
  • FCM - first class methods (with the related JCA proposal)

It is easy to get confused when evaluating these competing proposals. So much new syntax. So many new ideas. For this blog I'll summarise, and then deep dive into one area.

The first key point that I have been making recently is that closures is not 'all or nothing' - there are parts to the proposals that can be implemented separately. This table summarises the 5 basic parts in the proposals (I've split the last one into two rows):

  BGGA CICE+ARM FCM+JCA
Literals for reflection - - FCM member literals
References to methods - - FCM method references
One method callback classes BGGA closures CICE FCM inner methods
Function types BGGA function types - FCM method types
Library based Control Structure BGGA control invocation - JCA control abstraction
Language based Control Structure - ARM -

Of course a table like this doesn't begin to do justice to any of the three proposals. Or to express the possible combinations (for example, BGGA could add support for the first two rows easily). So, instead of talking generally, lets examine a key use case where all 'closure' proposals work.

One method callbacks

This issue in Java refers to the complexity and verbosity of writing an anonymous inner class with one method. These are typically used for callbacks - where the application code passes a piece of logic to a framework for it to be executed later in time. The classic example is the ActionListener:

  public void init() {
    button.addActionListener(new ActionListener() {
      public void actionPerformed(ActionEvent ev) {
        // handle the event
      }
    });
  }

This registers a callback that will be called by the framework whenever the button is pressed. We notice that the code in the listener will typically be called long after the init() method has completed. While this is an example from swing, the same design appears all over Java code.

Some of the issues with this code are immediately obvious. Some are not.

  1. The declaration of the listener is very verbose. It takes a lot of code to define something that is relatively simple, and importantly, that makes it harder to read.
  2. Secondly, information is duplicated that could be inferred by the compiler. Its not very DRY.
  3. The scope of the code inside actionPerformed() is different to that of the init() method. The this keyword has a different meaning.
  4. Any variables and methods from the interface take preference to those available in init(). For example, toString(), with any number of parameters, will always refer to the inner class, not the class that init() is declared in. In other words, the illusion that the code in actionPerformed() has full access to the code visible in init() is just that - just an illusion.

The three closure proposals differ in how they tackle this problem area. BGGA introduces full closures. These eliminate all the problems above, but introduce new issues with the meaning of return. CICE introduces a shorthand way of creating an inner class. This solves the first issue, and some of the second, but does not solve issue 3 or 4. FCM introduces inner methods. These eliminate all the problems above, and add no new surprises.

  // BGGA
  public void init() {
    button.addActionListener({ActionEvent ev =>
      // handle the event
    });
  }
  // CICE
  public void init() {
    button.addActionListener(ActionListener(ActionEvent ev) {
      // handle the event
    });
  }
  // FCM
  public void init() {
    button.addActionListener(#(ActionEvent ev) {
      // handle the event
    });
  }

In this debate, it is easy to get carried away by syntax, and say that one of these might look 'prettier', 'more Java' or 'ugly'. Unfortunately, the syntax isn't that important - its the semantics that matter.

BGGA and FCM have semantics where this within the callback refers to the surrounding class, exactly as per code written directly in the init() method. This is emphasised by removing all signs of the inner class. The scope confusion of inner classes (issues 3 and 4 above) disappears (as this is 'lexically scoped').

CICE has semantics where this within the callback still refers to the inner class. Only you can't even see the inner class now, its really well hidden. This simply perpetuates the scope confusion of inner classes (issues 3 and 4 above) and gains us little other than less typing. In fact, as the inner class is more hidden, this might actually be more confusing than today.

BGGA also has semantics where a return within the callback will return from init(). This generally isn't what you want, as init() will be long gone when the button press occurs. (It should be noted that this feature is a major gain to the overall power of Java, but comes with a high complexity factor)

FCM and CICE have semantics where return within the callback returns from the callback, as in an inner class. This enables easy conversion of code from today's inner classes. (For FCM, the return is considered to be back to the # symbol, a simple rule to learn)

And here is my personal summary of this discussion:

  // BGGA
  public void init() {
    button.addActionListener({ActionEvent ev =>
      // return will return to init() - BAD for many (most) use cases
      // this means the same as within init() - GOOD and simple
    });
  }
  // CICE
  public void init() {
    button.addActionListener(ActionListener(ActionEvent ev) {
      // return will return to caller of the callback - GOOD for most use cases
      // this refers to the inner class of ActionListener - BAD and confusing
    });
  }
  // FCM
  public void init() {
    button.addActionListener(#(ActionEvent ev) {
      // return will return to caller of the callback - GOOD for most use cases
      // this means the same as within init() - GOOD and simple
    });
  }

Summary

This blog has evaluated just one part of the closures proposals. And this part could be implemented without any of the other parts if so desired. (BGGA would probably find it trickiest to break out just the concept discussed, but it could be done)

The point here is that there are real choices to be made here. The simplicity and understandability of Java must be maintained. But that doesn't mean standing still.

Once again, I'd suggest FCM has the right balance here, and that anyone interested should take a look. Perhaps the right new features for Java are method references and inner methods, and no more?

Opinions welcome as always :-)

Saturday, 15 December 2007

Voting on Java 7 language changes

This year at Javapolis I again looked after the whiteboards. In this post, I'll discuss the results.

Whiteboard results

The whiteboards play an important part at Javapolis in bringing people together to discuss ideas and future trends.

This year, I chose to focus the whiteboard debate around a number of specific topics. This was achieved by having votes on 10 language changes proposals, plus areas for new ideas for Spring, JSF, JavaEE and JavaSE. I'm going to discuss the language change votes here, as the photos of the other boards aren't available yet.

Language change votes

Before the conference started, there were a number of language change proposals floating about, initiated by Neal and Josh at Google. These were discussed in a late night BOF on Thursday, but this was backed up by the votes on the whiteboards.

The full results are available (with more detailed examples), but here is the summary.

The strongest support was for Improving Generics, String Switch and Multi-catch. Attendees were giving a clear 'yes' to these being in Java 7.

The strongest opposition was to Typedef (type aliasing) and Extension methods and chaining. Attendees were giving a clear 'no' to these being in Java 7.

There was some support for Better null handling (with a new operator) and Properties. The balance was slightly against, but not overwhelming.

There was positive support for Using [] to access lists and maps. The balance was definitely in favour, yet there was a large minority against. This was perhaps the most surprising vote. Unfortunately, it wasn't possible to find out why there were so many 'no' votes. My view, is that it is still worth pursuing this for Java 7.

Summary

Java being open source isn't going to be enough going forwards - it will also be necessary to provide an open community. Votes and direct feedback like this have a really positive effect, allowing Java's growth to be influenced by real developers and their problems.

The Javapolis Experience 2007

So, Javapolis is over for another year! Big thanks and congratulations to the Javapolis team for making it such an excellent conference.

This year I, personally, had more of a focus on the networking aspect of the conference - trying to take time to chat to many of the delegates and speakers. This is often where you make the interesting and surprising connections that will come in useful in the future.

Whiteboards

The whiteboards again played a really interesting part in getting people talking. This year, I left half the whiteboards as freeform, available for any comments. The other half were carefully organised to get a snapshot of opinions on specific topics.

These fell in three areas. Firstly, boards for collecting ideas for changes in Spring, JSF, J2EE and Java 7. Next up, was a board gathering information on how many people are using each application server, and each web framework.

Finally, about 10 language change proposals were described. Each of these was then offered with a vote - "Do you support this change" - Yes/No/Maybe. In the region of 150 votes were received for each of these proposals, so hopefully the results should be pretty representative. I'll discuss the results in my next blog post.

Closures

The other big topic of the conference was again closures. The first talk on the topic was a well-attended BOF from Neal Gafter discussing the current status of BGGA.

The second was a last minute change - something you'd never see at JavaOne! Josh Bloch gave a talk on 'The Closures Controversy', where he showed some examples of complexity added to the language with BGGA closures. He also showed the syntax proposals of CICE and ARM. This talk went down well with the audience, however it should be born in mind that Neal had no opportunity to respond directly to the points made.

Finally, in another last minute change, I gave a 15 minute quick talk on FCM+JCA closures. Despite being at lunchtime on the Friday, there was still a reasonable group in the audience, and the feedback I received personally was good.

There was also a whiteboard discussing closures, with a vote - No closures vs CICE vs BGGA. The vote was about 2 to 1 in favour of BGGA before Josh's talk, while it was about 50/50 afterwards. This suggests to me that opinions on the topic are not fixed and can be swayed by the last good presentation people see.

Looking forwards, the hope is to have all three proposals (CICE, BGGA and FCM) discussed on JavaPosse.

Summary

Javapolis really is a great conference. It is extremely well run, and has a really good feel. Stefan has said that he intends to keep it at Metropolis, Antwerp next year, so since it sold out this year, I suggest you get your tickets early next year!

Monday, 10 December 2007

Javapolis 2007

I am just getting ready to go to Javapolis 2007 :-)

This year I am presenting a main conference talk and a BOF:

In addition, there will be whiteboards again this year, and I will be attempting to keep the discussions on them moving forwards.

Hope to see you there!

Saturday, 1 December 2007

Is the JCP broken?

Jean-Marie Dautelle has posted a question on Javalobby asking about the future of the JCP.

Java Community Process

This question leads into broader questions of governance in Java. Java is now opens source, yet we still have no real clarification on how that works or affects the JCP.

In particular, who is the 'architect' overseeing the development of Java, and its future direction? Is it a named person in Sun? A hidden magical process? Or hit and hope?

Some examples:

1) Language changes. I, and many others, have suggested numerous language changes for Java 7. But who decides which, if any, go forward? How do people influence the process? How do people register opposition to changes?

At present, the only approach seems to be to work for Sun in a senior position, or work in silicon valley. What I'm looking for is a direction, for example 'Java will add new language features, but only to fill in gaps in the current language', or 'Java will not add any new language features'.

Personally, I've argued strongly for language change. I believe that there are quite a lot of changes, many quite small, that would help 'complete' the language. But what I really need is that architectural statement that says whether there will be any language change. If not, then I can move on, perhaps to a different language.

2) Core library changes. What if I want to add a method to one of the core libraries, such as String, or Integer. How do I lobby for this to occur? Despite requests, it seems that this isn't worthy of a JSR. Yet, Google can propose a list of methods to add, and stand a big chance of succeeding.

Here, I'm arguing for an open JSR which examines all the core library utilities out there - Commons Lang/Collections/IO, Google Collections, Spring, Jodd, etc. and works out the commonality. Plus, anyone could propose their own ideas. But it needs organising, and it needs funding.

3) Modules. The shameless behaviour of Sun wrt OSGi shows the JCP in a very poor light. Anyone who believes that the JCP functions well should be examine this tragedy.

When JSR-277 started, the OSGi team were told that they were not to be invited onto the 14 person 'Expert Group' despite being probably the most experienced people on the planet in managing modules. Apparently, the expert group was full at 14. Today, it has 20 people. Pure politics.

So, instead of adopting a widely proven, top-notch module system suitable for everything from mobile phones to cars to enterprise, Sun has gone 'Not Invented Here'. It has desperately trying to reinvent OSGi and failed. The JCP now has 6 JSRs that could be solved simply by adopting OSGi. This is absurd.

Peter Kriens from OSGi has been repeatedly blogging about this, and how the weak requirements process at the JCP is a big issue.

And modules are the biggest change currently planned for Java 7, yet one that very few people are following or appreciating. Messing this one up will be a real blow to Java. Its like java.util.logging. Only far, far worse.

Issues

The issue is that JCP, and Sun's approach to Java, seems pretty broken at present.

1) Sun ignores the JCP legal agreements it signs (ie. wrt Apache Harmony). This destroys all trust.

2) The JCP still exists to rubber stamp proposals from its big company members. Basically, a company submits code they already have, and the 'Expert Group' really exists to peer review it.

3) Neither the JCP or Sun are giving any sense of direction to Java. There should be a clear sense of where Java is headed and what it is trying to be - a five to ten year vision.

4) The JCP is not eliminating duplicate JSRs (see OSGi). Worse, it is allowing the worse option to succeed.

5) Individuals have too small a say. They tend to provide the innovation and disruption that keeps the big companies honest.

Remedies

1) Sun must regain the trust of the community, and stop fracturing the Java landscape. It must offer Apache Harmony the correct testing kit.

2) I would suggest a slightly larger executive committee. This should guarantee 20%, or maybe 25%, of seats to individuals, yet still provide enough room for companies such as Spring Source to get in (who just missed out at the last ballot.

3) Sun needs to either lose its special status on the JCP, or have it properly acknowledged. The former would mean it having to face a ballot every 3 years. Given what Sun is to Java, this probably isn't realistic. So, I'd argue that the EC should be honest about Sun, and that it isn't counted in the members of the EC (ie. it is the chair). Thus, I'd suggest an EC of 1 chair (Sun), 15 other companies, and 5 individuals.

4) The EC needs to stand up to Sun, and force a few issues - notably licensing and OSGi.

5) The JCP needs to setup an architecture group that takes 3 months to create and publish a five to ten year vision for Java.

A real alternative

The real remedy may however be to kill the JCP. We may not need it anymore. Consider these words:

Modularization creates a market for solutions. It is no longer required to come up with a catch-all API in the JCP, which is too often designed under political pressure. For example, J2EE would never have been a success in a free market. It received its life because it was the “ordained” API for enterprise applications by virtue of its JCP origin. It was not until an external party like Interface21 showed with Spring that Enterprise applications can actually be lean and mean as well.

So, the real solution would be to redefine Java as a core kernel, and an OSGi module system, with a central repository of modules that could be downloaded on demand. Vendors could then group together ad hoc to build new modules for everything from date-time to JSF. The market would then decide the winners. Thus the JCP would simply be the guardian of the language syntax and core libraries, an actually manageable task!

Finally...

Finally, as co-spec lead of JSR-310, I have to say that money is a problem. Despite all of the above, I have given freely of my time to run a JSR (Date and Time) which the vast majority of developers appear to really want, but which Sun had no desire whatsoever to run. To achieve this I have taken a 40% pay cut for 6 months by working 3 days a week instead of 5.

So, my advice to anyone considering running a JSR is don't. If the JCP wants to encourage individuals to continue running JSRs, then perhaps it needs to start paying them to do so.

Tuesday, 27 November 2007

Java 7 - Extension methods

A recent document revealed some possible language changes that are being proposed. One of these is is extension methods.

Extension methods

Extension methods allow a user to 'add' a method to an interface or class that you don't control. The original document is linked with BGGA closures. And was followed up by Peter Ahe (broken link).

The classic example of the proposal is as follows (known as use-site extension methods):

// in the application
import static java.util.Collections.sort;
List list = ...
list.sort();

Thus, the sort method appears to be a method on the list, even though we haven't actually changed the List interface. When it is compiled, the extension method is removed, and replaced with the static method call.

Peter Ahe pointed out some flaws with this design. He also proposes declaration-site extension methods:

// in the JDK
public interface List {
  void sort() import static java.util.Collections.sort;
  ...
}

// in the application
list.sort();

In this case, it is easier for the user to find the implementation of the sort method, as it genuinely is a method on the interface, just one implemented elsewhere. Unfortunately, the downside is that now only the JDK authors can add methods to the List interface in this way.

My proposal is that the freedom given to the user by the use-site approach is far more desirable, but the possible side-effects are nasty. My preference would be to add a visible marker to show that this isn't a regular method:

// in the application
import static java.util.Collections.sort;
List list = ...
list.do.sort();

Note the '.do.'. This provides the user with the clear readability to show that something 'out of the ordinary' is going on.

In addition, I would argue that the static method should be marked to indicate that it can be used as an extension method, probably using an annotation:

// in the JDK
public class Collections {
  @ExtensionMethod
  public void sort(List list) { ...}
}

This provides the final piece of the puzzle, preventing innapropriate methods from being used as extension methods. While it does suffer from the same issue of pushing control back to the library author (the JDK), it avoids the issue of which methods get added to the List interface.

The point is that as many methods as appropriate can be tagged with @ExtensionMethod. There is no conceptual overhead in doing so, and it doesn't expand the conceptual weight or complexity of the ListInterface itself.

Summary

I've outlined an alternative proposal for extension methods, that tries to focus on freedom for users, without compromising readability or allowing confusing options.

Opinions welcome, as always :-)

Sunday, 11 November 2007

Kijaro - for Java language changes

I'd like to announce the creation of a new project - kijaro. Kijaro is designed as a place where ideas for changes to the Java language can be implemented.

Kijaro

The kijaro project has been setup following various discussions on blogs, mailing lists and email. Its aim is to provide a very open place for those interested in implementing a change to javac to gather and code.

Kijaro is similar in its scope to the Kitchen Sink Language. The KSL project has been open for quite a while, but has yet to see any new features. KSL also aims to have code reviews and experienced compiler writers involved, which can be seen as quite formal.

Kijaro aims to be lightweight in rules:

  • Documentation. Each new language feature must have some form of associated document, even if its just a blog. It doesn't have to be much, but should have an outline of why the feature is needed and the syntax implications.
  • Backwards compatibility. On svn TRUNK all existing Java code must compile.
  • Comments. Each change must have a comment so we can find it later, such as 'FCM-MREF'.

So far, the proponents of three language enhancements have expressed an interest in working at kijaro:

In fact, the FCM and Properties code is already checked in.

So, do you have a favourite language change that you want to see implemented in Java 7? Or, would you like to download and try out one of these changes? Then, please join us at kijaro! The more ideas we get implemented the better!

After all, real working prototypes tend to produce good feedback and really encourage the decision makers that any change is practical.