Monday 23 November 2009

More detail on Closures in JDK 7

This blog goes into a little more detail about the closures announcement at Devoxx and subsequent information that has become apparent.

Closures in JDK 7

At Devoxx 2009, Mark Reinhold from Sun announced that it was time for closures in Java. This was a big surprise to everyone, and there was a bit of a vacuum as to what was announced. More information is now available.

Firstly, Sun, via Mark, have chosen to accept the basic case for including closures in Java. By doing so, the debate now changes from whether to go ahead, to how to proceed. This is an important step.

Secondly, what did Mark consider to be in and what out? Well, he indicated that non-local returns and the control-invocation statement were out of scope. There was also some indication that access to non-final variables may be out of scope (this is mainly because it raises nasty multi-threading Java Memory Model issues with local variables).

In terms of what was included, Mark indicated that extension methods would be considered. This would be necessary to provide meaningful closure style APIs since the existing Java collection APIs cannot be altered. The result of what was in was being called "simple closures".

Finally, Mark offered up a possible syntax. The syntax he showed was very close to FCM:

  // function expressions
  #(int i, String s) {
    return i + str.length();

  // function expressions
  #(int i, String s) (i + str.length())
  // function types
  #int(int, String)

As such, its easy to say that Sun has "chosen the FCM proposal". However, with all language changes, we have to look at the semantics, not the syntax!

The other session at Devoxx was the late night BOF session. I didn't attend, however according to Reinier Zwitserloot, Mark indicated that Exception transparancy might not be essential to the final proposal.

According to Reinier, Mark also said "It is not an endorsement of FCM. He was very specific about that." I'd like to consider that in a little more detail (below).

The final twist was when a new proposal by Neal Gafter was launched which I'm referring to as the CFJ proposal. After some initial confusion, it became clear that this was written 2 weeks before Devoxx, and that Neal had discussed it primarily with James Gosling on the basis of it being a "compromise proposal". Neal has also stated that he didn't speak to Mark directly before Devoxx (and nor did I).

There are a number of elements that have come together in the various proposals:

  CICE BGGA 0.5 FCM 0.5 CFJ 0.6a Mark's announcement
(Devoxx summary)
Literals for reflection - - Yes - No
(No info)
Method references - - Yes Yes Worth investigating
(No info)
Closures assignable to
single method interface
Yes Yes Yes Yes Yes
(No info)
Closures independent of
single method interface
- Yes Yes Yes Yes
Access non-final local variables Yes Yes Yes Yes No
(Maybe not)
Keyword 'this' binds to
enclosing class
- Yes Yes Yes No info
(Probably yes)
Local 'return'
(binds to closure)
Yes - Yes Yes Yes
Non-local 'return'
(binds to enclosing method)
- Yes - - No
Alternative 'return' binding
- Yes - - No
Exception transparancy - Yes Yes Yes No info
(Maybe not)
Function types - Yes Yes Yes Yes
Library based Control Structure - Yes - - No
Proposed closure syntax Name(args) {block} {args => block;expr} #(args) {block} #(args) {block}
#(args) expr
#(args) {block}
#(args) (expr)

A table never captures the full detail of any proposal. However, I hope that I've demonstrated some key points.

Firstly, the table shows how the CFJ is worthy of a new name (from BGGA) as it treats the problem space differently by avoiding non-local returns and control invocation. Neal Gafter is also the only author of the CFJ proposal, unlike BGGA.

Secondly, it should be clear that at the high level, the CFJ and FCM proposals are similar. But in many respects, this is also a result of what would naturally occur when the non-local returns and control invocation is removed from BGGA.

Finally, it should be clear that it is not certain that the CFJ proposal meets the aims that Mark announced at Devoxx ("simple closures"). There simply isn't enough hard information to make that judgement.

One question I've seen on a few tweets and blog posts is whether Mark and Sun picked the BGGA or FCM proposal. Well, given what Mark said in the BOF (via Reinier), the answer is that neither was "picked" and that he'd like to see a new form of "simple closures" created. However, it should also be clear that the choices announced at Devoxx were certainly closer to the FCM proposal than any other proposal widely circulated at the time of the announcement.

At this point, it is critical to note that Neal Gafter adds a huge amount of technical detail to each proposal he is involved with - both BGGA and CFJ. The FCM proposal, while considerably more detailed than CICE, was never at the level necessary for full usage in the JDK. As such, I welcome the CFJ proposal as taking key points from FCM and applying the rigour from BGGA.

As of now, there has been relatively little debate on extension methods.


The key point now is to focus on how to implement closures within the overall scope laid out. This should allow the discussion to move forward considerably, and hopefully with less acrimony.

Friday 20 November 2009

Why JSR-310 isn't Joda-Time

One question that has been repeatedly asked is why JSR-310 wasn't simply the same as Joda-Time. I hope to expain some reasons here.

Joda-Time as JSR-310?

At its heart, JSR-310 is an effort to add a quality date and time library to the JDK. So, since most people consider Joda-Time to be a quality library, why not include it directly in the JDK?

Well, there is one key reason - Joda-Time has design flaws.

Now before everyone panics and abuses that line as a tweet, I need to say that Joda-Time is by far the best option curently available, and that most users won't appreciate the design flaws. But, I do want to document them, so the basis for the changes in JSR-310 is clear.

1) Human/Machine timelines

One element of clarity is a better understanding of the distinction between the two principle views of the timeline - Human and Machine.

Machines have one view - a single, ever increasing number. In Java we set zero as 1970-01-01T00:00Z and count in milliseconds from there.

Humans have a totally different view of time. We have multiple calendar systems (one primary, many others), which divide the timeline into years, months, days, hours, minutes and seconds. In addition, humans have time zones which cause the values to vary around the globe, and for there to be gaps and overlaps in the human timeline as DST starts and ends.

Much of the time, a conversion between the two timeline views is possible with a time zone, however in a DST gap or overlap, things are much less clear.

Joda-Time defines two key interfaces - ReadableInstant and ReadablePartial. Both the Instant class (simple instant in time) and the DateTime class (human view of an instant in time) are implementations of ReadableInstant. This is wrong.

DateTime is a human-timeline view of the world, not a machine-timeline view. As such, DateTime is much better thought of, and designed as, a LocalDateTime and a timezone rather than the projection of the machine timeline onto the human timeline. Thus, DateTime should not implement ReadableInstant.

2) Pluggable chronology

What is the range of values returned by this method in Joda-Time?:
int month = dateTime.getMonthOfYear();
The answer is not 1 to 12, but could be 1 to 13! (yet January is still 1 and December is 12)

The answer to this puzzler is down to pluggable chronologies. Each date/time class in Joda-Time has a pluggable chronology. This defines the calendar system that is in use. But most users of the API never check to see if the chronology is the standard ISO/ chronology before calling getMonthOfYear(). Yet, the Coptic chronology has 13 months in a year, and thus can return a range of 1 to 13.

A better solution would be to keep the date/time classes restricted to a single calendar system. That way, the result from each method call is clear, and not dependent on any other state in the class, like the chronology.

3) Nulls

Joda-Time accepts null as a valid value in most of its methods. For date/times it means 1970-01-01T00:00Z. For durations it means zero. For peiods it means zero.

This approach causes random bugs if your code happens to provide a null to Joda-Time that you hadn't originally planned for. Instead of throwing an error, Joda-Time continues, and the resulting date/time is going to be different from what you want.

4) Internal implementation

Certain aspects of the internal implementation are complex, and the result of having pluggable chronologies and a misunderstanding of the machine/human divide in the timeline. Changing this is a big change to the code.

One particular area of trouble is managing DST overlaps. The behaviour in Joda-Time of these isn't that well defined.


Joda-Time isn't broken!

It does the job it was designed for, and does it much better than the JDK. And it is widely used and without too many major issues. However, after a few years, it is now clear where it could be designed better.

I took the decision that I didn't want to add an API to the JDK that had known design flaws. And the changes required weren't just minor. As a result, JSR-310 started from scratch, but with an API 'inspired by Joda-Time'.

I hope that explains the thought process behind the creation of a new API in JSR-310.

Thursday 19 November 2009

Closures in JDK 7

So, on Wednesday, Mark Reinhold from Sun announced that it was time for closures in Java. This came as a surprise to everyone, but what was announced?

Closures in JDK 7

Mark took the audience through the new features of JDK 7 in his presentation at Devoxx. As part of this he showed the Fork Join framework.

Part of this framework features a set of interfaces to support functional style predicates and transforms. But, in order to implement this in Java required 80 interfaces, all generified, and that only covered 4 of the primitive types! Basically, Mark, and others, had clearly come to the conclusion that this code was important to Java, but that the implementation with single-method interfaces, generics and inner classes was too horrible to stomach for the JDK.

Thus, 'its time to add closures to Java'.

Mark announced that Sun would investigate, and intended to commit, closures to OpenJDK for JDK 7. While there will be no JSR yet, it isn't unreasonable to expect that there will be an open aspect to discussions.

Mark noted that the work in the closures discussion, and particularly the detail in the BGGA proposal, had allowed the options to be properly explored. He emphasised how this had resulted in the decision to reject key aspects of BGGA.

JDK 7 closures will not have control-invocation statements as a goal, nor will it have non-local returns. He also indicated that access to non-final variables was unlikely. Beyond this, there wasn't much detail on semantics, nor do I believe that there has been much consideration of semantics yet.

On syntax, Mark wrote up some strawman syntaxes. These follow the FCM syntax style:

  // function expressions
  #(int i, String s) {
    return i + s.length();

  // function expressions
  #(int i, String s) (i + s.length())
  // function types
  #int(int, String)

Mark also argued that a form of extension methods would be added to allow closures to be used with existing libraries like the collections API.

Although I must emphasise that everything announced was a proposal and NOT based on any specific proposal (BGGA, FCM or CICE). However, the syntax and semantics do follow the FCM proposal quite closely.

In addition, Neal Gafter has produced an initial formal writeup of what was announced derived from BGGA. Based on my initial reading, I believe that Neal's document represents a good place to start from. I also hereby propose that we refer to Neal's document as the CFJ proposal (Closures for Java), as BGGA rather implies control invocation.

So, lets hope the process for closures provides for feedback, and we get the best closures in the style most suitable for Java in JDK 7.

Tuesday 17 November 2009

Joda-Money 0.5

I've just released v0.5 of Joda-Money. It is a simple money and currency package along the lines of Joda-Time.

Joda-Money 0.5

Joda-Money is a project to provide a small, focussed monetary library for Java, similar to Joda-Time. I released v0.5 this morning.

Website, Javadoc, Download, Maven 2 repo

The code is all tested and there are no known bugs. But I wouldn't suggest using it in production code quite yet. I do want lots of feedback and testing though! So, all feedback is welcomed, here, the forum or the list.

The main API is as I originally intended. There is a Money and BigMoney class, the former decorates the latter to restrict the number of decimal places to the currency scale.

There is also CurrencyUnit and a provided data file of currency data. Finally, there is printing (but not parsing yet).

The goal of the library remains just these basic classes. Financial calculations (for whatever domain) or classes storing specific concepts like gross/net/tax are out of scope for the library.

This release is part of the 'release early' approach. Please provide as much feedback as you can!