Monday, 8 February 2010

New job - impact on JSR-310

This is a quick blog to outline my upcoming job change and how it affects JSR-310.

For many years I've worked for SITA, global leader in air transport communications and IT solutions. But the time has come to move on, so from the 1st of March I'm starting a new job at a London startup, Open Gamma.

So, what can I tell you about Open Gamma? Well not too much just yet as its only just coming out of stealth mode. I can say they're lead by Kirk Wylie, they're building technology for the financial industry, and I'm excited about their big idea! Oh, and they're hiring (London only).

And how does this affect JSR-310?

Well, OpenGamma will be actively supporting my work on JSR-310 in work time! Clearly this will have a big impact on development pace, and we may yet make JDK 7 (but of course thats up to the SunOracle).

In the meantime, watch out for the Early Draft Review of JSR-310 where I'll need maximum feedback!

Monday, 23 November 2009

More detail on Closures in JDK 7

This blog goes into a little more detail about the closures announcement at Devoxx and subsequent information that has become apparent.

Closures in JDK 7

At Devoxx 2009, Mark Reinhold from Sun announced that it was time for closures in Java. This was a big surprise to everyone, and there was a bit of a vacuum as to what was announced. More information is now available.

Firstly, Sun, via Mark, have chosen to accept the basic case for including closures in Java. By doing so, the debate now changes from whether to go ahead, to how to proceed. This is an important step.

Secondly, what did Mark consider to be in and what out? Well, he indicated that non-local returns and the control-invocation statement were out of scope. There was also some indication that access to non-final variables may be out of scope (this is mainly because it raises nasty multi-threading Java Memory Model issues with local variables).

In terms of what was included, Mark indicated that extension methods would be considered. This would be necessary to provide meaningful closure style APIs since the existing Java collection APIs cannot be altered. The result of what was in was being called "simple closures".

Finally, Mark offered up a possible syntax. The syntax he showed was very close to FCM:

  // function expressions
  #(int i, String s) {
    System.println.out(s);
    return i + str.length();
  }

  // function expressions
  #(int i, String s) (i + str.length())
  
  // function types
  #int(int, String)

As such, its easy to say that Sun has "chosen the FCM proposal". However, with all language changes, we have to look at the semantics, not the syntax!

The other session at Devoxx was the late night BOF session. I didn't attend, however according to Reinier Zwitserloot, Mark indicated that Exception transparancy might not be essential to the final proposal.

According to Reinier, Mark also said "It is not an endorsement of FCM. He was very specific about that." I'd like to consider that in a little more detail (below).

The final twist was when a new proposal by Neal Gafter was launched which I'm referring to as the CFJ proposal. After some initial confusion, it became clear that this was written 2 weeks before Devoxx, and that Neal had discussed it primarily with James Gosling on the basis of it being a "compromise proposal". Neal has also stated that he didn't speak to Mark directly before Devoxx (and nor did I).

There are a number of elements that have come together in the various proposals:

  CICE BGGA 0.5 FCM 0.5 CFJ 0.6a Mark's announcement
(Devoxx summary)
Literals for reflection - - Yes - No
(No info)
Method references - - Yes Yes Worth investigating
(No info)
Closures assignable to
single method interface
Yes Yes Yes Yes Yes
(No info)
Closures independent of
single method interface
- Yes Yes Yes Yes
Access non-final local variables Yes Yes Yes Yes No
(Maybe not)
Keyword 'this' binds to
enclosing class
- Yes Yes Yes No info
(Probably yes)
Local 'return'
(binds to closure)
Yes - Yes Yes Yes
Non-local 'return'
(binds to enclosing method)
- Yes - - No
Alternative 'return' binding
(Last-line-no-semicolon)
- Yes - - No
Exception transparancy - Yes Yes Yes No info
(Maybe not)
Function types - Yes Yes Yes Yes
Library based Control Structure - Yes - - No
Proposed closure syntax Name(args) {block} {args => block;expr} #(args) {block} #(args) {block}
#(args) expr
#(args) {block}
#(args) (expr)

A table never captures the full detail of any proposal. However, I hope that I've demonstrated some key points.

Firstly, the table shows how the CFJ is worthy of a new name (from BGGA) as it treats the problem space differently by avoiding non-local returns and control invocation. Neal Gafter is also the only author of the CFJ proposal, unlike BGGA.

Secondly, it should be clear that at the high level, the CFJ and FCM proposals are similar. But in many respects, this is also a result of what would naturally occur when the non-local returns and control invocation is removed from BGGA.

Finally, it should be clear that it is not certain that the CFJ proposal meets the aims that Mark announced at Devoxx ("simple closures"). There simply isn't enough hard information to make that judgement.

One question I've seen on a few tweets and blog posts is whether Mark and Sun picked the BGGA or FCM proposal. Well, given what Mark said in the BOF (via Reinier), the answer is that neither was "picked" and that he'd like to see a new form of "simple closures" created. However, it should also be clear that the choices announced at Devoxx were certainly closer to the FCM proposal than any other proposal widely circulated at the time of the announcement.

At this point, it is critical to note that Neal Gafter adds a huge amount of technical detail to each proposal he is involved with - both BGGA and CFJ. The FCM proposal, while considerably more detailed than CICE, was never at the level necessary for full usage in the JDK. As such, I welcome the CFJ proposal as taking key points from FCM and applying the rigour from BGGA.

As of now, there has been relatively little debate on extension methods.

Summary

The key point now is to focus on how to implement closures within the overall scope laid out. This should allow the discussion to move forward considerably, and hopefully with less acrimony.

Friday, 20 November 2009

Why JSR-310 isn't Joda-Time

One question that has been repeatedly asked is why JSR-310 wasn't simply the same as Joda-Time. I hope to expain some reasons here.

Joda-Time as JSR-310?

At its heart, JSR-310 is an effort to add a quality date and time library to the JDK. So, since most people consider Joda-Time to be a quality library, why not include it directly in the JDK?

Well, there is one key reason - Joda-Time has design flaws.

Now before everyone panics and abuses that line as a tweet, I need to say that Joda-Time is by far the best option curently available, and that most users won't appreciate the design flaws. But, I do want to document them, so the basis for the changes in JSR-310 is clear.

1) Human/Machine timelines

One element of clarity is a better understanding of the distinction between the two principle views of the timeline - Human and Machine.

Machines have one view - a single, ever increasing number. In Java we set zero as 1970-01-01T00:00Z and count in milliseconds from there.

Humans have a totally different view of time. We have multiple calendar systems (one primary, many others), which divide the timeline into years, months, days, hours, minutes and seconds. In addition, humans have time zones which cause the values to vary around the globe, and for there to be gaps and overlaps in the human timeline as DST starts and ends.

Much of the time, a conversion between the two timeline views is possible with a time zone, however in a DST gap or overlap, things are much less clear.

Joda-Time defines two key interfaces - ReadableInstant and ReadablePartial. Both the Instant class (simple instant in time) and the DateTime class (human view of an instant in time) are implementations of ReadableInstant. This is wrong.

DateTime is a human-timeline view of the world, not a machine-timeline view. As such, DateTime is much better thought of, and designed as, a LocalDateTime and a timezone rather than the projection of the machine timeline onto the human timeline. Thus, DateTime should not implement ReadableInstant.

2) Pluggable chronology

What is the range of values returned by this method in Joda-Time?:
int month = dateTime.getMonthOfYear();
The answer is not 1 to 12, but could be 1 to 13! (yet January is still 1 and December is 12)

The answer to this puzzler is down to pluggable chronologies. Each date/time class in Joda-Time has a pluggable chronology. This defines the calendar system that is in use. But most users of the API never check to see if the chronology is the standard ISO/ chronology before calling getMonthOfYear(). Yet, the Coptic chronology has 13 months in a year, and thus can return a range of 1 to 13.

A better solution would be to keep the date/time classes restricted to a single calendar system. That way, the result from each method call is clear, and not dependent on any other state in the class, like the chronology.

3) Nulls

Joda-Time accepts null as a valid value in most of its methods. For date/times it means 1970-01-01T00:00Z. For durations it means zero. For peiods it means zero.

This approach causes random bugs if your code happens to provide a null to Joda-Time that you hadn't originally planned for. Instead of throwing an error, Joda-Time continues, and the resulting date/time is going to be different from what you want.

4) Internal implementation

Certain aspects of the internal implementation are complex, and the result of having pluggable chronologies and a misunderstanding of the machine/human divide in the timeline. Changing this is a big change to the code.

One particular area of trouble is managing DST overlaps. The behaviour in Joda-Time of these isn't that well defined.

Summary

Joda-Time isn't broken!

It does the job it was designed for, and does it much better than the JDK. And it is widely used and without too many major issues. However, after a few years, it is now clear where it could be designed better.

I took the decision that I didn't want to add an API to the JDK that had known design flaws. And the changes required weren't just minor. As a result, JSR-310 started from scratch, but with an API 'inspired by Joda-Time'.

I hope that explains the thought process behind the creation of a new API in JSR-310.

Thursday, 19 November 2009

Closures in JDK 7

So, on Wednesday, Mark Reinhold from Sun announced that it was time for closures in Java. This came as a surprise to everyone, but what was announced?

Closures in JDK 7

Mark took the audience through the new features of JDK 7 in his presentation at Devoxx. As part of this he showed the Fork Join framework.

Part of this framework features a set of interfaces to support functional style predicates and transforms. But, in order to implement this in Java required 80 interfaces, all generified, and that only covered 4 of the primitive types! Basically, Mark, and others, had clearly come to the conclusion that this code was important to Java, but that the implementation with single-method interfaces, generics and inner classes was too horrible to stomach for the JDK.

Thus, 'its time to add closures to Java'.

Mark announced that Sun would investigate, and intended to commit, closures to OpenJDK for JDK 7. While there will be no JSR yet, it isn't unreasonable to expect that there will be an open aspect to discussions.

Mark noted that the work in the closures discussion, and particularly the detail in the BGGA proposal, had allowed the options to be properly explored. He emphasised how this had resulted in the decision to reject key aspects of BGGA.

JDK 7 closures will not have control-invocation statements as a goal, nor will it have non-local returns. He also indicated that access to non-final variables was unlikely. Beyond this, there wasn't much detail on semantics, nor do I believe that there has been much consideration of semantics yet.

On syntax, Mark wrote up some strawman syntaxes. These follow the FCM syntax style:

  // function expressions
  #(int i, String s) {
    System.println.out(s);
    return i + s.length();
  }

  // function expressions
  #(int i, String s) (i + s.length())
  
  // function types
  #int(int, String)

Mark also argued that a form of extension methods would be added to allow closures to be used with existing libraries like the collections API.

Although I must emphasise that everything announced was a proposal and NOT based on any specific proposal (BGGA, FCM or CICE). However, the syntax and semantics do follow the FCM proposal quite closely.

In addition, Neal Gafter has produced an initial formal writeup of what was announced derived from BGGA. Based on my initial reading, I believe that Neal's document represents a good place to start from. I also hereby propose that we refer to Neal's document as the CFJ proposal (Closures for Java), as BGGA rather implies control invocation.

So, lets hope the process for closures provides for feedback, and we get the best closures in the style most suitable for Java in JDK 7.

Tuesday, 17 November 2009

Joda-Money 0.5

I've just released v0.5 of Joda-Money. It is a simple money and currency package along the lines of Joda-Time.

Joda-Money 0.5

Joda-Money is a project to provide a small, focussed monetary library for Java, similar to Joda-Time. I released v0.5 this morning.

Website, Javadoc, Download, Maven 2 repo

The code is all tested and there are no known bugs. But I wouldn't suggest using it in production code quite yet. I do want lots of feedback and testing though! So, all feedback is welcomed, here, the forum or the list.

The main API is as I originally intended. There is a Money and BigMoney class, the former decorates the latter to restrict the number of decimal places to the currency scale.

There is also CurrencyUnit and a provided data file of currency data. Finally, there is printing (but not parsing yet).

The goal of the library remains just these basic classes. Financial calculations (for whatever domain) or classes storing specific concepts like gross/net/tax are out of scope for the library.

This release is part of the 'release early' approach. Please provide as much feedback as you can!

Saturday, 3 October 2009

JSRs submitted over time

The JCP is the body that looks after standards in Java, and a JSR is the individual request to create a standard. But how has the volume of JSRs changed over time?

JSRs over time

In tackling this question, I decided to focus on the simple statistic of how many JSRs were submitted per quarter over time. This does not address JSRs that were never finished, or how long the JSR took to complete, or whether it is still active and in maintenance mode.

The data as to when a JSR was submitted is publicly recorded on the JSR page, for example JSR-311 was submitted for voting on 13th February 2007.

So, without any further ado, here is the main graph of JSRs submitted over time. The red line is the total of all JSRs (Java SE, EE and ME) while the green line is only SE and EE:

Hopefully, the trend is clear - the number of JSRs submitted has been falling over time. To be clear - only 5 JSRs were submitted in the whole of 2008, and only 4 have been submitted so far this year (roughly 1 JSR per quarter).

The question is whether this is a worrying trend, or just normal for a maturing language. I suspect its a combination of factors.

In part, Java is now a platform where most innovation occurs outside the JCP. This is a good thing, and shows a healthy ecosystem that doesn't need the 'blessing' of an official standard to make a technology or product useful. Examples of this are Spring and OSGi - both were widely used without a JSR (although ironically both now have some connection to the JCP).

Another factor is that a lot of the Java EE work, and possibly others, appears to be going on as maintenance revisions to an existing JSR. This doesn't show up on this graph, as the no new JSR is created. This is fine, although it may suggest an unwillingness to go through the process of starting a new JSR.

One more point worth noting is the timeline. JavaOne 2006 was when the Open Sourcing of Java was announced, and Sun developers were taken off active development to begin the very long process of legally reviewing the entire Java codebase. This was then followed by the (mis)adventure of JavaFX, which drew even more developers away from core Java and new developments. There may be some signs of this in the graph, where the steady state of 2004-2006 declines at the end of 2006.

Finally of course, there is no doubt about the absence of Java SE JSRs for Java SE 7 (language changes and API changes). I've written before about the politics behind this.

Certainly I see the JCP as being at a crossroads. The Oracle acquisition is key here. Oracle could choose to reinvigorate the JCP, with industry partners, ideally separating the JCP from Oracle itself, as IBM did with Eclipse. However, given the money Oracle is spending, I suspect that won't happen, despite their previous sentiments.

Unfortunately, continuing the status quo doesn't wash either. At present, the JCP is deadlocked on how to proceed with the Java SE 7 specification. As a result, JDK 7, Project Jigsaw and Project Coin are proceeding outside the JCP. Plus of course, JavaFX isn't related to the JCP at all.

Summary

The JCP has considerably fewer JSRs being submitted now than the 2000 to 2003 period. This doesn't quite mean that the JCP is dead or no longer has value, but its another data point as to how Java is maturing and stabilising. Or perhaps less kindly - fossilising.

Feedback welcome.

Thursday, 10 September 2009

JDK 7 - Method suggestions

Joe Darcy has opened up a call for methods to add to the core JDK 7 lang/util package. The idea is to add methods that are "commonly-written utility methods".

JDK 7 utility methods

There is lots of prior art in the field of general utility method libraries for Java. Perhaps the best known is Apache Commons Lang. Lets choose some methods (other than string utilities) that might be appropriate for the JDK from ObjectUtils.

1) Null-safe equals check.
boolean Objects.equals(Object object1, Object object2)
Returns true if both are null, or both are non-null and equal.

2) Null-safe hash code.
int Objects.hashCode(Object object)
Returns the hash code of the object, or zero if the object is null.

3) Null-safe toString.
String Objects.toString(Object object)
Returns the toString of the object, or "" if the object is null.

4) Null-safe toString with specified default.
String Objects.toString(Object object, String defaultStr)
Returns the toString of the object, or defaultStr if the object is null.

5) Null-safe object defaulting.
T Objects.defaultNull(T object, T defaultValue)
Returns the object, or defaultValue if the object is null.
(Of course, the Elvis operator is a much clearer approach for this common case...)

6) Get maximum and minimum.
T Objects.max(T comparable1, T comparable2)
T Objects.min(T comparable1, T comparable2)
Returns the maximum/minimum object, returning the non-null object if either is null, or null if both are null.
Generics allow the return type to check the input is also a Comparable.

These would appear to be the obvious methods to add to an Objects utility class.

The following are missing methods on Integer/Long:

7) Compare two primitives.
int Byte.compare(byte value1, byte value2)
int Short.compare(short value1, short value2)
int Integer.compare(int value1, int value2)
int Long.compare(long value1, long value2)
Safely returns the comparison (-1/0/1) indicator for two primitives. (These methods already exist on Float/Double).

The methods from SystemUtils can be very useful in reducing arbitrary strings in code:

8) Get common environment variables.
File System.getJavaIoTempDir()
File System.getJavaHomeDir()
File System.getUserHomeDir()
File System.getUserDir()
Returns the java environment variables. There are lots more in SystemUtils that could be added.

Some more null handling methods for primitives - again, Elvis would be a better solution:

9) Convert wrapper to primitive avoiding NPE.
boolean Boolean.booleanValue(Boolean obj, boolean defaultValue)
char Character.charValue(Character obj, char defaultValue)
byte Byte.byteValue(Byte obj)
byte Byte.byteValue(Byte obj, byte defaultValue)
short Short.shortValue(Short obj)
short Short.shortValue(Short obj, short defaultValue)
int Integer.intValue(Integer obj)
int Integer.intValue(Integer obj, int defaultValue)
long Long.longValue(Long obj)
long Long.longValue(Long obj, long defaultValue)
float Float.floatValue(Float obj)
float Float.floatValue(Float obj, float defaultValue)
double Double.doubleValue(Double obj)
double Double.doubleValue(Double obj, double defaultValue)
Safe ways to convert a wrapper to a primitive.
The numeric ones return zero if the default argument isn't specified.
There is also a case for methods to convert arrays of wrappers to arrays of primitives.

These should probably be on a Booleans utility class as per BooleanUtils.

10) Boolean methods for clearer code.
boolean Booleans.isTrue(Boolean booleanObj)
boolean Booleans.isFalse(Boolean booleanObj)
Return true as per the method name, false if null.
boolean Booleans.isNotTrue(Boolean booleanObj)
boolean Booleans.isNotFalse(Boolean booleanObj)
Return true as per the method name, true if null.

11) Negate handling null.
Boolean Booleans.negate(Boolean booleanObj)
TRUE returns FALSE, FALSE returns TRUE, null returns null.

11) Boolean arithmetic.
boolean Booleans.and(boolean[] array)
boolean Booleans.or(boolean[] array)
boolean Booleans.xor(boolean[] array)
boolean Booleans.and(Boolean[] array)
boolean Booleans.or(Boolean[] array)
boolean Booleans.xor(Boolean[] array)
Performs the stated binary maths.

12) Character comparison.
boolean Character.equalsIgnoreCase(char ch1, char ch2)
Compares two characters ignoring case.

These should be constants for empty arrays on common classes:

13) Empty array constants.
Boolean[] Boolean.EMPTY_ARRAY
boolean[] Boolean.EMPTY_PRIMITIVE_ARRAY
Character[] Character.EMPTY_ARRAY
char[] Character.EMPTY_PRIMITIVE_ARRAY
Byte[] Byte.EMPTY_ARRAY
byte[] Byte.EMPTY_PRIMITIVE_ARRAY
Short[] Short.EMPTY_ARRAY
short[] Short.EMPTY_PRIMITIVE_ARRAY
Integer[] Integer.EMPTY_ARRAY
int[] Integer.EMPTY_PRIMITIVE_ARRAY
Long[] Long.EMPTY_ARRAY
long[] Long.EMPTY_PRIMITIVE_ARRAY
Float[] Float.EMPTY_ARRAY
float[] Float.EMPTY_PRIMITIVE_ARRAY
Double[] Double.EMPTY_ARRAY
double[] Double.EMPTY_PRIMITIVE_ARRAY
String[] String.EMPTY_ARRAY
Class[] Class.EMPTY_ARRAY
Object[] Objects.EMPTY_OBJECT_ARRAY
Return true as per the method name, true if null.

Or a better solution might be a method on Class:
T[] Class.emptyArray();
This allows code like:
Boolean.class.emptyArray();

14) Array size.
boolean Arrays.isEmpty(Object[] array)
int Arrays.size(Object[] array)
Where a null array is empty/size-zero.

15) Collection size.
boolean Collections.isEmpty(Collection coll)
int Collections.size(Collection coll)
boolean Collections.isEmpty(Map map)
int Collections.size(Map map)
Where a null collection is empty/size-zero.

Or, even better, add a new interface Sized with a single method to get the size. This would be retrofitted to Collection, Map, String and arrays (of course, technically, thats a language change...).

And some Class utilities for less NPEs when outputting the class name in debugging:

16) NPE safe way to get class name.
String Class.getName(Class cls)
String Class.getSimpleName(Class cls)
String Class.getPackageName(Class cls)
Returns the class name, or null if the input is null.
Again, the null-safe operators would avoid this kind of specific method.

Locale could do with some love, as per LocaleUtils:

17) Parse a locale string.
Locale parse(String localeStr)
Parses the locale string to a locale.

16) Country/Language lists.
List Locale.countriesByLanguage(String langaugeStr)
List Locale.languagesByCountry(String countryStr)
Extracts just countries or languages.

A better solution would be two new classes Country and Language. These would be useful in other places in the JDK, where a locale is used and it should be a country or language.

And the big one that I've missed - safe maths, as per Joda-Time FieldUtils:

18) Add/subtract/multiply/cast safely.
int Math.safeToInt(long value)
int Math.safeNegate(int value)
long Math.safeNegate(long value)
int Math.safeAdd(int value1, int value2)
long Math.safeAdd(long value1, int value2)
long Math.safeAdd(long value1, long value2)
int Math.safeSubtract(int value1, int value2)
long Math.safeSubtract(long value1, int value2)
long Math.safeSubtract(long value1, long value2)
int Math.safeMultiply(int value1, int value2)
long Math.safeMultiply(long value1, int value2)
long Math.safeMultiply(long value1, long value2)
These perform the specified mathematical operation, but throw ArithmeticException, rather than failing quietly, if overflow occurs.

I should also note that there are a whole heap of BigDecimal, BigInteger and String utilities that could be added.

Finally, of course, there are lots of other utility class libraries apart from Commons-Lang. Its simply that Commons-Lang provides a good point to start the discussion.

Feel free to (a) complain about my choices, and (b) suggest your own ideas. Remember to focus on non-string core classes for now.