No more Java 7

"No more Java 7". Yes, I know its a strong statement, but I don't mean it in the way you probably think.

Update: I've followed up this blog with a whole series of articles:
* a description of how IP works in the JCP (pretty picture!)
* a deep dive into the behind-the-scenes world of the JCP to provide a clearer explanation of the lack of a Java SE 7 specification and how this affects the whole Java industry, not just Apache and Sun
* more info from the ASF board meeting minutes
* an exploration of how the JCP doesn't really exist
* a take on the Oracle-Sun deal

What is Java?

Well, the term 'Java' is a complex and overloaded one, including:

• the stock symbol of Sun Microsystems
• the trademark of Sun Microsystems
• the term for a raft of technologies based around the JVM
• a platform (Java SE) based around the Java programming language (a programming language)
• the specification for the Java SE platform (the specification for the programming language)

This blog post is about the last of these - the Java SE platform specification.

Back in 1997 Sun talked about making the Java programming language into a formal open standard. At that time, the ECMA was the standards body approached. However, Sun withdrew from the standards process and created the Java Community Process instead. Everyone breathed a little easier, as Java now appeared to be an effectively open standard.

The job of the JCP is to "... develop and revise the Java technology specifications, reference implementations, and test suites, the Java Community Process (JCP) program has fostered the evolution of the Java platform in cooperation with the international Java developer community."

Over the years, and with a few hiccups, the process hasn't gone too badly. There are multiple implementations of many of the JSRs, creating a healthy eco-system of competition and improvement. That is after all a key goal of any standard - to define what is necessary to produce a conforming implementation.

The JCP is, in fact, an unusual standards body in that it goes further than most to ensure that every specification must be tested before it is officially allowed to be released as an implementation of the specification. Each JSR (the individual standard) must provide an appropriate specification document, reference implementation and, crucially, a testing kit.

One group that has implemented many JSR specification is Apache. From Tomcat to Geronimo to JMS to the whole of Java EE. (And back in 2002, Apache had to fight to allow any Open Source implementations of JSR specifications.)

So, what has this to do with the end of Java SE 7? Well, as I said, I'm specifically referring to Java the specification. Now many people don't realise this, but as well as the JSRs for Servlets, EJBs and JMS, there are also JSRs for the whole of Java EE and for the whole of Java SE.

The relevant specifications are JSR-176 for Java SE 5 and JSR-270 for Java SE 6. These specifications are no different to any other JSR. The specification documents exist (mostly Javadoc). The testing kit exists. As does the reference implementation (the Sun JDK).

So, is it an open specification and open standard, as promised back in 1998?

Well, the Java SE specification has been implemented by other companies - IBM has a JDK amongst others. However, IBM pay money to Sun for the rights to do this. The question is whether Apache can implement it, just like they implement 25 other JSRs.

Apache's implementation of the Java SE 5 JSR specification is Apache Harmony. However, when Apache came to obtain the testing kit for the specification a whole political game started. Instead of Sun offering the testing kit as normal for each of the other 25 JSRs, they offered a testing kit where the results of the tested code would not be Open Source

Obviously, Apache couldn't accept this limitation, which is against the legal agreement signed between Sun and Apache. Apache complained 2 years ago, but has yet to receive an acceptable response. And no, there is no way a charity not-for-profit like Apache is going to sue a multi-national corporation - who do you think would get the better lawyers?

The key point is that Sun's tactics here were quite deliberate. They knew exactly what they were doing by offering a testing kit with a restrictive license. They wanted to ensure that Apache Harmony couldn't be certified as complete. Sun was going out of its way to ensure that there was no competition to its own JDK.

In the meantime, Sun played the OpenJDK card. It announced the release of the JDK under the GPL license.

Lots in the community (notably those from a GPL background) suddenly became best friends with Sun. Some joined Sun, some now work closely with Sun. Many of these people now believe that there is no danger around Java, and that Richard Stallman's Java trap is over. I think this shows a gross lack of perspective - the code may now be GPL open source, but the specification is now no longer open. Which is more important?

If you can't freely implement a specification*, then I'd argue its not worth the paper its written on

* There are two ways you can implement the Java specification - (1) pay Sun money, or (2) ensure your code is "mostly derived from the OpenJDK".

What about Java SE 7? Everyone isn't noticing the obvious

I started with a contentious statement - that there may not be a Java SE 7 (open specification). So, lets look at the evidence.

Blogs, blogs and more blogs - Every blog from a key Sun employee in recent times has referred to JDK 7, not Java 7. Read them carefully!!! Still think there is guaranteed to be a Java 7?

To emphasise the point - JDK 7 is the implementation, Java SE 7 is the specification. But everyone at Sun is talking about the JDK (Open JDK), not the specification (Java) and this isn't an accident.

Project Jigsaw - Modularisation of the core of Java - No JSR planned. Funny, how such a major piece of infrastructure has managed to slide by without becoming an open specification...

To be fair, Mark Reinhold's recent blog is more positive - "The JDK 7 Project is creating a prototype of what might-or might not-wind up in the Java SE 7 Platform Specification. When the SE 7 Platform JSR is submitted then the features under development in JDK 7 will be proposed for inclusion therein, except for those that are VM-level or implementation-specific." (He uses the 'when the JSR is submitted phrase', but note the 'implementation-specific' phrase which conveniently covers Jigsaw at a minimum).

However, unless anyone is being particularly naive, it should be clear that if there is to be a Java SE 7 specification then it would be fully defined by what is in Open JDK, not by the JSR process. Do you think that Sun would remove things already in OpenJDK were the JCP to so decide? Er, no.

And look at the timeline. The full JDK contents are defined by September, yet there is no JSR for Java SE 7 yet. The first two milestones have already happened for goodness sake!!!

If the JSR happens it will be a rubber stamp. So why bother?

Next release - JDK 7, not Java 7

Yes, this is what I mean by the title of this blog. The evidence I see, and have shown here, is that the next release will be JDK 7, not Java 7. ie. there won't be an open specification.

Of course, since Sun own the Java trademark, Sun puts in all the investment and Sun has all the mindshare, the vast majority of developers won't notice this. In fact, since its now Open Source as the OpenJDK they might even think all is well. Sun's strategy to quosh all competition is working well isn't it?

What won't be realised is how 10 years of open standards have been lost. What won't be realised is how everyone is now locked into a new Java trap - a single vendor of anything that can truly be certified as Java. Its just a trap that Richard Stallman isn't interested in arguing against.

Personally, I would have thought that given all our experiences with the benefits of open Standards on the internet we would be demanding it of Java too - especially since it was promised back in 1997. We need to fight to retain Java as an open standard.

Summary

All the evidence I see is that Java SE is no longer an open standard and the next release will be JDK 7, not Java 7. That is something that should matter to all of us who care about the Java eco-system.

After all, if Sun can get away with making Java SE not an open standard, then why not Java EE, or Servlets or JMS? Its time to stand up and be counted and demand our open standard back!

Feel free to tell me why you think my analysis is wrong... I suspect you won't hold back...

For the record, I'm a member of the Apache Software Foundation, speaking personally here to point out something someone else really should have noticed...

Update 1: Please tag any follow up blogs or tweets with 'nojava7' so the feedback can be seen!

Update 2: For info, I've never contributed to either Apache Harmony or the Open JDK, although I have used the OpenJDK in Kijaro and work on JSR-310. As such, I'm pretty dispassionate about the quality or usefulness of Harmony/OpenJDK. I care more about the lack of an open specification for Java 7, which is what this blog is primarily about.

Update 3: I haven't seen any evidence that the engineering team at Sun don't want a Java SE specification. The evidence I see is that this is driven from a higher business/management level.

Update 4: I've clarified the initial 'meanings of Java' section to place the emphasis on the Java Platform, as explained here. (The licensing terms are well worth a read - particularly number 7)

Update 5: I've updated Java 7 to Java SE 7 in a few places to aid clarity. Most people probably understand that Java EE and Java ME are different, but it helped in a couple of places to be extra sure.

Update 6: Fixed "offering Sun the testing kit" to "Sun offering the testing kit"

Speaking on Fantom tonight in London

Quick note to say I'm speaking on the Fan Fantom programming language tonight in London. The evening will also include a brief intro to OSGi.

Its at the JavaWUG user group, hosted by Skills Matter, registration required to attend.

Renaming the Fan programming language?

I've recently started a poll to discuss the possibility of renaming the Fan programming language. The reason for a possible name change is simple - you can't google "Fan" and get hits about the language, and you never will (unlike Java).

If you're interested in this new language at all, please take the poll. We're also gathering alternative names, which you can leave in comments here if you prefer.

JDK 7 language changes - Everyone votes!

A third vote on language changes has just completed, this time it was online. I'll summarise the results from the three votes in this post.

Three votes

The three votes in question were:

• Devoxx
• JavaEdge
• Online

At Devoxx, the topics voted on were:

• Properties
• Multi-catch of exceptions
• Null handling
• List/Map access using []
• Extension methods
• Method pointers
• Multi line strings
• Infer generics on the RHS

At JavaEdge and Online the features voted on were:

• For-each loop over a Map
• Control over for-each loops (index/remove)
• List/Map access using []
• Infer generics on the RHS
• Multi-catch of exceptions
• String switch
• String interpolation
• Multi line strings
• Resource management
• Null handling

The later polls excluded properties as they are specifically excluded for Java 7. They also excluded extension methods and method pointers as these were the least popular options at Devoxx.

Here are the first preference votes for Devoxx, JavaEdge (ranked), JavaEdge, (not-ranked) and Online:

Considering the first to fourth preference votes (first at the bottom in dark green):

Totals

Unfortunately, we can't add the Devoxx figures directly to the rest as the data is incomplete. However, it is useful to add all JavaEdge and Online votes together:

So, what do these results tell us? As always, absolute answers are difficult, because not all voters followed the same rules (online voters clearly didn't rank properly), and Devoxx was asked a different question. However, some things can be established, based not only on total votes but also how the preferences went from 1 to 10.

• Devoxx voters favoured Infer-generics, Multi-catch and Null-handling.
• JavaEdge voters favoured Null-handling, String-switch and Multi-catch.
• Online voters favoured Multi-catch, Null-handling, Infer-generics and String-switch.

So, these four options polled consistently highly in each of the different votes (except Infer-generics which did badly at JavaEdge). This now lends quite significant weight to the question of 'what does the community want' and 'what pain points need addressing'. The small language changes for Java 7 project now has some solid data to work from.

My take is that Multi-catch looks like a strong favourite for inclusion, as does String-switch. Infer-generics had a slightly more mixed reception in JavaEdge, but is popular elsewhere.

The big question is over Null-handling. Three separate votes has ranked it highly, with it taking the highest number of first preference votes in three of the four measured graphs above. There is a clear desire for change and a clear proposal. I hope the opportunity is not missed.

Summary

As more votes stack up, the statistics become more secure. We can now say that out of the ten polled items, the top four are Null-handling, Multi-catch, String-switch and Infer-generics.

Any feedback or thoughts are welcome!

Java 7 - Null-default and Null-safe operators

The most popular small language change request is better handling of nulls. As a result, I've updated my null-handling proposal.

Enhanced null-handling

The votes from Devoxx and JavaEdge were clear. Ordinary developers find handling nulls to be a pain and they would like to see language change to address it. At JavaEdge in particular almost a third of the first preferences and two thirds of the top four preferences went to null-handling.

I blogged my original thoughts two years ago. The updated null-default and null-safe invocation proposal is now available (v0.2).

The proposal covers two new operators, which follow Groovy for syntax (and thus consistency).

The null-default operator ?: returns the LHS unless that is null in which case it returns the RHS:

// today
  String str = getStringMayBeNull();
  str = (str != null ? str : "");
  
  // with null-default operator
  String str = getStringMayBeNull() ?: "";

The null-default operator also works particularly well with auto-unboxing.

Integer value = getIntegerMayBeNull();
  int val = value ?: -1;

In fact, I hope that tools (IDEs and static analysis) would combine to effectively make the null-default operator mandatory when unsafe unboxing is occuring.

The null-safe operator ?. is an alternative form of method/field invocation. If the LHS is null, then the result of the whole method/field invocation is null:

// today
  String result = null;
  Foo foo = getFooMayBeNull();
  if (foo != null) {
    Bar bar = foo.getBarMayBeNull();
    if (bar != null) {
      result = bar.getResult();
    }
  }
  
  // with null-safe operator
  String result = getFooMayBeNull()?.getBarMayBeNull()?.getResult();

There is an interesting case if the last segment of the field/method expression returns a primitive. However, this can be handled by combining the two new operators (hence why its one proposal):

// today
  int dotIndex = -1;
  if (str != null) {
    dotIndex = str.indexOf(".");
  }
  
  // with null-safe and null-default operators
  int dotIndex = str?.indexOf(".") ?: -1;

I considered mandating this when a primitive occurs, but it would have undesirable side effects, so this is one best left to the tools to handle.

More details on all of this in the proposal.

Summary

Nulls remain a pain in Java. There simply isn't enough language support for this most common task. Every day, developers write logic to check and handle nulls and this obscures the meaning of the real code. And every day production systems go down due to NullPointerExceptions that weren't caught. This proposal provides a simple enhancement to Java that tackles the heart of the null issue.

Opinions welcome.

JDK 7 language changes - JavaEdge votes!

For the past few days I have been in the fabulous country of Israel at the JavaEdge conference. Duing the keynote speech, I presented 10 possible language changes for JDK7, and asked the audience to vote on them.

These results are in more depth than the Devoxx figures simply because we could extract information about each persons preferences. For academics or statisticians the raw data is available in Open Office format.

Given ten language changes, rank them according to priority

So, the question is the same basic question as we asked at Devoxx, but with a different selection of changes to pick from. The results show that 171 people voted correctly (starting from 1 for the first preference, 2 for the second and so on). Another 41 attendees voted mostly corrrectly, skipping one of the prefereces (for example marking a 1st, 2nd, 3rd and 5th preference, but not a 4th). A final 113 attendees didn't follow the instructions at all, and marked multipe 1st preferences, multiple 2nd preferences and so on.

The features voted on were:

• For-each loop over a Map
• Control over for-each loops (index/remove)
• List/Map access using []
• Infer generics on the RHS
• Multi-catch of exceptions
• String switch
• String interpolation
• Multi line strings
• Resource management
• Null handling

These results are for those who completed the voting according to the rules (171 attendees).

Here are the first preference votes:

Adding the second preference votes:

Adding the third preference votes:

Adding the fourth preference votes:

Adding the fifth preference votes:

Adding the remaining votes (white is didn't vote, black is 'this is a bad idea':

Now those that almost voted according to the rules - votes are adjusted to move everything beyond the missing column up (41 attendees).

All votes:

Now those that voted based on general preference (103 attendees).

All votes:

So, what do these results tell us? (I'm focussing the analysis on those who voted fully according to the rules, as it is hard to interpret the results from those who simply gave arbitrary rankings).

As with the Devoxx results, there is a clear winner - null handling. Null handling had 50 first preference votes, double that of second place string switch and almost a third of all first preferences. This trend continued with almost two thirds placing it in their top four.

Other popular options were String Switch, Multi-catch of exceptions, enhanced for-each loop for Maps, enhancing the for-each loop to be able to remove or find the index and ARM-style resource management.

Unpopular options (especially considering the black 'bad idea' values) were List/Map access using [] and String interpolation (${variable} in strings).

Infer generics and multi-line strings were deemed of relatvely low importance but not particularly objectionable.

The other 144 votes that were not completely according to the rules are harder to interpret. If someone wants to give it a go, feel free to interpret the raw data.

Summary

Firstly, a big thank you to the JavaEdge voters, and to the AlphaCSP team in Israel. Again, we have some really interesting figures from this vote, with null handling again coming out top. Clearly, there needs to be some real thought about whether some form of null handling can be achieved.

Any feedback or thoughts are welcome!

JDK 7 language changes - Devoxx votes!

Once again, the great Devoxx conference allowed ordinary developers to express an opinion on the future of the Java language. This time, the focus was on prioritising which changes should happen.

The figures given below are of course indicative only. They are based on those Devoxx attendees who actually participated on the whiteboards. Also, the figures are as recorded at the end of Thursday (Friday scores not included). The whiteboard photos are available which provide additional information. Bear in mind that my figures may disagree with the photos in some small way based on when I counted the results relative to when the photos were taken.

Given eight language changes, rank them according to priority

This year, rather than simple "do you like this yes/no" questions, we posed something tougher. Prompted by Alex Buckley, we asked participants to rank eight language change proposals. Voters had to mark the number one against their highest priority, two against their next highest and so on until they had no more priorities. Attendees could also vote against a proposal by marking an 'X'.

The voting system was not the simplest, and we know a few people didn't vote correctly. However, we do believe that the vast majority did vote according to the rules making these results valid. The total number of voters was around 220, so the results have good validity.

Feature	Properties	Multi catch	Null handling	List/Map syntax	Extension meths	Method pointers	Multiline Strs	Infer generics
1	33	47	70	10	6	8	9	61
2	21	56	38	16	8	10	12	49
3	30	35	27	35	10	15	21	35
4	25	26	15	42	17	13	28	19
5	10	20	15	29	13	25	29	19
6	14	15	8	17	23	21	27	15
7	14	9	2	18	24	25	24	11
8	26	19	2	9	19	23	32	8
X	29	1	12	8	28	24	15	0
Total Votes	202	228	189	184	148	164	197	217
Total votes (excluding X)	173	227	177	176	120	140	182	217
Weighted average	4.08	3.41	2.49	4.32	5.36	5.25	5.16	3.07

Here are the first preference votes:

Adding the second preference votes:

Adding the third preference votes:

Adding the fourth preference votes:

Adding the remaining votes:

So, what do these results tell us?

Well, they are in fact amazingly clear. Null-handling was the first preference favourite by a small margin, ahead of Infering RHS generics and Multi-catch of exceptions. However, if particpants had been given just three votes, then these three would have scored almost exactly equal (135/145/138) and over double that of List/Map access (61).

Next in line (ignoring properties) was List/Map access using [], which can be seen by the big jump in third and fourth preference votes. Fifth was Mult-line strings, which developers seem to not be that fussed about.

Finally, extension methods and method pointers (method references) performed poorly in the vote.

Properties was shown to be the most divisive, with both a large number of votes for and against, and fewer in the middle preferences. For example, notice the high number of first preferences, but dropping back when adding second and third preferences. In addition, properties has a high number of low preferences and votes against (coloured black). Since properties isn't going to be included in JDK 7, the properties column is really just an interesting comparison data point.

Obviously in a free vote like this without any explanations, it is easy for people to not vote for items they don't understand. This could explain some the lower scores for Extension methods and Method references (see the total number of votes cast to confirm this). However, even taking this into account, these two don't seem to have as much support as the big three of Null-handling, Infering RHS generics and Multi-catch exceptions

It should also be noted that the graphs and the weighted averages are in line with one another.

Other language changes

In addition to the ranking above, there were some other votes.

The vote on closures was split 50/50:

Yes	41
No	41

The vote on abstract enums showed a majority in favour:

Yes	24
No	18

The vote on ARM (resource management) showed a large majority in favour:

Yes	64
No	11

The vote on some form of delegation showed a majority in favour:

Yes	36
No	9

The vote on for each over a string showed a majority in favour:

Yes	34
No	21

Summary

Firstly, a big thank you to the Devoxx voters, and to Stephan and the Devoxx organisers. I think this exercise was a huge success in participation. I'm certainly not in favour of language design by democracy, but I am very much in favour of gathering large scale information on what really causes developers pain. Clearly, handling nulls, RHS generics and catching exceptions are three big issues.

I hope to repeat the exercise again soon. And if any JUG (public or company internal) wants to do the same (maybe with different proposals) then I'd recommend it (and I've even got an Open Office presentation if you want some material - just drop me a line).

Any feedback or thoughts are welcome!