Shedding new light on No Java SE 7 JSR

There is a direct connection between the 'Sun vs Apache Harmony' dispute and the lack of a Java SE 7 platform JSR. Using newly available evidence I hope to shed new light on what that link is.

Back in the day

Its interesting to read again how Harmony was welcomed when it first started:

Part of the reason these Apache projects are possible is that the JCP has been working over the last few years to clarify the role of open source projects and also to improve overall transparency. For example there were changes to the intellectual property rules in JCP 2.5 specifically designed to allow open source implementations. There have also been a number of transparency improvements in JCP 2.6 to allow earlier access to specification information. In addition to those JCP updates, Sun created a scholarship program under which we have provided Apache (and other not-for-profit groups) with free access and free support for Sun's Java compatibility test suites.
...
The licensing rules for J2SE 5.0 were carefully designed to allow independent, compatible open-source implementations of the J2SE specification. Personally, I am not entirely sure if the world really needs a second J2SE implementation, but at the same time I am also glad to see that all the effort we put into getting the rules and the licensing issues straightened out is actually proving useful!
Graham Hamilton's blog VP and Fellow at Sun responsible for Java at the time, written May 7, 2005.
(My emphasis)

And lets look at how that was reiterated in the business terms for Java SE 6:

JSR 270: JavaTM SE 6 Release Contents
...
2006.10.24:
2.18 Please provide a description of the business terms for the Specification, RI and TCK that will apply when this JSR is final.
...
7. Nothing in the licensing terms will prevent open source projects from creating and distributing their own compatible open source implementations of Java SE 6, using standard open source licenses. (Yes, you can create your own open source implementation of Java SE 6 if you really want to. But we're also doing everything we can to make it easy for you to use the original RI sources! See http://jdk6.dev.java.net.)

JSR-270
(My emphasis)

How times change. Update: The 'how times change' reference was a comparison of both of the above quotes to the dispute that followed, not a comparison of the two quotes. Sorry for the confusion.

Trying to resolve the dispute

As is clear, Apache claim that Sun is in violation of the JSPA contract for both Java SE 5 and Java SE 6 platform specifications. The comments above also indicate how matters changed over the course of time.

Given the dispute, it is clear that Apache would object to there being a Java SE 7 platform JSR without changes being made to resolve the issue. However, Apache is just one voice in the industry. The power in the JCP is held in two 16 member Executive Committes - one for Java SE/EE and one for Java ME. Clearly, therefore, Apache could be outvoted 15-1 on the SE/EE committee, and it has no vote on the ME committee.

So, why hasn't Sun just submitted the Java SE 7 platform JSR and accepted/ignored Apache's No vote?

Well, until recently, I thought that the reasons were all private information. However, it turns out that since September 2008, the Executive Committee meeting minutes have been openly published. As such, we can see how the Executive Committees have tried to resolve the issue. As far as I know, no journalist or blog has yet picked up on this.

Given the controversial nature of this, I strongly encourage anyone following this to read the minutes themselves. Its a good insight into the the way in which the JCP operates (a style probably typical of any commercial based standards body). In this section I will extract some key elements, and make some minimal comment, but please read the full documents to make up your own mind.

Meeting summary - February 27th 2007
Geir Magnusson presented on a hypothetical situation regarding a company pursuing a certain Java-related business opportunity and whether the JSPA allows licenses that could affect that opportunity. During the meeting Geir asked for a straw poll on the following statement:
"The Executive Committee of the JCP supports the Apache Software Foundation's assertion that placing limitations on field of use for compatible implementations of Java specifications is contrary to the letter of the JSPA and the spirit of the JCP as a creator of open specifications."
Voting "yes": 17
Voting "no": 2
Voting "abstain": 3

(Geir Magnusson is the Apache representative on the Executive Committee)
(Note that the official document status is draft and unapproved, however it is publicly linked from the JCP website)

Note the 'straw poll' explicitly mentions the field of use restriction that Sun used against Harmony. This vote represents the collective wisdom of the two Executive Committees - Java ME and Java SE/EE - 32 votes in total, however there were 10 absentees.

Note that there were two 'no' votes.
Also note that Sun sits on both executive committees and thus has two votes.
But there is no other information to tie those two facts together.

Meeting summary - December 7th 2007
Resolution 1 (proposed by Oracle, seconded by BEA)
"It is the sense of the Executive Committee that the JCP become an open independent vendor-neutral Standards Organization where all members participate on a level playing field with the following characteristics:
* members fund development and management expenses
* a legal entity with by-laws, governing body, membership, etc.
* a new, simplified IPR Policy that permits the broadest number of implementations
* stringent compatibility requirements
* dedicated to promoting the Java programming model
Furthermore, the EC shall put a plan in place to make such transition as soon as practical with minimal disruption to the Java Community."

Vote tally from SE/EE Executive Committee:
Voting "yes": 13 (Apache, BEA, Fujitsu, Google, HP, IBM, Intel, Doug Lea, Nortel, Oracle, Red Hat, SAP, Hani Suleiman)
Voting "abstain": 1 (Sun)
Vote tally from ME Executive Committee:
Voting "yes": 13 (BenQ, Jean-Marie Dautelle, Ericsson, IBM, Intel, Nokia, Orange, RIM, Samsung, Siemens, Sony-Ericsson, Time Warner, Vodafone)
Voting "abstain": 1 (Sun)

Note how the proposal is explicit about the JCP becoming 'independent' and 'vendor-neutral'. Note also how the proposal accepts that the costs of such a body would have to be shared by its members. Again, the two Executive Committees - Java ME and Java SE/EE - clearly voted in favour of a resolution to further open the JCP. The sole exception was Sun, who abstained.

Meeting summary - December 7th 2007
Resolution 2 (proposed by BEA, seconded by Intel)
"The SEEE EC requests Sun to submit a Java SE7 JSR with the following characteristics:
* EG members will share costs by contributing to the TCK and RI submitted under common licensing terms that will allow any implementer to use them;
* accompanied by public spec, tck, ri licenses at JSR initiation;
* licenses do not contain Field of Use restrictions on spec implementations.
* the JSR is operated by the Spec Lead under democratic principles (e.g. Java SE6)
Furthermore, from the time of submission, TCK license(s) for Java SE5 and later will be offered without field of use restrictions on spec implementations enabling the TCK to be used by organizations including Apache."

Vote tally from SE/EE Executive Committee:
Voting "yes": 10 (Apache, BEA, Fujitsu, Google, Intel, Doug Lea, Nortel, Oracle, SAP, Hani Suleiman)
Voting "abstain": 4 (Sun, HP, IBM, Red Hat)

So, the SE/EE Executive Committee clearly voted in favour (10-4) of a resolution in favour of an open Java SE 7 specification, and for Apache to receive the testing kit for Java SE 5 and 6. There is no information available as to the details of the discussion, nor or the reasons why individual participants voted for or against.

Meeting summary - April 8th 2008
Sun's commitments on JCP reform
Onno Kluyt gave a presentation in which he outlined a series of JCP reforms that Sun is willing to commit to. The presentation was followed by an extensive discussion.
Java SE 7 JSR
Roberto Chinnici summarized Sun's plans for the Java SE 7 JSR, expressing the hope that this would be submitted and approved soon.

We can note that back in April 2008 Sun were making efforts to resolve the issues. Also note that Sun expressed the desire to submit the Java SE 7 JSR 'soon'. I'd also note that April 2008 is just before JavaOne 2008, which was in May - a time when Sun frequently makes big announcements.

Meeting summary - June 24th 2008
Geir Magnusson and Onno Kluyt reported that Sun and Apache have failed to reach agreement on the terms under which the JCK would be licensed to Apache for use in the Harmony project. They also reported that no further discussions are planned. Onno stated that Sun has no immediate plans to submit the JSR for Java SE 7. Rather, Sun plans to continue SE 7 development work through the OpenJDK project and through component JSRs that are already in progress. He expressed the hope that when the JSR is submitted at some point in the future the EC will approve it.

EC members expressed their disappointment at this outcome, and discussed the possible implications without reaching any firm conclusions.

Sun now have 'no immediate plans' to submit the Java SE 7 platform JSR. It is fair to ask what happened between April and June to cause the submission of the JSR to go from 'soon' to 'no immediate plans'. Sadly, there is no published information to answer that question, so I'll leave that open to debate (I can't comment further).

Further, it can be seen that Sun are beginning to focus on working towards version 7 using Open JDK instead.

Meeting minutes - September 25th 2008
Patrick gave an update on the upcoming elections, and informed the ECs of Sun's nominations:
...
Several members expressed concern that they weren't consulted about Sun's nominations. Patrick said that Sun was unwilling to make concessions on "control points" (such as the right to nominate for EC elections) without something in return. When members asked what concessions Sun wanted, Patrick responded that Sun expressed the desire to feel confident that the Java SE7 JSR would be considered by EC members without allowing the dispute about TCK licensing between Apache and Sun to obstruct it.

This passage shows Sun as being concerned about whether the Java SE 7 JSR would be considered without the TCK dispute being resolved. Further, they link certain aspects of Sun control in the JCP (ie. who gets a guaranteed seat on the JCP) to the Java SE 7 debate. I'm not going to speculate on the details of what is clearly a politically-charged passage.

Meeting minutes - September 25th 2008
Private Session
The ECs then went into private session for a discussion on the negotiations between Apache and Sun.
After these discussions it was agreed that Apache and Sun would explore the possibility of arbitration to resolve their differences.

Arbitration is mentioned as one possible solution.

Compatibility Rules discussed at the meeting on September 25th 2008
Slide 13:
Compatibility is a contractual obligation.
- You must not ship incompatible products.
-- Compatibility is binary.
- You can't be "almost compatible" or "a little bit incompatible".
- The JCK contains 120,000 test cases.
-- If you pass 119,999 you are not compatible.
-- If you pass 120,000 and don't meet all the other requirements you are not compatible.

This hammers home the point that any implementation of any JSR must be fully compatible. And that it is a contractual obligation. It also shows the size of the Java SE platform JSR testing kit.

Meeting minutes - January 13th 2009
Private session
The ECs then went into private session to discuss the status of the Sun-Apache negotations. During this session members had a lively discussion, and made some proposals that will be brought to the attention of Sun's management

Again, discussions to resolve the issue by the Executive Committee are still ongoing in January 2009.

There are no further minutes publicly available - January 2009 is the last published date. However, the next meeting is on April 9th 2009, so we should expect to see the minutes from February 25th approved at the April meeting very soon.

Any clearer?

I hope that these quotes, and the original minutes, provide a deeper insight into the dispute, and into how the Java SE 7 is affected. I'll draw out three points:

Firstly, that this isn't just Apache vs Sun. Other members of the Executive Committee have been willing to side with Apache and vote against Sun (February and December 2007). Bear in mind that those members have access to even more information than is public here in order to judge which way to vote. Remember though that those votes are not-binding. Due to the structure of the JCP, the other Executive Committee members cannot actually intercede on behalf of Apache.

Secondly, from the September 2008 meeting, there is distinct evidence (but not proof) that Sun feels unable to get a Java SE 7 JSR passed until the Harmony dispute is resolved. We can also note the difference in tone between April 2008 and June 2008, where a Java SE 7 platform JSR went from 'soon' to 'no immediate plans', and Sun's focus became clearer around using Open JDK instead of a JSR.

Finally, it is clear that discussions have continued in one form or another for the entire time of the dispute. It isn't the case that the Apache open letter was published and then forgotten or ignored.

Despite all the efforts, there has been no solution. And as such there is still no Java SE 7 platform specification. Thus, I hope its a little clearer why I'm arguing that, in all likelihood, we will get a JDK 7 implementation, but not a Java SE 7 platform open specification.

Summary

Personally, I'm glad these meeting minutes are now public. Everyone can now see the behind-the-scenes discussion for themselves and make up their own minds as to the true state of the debate. In addition, you can freely judge if my blogging has been factual or not.

Finally, for anyone still wondering why Apache Harmony might be of any value at all, consider this:

Apache Harmony: Thanks for the TreeMap Work!
I'd like to thank Apache Harmony for their JDK library performance efforts. We were given a tip that the Harmony folks were doing some interesting work with the TreeMap collection class, and low and behold they were.
...
Controlled measurements showed the "fat" TreeMap significantly faster with in order traversals, and improved our SPECjbb2005 score by a solid 3-5% depending on the platform.
...
Considering the potential performance gain we started the effort of porting the Apache Harmony TreeMap code to JDK 6.
...
The new TreeMap is included in JDK 6 Update 6 Performance Release
David Dagastine

So, Sun's JDK runs faster today as a direct result of code written in Apache Harmony. If you use Sun's JDK 6 update 6 or later, Apache Harmony has already benefited you. And that just one of the benefits of open specifications and competing implementations.

Stephen Colebourne
Apache Software Foundation member, speaking personally
Sun Java Champion, speaking personally
Not a committer on Harmony or OpenJDK