Friday, 10 December 2010

Is the JCP dead?

Today, the Apache Software Foundation (ASF) joined Doug Lea and Tim Peierls in resigning from the Java Community Process (JCP) Executive Committee (EC). The message from the ASF was simple - The JCP is Dead. But is it?

Is the JCP dead?

The JCP, is the Java Community Process, setup in 1998. But why did it get created at all?

Well, back in 1996 - 14 years ago - Sun started on the road of creating an globally recognised Open Standard for Java at the International Standards Organisation (ISO). This standard was a promise to the community, in part to gain adoption of the platform.

If you weren't involved in Java at the time this might seem unimportant. Yet I can remember using the promise of an Open Standard as one of the arguments for adopting Java as the principal language in the insurer I worked at. I'm sure other developer's similarly found the Open Standard argument useful in adoption discussions. That promise was responsible in part for the rapid adoption of Java.

Sadly, the tale of standardisation through ISO, and then ECMA, is a sordid one. This was a time when Microsoft feared and abused Java resulting in a lawsuit. Sun faced two main hurdles. Firstly, that it wouldn't give up its copyright, patents, trademarks or rights to the compatibility testing suite (which ISO wanted them to). And secondly, the very rational fear that its competitors, notably Microsoft and HP, would abuse the standards process to gain control of, and destroy, Java. (For full details, see this excellent academic paper).

The JCP was setup effectively after the efforts at ISO and before the efforts at ECMA. But initially, it was very focussed on vendors, with a minimum $2000 price tag per year. Some early criticism is still interesting:

To me this makes it pretty clear, that this process is not an open one. If you're still not convinced, ask yourself these questions:

0. Can anyone tell Sun No? Can anyone keep Sun from putting something into the spec they want to put it in? Or put something in that Sun wants to keep out?

1. Can Sun's enemies (e.g. Microsoft, HP, etc.) participate in this process on an equal footing with Sun? Can they even participate at all?

When you actually read the fine print, what the Java Community Process says is that Sun agrees to let other companies contribute their time, money, and knowledge to help Sun do what it wants to do anyway. That may be intelligent business, but it's not an open, community based process for developing standards.

In hindsight, Sun's choices to avoid ISO and ECMA look reasonable. Java was not that established, and fast moving technology often doesn't fit well in standards bodies. And the threat from Microsoft was very real. Thus the JCP was better than nothing. It wasn't an open standards body though.

The Apache Software Foundation (ASF) did not participate initially but later played a part. In the lead up to JavaOne 2002, the Apache Software Foundation worked with Sun to ensure that specifications from the JCP could be implemented in Open Source. On stage announcement (large mp4 file) and Press reports:

The biggest news of this year's JavaOne was delivered Tuesday morning by Jason Hunter, an Apache Software Foundation vice president, co-creator of JDOM, and author of the popular O'Reilly Servlets book. Flanked by Sun CEO Scott McNeally and Sun vice president Rob Gingell, Hunter outlined an agreement negotiated between Sun and Apache that has broad-ranging implications to developers and to the future of Java itself. First, all in-progress and future Sun-led Java Specification Requests (JSRs) will be made available under a license that allows for open source.

In addition, key JSRs that have already been released will have their license altered to this newer, more open license. Even JSRs not being led by Sun can release their reference implementations under an open source license. Finally, Test Compatibility Kits (TCKs) for Sun-led JSRs will be made available for free to qualifying open source and academic groups.

There is a further important implication of Sun's statement that "Sun will modify the specification licenses of all the JSRs currently in progress to reflect Apache's requirements as met in the new draft JSPA." The JSPA is the legal agreement signed by members when they join the Java Community Process. Hunter points out that the draft JSPA includes the requirement that the TCK be licensed separately from the Reference Implementation (RI). In the past, commercial interests, which used to have to license the RI and TCK together, were bound by some interesting legal issues by having the RI source. Under the new agreement, they can now pay just for the TCK. This will help JBoss, for example, which can't get the TCK because it's currently only provided with the RI; they can't see the RI and remain independent.

This agreement from 2002, and the associated updated JSPA legal agreement, are the defining legal basis for the ASF's actions today.

The actual JSPA says.

Other than as set forth above, the Spec Lead agrees not to impose any contractual condition or covenant that would limit or restrict the right of any licensee to create or distribute such Independent Implementations.

Since 2002, the ASF, and many others, have implemented JSRs using these rights. This includes Apache Tomcat, Geronimo, OpenEJB, MyFaces and many more. The ecosystem worked! And surely the community now had an Open Standards body for Java equivalent to ISO?

But, when the ASF started the Harmony project, Sun was not happy.

It decided to stop Harmony using a Field Of Use restriction on the testing compatibility kit. The aim was simple, if Harmony passed the tests, then it wouldn't be open source software. Thus the ASF couldn't release Harmony. It was legally clever. Except that it conflicted with the JSPA legal agreement, as outlined above.

After private negotiations went nowhere, the ASF talked to the JCP Executive Committee (EC). The EC supported the position of the ASF in two critical votes. Oracle voted in support of the position of the ASF in both votes.

By this stage, Sun was in financial trouble. The apparent threat from Harmony was that the Java ME licensing revenue would disappear, and since every last dollar mattered, Sun did not feel able to concede. Instead, Sun released OpenJDK as the GPLv2 implementation of Java, placating some like Richard Stallman.

But the dispute itself dragged on. And on. And on.

Until Oracle bought Sun.

At that point there was hope that the issue would be resolved. After all, Oracle's public position was in support of the ASF. But over time it became apparent that Oracle intended to make no change to the licensing offered. It had done a U-turn.

So why didn't the ASF sue Sun, then Oracle? Well, basically its a really bad idea for a not-for-profit to sue a mega corporation, no matter how good the case. (That said, it might still have been the better course of action)

And why did the Executive Committee fail to fix the problem? Because there is no dispute resolution process in the JCP. All legal agreements are between the member and Sun, now Oracle - because the JCP itself is just a department, not an actual legally recognised organisation.

What the EC could do was prevent the approval of the Java SE 7 JSR. And they did. While this isn't the whole reason why its been so long since Java SE 6, it is a major part of the story.

Once Oracle finally got to grips with the issue it made a decision to force Java SE 7 and 8 through, no matter what. Tim Peirels captured this in his resignation statement:

Add to that Oracle's expressed intent to proceed with the SE7/8 JSRs whatever the outcome of the vote, and one can only conclude that the SE/EE EC is never going to be more than a rubber stamp for Oracle. (The belligerent tone with which this message was delivered does not come across in the public minutes, but it was loud and clear over my phone connection.)

At the same time, the second strand of Oracle's plan was in action. IBM and Apple were being invited to join OpenJDK in leadership positions. Effectively, this was the sign that IBM would now vote through the Java SE 7 and 8 JSRs, irrespective of the ASF dispute.

However, even I was surprised at just how clear the mismatch between the Java SE 7 license and the JSPA was. Not only that, the license terms are inconsistent with the stated uses of the JSR itself.

Of course, none of this mattered. The EC corporate members, and their lackie Eclipse, had decided to pass the JSRs even though they were in violation of the legal agreements. This was now big company politics, and if the ASF had to go, then it had to go.

Doug Lea resigned just before the vote. Tim Peirels just after. And today, the ASF joined them.

After all, there is no point helping to write specifications that you aren't allowed to implement.

But were they right to declare the JCP is dead?

The first ASF claim is that the JCP is not an open standards organisation because there are restrictions made in the commercial interests of one party that prevent another from producing an implementation. The second ASF claim is that the EC failed to perform its duty to enforce the rules and to defend a fellow member.

By approving Java SE 7, the EC has failed on both counts : the members of the EC refused to stand up for the rights of implementers, and by accepting Oracle's TCK license terms for Java SE 7, they let the integrity of the JCP's licensing structure be broken.

The Apache Software Foundation concludes that that JCP is not an open specification process - that Java specifications are proprietary technology that must be licensed directly from the spec lead under whatever terms the spec lead chooses; that the commercial concerns of a single entity, Oracle, will continue to seriously interfere with and bias the transparent governance of the ecosystem; that it is impossible to distribute independent implementations of JSRs under open source licenses such that users are protected from IP litigation by expert group members or the spec lead; and finally, the EC is unwilling or unable to assert the basic power of their role in the JCP governance process.

But none of this definitively means that the JCP is dead.

Oracle's public argument is that Java needs to move forward:

Last month Oracle renominated Apache to the Java Executive Committee because we valued their active participation and perspective on Java. Earlier this week, by an overwhelming majority, the Java Executive Committee voted to move Java forward by formally initiating work on both Java SE 7 and SE 8 based on their technical merits. Apache voted against initiating technical committee work on both SE 7 and SE 8, effectively voting against moving Java forward. Now, despite supporting the technical direction, Apache have announced that they are quitting the Executive Committee. Oracle has a responsibility to move Java forward and to maintain the uniformity of the Java standard for the millions of Java developers and the majority of Executive Committee members agree. We encourage Apache to reconsider its position and remain a part of the process to move Java forward. ASF and many open source projects within it are an important part of the overall Java ecosystem.

For me, this is very hollow. The reference to "technical merits" covers the fact that many members used their comments to object to the licensing. And frankly, Oracle were going to proceed whether or not the EC voted "Yes".

My conclusion is that the EC should have voted "No" and forced Oracle to run Java SE 7 and 8 outside the JCP, thus preserving the integrity of the EC and the JSPA license. And this action would have had zero effect on "moving Java forward".

Since the EC voted "Yes", they tacitly accepted Oracle's right to break the JSPA legal agreement when it is in their commercial interest.

Given that the ASF has no commercial axe to grind, they made the sensible choice to leave. And since the EC failed to act as the leaders of the industry it is entirely reasonable to say that the JCP is now a sham body where legal agreements mean little.

Thus essentially whether you consider the JCP is dead or not depends on what you believe the JCP is.

Since 2002, I have essentially seen the JCP as a vendor-neutral open standards body, the Java specific equal of ISO. Clearly, it can no longer be viewed as that.

But others have viewed it more as a useful tool for influencing the owner of Java, where the output sometimes allows the creation of a competitive ecosystem around a specification. Viewed this way, the JCP is still entirely feasible.

Certainly, Doug Lea saw it as the former:

I believe that the JCP is no longer a credible specification and standards body

So, is the JCP dead? Well, it will probably limp on for a while. And if the second view of what it is takes hold, then it may have some future as far as big corporates are concerned.

But it cannot now be seen as an Open Standards Body.

So its both dead and undead. A zombie.

Summary

Its the end of an era. An era of hope that the JCP would be Java's ISO, producing truly Open Standards.

So as that era closes we look forward to the new closed era of Java. Where its Oracle's way or the highway.


Stephen Colebourne
Apache Software Foundation member, speaking personally
Oracle Java Champion, speaking personally
Not a committer on Harmony or OpenJDK

25 comments:

  1. The FAQ to Apache's Open Letter to Sun [1] also provides some historical perspective. One section that I find particularly interesting is [2].

    [1]http://www.apache.org/jcp/sunopenletterfaq.html
    [2]http://pelegri.wordpress.com/2010/12/09/from-the-faq-on-apaches-open-letter-to-sun/

    ReplyDelete
  2. Very informative, Stephen, thanks. This makes me wonder if Oracle expected the move by ASF, aware that it potentially splits the server-client markets for Java. ASF projects would now be very wary of implementing the future roadmap, but that would frustrate Oracle's plans too. Unless they're also trying to divide and conquer ASF (and not a few vendors would want that, I suspect)?

    ReplyDelete
  3. Will you be withdrawing from the Java date and time API and leaving the JCP as an Apache member?

    ReplyDelete
  4. Will you be abandoning the date and time JSR as the ASF are suggesting all Apache members abandon their JSRs?

    ReplyDelete
  5. While I agree that they *did* fail to meet their legal obligations, I do see their side.

    If I remember correctly, at the time Sun was a bad name in open source. Harmony was created with express purpose to get around Sun's control of the java ecosystem, and was financially manned by IBM, Intel and other major companies who had a direct interest in removing Sun from the Java equation.

    This was about Sun vs the big guys, and apache was the front for a 100billion+ market cap conglomerates scheme to take Java away from Sun. The only thing I was surprised about was that they didn't fund the lawsuit too.

    Would you disagree?

    ReplyDelete
  6. Stephen Colebourne10 December 2010 at 12:12

    @Eduardo, I've avoided commenting on the Google lawsuit beyond saying that they sailed "close to the wind"

    @Alex, I don't lead JSR-310 via the ASF, I lead it as an individual. Thus the decision I take will be solely mine. I expect to clarify my position in the next week or so.

    @Ben, The IBM/Intel control theory is a conspiracy theory until there is public information to back it up. Find me the information and I will publish (even if it isn't favourable to the ASF).

    My view is that the a far better resoluton would have been a public statement by Sun or Oracle to say "we were wrong to day that Java SE is an open specification, so we're withdrawing it from the JCP". Doing so would not have been pleasant, but it would have retained the integrity of the JCP and probably retained the input of the ASF.

    ReplyDelete
  7. > This was a time when Microsoft feared and abused Java

    In retrospect, it's difficult to tell how Microsoft abused Java. They implemented their own GUI libraries (a la SWT), they implemented language/VM extensions (a la closures). And they had (at the time) the highest performing JIT.

    ReplyDelete
  8. If Java is going to be "closed" and I am fine with that, I don't see community input being taken any less seriously. It just won't be as formal a process. I am hoping that since Oracle wants to make money with Java, then that is going to equate to them putting resources behind it and making it a "better" language going forward.

    ReplyDelete
  9. My comment and highlight points to the economic context for all these discussions. At the point of the letter to the Sun, ASF recognized the potential impact of what they were asking and they addressed it in their FAQ. Although history has proven the analysis may not have been complete, I applaud the ASF for taking that into consideration at the time. I am disappointed that most current analysis seems to ignore these angles.

    ReplyDelete
  10. Correct me if I'm wrong, but wasn't the vote to accept the JSR, with any changes needed occurring once the JSR had been authorized to proceed?

    With all the troubles in the JCP community((1), (2), (3), and (4)), I thought I would provide some of my thoughts on the JCP, JSRs, RIs, TCKs, etc.

    I believe:
    (1) RIs and TCKs should be separate from the JSR and the JCP.
    (2) JSR Specs (not RI) and Test Specs (not TCKs) should be provided by the JCP.
    (3) Control of Java (TM) and Patents directly effecting a JSR should be turned over to the JCP.
    (4) Additional patents in an implementation (not spec) would then be part of the RI/TCK and not limited in compliance of the Specs. So then any patent issues would be against the implementors (not the JCP).
    (5) Oracle (Sun) should not be given a permanent veto controlling leadership position in the JCP. The leadership role should be elected and voted just like everything else.
    (6) JCP (or a Validation WG) can be a validating body if they so chose to take up that role. Implementors can provide their implementation for acceptance against the Test Spec (not TCK).
    (7) Commercial interests addressed would be addressed by individual, groups, and companies with their own implementations, independent of the JCP with their own licensing for their own implementation. The public could then identify which implementation they like best in the market.
    (8) Oracle/Sun still owns (an open source based) OpenJDK code base implementation.

    Excert from http://eblog.bresie.com/2010/12/thoughts-on-jcp-jcrs-and-many-things.html for

    ReplyDelete
  11. Stephen, I don't think that this is a conspiracy theory. Everyone knows it would be in IBM's best interest to have a jvm that they control, IBM have been pushing for it for years.

    An "im feeling lucky" gave me:
    http://harmony.apache.org/contributors.html

    ReplyDelete
  12. Stephen,

    I've seen you write several times that it would be a bad idea for Apache to sue Sun/Oracle. Can you please explain why?

    ReplyDelete
  13. Stephen Colebourne11 December 2010 at 11:00

    @Robert, Yes there will still be investment. Having Java SE outside the JCP would have been no bad thing as it still would have evolved.

    @Eduardo, OK I think I see where you're driving.

    @Eric, Yes this is just the vote to proceed on Java SE 7, but effectively the deed is done. Oracle don't want to change these licenses.

    @Ben, Showing that it is is IBMs interests is meaningless. You need to find public evidence of a specific IBM plan. If a theory meets the set of publicly known facts, but there is no other evidence to validate the theory, then its a conspiracy theory. One which might be true, but frequently isn't.

    @Eugene, The answer to your question is in the article.

    ReplyDelete
  14. "In hindsight, Sun's choices to avoid ISO and ECMA look reasonable."

    Who are you kidding? Sun has been acting in bad faith for over a decade: their licenses have been cleverly chosen to give the appearance of open source software, all the while Sun has been creating a thicket of contracts and patents, and all the while they have been threatening anybody who didn't use Java in the way they liked.

    To put it differently, if Sun had the means of stopping Microsoft, they had the means of stopping anybody they liked, and that made Java closed and proprietary. Furthermore, Sun's entire business had been founded on taking open source software proprietary and McNealy was hobnobbing with Gates and others; lying to the open source community was entirely in character for the company.

    And Java has suffered enormously from how Sun managed it: the Java language has deteriorated steadily from a simple if limited language to complex bloatware, and the language still is stuck somewhere in the 1980's. The Java libraries are bloated, inelegant, and unoriginal. Java is unsuitable for high performance computing, numerics, graphics, and tons of other important fields. And Java isn't even suitable for serious cross-platform development, except perhaps on the server side (but even C allows cross-platform server side development). Part of the reason is that the JCP allowed a few so-called experts to realize their pipe dreams without much real-world experience and feedback with the resulting libraries; the JCP is a good example of how to design a process that produces bad standards.

    What's happening now with Oracle/ASF has been in the works for more than a decade. You had to be blind not to see it coming. The only rational response is to drop Java like the hot potato it is and move on to something else. Fortunately, there are plenty of better programming languages around.

    ReplyDelete
  15. Stephen Colebourne11 December 2010 at 12:26

    @Mike, You are entitle to your opinion, I just hope you've done as much research as I have to back your claims up. I don't recognise most of your technical summary of the Java platform - by most accounts its the world's most popular language because people realise its the ecosystem thats valuable, not the language itself.

    ReplyDelete
  16. Hi Stephen,

    if you decide to withdraw from the JSR-310, will you be able to take the code with you (considering that most parts of the code have your copyright on it) or will Oracle still be able to just "steal" the code and go on?

    Does JSPA 4.D help here? ("Withdrawal of Contributions due to Change in Announced License Terms.")

    ReplyDelete
  17. Calling it a conspiracy theory is a poor logical argument. If it were not in IBM's best interests to pay people to work on harmony, why would they do it?

    Of course it was in their best interests:

    1) IBM bet big on Java - all their software is Java based.

    2) IBM were considering BUYING Sun, and it was stated that it was primarily to get Java.

    3) IBM paid employees to work on a JVM independent of Sun's after it was "clear" Sun would license the TCK.

    4) IBM contributed all the code it legally could (that wasn't tainted by licensing with Sun) to Harmony

    5) IBM is paying Sun for TCK of J9 and Jikes.

    6) IBM has a history of using open source as a weapon in business: eclipse.

    Any business worth their salt would conclude that funding an open-licensed JVM would be in their best interest. Of course it was: They bet their business on it, have their own runtimes, but pay Sun for the privelege. If Sun got bought by someone else or changed terms, then this would pose a significant risk to IBM.

    Why they have to come out and publically say something like that is beyond me. It's obvious, and is a simple explanation for the facts. What's your justification for their actions - IBM *never* thought of this, and simply decided out of the blue to fund full time employees on the project because IBM love open source? This was a strategic decision, plain and simple. It's what you would expect them to do in the situation.

    ReplyDelete
  18. Stephen Colebourne11 December 2010 at 18:42

    @Steve, I'll be commenting on JSR-310 soon.

    @Ben, My point is that there is a difference between it being in IBMs interests to support Harmony, and it being a deliberate strategy to use and abuse Apache to wrest control of Java. You might think that distinction is subtle, but IMO it is there. And finally, your suggestion is tangental to the core issue discussed in this blog (relevent, but tangental). After all, the JSPA does allow for an Apache licensed version of Java SE.

    ReplyDelete
  19. There is a "response" from Oracle. IMHO that response is a shame

    http://blogs.oracle.com/henrik/

    The content is a shame. Ståhl acting as a mouthpiece is a shame. Messinger (and Deutsch) not talking directly to the community, but using a mouthpiece is a shame. Not allowing comments in that blog is a shame.

    It clearly shows what Oracle is about. Some vice presidents fart in our general direction and we should say "hmmm, smells like roses".

    BTW, I also think Sun acted in bad faith for a long time. The JSPA and SCA were designed to keep the normal developer out and to allow the big boys to play their game. Regularly Sun claimed they "will look into it" and "simplify them". Sun never did. The JCP deliberately excluded and excludes the community. Somewhere on java.net, in a discussion about a JCP reform, a JCP bigwig even admitted that they don't want to hear from every Tom, Dick and Harry.

    ReplyDelete
  20. Hi Stephen,

    as far as I know Scala isn't your preferred language, but maybe this looks like an interesting way of Date&Time going forward: https://github.com/soc88/scala-datetime ?

    At the moment the project is more about fun and pushing IntelliJ's Java-to-Scala conversion to its limits and it doesn't even compile yet (because I didn't dare to touch all the "rule magic" yet), but the translation from Java to Scala was pretty easy in many cases as most of the Date&Time classes do already have a "Scala-like" design.

    Refactoring with IntelliJ's Scala plugin worked rather well, but the Java-to-Scala conversion still doesn't preserve the order of methods/fields/classes (very bad!).

    Let me know what you think (if you want)!

    Thanks!

    Bye,

    Simon

    ReplyDelete
  21. Yes, the conspiracy theory is that IBM didn't want to pay the licensing for the Java SE TCK and hid behind a not-for-profit foundation to do so.

    People who wanted a 'free' solution on platforms not supported by Sun already had a working solution in the form of Kaffe, Cacao, GCJ, JamVM and other users of GNU Classpath.

    ReplyDelete
  22. Stephen,
    As many, not just yourself seem to feel uncomfortable with the "Umbrella" or "Distribution" JSRs for SE 7 and beyond (I don't think ME was immune to that either, it's just not there to discuss any Next Generation yet, although news of SE6 for Embedded are arising these days)

    At least following a working Modularization for Java (so maybe for SE7 we might have to live with that compromise or "zombie" as referred to here) couldn't a system that works perfectly well on top of Linux be found? Having commercial vendors, not just Oracle itself bundle "Distributions" which then the JCP may not need to worry about any more...?

    Linux neither has a problem with Oracle's "Unbreakable Linux" nor those parts Android or LiMo derived from it, as long as those distributions don't violate the GPL, which all of OpenJDK is also lisensed under.

    Is that idea too foolish or as Martin Fowler told us this week during a speech out of a "Fantasy World" ?
    (he referred to a "Java Toaster" people claiming to know and fully understand the JSPA say wasn't even possible under its current form and FOU Restrictions ;-)

    ReplyDelete
  23. @Julian, what did offer prison guards to get access to the Internet in Wandsworth ?;-)

    ReplyDelete
  24. ok, so finally the day came.

    1.
    Hey ASF, it is time to create our own specs of a new vm and language and divert all resource to it of cource stop further java based dev also.

    2.
    a new license - this new license should be "paid for paid and free for free" model.
    means if source code or binary is included in a commercial (revenue generated) then the company has to pay to ASF otherwise if it is used by enduser or nonprofit purpose it is free.
    so as to avid taken for granted by mnc and also ask for fair share of revenue to keep all free coders and best minds/ solutions keep living their dreams.

    Thanks
    Pradeep Jindal

    ReplyDelete
  25. oh! yes.. and we already hav an alternative vm http://llvm.org/

    ReplyDelete