Thursday, 6 September 2018

From Java 8 to Java 11

Moving from Java 8 to Java 11 is trickier than most upgrades. Here are a few of my notes on the process.

(And here are a couple of other blogs - Benjamin Winterberg and Leonardo Zanivan.)

Modules

Java 9 introduced one of the largest changes in the history of Java - modules. Much has been said on the topic, by me and others. A key point is sometimes forgotten however:

You do not have to modularise your code to upgrade to Java 11.

In most cases, code running on the classpath will continue to run on Java 9 and later where modules are completely ignored. This is terrible for library authors, but great for application developers.

So my advice is to ignore modules as much as you can when upgrading to Java 11. Turning your application into Java modules may be a useful thing to consider in a few years time when open source dependencies really start to adopt modules. Right now, attempting to modularise is just painful as few dependencies are modules.

(The main reason I've found to modularise your application is to be able to use jlink to shrink the size of the JDK. But in my opinion, you don't need to fully modularise to do this - just create a single jar-with-dependencies with a simple no-requires no-exports module-info.)

Deleted parts of the JDK

Some parts of the JDK have been removed. These were parts of Java EE and Corba that no longer fitted well with the JDK, or could be maintained elsewhere.

If you use Corba then there is little anyone can do to help you. However, if you use the Java EE modules then the fix for the deleted code should be simple in most cases. Just add the appropriate Maven jars.

On the Java client side, things are more tricky with the removal of Java WebStart. Consider using Getdown or Update4J instead.

Unsafe and friends

Sun and Oracle have been telling developers for years not to use sun.misc.Unsafe and other sharp-edge JDK APIs. For a long time, Java 9 was to be the release where those classes disappeared. But this never actually happened.

What you might get with Java 11 however is a warning. This warning will only be printed once, on first access to the restricted API. It is a useful reminder that your code, or a dependency, is doing something "naughty" and will need to be fixed at some point.

What you will also find is that Java 11 has a number of new APIs specifically designed to avoid the need to use Unsafe and friends. Make it a priority to investigate those new APIs if you are using an "illegal" API. For example, Base64, MethodHandles.privateLookupIn, MethodHandles.Lookup.defineClass, StackWalker and Variable Handles.

Tooling and Libraries

Modules and the new six-monthly release cycle have conspired to have a real impact on the tooling and libraries developers use. Some projects have been able to keep up. Some have struggled. Some have failed.

When upgrading to Java 11, a key task is to update all your dependencies to the latest version. If there hasn't been a release in since Java 9 came out, then that dependency may need extra care or testing. Make sure you've updated your IDE too.

But it is not just application dependencies that need updating, so does Maven (and Gradle too, but I don't know much about Gradle myself). Most Maven plugins have changed major versions to a v3.x, and upgrading Maven itself to v3.5.4 is also beneficial.

Sadly, the core maven team is very small, so there are still some bugs/issues to be solved. However, if your Maven build is sensible and simple enough, you should generally be OK. But do note that upgrading a plugin from a v2.x to a v3.x may involve changes to configuration beyond that just associated with modules. For example, the Maven Javadoc plugin has renamed the argLine property.

A key point to note is the way Maven operates with modules. When the Maven compiler or surefire plugin finds a jar file that is modular (ie. with a module-info.class) it can place that jar on the modulepath instead of the classpath. As such, even though you might intend to run the application fully on the classpath, Maven might compile and test the code partly on the classpath and partly on the modulepath. At present, there is nothing that can be done about this.

Sometimes your build will need some larger changes. For example, you will need to change Findbugs to SpotBugs. And change Cobertura to JaCoCo.

These build changes may take some time - they did for me. But the information available by a simple web search is increasing all the time.

Summary

I've upgraded a number of Joda/ThreeTen projects to support Java 9 or later now. It was very painful. But that is because as a library author I have to produce a jar file with module-info that builds and runs on Java 8, 9, 10 and 11. (In fact some of my jar files run on Java 6 and 7 too!)

Having done these migrations, my conclusion is that the pain is primarily in maintaining compatibility with Java 8. Moving an application to Java 11 should be simpler, because there is no need to stay tied to Java 8.

Comments welcome, but note that most "how to" questions should be on Stack Overflow, not here!

Monday, 3 September 2018

Time to look beyond Oracle's JDK

From Java 11 its time to think beyond Oracle's JDK. Time to appreciate the depth of the ecosystem built on OpenJDK. Here are some of the key builds available.

This is a quick follow up to my recent zero-cost Java post

OpenJDK builds

In practical terms, there is only one set of source code for the JDK. The source code is hosted in Mercurial at OpenJDK.

Anyone can take that source code, produce a build and publish it on a URL. But there is a distinct certification process that should be used to ensure the build is valid.

Certification is run by the Java Community Process, which provides a Technology Compatibility Kit (TCK, sometimes referred to as the JCK). If an organization produces an OpenJDK build that passes the TCK then that build can be described as "Java SE compatible".

Note that the build cannot be referred to as "Java SE" without the vendor getting a commercial license from Oracle. For example, builds from AdoptOpenJDK that pass the TCK are not "Java SE", but "Java SE compatible" or "compatible with the Java SE specification". Note also that certification is currently on a trust-basis - the results are not submitted to the JCP/Oracle for checking and cannot be made public. See Volker's excellent comment for more details.

To summarise, the OpenJDK + Vendor process turns one sourcebase into many different builds.

In the process of turning the OpenJDK sourcebase into a build, the vendor may, or may not, add some additional branding or utilities, provided these do not prevent certification. For example, a vendor cannot add a new public method to an API, or a new language feature.

Oracle JDK

http://www.oracle.com/technetwork/java/javase/downloads/

From Java 11 this is a branded commercial build with paid-for support. It may be available for free for development use, but not for production. Oracle plans to provide full paid support until 2026 or later (details). Note that unlike in the past, the Oracle JDK is not "better" than the OpenJDK build (provided both are at the same security patch level). See here for more details of the small differences between Oracle JDK and the OpenJDk build by Oracle.

OpenJDK builds by Oracle

http://jdk.java.net/

These are $free pure unbranded builds of OpenJDK under the GPL license with Classpath Extension (safe for use in companies). These builds are only available for the first 6 months of a release. For Java 11, the expectation is there will be Java 11.0.0, then two security patches 11.0.1 and 11.0.2. To continue using the OpenJDK build by Oracle with security patches, you would have to move to Java 12 within one month of it being released. (Note that the provision of security patches is not the same as support. Support involves paying someone to triage and act upon your bug reports.)

AdoptOpenJDK builds

https://adoptopenjdk.net/

These are $free pure unbranded builds of OpenJDK under the GPL license with Classpath Extension. Unlike the OpenJDK builds by Oracle, these builds will continue for a much longer period for major releases like Java 11. The Java 11 builds will continue for 4 years, one year after the next major release (details). AdoptOpenJDK is a community group. They will provide builds provided that other groups create and publish security patches in a source repository at OpenJDK. Both IBM and Red Hat have indicated that they intend to provide those security patches.

AdoptOpenJDK OpenJ9 builds

https://adoptopenjdk.net/

In addition to the standard OpenJDK builds, AdoptOpenJDK will also be providing builds with OpenJ9 instead of HotSpot. OpenJ9 was originally IBM's JVM, but OpenJ9 is now Open Source at Eclipse.

Red Hat OpenJDK builds

Red Hat provides builds of OpenJDK via Red Hat Enterprise Linux (RHEL) which is a commercial product with paid-for support (details). They are very good at providing security patches back to OpenJDK, and Red Hat has run the security updates project of Java 6 and 7. The Red Hat build is integrated better into the operating system, so it is not a pure OpenJDK build (although you wouldn't notice the difference).

Other Linux OpenJDK builds

Different Linux distros have different ways to access OpenJDK. Here are some links for common distros: Debian, Fedora, Arch, Ubuntu.

Azul Zulu

https://zulu.org/

Zulu is a branded build of OpenJDK with commercial paid-for support. In addition, Azul provides some Zulu builds for $free as "Zulu Community", however there are no specific commitments as to the availability of those $free builds. Azul has an extensive plan for supporting Zulu commercially, including plans to support Java 9, 13 and 15, unlike any other vendor (details).

IBM

IBM provides and supports a JDK for Java 8 and earlier. They also provide commercial paid-for support for the AdoptOpenJDK builds with OpenJ9.

SAP

https://sap.github.io/SapMachine/

SAP provides a JDK for Java 10 and later under the GPL+CE license. They also have a commercial closed-source JVM. I haven't found any information on support lifetimes.

Others

There are undoubtedly other builds of OpenJDK, both commercial and $free. Please contact me if you'd like me to consider adding another section.

Summary

There are many different builds of OpenJDK, the original upstream source repository. Each build offers its own unique take - $free or commercial, branded or unbranded.

Choice is great. But if you just want the "standard", currently my best advice is to use the OpenJDK builds by Oracle, AdoptOpenJDK builds or the one in your Operating System (Linux).

Tuesday, 28 August 2018

Java is still available at zero-cost

The Java ecosystem has always been built on a high quality $free (zero-cost) JDK available from Oracle, and previously Sun. This is as true today as it always has been - but the new six-monthly release cycle does mean some big changes are happening.

Six-monthly releases

Java now has a release every six months, something which greatly impacts how each version is supported. By support, I mean the provision of update releases with security patches and important bug fixes.

Up to and including Java 8, $free security updates were provided for many years. Certainly up to and beyond the launch of the next version. With Java 9 and the six-monthly release cycle, this $free support is now much more tightly controlled.

In fact, Oracle will not be providing $free long-term support (LTS) for any single Java version at all from Java 11 onwards.

VersionRelease dateEnd of $free updates from Oracle
Java 8March 2014January 2019 (for commercial use)
Java 9Sept 2017March 2018
Java 10March 2018Sept 2018
Java 11Sept 2018March 2019 (might be extended, see below)
Java 12March 2019Sept 2019

The idea here is simple. Oracle wants to focus its energy on moving Java forward with the cost of long-term support directly paid for by customers (instead of giving it away for $free). To do this, they need developers to continually upgrade their version of Java, moving version every six months (and picking up the patch releases in-between). Of course, for most development shops, such rapid upgrade is not feasible. But Java is now developed as OpenJDK, which means that Oracle's support dates are not the only ones to consider.

OpenJDK

A key point to grasp is that most JDK builds in the world are based on the open source OpenJDK project. The Oracle JDK is merely one of many builds that are based on the OpenJDK codebase. While it used to be the case that Oracle had additional extras in their JDK, as of Java 11 this is no longer the case.

Many other vendors also provide builds based on the OpenJDK codebase. These builds may have additional branding and/or additional non-core functionality. Most of these vendors also contribute back to the upstream OpenJDK project, including the security patches.

The impact is that the JDK you use should now be a choice you actively make, not passively accept. How fast can you get security patches? How long will it be supported? Do you need to be able to apply contractual pressure to a vendor to help with any issues?

In addition, there are two main ways that the JDK is obtained. The first is an update mechanism buit into the operating system (eg. *nix). The second is to visit a URL and download a binary (eg. Windows).

To examine this further, lets look at Java 8 and Java 11 separately.

Staying on Java 8

If you want to stay on Java 8 after January 2019, here are the choices as I see them:

1) Don't care about security.

It is entirely possible to remain on the last $free release forever. Just don't complain when hackers destroy your company.

2) Rely on Operating System updates.

On *nix platforms, you may well obtain your JDK via the operating system (eg. Red Hat, Debian, Fedora, Arch, etc.). And as such, updates to the JDK are delivered via the operating system vendor. This is where Red Hat's participation is key - they promise Java 8 updates until June 2023 in Red Hat Enterprise Linux - but they also have an "upstream first" policy, meaning they prefer to push fixes back to the "upstream" OpenJDK project. Whether you get security patches to the JDK or not will depend on your operating system vendor, and whether they need you to pay for those updates.

3) Pay for support.

A number of companies, including Azul, IBM, Oracle and Red Hat, offer ongoing support for Java. By paying them, you get access to the stream of security patches and update releases with certain guarantees (as opposed to volunteer-led approaches). If you have cash, maybe it is fair and reasonable to pay for Java?

4) Use the non-commercial build in a commercial setting.

Oracle will provide builds of Java 8 for non-commercial use until December 2019, so you could use those. But you don't want Oracle's software licensing teams chasing you, do you?

5) Build OpenJDK yourself.

The stream of security patches * is published to a public Mercurial repository under the GPL license. As such, it is perfectly possible to build OpenJDK yourself by keeping track of commits to that repository. I suspect this not a very realistic choice for most companies.

6) Use the builds from AdoptOpenJDK.

The community team at AdoptOpenJDK has been busy over the past few years creating a build farm and testing rig. As such, they are now able to take the stream of security patches * and turn them into releases, just like you would get from the commercial offerings. They are also running the Java TCK (testing compatibility kit) to allow these builds to be fully certified as being compatible with the Java SE specification. Their plan is to produce Java 8 builds until September 2022, one year after Java 17 comes out. Obviously, you don't get a warranty or genuine support - its a community build farm project. But for most users that want to use Java 8 without paying, this is likely the best choice.

Note that Azul also offers $free OpenJDK release builds at zulu.org.

* The last two options assume that a group actually will step forward and take over the "JDK 8 updates" OpenJDK project once Oracle stop. While the exact project details are not yet confirmed, this IBM statement indicates real backing for the approach:

Recognizing the impact that the release cycle changes will have with Java developers, IBM will partner with other members of the OpenJDK community to continue to update an OpenJDK Java 8 stream with security patches and critical bug fixes. We intend to keep the current LTS version secure and high quality for 4 years. This timescale bridges the gap between LTS versions with 1 year to allow for a migration period. IBM has also invested in an open build and test project (AdoptOpenJDK.net) along with many partners and Java leaders to provide community binaries across commonly used platforms of OpenJDK with Hotspot and OpenJDK with Eclipse OpenJ9. These community binaries are TCK (Java SE specification) compliance tested and ready for developers to download and use in production.
IBM statement

And here is further indication of Red Hat's support for the June 2023 date, based on their "upstream first" policy.

> Red Hat has said it may step forward to be the maintainer for JDK 11 - might it also > step forward to be the maintainer for JDK 8?
Yes.
> How long will JDK 8 be maintained?
June 2023 is right for JDK 8, but I wouldn't be surprised if it goes on beyond that.
Andrew Haley, Red Hat

Note that the process around security issues will be managed by the newly formed vulnerability group (which formally codifies what was happening anyway).

Staying on Java 9 or Java 10

Don't.

No-one wants to provide builds or support for Java 9 or Java 10. And anyway, there is no good reason not to upgrade to Java 11 in my opinion.

(Actually, Azul are providing medium-term support for Java 9, if you really need it.)

Staying on Java 11

It is a brave new world and not 100% clear, but this is how it looks like things will happen.

Firstly, it is not clear that there will be an Oracle JDK that is $free to download. Despite my best attempts, I could not get 100% clarity on this point. However, this tweet and other sources indicate that it will be accessible for development purposes, just not for use in production (which is a bit of a trap for the unwary).

But in reality, it is now irrelevant as to whether Oracle JDK is $free to download or not. That is because from Java 11, developers can treat Oracle JDK and OpenJDK as being equivalent. (See here for the detailed differences.) It is no longer appropriate or correct to consider the OpenJDK build to be secondary or less important. In fact, the most important build is now the OpenJDK one.

To be more specific, as of the release date, Java 11 developers should consider using jdk.java.net to obtain a binary download, not any page on oracle.com.

So, for how long will Oracle provide security patches to Java 11?

Again, the answer to this is not 100% clear:

> What will Java 11 get from Oracle?
At least six months of free, GPL-licensed updates with binaries at http://jdk.java.net.
(I say “at least” because that’s the current plan. The plan could change, to a longer time period, if the situation warrants.)
Mark Reinhold, Oracle

Clearly, six months of security updates is not long enough to treat Java 11 as a "long term support" (LTS) release. The promise of the period being potentially longer doesn't really help, as companies need longer timelines and more certainty. So the working assumption should be that Java 11 has just 6 months of releases containing security patches from Oracle.

At this point, things move into the realms of probabilities. In all likelihood, when Oracle steps down from managing the "JDK 11 updates" project at OpenJDK (the one containing the security patches), someone else will take over. This has happened before with Java 6 and 7. And the evidence is that it will happen for Java 11 too:

OpenJDK is a community project. It's up to the community to support it. In practice this means that a group of organizations and individuals will maintain each OpenJDK LTS release for some period (TBA for 11, but it's sure to be a *lot* longer than six months.) I am certain that there will be a jdk11u project, and it will be properly and professionally run. I think it's likely that I'll be leading the project, but someone else may be chosen.
Andrew Haley, Red Hat

That covers the repository containing the security patches. (Red Hat have an excellent record in maintaining old releases of OpenJDK for the wider community.) But there is still the question of producing actual releases to download that have been certified as passing the Java SE testing TCK.

This is where the AdoptOpenJDK build farm is critical:

As part of the discussions Andrew mentioned, AdoptOpenJDK offered to build, test and make available OpenJDK LTS binaries for the major (and several minor) platforms. This isn't yet set in concrete but folks broadly thought that was a good idea. So the challenge of having a build and test farm for this joint effort is solved.
Martijn Verburg, AdoptOpenJDK

And AdoptOpenJDK are currently planning to do create releases until September 2022, one year after Java 17 comes out.

If people do what they say they will, then we can therefore conclude that it will be possible to use Java 11 for 4 years from release, with security patches, for $free (zero-cost). (I would imagine that if volunteers came forward, the September 2022 date could be moved even further into the future.)

Of course, only you and your company can decide if it is right and proper to use Java without giving back to the ecosystem. It could be argued that it is more ethical to either pay for support, or assist either the AdoptOpenJDK or "JDK 11 updates" OpenJDK project.

This is therefore the updated table of $free updates:

VersionRelease dateEnd of Oracle $free updatesEnd of OpenJDK-based $free updates
Java 8March 2014January 2019June 2023 (or later)
Java 9Sept 2017March 2018N/A
Java 10March 2018Sept 2018N/A
Java 11Sept 2018March 2019 (or later)September 2022 (or later)
Java 12March 2019Sept 2019N/A

(June 2023 is the date Red Hat has provided for the end of JDK 8 security patches, September 2022 is the date AdoptOpenJDK have provided - one year after the expected release of the next LTS (long-term support) version, Java 17).

Platforms

The OpenJDK builds by Oracle at http://jdk.java.net/ only cover three platforms. But this doesn't mean that they are the only platforms OpenJDK runs on. For example, AdoptOpenJDK provides Java 8 builds on 7 platforms with Hotspot (including ARM) and more platforms with OpenJ9 (including Windows 32 bit).

Summary

All the pieces are in place for Java 11 to be maintained as a long-term support release. However, unlike Java 6, 7 and 8, Oracle will not be leading the long-term support effort. In all likelihood, Red Hat will take over this task - they have said publicly that they would like to.

For the first 6 months of Java 11's life, Oracle will be providing GPL+CE licensed $free zero-cost downloads at jdk.java.net with security patches.

To get GPL+CE licensed $free zero-cost update releases of Java 11 after the first six months, you are likely to need to obtain them from a different URL and a different build team. Currently, my view is that AdoptOpenJDK build farm is the place to look for those builds. But zulu.org is another possibility.

Feel free to comment if I've missed something obvious.

Monday, 9 July 2018

Upgrading to Eclipse Photon

I use Eclipse as my Java IDE. And the new release, Photon is now out.

Photon is a large release, with lots of new features. The most important is the separation of the test and main classpaths, which has always been a point of pain in the IDE. Now it just works as you would expect, and the Maven plugin M2E correctly sets it up:

Note the darker colour of the src/test classpath elements.

Support for Java 9 (modules) and Java 10 (local variable type inferenece) is also present, ready for Java 11 in September. You can also use JUnit 5. It even tries to help you reach 100% code coverage!

All in all, I feel this is a release where upgrading will make a difference to everyday coding.

I've upgraded my own Eclipse installations, and it all went pretty well. You can either start from a clean install (I prefer the basic IDE without plugins so I can choose which ones to add). Or you can add Photon as an update site, and let Eclipse update itself.

One problem I had was the plugin that connects Maven (M2E) to Checkstyle (Eclipse-CS), known as m2e-code-quality. Fortunately, the team at GEBIT have been maintaining a fork of the original plugin. However, they don't release it in binary form. As such, I had to build the plugin locally (no big deal - its a simple build).

To simplify the process however, I've created a repository on GitHub with my Eclipse setup files, and a binary zip of the GEBIT forked plugin.

To use just the m2e-code-quality GEBIT fork, download the zip file and add it as an update site. Here are some instructions.

Thank you Eclipse team for a great release!


PS. I won't be answering "how to" questions about upgrading Eclipse or the eclipse-setup repository. There are plenty of other places to ask questions, such as Stack Overflow or the Eclipse Forums.

Thursday, 22 March 2018

JPMS modules for library developers - negative benefits

Java 9 introduced a major new feature - JPMS, the Java Platform Module System. After six months I've come to the conclusion that JPMS currently offers "negative benefits" to open source library developers. Read on to understand why.

Modules for library developers

Java 8 is probably the most successful Java release ever. It is widely used and widely liked. As such, almost all open source libraries run on Java 8 (as library authors want their code to be used!). Some libraries with a long history also still run on older versions. Joda-Convert has a Java 6 baseline, while Joda-Time has a Java 5 baseline. Others have a Java 8 baseline, such as ThreeTen-Extra.

Java 9 was released in September 2017, but it is not a release that will be supported for a number of years. Instead, it had a lifetime of six months and is now obsolete because Java 10 is out. And in six months time Java 11 will be out making Java 10 obsolete, and so on.

While most releases last six months, some are luckier. Java 11 will be a "long term support" (LTS) release with security and bug support for a few years (Java 8 is also an LTS release). Thus, even though Java 10 is out, Java 8 is still the sensible Java version for open source library developers to target right now because it is the current LTS release.

But what happens when Java 11 comes out? Since Java 8 will be unsupported relatively soon after Java 11 is released, you'd think that the sensible baseline would be 11. Unfortunately I believe many companies will be sticking with Java 8 for a long time. An aggressive open source project might move quickly to a Java 11 baseline, but doing so would be a risky strategy for adoption.

The module-path

Before discussing the JPMS options for open source library developers, it is important to cover the distinction between the class-path and the module-path. The class-path that we all know and love is still present in Java 9+, and it mostly works in the same way.

The module-path is new. When a jar file is on the module-path any module-info is used to apply the new stricter JPMS rules. For example, a public method is no longer callable unless it has been exported from the module it is contained in (and required by the caller's module).

The basic idea is simple, you put old fashioned non-modular jar files on the class-path, while you put modular jar files on the module-path. Nothing enforces this however, and it turns out this is a bit of a problem. There are thus four possibilities:

  • modular jar on the module-path
  • modular jar on the class-path
  • classic non-modular jar on the module-path
  • classic non-modular jar on the class-path

To be sure your library works correctly, you need to test it both on the class-path and on the module-path. For example, service loading is very different on the module-path compared to the class-path. And some resource lookup methods also work completely differently.

To complicate this further, JPMS has no explicit support for testing. In order to test a module on the module-path (which is a tightly locked down set of packages) you have to use the --patch-module command line flag. This flag effectively extends the module, adding the testing packages into the same module as the classes under test. (If you only test the public API, you can do this without using patch-module, but in Maven you'd need a whole new project and pom.xml to achieve that approach, so its likely to be rare.)

In the latest Maven surefire plugin (v2.21.0 and later) the patch-module flag is used, but if your module has optional dependencies, or you have additional testing dependencies, you may have to manually add them, see this issue and this issue.

Given all this, what should an open source library developer do?

Option 1, do nothing

In most cases, but not all, code that is compiled on Java 8 or earlier will run just fine on the class-path in Java 9+. So, you can do nothing and ignore JPMS.

The problem is that other projects will depend on your library. By not adopting JPMS at all, you block those projects from progressing in their modularization. (A project can only choose to fully modularize once all of its dependencies are modularized.)

Of course if your code doesn't run on Java 9+ because you've used sun.misc.Unsafe or something else you shouldn't have done then you've got other things to fix.

And don't forget that a user could put your jar file on the class-path or the module-path. Have you tested both? ie. The truth is that "do nothing" is not possible - at a minimum you have extra testing to do, even if it just to document that your project does't work on the module-path.

Option 2, add a module name

Java 9+ recognises a new entry in the MANIFEST.MF file. The Automatic-Module-Name entry allows a jar file to declare what name it will use if/when it is turned into a proper modular jar file. Here is how you can use Maven to add it:

 <plugin>
  <groupId>org.apache.maven.plugins</groupId>
  <artifactId>maven-jar-plugin</artifactId>
  <configuration>
   <archive>
    <manifestEntries>
     <Automatic-Module-Name>org.foo.bar</Automatic-Module-Name>
    </manifestEntries>
   </archive>
  </configuration>
 </plugin>

This is a nice simple way to move forward. It reserves the module name and allows other projects that depend on your jar file to fully modularize if they wish.

But because its so simple, its easy to forget the testing aspect. Again, your jar file might be placed on the class-path or on the module-path, and the two can behave quite differently. In fact, now that it has some module information, tools may treat it differently.

When Maven sees an Automatic-Module-Name it will normally place the classes on the module-path instead of the class-path. This may have no effect, or it may show up a bug where your code works on the class-path but not on the module-path. With Maven right now, you have to use surefire plugin v2.20.1 to test on the class-path (an old version that doesn't know about the module-path). To test on the module-path, use v2.21.0. Swapping between these two versions is of course a manual process, see this issue for a request to improve this.

While upgrading some of my projects I added Automatic-Module-Name without testing on the module-path. When I did eventually test on the module-path the tests failed, as the code simply didn't work on the module-path. Unfortunately, I now have some releases on Maven-Central that have Automatic-Module-Name but don't work on the module-path, happy days...

To emphasise this, just adding something to the MANIFEST.MF file can have an effect on how the project is run and tested. You need to test on both the class-path and module-path.

Option 3, add module-info.java

This is the full modularization approach described in numerous web pages and tutorials on JPMS.

 module org.foo.bar {
   requires org.threeten.extra;
   exports org.foo.bar;
   exports org.foo.bar.util;
 }

So, what are the implications of doing this to the open source project?

Unlike option 2, your code now has a baseline of Java 9+. The Java 8 compiler won't understand the file. What we really want is a jar file that contains Java 8 class files, but with just the module-info.java file compiled under Java 9+. In theory, when running on Java 8 the module-info.class file will be ignored if it is not used.

Maven has a technique to achieve this. While the technique works OK, it turns out to be nowhere near sufficient to achieve the goal. To actually get a single jar file that works on both Java 8 and 9+ you need:

  • use the release flag on Java 9+ to build for Java 8
  • add an OSGi require capability filter to inform it that its still Java 8 compatible
  • exclude module-info.java from maven-javadoc-plugin when building on Java 8
  • use maven-javadoc-plugin v3.0.0-M1 (not later), manually copy dependencies to a directory and refer to them using additional Javadoc command line arguments, see this issue
  • exclude module-info.java from maven-checkstyle-plugin
  • exclude module-info.java from maven-pmd-plugin
  • manually swap the version of maven-surefire-plugin to test both the module-path and the class-path

And probably some more I've forgotten about. Here is one pom.xml before integrating Java 9. Here it is after integrating Java 9. An increase from 650 to 862 lines, with lots of complexity, profiles and workarounds.

With a Java 11 baseline, the project would be simpler again, but that baseline isn't going to happen for a number of years. Note that my comments should not be interpreted as anti-Maven. A small team there is working hard to do the best they can - JPMS is complex.

And just for kicks, your project can no longer be used by Android (as the team there seems to be very slow in adding a simple "ignore module-info" rule). And many tools with older versions of bytecode libraries like ASM will fail too - I had a report that a particular version of Tomcat/TomEE could not load the modular jar file. I've ended up having to release a "classic" non-modular jar file to cope with these situations, something which is profoundly depressing.

While I've added module-info.java to some of my projects, I cannot recommend others to do so - its a very painful and time-consuming process. The time to consider it would appear to be once Java 11 or beyond is widely adopted and the baseline of your project.

Negative benefits

Now for the controversial part.

It is my conclusion that JPMS, as currently designed, has "negative benefits" for open source libraries.

As explained above, the cost of full modularization is high for library developers. The need to retain Java 8 compatibility makes JPMS really hard to use (module information should have been textual, not a class file). The tooling is still incomplete/buggy. Many older projects can't cope with the new jar files if you do go for it. Much of this will improve over time, but we're talking a number of years before Java 11 is widely adopted. But don't be lulled into just believing waiting will solve the key problem.

The split (bifurcation) of the module-path from the class-path is an absolute nightmare. At a stroke, there are now two different ways that your library can be run, and the two environments have quite different qualities. Code that compiles and runs on the class-path will often not compile or not run on the module-path. And vice versa. As a library author, you cannot control whether the class-path or module-path is used. You have no choice - you must test both, which you probably won't think to do. (And Maven currently provides no way to test both in one pom.xml)

Given all this effort and extra complexity, we should be getting some great benefits, right? Well no.

JPMS is supposed to ensure reliable configuration (that all your dependencies are available at startup) and strong encapsulation (that other code can't see or use packages that you want to keep hidden). But since there is no way to stop your modular jar file being used on the class-path, you get none of these benefits.

Did you put lots of effort into choosing which packages to hide? Meaningless, as the user can just put the jar file on class-path and call your internal packages. Did you believe that the JVM will check all your dependent modules are available before starting? Afraid not, no checks performed when the user puts the jar file on class-path.

Since we get none of the claimed benefits of JPMS, but get lots of extra work in testing and complexity in the build tools, I feel "negative benefits" is a pretty accurate summary.

Summary

As of today, JPMS is a pain for library authors. The split of the module-path from the class-path is at the heart of the problem. You really can't afford to release a library without testing on both module-path and class-path - there are subtle corner cases where the environments differ.

What is desperately needed is a small change to JPMS. There needs to be a way for a library author to insist that the modular jar file they are producing can only be run on the module-path (with any attempt to use it on the class-path preventing application startup). This would eliminate the need for testing both class-path and module-path. Together with the passage of time, JPMS might yet achieve its goals and go from negative to positive benefits.

Monday, 5 February 2018

Java 9 has six weeks to live

Java 9 is obsolete in just six weeks (20th March 2018). What? You haven't upgraded yet? Well, Java 10 is only going to last six months before it is obsolete too.

Update 2018-03-20: Java 10 is released. Java 9 is obsolete.

Release train impact

The new Java release train means that there will be a new release of Java every six months. And when the next release comes out, the previous release is obsolete.

What do I mean by obsolete?

In practical terms it means that there are no more security updates from Oracle. (Theoretically, the OpenJDK community could release security updates, but there is no sign of this yet). And since you don't want to run your software without the latest security updates, you are expected to upgrade to Java 10 as soon as it is released.

As a user of Java, here are three possible ways to approach the release train:

  1. Stay on Java 8, the current LTS (long term support) release, until the next LTS release occurs (Java 11)
  2. Move from Java 9 to Java 10 to Java 11, making sure you update rapidly to get the security updates
  3. Stay on Java 9 (or Java 10) and don't worry about security updates

If you have already moved to Java 9, you have effectively committed to option 2 or 3. If you care about security updates, you need to be prepared to switch to Java 10 shortly after it is release on 20th March. To do this, you probably should be testing with a Java 10 pre-release now. If you find that to be a challenge, you have to stop caring about security, or consider going back to Java 8 LTS.

However you look at it, being on the release train is a big commitment.

  • Will your dependencies work on the next version?
  • Will your IDE be ready?
  • Will your build tool (Maven, Gradle etc.) be ready?
  • Will your other tools (spotbugs, checkstyle, PMD etc.) be ready?
  • How fast are you going to be able to update when the release you are on is obsolete?

Lots to consider. And given the number of external tools/dependencies to consider, I think its fair to say that its a bold choice to use Java 9 or 10.

Summary

With a release every six months, it is important to decide on an approach to the release train. If you want to upgrade every six months, great! But you'll need to test pre-releases of Java with your whole toolchain in advance of the release to ensure you don't get stuck on an unpatched obsolete release of Java.