Saturday, 15 November 2014

Devoxx 2014 - Whiteboard votes

As is traditional, the attendees of Devoxx vote on certain topics on whiteboards. Each result must be treated with care - there is no validation process, and only a fraction of the 3500 attendees actually bother to vote. Nevertheless, the results can be interesting.

What version of Java do you use at work right now?

A high level question focussed on Java at the day job. Java 8 hasn't surpassed Java 6 yet, but it is doing pretty well. Remember that Java 7 is End-Of-Life in April 2015!

Java 91
Java 863
Java 7160
Java 681
Java 511
Java 43
Java 31
Dalvik6
Other4


What IDE do you use?

A sense of how the IDE market is looking. The interesting data point is how Eclipse is falling - compare with 2008, where Eclipse led IntelliJ by almost 2 to 1, and NetBeans by almost 4 to 1. Sadly, the absolute number of votes is a lot lower than 6 years ago, so be careful with comparisons.

Eclipse36
NetBeans26
IntelliJ75
Vi/vim8
Other1

Beyond this, I heard various people comment on IBM's apparent withdrawal of funding for Eclipse IDE and what that might mean looking forward.

What JVM language are you using?

An attempt to see what languages people are using. Whenever this is asked, it is always an opportunity for advocates of a language to push hard, so again take the results with a good pinch of salt:

Java214
Scala64
Groovy62
Clojure15
Rhino/JS6
Ceylon3
Kotlin1
Cobol1


Is merging Scala into Java a good idea?

An odd question with a predictable answer - WTF!

Yes3
No26
WTF?! seriously?52
Depends2


What JavaScript framework/library do you use?

An open question that showed a clear dominance for Angular.

Angular51
Ember4
Reactive2
Backbone6
Knockout7
Angular-Dart3
Dojo5
JQuery9
Meteor2


Is this the end for Angular?

Angular 2.0 - "no migration path, not backwards compatible". Whether this is factually correct or not, this was the question asked.

Yes4
No47
Maybe12
Hate JavaScript24
Follow GWT1


What desktop UI technology have you used in the last 3 years?

A sense of where the desktop UI is today. Given that JavaFX has only been truly useful in Java SE 8, its not a bad showing for FX.

None39
Swing41
JavaFX15
MS .NET10
Chrome apps2
Eclipse RCP14
NetBeans platform2
QT5
VCL1
CLI4
OGLPHI1
Applet2
Broswer/HTML7


How do you create WebApps?

An open question on web technologies. Bear in mind that some options were added after others, notably JAX-RS+JavaScript, which was added long after REST+JavaScript. I think its fair to say that there are an odd mix of answers, but a clear leader in REST + JavaScript.

JSF+JavaEE44
REST+JavaScript101
JAX-RS+JavaScript9
Play23
Swing MVC76
Wicket26
JavaScript library50
Grails8
GWT30
Vaadin20
Apache Sling6
WebSockets+JavaScript6
Spray+AkkaHTTP6
Ruby on Rails2
ZK framework4
OmniFaces4
PrimeFaces10
REST+Dart2
Fluent-HTTP1
AngularJS23
Polymer3
By hand, with love2
Web motion3
Struts v1 or v211
Beehive2
JRuby3
Bootstrap2


What one thing would you add to Java (your top priority)?

A traditional Devoxx question, with a long shopping list of answers, as always. Note however that Brian Goetz indicated strongly that properties as a language feature is not happening.

Properties54
Better handling of null33
Multi-line strings33
Safe null dereference23
Interpolation of strings22
Immutability21
Value types16
Unsigned int11
Too bloated!8
Better logging7
Pattern matching8
Sub-package scope4
Tail recursion3
Solve expression problem2
Algorithmic data types2
Class versioning2
Cluster aware JVM1
Performance1
Free$ embedded support1
Inner functions1
Generators and co-routines1
Operator overloading1 (plus 2 minus 1 !)
Enterprise modularity1


Summary

Lots of interesting, and no doubt slightly dubious, statistics to examine. Use with care, and congratulations to the Devoxx team for another great conference!

Thursday, 13 November 2014

No properties in the Java language

Here at Devoxx 2014, Brian Goetz went a little further than he has before about whether properties will ever be added to Java as a language feature.

No properties for you

Properties as a language feature in Java has been talked about for many years. But ultimately, it is Oracle, the Java stewards, who get to decide whether it should be in or out.

Today, at Devoxx, Brian Goetz, Java Language Architect, went a little further than he has before to indicate that he sees little prospect of properties being added to the language at this point.

Oracle have chosen to prioritize lambdas, modularization and value types as key language features over the JDK 8 to 10 timescale. As such, there was no realistic prospect of properties before JDK 11 anyway. However, the indications today were reasonably clear - that it isn't on the radar at all.

When quizzed, Brian indicated two main reasons. First, that it is perhaps too late for properties to make a difference as a language feature, with compatibility with older getter/setter beans being a concern. Second, that there are many different, competing visions of what properties means, and there is an unwillingness to pick one.

Competing visions include:

  • Just generate getters, setters, equals, hashCode and toString
  • Generate observable properties for GUIs
  • Raising the level of abstraction, treating a property as a first class citizen
  • Immutability support, with builders for beans
  • New shorter syntax for accessing properties

While it is easy to argue that these are competing visions, it is also possible to argue that these are merely different requirements, and that all could be tackled by a comprehensive language enhancement.

(Just to note that "beans and properties" does not imply mutable objects. It is perfectly possible to have immutable beans, with or without a builder, but immutable beans are harder for frameworks to deal with. Value types are intended for small values, not immutable beans.)

As is well known, I have argued for properties as a language feature for many years. As such, I find the hardening of the line against properties to be intensely disappointing.

Every day, millions of developers interact with beans, getters, setters and the frameworks that rely on them. Imagine if someone today proposed a naming convention mechanism (starts with "get") for properties? It would be laughable in any language, never mind the world's most widely used enterprise development language.

Instead, developers use IDEs to generate getters, setters, equals and hashCode, adding no real value to their code, just boilerplate of the kind Java is notorious for.

Consider that with 9 million Java developers, there could easily be 50,000 new beans created every day, each with perhaps 100 lines of additional boilerplate getters/setters/equals/hashCode/toString - easily 5 million lines of pointless code created, per day.

(And plenty more code in serialization and mapping code that either tries to deduce what the properties are, or results in lots of additional manual mapping code)

Realistically however, we have to accept that in all likelihood Java will never have properties as a language feature.

Summary

The Java language is not getting properties.

Comments welcome, but please no "Foo language has properties" comments!

Wednesday, 12 November 2014

Converting from Joda-Time to java.time

What steps are needed to upgrade from Joda-Time to Java SE 8 date and time (JSR-310)?

From Joda-Time to java.time

Joda-Time has been a very successful date and time library, widely used and making a real difference to many applications over the past 12 years or so. But if you are moving your application to Java SE 8, its time to consider moving on to java.time, formerly known as JSR-310.

The java.time library contains many of the lessons learned from Joda-Time, including stricter null handling and a better approach to multiple calendar systems. I use the phraseology that java.time is "inspired by Joda-Time", rather than an exact derivation, however many concepts will be familiar.

Converting types

Classes not including a time-zone:

Joda-Time java.time Notes
LocalDate LocalDate Same concept - date without time or time-zone
YearMonthDay
LocalTime LocalTime Same concept - time without date or time-zone
TimeOfDay
LocalDateTime LocalDateTime Same concept - date and time without time-zone
MonthDay MonthDay Same concept - month and day without time-zone
YearMonth YearMonth Same concept - year and month without time-zone
- Year New concept - a value type for the year
- Month New concept - an enum for the month-of-year
- DayOfWeek New concept - an enum for the day-of-week
Partial - Not included in java.time

Classes including a time-zone or representing an instant:

Joda-Time java.time Notes
DateTime ZonedDateTime Same concept - Date and time with time-zone
OffsetDateTime New concept - Date and time with offset from Greewich/UTC
MutableDateTime - Not included in java.time, use immutable ZonedDateTime instead
DateMidnight - Deprecated as a bad idea in Joda-Time, Use LocalDate or ZonedDateTime
- OffsetTime New concept - Time with offset from Greewich/UTC
Instant Instant Same concept - Instantaneous point on timeline
DateTimeZone ZoneId Same concept - Identifier for a time-zone, such as Europe/Paris
ZoneOffset New concept - An offeset from Greenwich/UTC, such as +02:00

Amounts of time:

Joda-Time java.time Notes
Duration Duration Same concept - Amount of time, based on fractional seconds
Period Period and/or Duration Similar concept - Amount of time
Joda-Time Period includes years to milliseconds, java.time only year/month/day
MutablePeriod - Not included in java.time, use immutable Period or Duration instead
Years - Not included in java.time, use Period instead (or ThreeTen-Extra)
Months
Weeks
Days
Hours - Not included in java.time, use Duration instead
Minutes
Seconds
Interval - Not included in java.time
MutableInterval - Not included in java.time

Formatting:

Joda-Time java.time Notes
DateTimeFormatter DateTimeFormatter Same concept - an immutable formatter
DateTimeFormatterBuilder DateTimeFormatterBuilder Same concept - a builder of the formatter
DateTimeFormat DateTimeFormatter Concepts exposed as static methods on DateTimeFormatter
ISODateTimeFormat
PeriodFormatter - Not included in java.time
PeriodFormatterBuilder
PeriodFormat
ISOPeriodFormat

Other classes and interfaces:

Joda-Time java.time Notes
- Clock New concept - Standard way to pass around the current time
Chronology Chronology Similar concept - Very different API and implementation
ISOChronology IsoChronology Similar concept - Very different API and implementation
DateTimeFieldType ChronoField Similar concept - Very different API and implementation
DateTimeField
DurationFieldType ChronoUnit Similar concept - Very different API and implementation
DurationField
PeriodType - Not included in java.time
Readable* Temporal* The Readable* interfaces are most closely replaced by the Temporal* interfaces
It is strongly recommended to use the value types, not the temporal interfaces

Odds and Ends

In most cases, the table above will be sufficient to identify the class to convert to. After that, in most cases the method needed will be obvious. Some special cases are noted below.

Rounding. Joda-Time has considerable support for rounding, java.time has less, based only on truncation. See the truncatedTo() methods.

End of month. Joda-Time has the withMaximumValue() method on the property object to change a field to the last day of the month. java.time has a more general mechanism, see TemporalAdjusters.lastDayOfMonth().

Nulls. Joda-Time accepts null as an input and handles it with a sensible default value. java.time rejects null on input.

Construction. Joda-Time has a constructor that accepts an Object and performs type conversion. java.time only has factory methods, so conversion is a user problem, although a parse() method is provided for strings.

Time between. Joda-Time provides methods on amount classes such as Days.between(a, b). java.time provides a similar method on the ChronoUnit classes - ChronoUnuit.DAYS.between(a, b).

Additional fields. Joda-Time provides a fixed set of fields in DateTimeFieldType. java.time allows the set of fields to be extended by implementing TemporalUnit, see IsoFields, JulianFields and WeekFields.

Summary

This blog post has outlined the key mappings between Joda-Time and java.time in Java SE 8. If you are writing code in Java SE 8, its time to migrate to java.time, perhaps using ThreeTen-Extra to fill any gaps.

Monday, 10 November 2014

One more library for JDK 9?

On Tuesday night in Belgium at Devoxx I'm asking the question "Is there room for one more library in JDK 9"?

With JDK 9 full of modules, and the long wait until JDK 10 for more interesting things like value types, I pose this question to the community to ask what would you add to JDK 9 if you could?

But, I'm asking more than that. Any additional work for JDK 9 would have to be led by someone outside Oracle capable of drawing together a community. And whatever it is it would need to be tightly focussed - the timescale available is limited.

Think of something that could be expressed in 20 classes or less. Involves only code and no library or JVM changes. Something that the JDK doesn't have, but you think it should. Think about whether you'd put in your time to try and make it happen.

But also think about whether your idea should be in the JDK? Is it better as an external library? What makes it necessary to be in the JDK, especially a modularized JDK? Will Oracle's Java stewards support the idea?

As an example, I'll put forward Joda-Convert, a very small and simple library that provides a mechanism to convert and round-trip a simple object (like a VALJO) to a formatted string, ideal for use in serialization, such as JSON or XML (and noting that JSON is coming to JDK 9...). I'm sure that if you're reading this, you've got some other ideas!

If you've got any thoughts, ideas, or want to suggest something, please add a comment. But remember the rules above - there is no free lunch here. Are you willing to take a lead on your idea?

Thursday, 6 November 2014

Better nulls in Java 10?

Rethinking null when Java has value types.

Null in Java

Was null really the billion dollar mistake? Well, thats an interesting question, but what is certain is that null is widely used in Java, and frequently a very useful concept. But, just because it is useful doesn't mean that there isn't something better.

Optional everywhere!

There are those that would point you to adopt the following strategy - avoid using null everywhere, and use Optional if you need to express the possible absence of something. This has a certain attraction, but I firmly believe the Java is not yet ready for this (in Java SE 8) as per my previous post on Optional.

Specifically, all the additional object boxes that Optional creates will have an effect of memory and garbage collection unless you are very lucky. In addition, its relatively verbose in syntax terms.

Nullable annotations

A second approach to handling null is to use the @Nullable and @NonNull annotations. Except that it is only fair to point out that from an engineering perspective, annotations are a mess.

Firstly, there is no formal specification for nullable annotations. As such, there are multiple competing implementations - one from FindBugs, one from JSR-305, one from IntelliJ, one from Eclipse, one from Lombok and no doubt others. And the semantics of these can differ in subtle ways.

Secondly, they don't actually do anything:

  @javax.annotation.ParametersAreNonnullByDefault
  public class Foo {
    public StringBuilder format(String foo) {
      return new StringBuilder().append(foo);
    }
    public StringBuilder formatAllowNull(@Nullable String foo) {
      return new StringBuilder().append(foo);
    }
  }

Here, I've use @javax.annotation.ParametersAreNonnullByDefault to declare that all parameters to methods in the class are non-null unless otherwise annotated, that is they only accept non-null values. Except that there is nothing to enforce this.

If you run an additional static checker, like FindBugs or your IDE, then the annotation can warn you if you call the "format" method with a null. But there is nothing in the language or compiler that really enforces it.

What I want is something in the compiler or JVM that prevents the method from being called with null, or at least throws an exception if it is. And that cannot be switched off or ignored!

The nice part about the annotations is that they can flip the default approach to null in Java, making non-null the default and nullability the unusual special case.

Null with value types in JDK 10

The ongoing work on value types (target JDK 10 !!) may provide a new avenue to explore. Thats because Optional may be changed in JDK 10 to be a value type. What this might mean is that an Optional property of a Java-Bean would no longer be an object in its own right, instead it would be a value, with its data embedded in the parent bean. This would take away the memory/gc reasons preventing use in beans, but there is still the syntactic overhead.

Now for the hand-waving part.

What if we added a new psuedo keyword "nonnull" that could be used as a modifier to a class. It would be similar to @javax.annotation.ParametersAreNonnullByDefault but effectively make all parameters, return types and variables in the class non-null unless modified.

Then, what if we introduce the use of "?" as a type suffix to indicate nullability. This is a common syntax, adopted by Fantom, Ceylon and Kotlin, and very easy to understand. Basically, wherever you see String you know the variable is non-null and wherever you see String? you know that it might be "null".

Then, what if we took advantage of the low overhead value type nature of Optional (or perhaps something similar but different) and used it to represent data of type String?. Given this, a "nonnull" class would never be able to see a null at all. This step is the most controversial, and perhaps not necessary - it may be possible for the compiler and JVM to track String and String? without the Optional box.

But the box might come in useful when dealing with older code not written with the "nonnull" keyword. Specifically, it might be possible to teach the JVM to be able to auto box and un-box the Optional wrapper. It might even be possible to release a standard annotation jar that projects could use on codebases that need to remain compatible with JDK 8 or 9, where the annotation could be interpreted by a JDK 10 JVM to provide similar nullability information.

  public nonnull class Foo {
    public StringBuilder format(String foo) {
      return new StringBuilder().append(foo);
    }
    public StringBuilder formatAllowNull(String? foo) {
      return new StringBuilder().append(foo);
    }
  }

The example above has been rewritten from annotations to the new style. It is clearly a shorter syntax, but it would also benefit from proper integration into the compiler and JVM resulting in it being very difficult to call "format" with a null value. Behind the scenes, the second method would be compiled in bytecode to take a parameter of Optional<String>, not String.

The hard part is the question of what methods can be called on an instance of String?. The simple encoding suggests that you can only call the methods of Optional<String>, but this might be a little surprising given the syntax. The alternative is to retain the use of null, but with an enforced check before use.

The even harder part is what to do with things like Map.get(key) where null currently has two meanings - not found and found null.

This concept also appears to dovetail well into the value type work. This is because value types are likely to behave like primitives, and thus cannot be null (the value type has to be boxed to be able to be null). By providing proper support for null in the language and JVM, this aspect of value types, which might otherwise be a negative, becomes properly handled.

It should also be pointed out that one of the difficulties that current tools have in this space is that the JDK itself is not annotated in any way to indicate whether null is or is not allowed as an input or return type. Using this approach, the JDK would be updated with nullability, solving that big hurdle.

(As a side note, I wrote up some related ideas in 2007 - null-safe types and null-safe invocation.)

Just a reminder. This was a hand-wavy thought experiment, not a detailed proposal. But since I've not seen it before, I thought it was worth writing.

Just be careful not to overuse Optional in the meantime. Its not what it was intended for, despite what the loud (and occasionally zealot-like) fans of using it everywhere would have you believe.

Summary

A quick walk through null in Java, concluding with some hand-waving of a possible approach to swapping the default for types wrt null.

Feel free to comment on the concept or the detail!

Wednesday, 5 November 2014

Optional in Java SE 8

Java SE 8 added a new class to handle nullability - Optional - but when should it be used?

Optional in Java SE 8

Optional is not a new concept, but an old one which often results in polarized opinions. It was added to Java 8 and is used in limited places at the moment.

The class itself is a simple box around an object, which has a special marker for empty instead of using null:

  Optional<String> optional = Optional.of("Hello");
  Optional<String> empty = Optional.empty();

The goal of the class is to reduce the need to use null. Instead, Optional.empty() takes the place of null. One way to think of it is as a collection that holds either zero or one objects.

When you want to get the data out of the optional, there is a nice simple method, get(), but this will throw an exception if the optional is empty. In most cases, you will want to call one of the other methods, for example to return a default value:

  Optional<Person> personResult = findPerson(name);
  Person person = personResult.orElse(Person.GUEST);

One approach to using Optional is to use it everywhere in your entire codebase and eliminate null altogether. I don't subscribe to this approach, and nor does Brian Goetz.

Of course, people will do what they want. But we did have a clear intention when adding this feature, and it was not to be a general purpose Maybe or Some type, as much as many people would have liked us to do so. Our intention was to provide a limited mechanism for library method return types where there needed to be a clear way to represent "no result", and using null for such was overwhelmingly likely to cause errors.

The key here is the focus on use as a return type. The class is definitively not intended for use as a property of a Java Bean. Witness to this is that Optional does not implement Serializable, which is generally necessary for widespread use as a property of an object.

Where it is used as a return type, Optional can work very well. Consider the case where you are finding the first matching element in a stream. In this case, what should the stream API do if the stream is empty? Returning Optional makes it clear that this case should be considered when using the API.

For me, the pleasant surprise about Optional is that it has three friends - OptionalInt, OptionalLong and OptionalDouble that wrap an int, long and double. I've made good use of these as return types where previously I would have used Integer, Long and Double. Not only is the increase in meaning helpful, but the ability to call orElse(0) to default the missing value is a useful feature for API users.

My only fear is that Optional will be overused. Please focus on using it as a return type (from methods that perform some useful piece of functionality) Please don't use it as the field of a Java-Bean.

(As a side note, one reason not to use it in this way is that it creates an extra object for the garbage collector. If used as a method return type, the extra object is typically short-lived, which causes the gc little trouble. But when used in a Java-Bean, the object is likely to be long-lived, with a memory/gc overhead. I was bitten by something very similar to this about 13 years ago, and I can categorically say that for server systems running at scale, you don't want lots of little object boxes within your beans. Interestingly, if Optional becomes a value type in a future Java version, then the memory/gc issue goes away.)

Optional in Java SE 8

Java 8 Optional, and its three primitive friends, is very useful when used as a return type from an operation where previously you might have returned null. As written, the class is is not intended for use as a Java-bean property - please don't use it for that.

Sunday, 7 September 2014

Oracle RDBMS empty string and null

The Oracle database (RDBMS) has a "wonderful quirk" - it treats null and empty string as one and the same.

Oracle database empty string

If you have never used Oracle RDMBS you may not be aware of this particular "feature". Basically, if you insert an empty string into the database it is treated as null. Thus, a column declared as "NOT NULL" will reject the insertion of an empty string. For me, and probably most other developers, this isn't what you expect.

The Stack Overflow explanation is "I believe the answer is that Oracle is very, very old." As well as being amusing, it is as good an explanation as any other.

Last week, I had the dubious pleasure of porting the OpenGamma database infrastructure to work on Oracle. It already worked on various other databases (including HSQLDB, Postgres and SQL Server) but Oracle is rather an outlier in the world of SQL.

Most of the conversion went fine with ElSql able to handle any deviations from standard SQL. But the problem of empty strings and null was a little more vexing.

The first question was whether we could avoid needing to identify the difference between null and an empty string? I believe that if Oracle is your primary database, then you could code your application in that way. However, if you are coding the application to work on multiple different databases then this is simply not an option.

The second question was of course how to handle the problem? My research threw up three realistic possibilities:

  1. Prefix all strings by the same character, such as '@'. The string 'Stephen' would be stored in the database as '@Stephen', while the empty string would be stored as '@'.
  2. Replace the empty string with a magic value, such as '!!! EMPTY !!!'.
  3. Store an additional flag column indicating the difference between null and the empty string.

Option 1 means that every string in the database is polluted, making it hard to use the data directly by SQL (bypassing the application). Option 2 relies on the magic value never occurring in real data, and direct SQL access is again compromised (though less than option 1). Option 3 is the most "pure" solution, but would have been very hard to adopt when writing one piece of code for multiple different databases.

My chosen solution was a variation on option 2 - encoding using "one extra space". I'm documenting it here in case anyone else finds it to be a useful strategy:

  • The empty string is stored in the database as a single space (ASCII 32).
  • Any string that consists only of spaces (ASCII 32) is stored with one additional space added.
  • Any other string is stored without change

The advantage of this encoding is that the vast majority of strings are stored without being changed. This makes access by direct SQL simple and obvious. The only strings to be encoded are the empty string, and the unlikely "all space" strings. The encoding will make little difference to most algorithms or displays even if it is not decoded. The encoding is also fully reversible, provided that column length limits are not hit.

Fortunately, I was able to encode and decode the data in relatively few places. For decoding, a decorated ResultSet worked effectively. For encoding, it was possible to create a subclass of Spring's NamedParameterJdbcTemplate and JdbcTemplate to do the trick. See this commit for details.

As a final note, there are other complications with data storage in Oracle. As I understand it, NaN and null also cannot be separated. This blog only covers problems of strings, which is tricky enough!

Summary

I will continue to believe that an empty string and null are two different concepts. Oracle RDBMS disagrees. If you face the problem of creating a workaround, perhaps the option I used is worth considering.