Tuesday, 6 January 2009

Java 7 - Null-default and Null-safe operators

The most popular small language change request is better handling of nulls. As a result, I've updated my null-handling proposal.

Enhanced null-handling

The votes from Devoxx and JavaEdge were clear. Ordinary developers find handling nulls to be a pain and they would like to see language change to address it. At JavaEdge in particular almost a third of the first preferences and two thirds of the top four preferences went to null-handling.

I blogged my original thoughts two years ago. The updated null-default and null-safe invocation proposal is now available (v0.2).

The proposal covers two new operators, which follow Groovy for syntax (and thus consistency).

The null-default operator ?: returns the LHS unless that is null in which case it returns the RHS:

// today
  String str = getStringMayBeNull();
  str = (str != null ? str : "");
  
  // with null-default operator
  String str = getStringMayBeNull() ?: "";

The null-default operator also works particularly well with auto-unboxing.

Integer value = getIntegerMayBeNull();
  int val = value ?: -1;

In fact, I hope that tools (IDEs and static analysis) would combine to effectively make the null-default operator mandatory when unsafe unboxing is occuring.

The null-safe operator ?. is an alternative form of method/field invocation. If the LHS is null, then the result of the whole method/field invocation is null:

// today
  String result = null;
  Foo foo = getFooMayBeNull();
  if (foo != null) {
    Bar bar = foo.getBarMayBeNull();
    if (bar != null) {
      result = bar.getResult();
    }
  }
  
  // with null-safe operator
  String result = getFooMayBeNull()?.getBarMayBeNull()?.getResult();

There is an interesting case if the last segment of the field/method expression returns a primitive. However, this can be handled by combining the two new operators (hence why its one proposal):

// today
  int dotIndex = -1;
  if (str != null) {
    dotIndex = str.indexOf(".");
  }
  
  // with null-safe and null-default operators
  int dotIndex = str?.indexOf(".") ?: -1;

I considered mandating this when a primitive occurs, but it would have undesirable side effects, so this is one best left to the tools to handle.

More details on all of this in the proposal.

Summary

Nulls remain a pain in Java. There simply isn't enough language support for this most common task. Every day, developers write logic to check and handle nulls and this obscures the meaning of the real code. And every day production systems go down due to NullPointerExceptions that weren't caught. This proposal provides a simple enhancement to Java that tackles the heart of the null issue.

Opinions welcome.

20 comments:

  1. I question that this "tackles the heart of the null issue".

    It's just a bit of extra syntactic sugar... I would suggest that perhaps something like functionaljava.org 's Option is a better solution:

    http://functionaljava.googlecode.com/svn/artifacts/2.17/javadoc/fj/data/Option.html

    ReplyDelete
  2. It really should cut down on the boiler code that is induced for NULL checks over.

    Using functionaljava's Option is a only a solution when you are dealing with your own code, but development spanning multiple third-party libraries, it probably won't be an always an option. Second, as shown in the above example primitive wrappers and (un)boxing is taken care of, but with Option it won't be possible.

    ReplyDelete
  3. I say we go whole hog and nail this problem once and for all: Move nullity into the type system. This is more complex than simply having 'allows null' and 'does not allow null' in the syntax, just like generics has wildcars, you also need 'I don't care if null is allowed or not', which is different from 'allows null'.

    But, the good news is, the strangeness is no different from what we're already used to with generics, so in the end its not hard to grok, and it definitely solves the problem. Full proposal writeup here: http://www.zwitserloot.com/2008/10/23/non-null-in-static-languages/

    ReplyDelete
  4. +1 for enhanced null handling
    -1 for "move nullity into the type system"

    ReplyDelete
  5. Stephen Colebourne6 January 2009 at 12:57

    Daniel: One look at that javadoc URL would scare away 99% of developers IMHO. Functional programming is far from the norm today, and far from what Java is intended to be (ie. forcing functional paradigms into Java doesnn't make sense).

    Reinier: I discuss nullity in types in the proposal. I believe that this are a good idea, however it is specifically not possible for Java 7 (type changes are not permitted as they are not 'small'). Further, the Fan language demonstrates that the proposed null-safe and null-default features are still needed even when you have nullity in types. I also think that your proposal of 'don't care' adds additional complexity and is probably not needed.

    ReplyDelete
  6. I think JSR 308 _is_ about adding Nullity to the type system... with the @Nullable and @NonNull annotations. You can set a default nullity for your source and then effective have all fields and variables marked @NonNull unless you specifically override.
    I realize you're asking for something slightly different than this... but I spend a few hours looking at the JSR 308 spec and FAQ and came away impressed with it.

    ReplyDelete
  7. @Daniel /me vomits a little in his mouth

    Every time someone posts that class as a solution I go cross-eyed.

    To abuse a JWZ quote:

    'Some people, when confronted with a problem, think "I know, I'll use Option from functionaljava!" Now they have two problems.'

    ReplyDelete
  8. 1.
    Nullity is wrong path.
    We can use if, or add validators to ArrayList.


    2.
    null-default operator in current look is bad idea.
    func(a,b,c,d){
    return ((a?.b?:b)?.c:?c):?d;
    }
    realy bad to read in some cases.


    3.
    The rule:
    The RHS expression is only executed if the LHS expression yields null
    Need to be considered, cos it will give a little speed but it will kill language evolution.
    U see that?

    This is a personal weblog, I do not speak for my employer.

    ReplyDelete
  9. Stephen Colebourne6 January 2009 at 18:01

    Hamlet: Using annotations for nullity is a hack IMHO. A proper language level integration provides much more power and control (and interaction with other features). The typical syntax (backwards compatible) for Java would be:

    String str = ... // string that can hold null
    String! str = ... // string that cannot hold null

    ReplyDelete
  10. I really like the null-default operator. My only question is what do you do after a line like this?

    String result = getFooMayBeNull()?.getBarMayBeNull()?.getResult();

    If result is null what operation failed?

    Assuming you don't care and want to keep on processing. Either I need to assign a default value or every time I use result I must use '?.' operator.
    There is nothing to insulate the code that comes after from the null.
    The null handling just gets passed on further and further down the chain until someone decides to assign a value, or you get an NPE.
    I think it would be helpful if a class could implement an interface that provided a nullObject() method.
    This value would get returned by default by the '?.' operator.
    So for a List the implementation of nullObject() would return Collections.emptyList().

    Making code like this work.

    foreach(Person p : group.?getPeople()){
    //.. code
    }

    Also String might return "" for .? making this safe.

    String result = getFooMayBeNull()?.getBarMayBeNull()?.getResult();
    if(result.equals("something"){
    ... logic
    }

    The only problem is if you actually want a null. But do you ever want a null if you are using '.?' ?

    The only other issue is that the nullObject() method would have to come from a static interface method or be added to object.

    ReplyDelete
  11. My money is on non-null types (e.g., String!).
    If redundant null-checking is the problem, then removing the nullability is the most effective "roots" solution, not ill-specified syntactic sugar.

    Let's demonstrate the importance of non-null types via an analogy. You write this every day:

    if(arg == null) {
    throw new IllegalArgumentException("arg == null");
    } else {
    return arg.substring(4); // or omsething
    }

    But this is conceptually similar to:

    if(!(arg instanceof String)) {
    throw new IllegalArgumentException("!(arg instanceof String)");
    } else {
    return ((String)arg).substring(4);
    }

    You would never write the latter kind of snippet; why do you keep up with writing the former?

    ReplyDelete
  12. +1 for this and
    +1 for fcm

    ReplyDelete
  13. Stephen Colebourne7 January 2009 at 08:02

    Collin: In v0.1 of the proposal I outlined a possible null object design. However, I removed this as it makes the overall solution much more complex.

    Collin: "Assuming you don't care and want to keep on processing ... there is nothing to insulate the code that comes after from the null.

    Most of the time I would expect to write something like:

    if (a.?b?.c != null) {
    // process c
    }

    Mikko: I have nothing against nullity in types. In fact I prefer it. But it is a complex change for Java (easy for new languages like Fan). And as Fan shows, you still need these two operators even when you have nullity in types.

    ReplyDelete
  14. Kieron Wilkinson7 January 2009 at 12:56

    Perhaps my use of Java is unusual, but I really don't see a big need for this sort of null handling.

    The only support for null I would like (actually, love) in Java is being able to say that something should or not should not be null, and let the compiler complain against possible breaches.

    Virtually all of our software does not use nulls, and so they have ceased to be a problem for a long time now.

    Apart from using 3rd party libraries (fairly rare here), the only need I see with this is for legacy code? After all, who uses null in an API these days? Is that is generally the case, should we add something that only satisfies a legacy use?

    ReplyDelete
  15. Hi!
    I forget to mention it again but it would be really bad if some thing like:
    array.?.[index]
    would be not possible.
    ^__^

    I have some work now, but after it's done I'll prepare construction for methods that are able to return more than one value.

    ReplyDelete
  16. I agree with many commenters that null should be avoided in APIs.

    I like "?:" because it helps handle calls to APIs that return null inappropriately. In this example, getHistory() should return a null object but returns null instead. The "?:" operator makes it easy to fix that at the call site if you can't change the API:

    History history = user.getHistory() ?: NULL_HISTORY;
    List dates = history.getDates(); // NULL_HISTORY returns an empty list

    I'm leery of "?." because it implies that the correct null object for a type is one that returns null for every method, so it propagates nulls instead of eliminating them. The above example would become:

    List dates = user.getHistory()?.getDates();

    and you still have to deal with dates == null.

    There are times when returning null is reasonable, e.g. when searching for something. It implies the need for special processing. For example, you may design your API so that findUser can return null but the other methods can't:

    User user = userManager.findUser(userName);
    if (user == null) {
    // special handling because user can't be found
    } else {
    List dates = user.getHistory().getDates();
    // dates cannot be null
    }

    This would be the wrong way to do it (in my opinion):

    List dates = userManager.getUser(userName)?.getHistory().?getDates();
    if (dates == null) {
    // why are we here?
    // user wasn't found or user doesn't have history or history doesn't have any dates
    }

    ReplyDelete
  17. Is there some way to use the null-default to also throw exceptions? For instance

    return map.get(key) ?: throw new IllegalArgumentException("Cannot find " + key);

    Currently, since throw is not an expression, you can't use

    Object result = map.get(key);
    return result != null ? result : throw new IllegalArgumentException("Cannot find " + key);

    But since we are adding sugar, we could have the compiler generate the necessary bytecode if it finds a throw keyword.

    ReplyDelete
  18. Is "?:" (null-safe operator) similar to C# "??" (null coalescing operator)?

    ReplyDelete
  19. Kieron: Yes, I think your situation is unusual. Lots of code is written to handle nulls all the time (and using null in an API can be a valid choice if used carefully).

    Lasu: I hadn't considered array access, but there may be a case to support it.

    Tim: I'd argue its pretty common when dealing with deeply nested bean/POJO structures to have to handle any of the layers potentially being null, and you not actually caring.

    Roel: I believe that converting throw to an expression would cause a non-trivial piece of work in the spec wrt definite assignment and the like. As such, I don't see that happening in Java 7.

    Vineet: Yes, ?: is the C# ?? operator IIUC.

    ReplyDelete
  20. Stephen, in the case of a null-default operator that contains a throws, it could be rewritten in existing java keeping definite assignment intact.

    In my example, the code

    return map.get(key) ?: throw new IllegalArgumentException("Cannot find " + key);

    could be rewritten as

    final Object result;
    Object temp = map.get(key);
    if (temp != null) {
    result = temp;
    }
    else {
    throw new IllegalArgumentException("Cannot find " + key);
    }
    return result;

    As I see it, at the return statement, the compiler has made sure that every code path either assigned the variable result exactly once or thrown an exception.

    On the other hand, I agree it would complicate the spec. I think however it would be a good idea to at least see if we could keep this option open for a next java release.

    ReplyDelete