Wednesday, 10 January 2007

Java 7 - Null-ignore invocation

Have you ever had an NPE in production? Or felt that handling null is a right pain? Well perhaps this Java 7 language change proposal - Null-ignore invocation - will interest you.

The concept is to provide the developer with a tool to handle null, that will reduce NPEs and improve code clarity. To explain this, lets consider this method:

  public String getPostcode(Person person) {
    return person.getAddress().getPostcode()
  }

The problem is that this code may throw an NPE if person or getAddress() is null. Here is the 'correct' way to code this:

  public String getPostcode(Person person) {
    if (person != null) {
      Address address = person.getAddress();
      if (address != null) {
        return address.getPostcode();
      }
    }
    return null;
  }

The problem with the 'correct' solution is that the intent of the code has been lost. The null-checks are really getting in the way of reading and understanding the method.

This proposal aims to tackle this by adding compiler syntax sugar to generate the null pointer checks for developers. It achieves this by adding an alternative to the dot for method and field invocation, the hash - #. Using the hash invokes a method or field in the same way as using the dot, but with one change. If the expression on the left hand side of the hash is null, then the result of the expression is null. If the expression type is a primitive then zero or false is returned instead of null. If the expression is a void, then no action occurs.

  public String getPostcode(Person person) {
    return person#getAddress()#getPostcode()
  }

This code is now null-safe and won't throw an NPE. (Of course the interals of getAddress() or getPostcode() might throw NPE.) In addition the code is now back to its former clarity - you can now read it and clearly see the intent without any visual clutter.

I know that some reading this will cry 'Null object design pattern'. Well maybe thats fine in your closed sourcebase, where you can enforce its use with code reviews and so on. But as soon as your code deals with an external library, or even the JDK, then your code has to handle null. Which leads you right back to this proposal.

If you want to read the full details, I've written up the formal Null-ignore invocation proposal. If you've got any suggested enhancements or fixes to the proposal, let me know. All feedback welcomed.

30 comments:

  1. Jonatan Kazmierczak10 January 2007 15:17

    NPE is thrown, when you try to invoke method on the not initialized pointer (which is originally null). This is expected and very useful behavior - it prevents from something like "Segmentation fault", which can be seen in programs written in C++.
    Simple method to avoid problem you described is to use try/catch.
    You can do it like this:

    public String getPostcode(Person person) {
    try {
    return person.getAddress().getPostcode();
    } catch (NullPointerException e) {
    return null;
    }
    }

    ReplyDelete
  2. Jonatan, I think you haven't read this article so why are you giving an opinion on it ?

    ReplyDelete
  3. Stephen Colebourne10 January 2007 15:36

    @Jonatan: Exceptions are not intended to be part of the normal program flow. Your suggestion would be considered by many to be bad coding style.

    ReplyDelete
  4. I wonder if the null-ignore-invocation behaviour is not "normal" enough to provide it as the default behaviour. I.e. without needing another syntax.

    I'm trying to imagine how this would affect existing code/libraries.

    Either they ignore NPE-catching and then this null-chaining will probably not make things worse. The NPE might occur in another spot, the moment the library tries to work with the result (e.g. postcode above), but crash it will anyway.

    Or they already do some NPE checking and then it could be a problem if it's not done consistently. With the change, an NPE could be thrown elsewhere now... But also there one could argue that in the current situation such a library will/might eventually fail anyway...

    Your concept of null handlers seems very interesting as an efficient way to implement a null-object pattern.

    Also there, I'm a bit afraid of adding new syntax again to Java. I'm not a big fan of annotations, but they're here. Maybe we can use that for a null-handler?

    cheers

    erwin

    ReplyDelete
  5. Michael Slattery10 January 2007 15:44

    I think it would be MUCH better to do a COMPILE-TIME check as Nice and FindBugs do. Annotations could help here.

    http://nice.sourceforge.net/safety.html#id2448199
    http://findbugs.sourceforge.net/bugDescriptions.html#NP_NONNULL_PARAM_VIOLATION

    ReplyDelete
  6. A compile-time check is of course usefull for many reasons, but does not address the fact that in order to pass the tests, you currently need to write tons of null-checks for chained getters...

    While in general you don't care on which level the null occurs in the chain.

    ReplyDelete
  7. JSR-305 is standardizing annotations to help deal with this problem. Actually, I would favor a language solution in addition to annotations, but a very different one. Nice has exactly the answer that Java needs:

    Option Types
    http://nice.sourceforge.net/manual.html#optionTypes

    ReplyDelete
  8. Stephen Colebourne10 January 2007 16:09

    @Michael, I know of the compile-time NPE checks in Nice, and have started writing a similar document to cover them. However, they actually turn out to be a lot more difficult to integrate into the existing Java language without breaking anything. I'm not talking about implementation here, just the look and feel of the syntax.

    @Erwin, changing the default behaviour of the dot operator would appear to be a serious backwards compatibility issue, changing the meaning ad behaviour of existing code.

    Annotations are, according to the Sun guys, not to be used for language features. I agree.

    Oh, and the unit test argument is a good point too.

    ReplyDelete
  9. I would have to disagree with this approach. I think the NPE exception check that Jonathan suggested is better.

    It might still be ugly, but if you have to check for nulls, then something is definitely wrong.

    I would rather have something like...

    void foo(Object abc notNull)

    or

    Boo notNull guaranteed()

    to enforce that the return or inputs are not null. That way if there is an NPE it will be from the caller of the method rather than the actual method itself. This cleans up the code for the method.

    ReplyDelete
  10. Stephen Colebourne10 January 2007 17:10

    There are a number of suggestions/confusions here about Nice style option-types. This proposal is about something much simpler - just adding null checks. This can be achieved in Java easily as shown in the formal proposal.

    Nice-style option-types turn out to be much more complicated to integrate with Java. (For a new language like Nice they were easy to add.) If I can work out a syntax that does look good in Java for option-types then I might publish it too, but that will be a more radical change, and thus less likely to succeed than this proposal. (Help appreciated, by email)

    @Archimedes, Having to deal with null does not mean that something is definitely wrong. Lots of APIs return null (often without good documentation), and lots of domain models use null to indicate no knowledge about that property. A 'notnull' syntax doesn't help here.

    ReplyDelete
  11. I usually set default values for all database columns, thus rarely get the NPE when retrieving domain object fields. On the other hand, I agree that the whole thing with checking for null all the time is a bit tedious. I rarely write a single method without a precondition where I check for null.

    ReplyDelete
  12. Stephen, I can't say that I really understand why this would be so much easier to implement than Nice-style option types. I prefer to detect and deal with nulls as early as possible. Nice-style option types force you to do that. Your proposal encourages developers to "let nulls ride", carrying the exception that will eventually be thrown farther and farther from the null that is ultimately the problem.

    "But as soon as your code deals with an external library, or even the JDK, then your code has to handle null. Which leads you right back to this proposal."

    I prefer to handle nulls by eliminating or replacing them where possible. Whenever I can't do that, I would rather be reminded that the value could be null, in order to force me to think about the consequences.

    ReplyDelete
  13. Hash is a bit ugly. Colon would be prettier.

    public String getPostcode(Person person) {
    return person:getAddress():getPostcode()
    }

    Won't this conflict with the new -> operator proposal?

    public String getPostcode(Person person) {
    return person:>address:>postcode
    }

    ReplyDelete
  14. Want about using a closure?

    public String getPostcode(Person person) {
    nullable {
    return person.getAddress().getPostcode();
    }
    }
    where nullable is:

    R nullable({R} block) {
    try {
    return block.invoke();
    } catch (NullPointerException x) {
    return null;
    }
    }

    ReplyDelete
  15. That should be:
    public String getPostcode(Person person) {
    return nullable {
    person.getAddress().getPostcode()
    }
    }

    ReplyDelete
  16. Not a good idea, IMHO. If an API returns a null, the caller should be prepared and take the proper action.

    This proposal goes in the wrong direction and would encourage the use of null values.

    The proposal overloads the meaning of null and 0/false (in the case of primitives) which can lead to confusion. Is an expression null because the method returned it, or because the object was null? null != nil. This could have the effect of delaying NPEs even further in execution and make them harder to debug.

    And, why stop at null pointers. How about making #= return null on class cast exceptions. Or #/ return 0 on divide by zero. Make # a "do the right thing" modifier.

    ReplyDelete
  17. @Stephen
    Like you said there are lots of functions and API that return a null with no documentation. Nulls I have to admit are quite convenient since they are already there. I tend to use them when I don't feel like writing a NullObject pattern.

    However, nulls are also not typed and you can't get any more information about it.

    It is better to just have something that follows the NullObject pattern. Of course just because it is better doesn't always mean it would always be the case.

    As @gjfdh concurred, the handling of nulls should be the responsibility of the caller. For all intents and purposes your code should assume that the data sent to you is correct.

    With the exception of data coming external from your system, those need to be checked by something in your facade layers.

    Per @gjfdh, I'd rather discourage the use of nulls in the first place whenever possible. Although don't take it to an extreme where you create a null object that does not contain any data or you some thing can't operate on.

    Bad use of a null object pattern is

    class NullFoo implements Foo {
    // but has nothing
    }

    You basically want to try and avoid code like

    if (foo == null) {
    // do something A
    // null op
    } else {
    // do something B
    foo.op();
    }

    instead you'd just want to do

    foo.op()

    because you know it won't be null and if it is a null object the op() should do something that a null object would do like nothing.

    ReplyDelete
  18. Stephen Colebourne10 January 2007 22:28

    @Dan, A colon is too visually similar to a dot. The eye would skip the important information too easily.

    @Peter, A closure has a different meaning to the proposal I present. With a closure, if getAddress() or getPostcode() throw an NPE internally, then that exception is caught. The # proposal only stops the single immediate invocation from failing.

    @gjfdh, This *is* a clear way for the caller to handle null results from an API. In other words, the caller *is* making an active decision to handle null by choosing #. Or perhaps you enjoy having != null checks cluttering up the real logic of the method.

    @Archimedes, If you feel that your code can rely on the data it receives being correct then your a lot braver than I am! I suspect you may be in denial about the number of NPEs you hit.

    As expected, the most common argument against this is to use the null-object pattern. I'll state this again - I believe most development shops can't or won't do that, its impractical for them. There's nothing enforcing it. If you read the document, you'll find that null-handlers cover some of the ground of null-object pattern anyway, but in a more enforced way.

    ReplyDelete
  19. I think this is an interesting possibility. I'm not so sure that I like the "null handlers" suggestion in the proposal (for handling e.g. isEmpty() and equals()). Doesn't "feel" like Java... but I don't currently have a better suggestion.... except to disallow the null-ignore invocation if return type is not an Object.

    As Stephen points out, Jonathan's try/catch solution and PeterReilly's closure variant both have an important flaw: they both assume that the NPE was thrown one of the dot operators in the visible line of code. The NPE could just have easily been thrown from WITHIN getAddress() or getPostCode()... then your "catch" could be catching something you didn't want to catch!

    Might a "?." operator be more clear?

    Like this:
    public String getPostcode(Person person) {
    return person?.getAddress()?.getPostcode();
    }

    In a sense, we're simulating:
    public String getPostcode(Person person) {
    Address address;
    return person==null?null:((address=person.getAddress())==null)?null:address.getPostCode();
    }

    ReplyDelete
  20. O Boy!
    Let's not complicate the language. Please...

    I solved this pretty universally in my apps by having a CoerceTo singleton that deals with most of the primitives and has overloaded methods to allow for default values.

    Do not pass primitives around, except in truly mathematical methods.

    Catch Exceptions/NPE's at the lowest level of code/lib and either return with a default value or throw a business/persistence oriented exception.

    ReplyDelete
  21. I also like Nice's style of declaring nullable objects (?String param). Other way to do this would be generics (Nullable param).

    http://nice.sourceforge.net/safety.html

    ReplyDelete
  22. You give an absolutely non-OO example to put forward your proposal of a language change?
    You must be joking.

    ReplyDelete
  23. @Stephan, @Natan, yes the closure/try-catch
    will wipe out the internal NPEs so it has
    different semantics from the nullable operator
    - the groovy ?. operator looks the best for this.

    ReplyDelete
  24. I feel that this proposal solves a problem, but without being able to guarantee non-null on references, we'll just see what is effectively a global search and replace of . for #, and therefore hidden bugs.

    I'd rather see discussion about a 'notnull' keyword, that is checked by the compiler and verified by the, erm, verifier.

    Of course, even with 'notnull', at least because Java doesn't have tail recursion optimisation, null is still necessary (I can go into details on that leap of faith), so the # operator would be useful.

    I'd rather emphasise removing null from your work than being able to work with it more easily - not to say that the latter isn't important.

    ReplyDelete
  25. Sounds interesting, *could* be an interesting addition but I think it would bring more harm than good.
    some points:
    1. Too limited. If it is implemented then it should be extensable for defined exceptions. IOW the developer would need to define what exceptions the # would apply to.
    2. Why have exceptions then? It is in the name "Exception", basically a not normal situation. Something is wrong when a nullpointer comes unexpected. Plus, nulls may be intentionally included in various libraries (they should be documented so)
    3. Less need for testing when 'default exception handling methods' are used. Just imaginge a banking program that previously ended the transaction and rolled the db back having a new feature that sets all values to '0' whenever a NPE occurs. Ouch. This could be seriously misused.

    ReplyDelete
  26. I totally agree with gjfdh.

    It's like hiding a problem. As others said if you put this # operator, you are simply hiding the cause of the problem. It will show up somewhere later in the execution and it will be much much harder to debug it. With this operator you are simply destroying a very useful debug information. It is exactly like finding a couple of drops of rocket coolant fluid in a chamber of the space shuttle and cleaning it up, just because this chamber should be clean! ...But what about the the broken pipe that caused the leak?

    If something went wrong in some point of your program, it will not be fixed this way. If it will be, then it's just luck. You will not go far this way (except if you are Gladstone Gander).

    ReplyDelete
  27. >Is an expression null because the method returned
    >it, or because the object was null?

    gjfdh, the method returned null because the object *was* null. If the object were not null then it would have returned the object's value. There is *only one way* to return null and this is only when the object is null. Plain and simple.

    ReplyDelete
  28. Morten Knudsen11 January 2007 17:22

    Why not use the same operator as Groovy? It uses "?"

    ReplyDelete
  29. Stephen Colebourne11 January 2007 17:37

    @Morten, See the next blog - http://jroller.com/page/scolebourne?entry=java_7_null_safe_types - where it becomes more obvious why the question mark is unsuited for my proposal.

    ReplyDelete
  30. >>Is an expression null because the method returned
    >>it, or because the object was null?

    >gjfdh, the method returned null because the object
    >*was* null. If the object were not null then it
    >would have returned the object's value. There is
    >*only one way* to return null and this is only when
    >the object is null. Plain and simple.
    Panos, if getPostCode() returns null, it may either mean that the person was null, or the person's address was null, or the address' post code was null. That's three ways.

    Apart from this, I don't think overloading of null or zero or false is such a universally common way of handling null pointers, to warranty complicating the language. Even if it was, I wouldn't wish to make it easier to overload null with all sorts of meanings.

    ReplyDelete