Thursday, 17 June 2010

Exception transparency and Lone-Throws

The Project Lambda mailing list has been considering exception transparency recently. My fear with the proposal in this area is that the current proposal goes beyond what Java's complexity budget will allow. So, I proposed an alternative.

Exception transparency

Exception transparency is all about checked exceptions and how to handle them around a closure/lambda.

Firstly, its important to note that closures are a common feature in other programming languages. As such, it would be a standard approach to look elsewhere to see how this is handled. However, checked exceptions are a uniquely Java feature, so this approach doesn't help.

Within Neal Gafter's BGGA and CFJ proposals, and referenced by the original FCM proposal is the concept and solution for exception transparency. First lets look at the problem:

Consider a method that takes a closure and a list, and processes each item in the list using the closure. For out example, we have a conversion library method (often called map) that transforms an input list to an output list:

  // library method
  public static <I, O> List<O> convert(List<I> list, #O(I) block) {
    List<O> out = new ArrayList<O>();
    for (I in : list) {
      O converted = block.(in);
      out.add(converted);
    }
    return out;
  }
  // user code
  List<File> files = ...
  #String(File) block = #(File file) {
    return file.getCanonicalPath();
  };
  List<String> paths = convert(list, block);

However, this code won't work as expected unless we specially handle it in closures. This is because the method getCanonicalPath can throw an IOException.

The problem of exception transparency is how to transparently pass the exception, thrown by the user supplied closure, back to the surrounding user code. In other words, we don't want the library method to absorb the IOException, or wrap it in a RuntimeException.

Project Lambda approach

The approach of Project Lambda is modelled on Neal Gafter's work. This approach adds addition type information to the closure to specify what checked exceptions can be thrown:

  // library method
  public static <I, O, throws E> List<O> convert(List<I> list, #O(I)(throws E) block) throws E {
    List<O> out = new ArrayList<O>();
    for (I in : list) {
      O converted = block.(in);
      out.add(converted);
    }
    return out;
  }
  // user code
  List<File> files = ...
  #String(File)(throws IOException) block = #(File file) {
    return file.getCanonicalPath();
  };
  List<String> paths = convert(list, block);

Notice how more generic type information was added - throws E. In the library method, this is specified at least three time - once in the generic declaration, once in the function type of the block and once on the method itself. In short, throws E says "throws zero-to-many exceptions where checked exceptions must follow standard rules".

However, the user code also changed. We had to add the (throws IOException) clause to the function type. This actually locks in the exception that will be thrown, and allows checked exceptions to continue to work. This creates the mouthful #String(File)(throws IOException).

It has recently been noted that syntax doesn't matter yet in Project Lambda. However, here is a case where there is effectively a minimum syntax pain. No matter how you rearrange the elements, and what symbols you use, the IOException element needs to be present.

On the Project Lambda mailing list I have argued that the syntax pain here is inevitable and unavoidable with this approach to exception transparency. And I've gone further to argue that this syntax goes beyond what Java can handle. (Imagine some of these declarations with more than one block passed to the library method, or with wildcards!!!)

Lone throws approach

As a result of the difficulties above, I have proposed an alternative - lone-throws.

The lone-throws approach has three elements:

  1. Any method may have a throws keyword without specifying the types that are thrown ("lone-throws"). This indicates that any exception, checked or unchecked may be thrown. Once thrown in this manner, any checked exception flows up the stack in an unchecked manner.
  2. Any catch clause may have a throws keyword after the catch. This indicates that any exception may be caught, even if the exception isn't known to be thrown by the try block.
  3. All closures are implicitly declared with lone throws. Thus, all closures can throw checked and unchecked exceptions without declaring the checked ones.

Here is the same example from above:

  // library method
  public static <I, O> List<O> convert(List<I> list, #O(I) block) {
    List<O> out = new ArrayList<O>();
    for (I in : list) {
      O converted = block.(in);
      out.add(converted);
    }
    return out;
  }
  // user code
  List<File> files = ...
  #String(File) block = #(File file) {
    return file.getCanonicalPath();
  };
  List<String> paths = convert(list, block);

If you compare this example to the very first one, it can be seen that it is identical. Personally, I'd describe that as true exception transparency (as opposed to the multiple declarations of generics required in the Project Lambda approach.)

It works, because the closure block automatically declares the lone-throws. This allows all exceptions, checked or unchecked to escape. These flow freely through the library method and back to the user code. (Checked exceptions only exist in the compiler, so this has no impact on the JVM)

The user may choose to catch the IOException, however they won't be forced to. In this sense, the IOException has become equivalent to a runtime exception because it was wrapped in a closure. The code to catch it is as follows:

  try {
    paths = convert(list, block);  // might throw IOException via lone-throws
  } catch throws (IOException ex) {
    // handle as normal - if you throw it, it is checked again
  }

The simplicity of the approach in syntax terms should be clear - it just works. However, the downside is the impact on checked exceptions.

Checked exceptions have both supporters and detractors in the Java community. However, all must accept that given projects like Spring avoiding checked exceptions, their role has been reduced. It is also widely known that other newer programming languages are not adopting the concept of checked exceptions.

In essence, this proposal provides a means for the new reality where checked exceptions are less important to be accepted. Any developer may use the lone-throws concept to convert checked exceptions to unchecked ones. They may also use the catch-throws concept to catch the exceptions that would otherwise be uncatchable.

This may seem radical, however with the growing integration of non-Java JVM languages, the problem of being unable to catch checked exceptions is fast approaching. (Many of those languages throw Java checked exceptions in an unchecked manner.) As such, the catch-throws clause is a useful language change on its own.

Finally, I spent a couple of hours tonight implementing the lone-throws and catch-throws parts. It took less than 2 hours - this is an easy change to specify and implement.

Summary

Overall, this is a tale of two approaches to a common problem - passing checked exceptions transparently from inside to outside a closure. The Project Lambda approach preserves full type-information and safeguards checked exceptions at the cost of horribly verbose and complex syntax. The lone-throws approach side-steps the problem by converting checked exceptions to unchecked, with less type-information as a result, but far simpler syntax. (The mailing list has discussed other possible alternatives, however these two are the best developed options.)

Can Java really stand the excess syntax of the Project Lambda approach?
Or is the lone-throws approach too radical around checked exceptions?
Which is the lesser evil?

Feedback welcome!

14 comments:

  1. I totally agree with you, java should include a mechanism to "uncheckify" checked exceptions (that way we could have best of both worlds: people who are not happy with checked exceptions would have a way to bypass them and the others could continue the standard way).

    See http://beust.com/weblog/2010/04/25/improving-exceptions#comment-9051, where I was suggesting to re-use the 'transient' keyword (re-using keywords is a common practice in java) in the throws clause of a method to tell the compiler the marked exception should be treated like a runtime one.

    I really hope this will make it in a future version of java.

    ReplyDelete
  2. I understand you point of view, which is valid in general.
    But in most cases, you probably will end-up defining the closure very close to when it will be used. And the closure itself definitely know which exceptions can be thrown, so this information could be used.

    In your case, convert() cannot possibly know which exceptions are thrown (and it will deal with them as unchecked), but the user code in your example could be forced to handle them, or at least get a warning.

    As a rule of thumb, if you declare a closure and pass it to a method, then you should handle the exceptions, even if not in all cases they can reach your level. This does not apply to the closure that you receive as a parameter, because you cannot know the list of exceptions.

    This in my opinion gives more control to the developer, with a minimum impact of the syntax.

    I feel that checked exceptions are quite important, above all in code that needs to be reliable. I would feel very uncomfortable to use unchecked exceptions for complex projects and in enterprise environment.

    ReplyDelete
  3. Stephen Colebourne17 June 2010 at 10:52

    @Luca, I found this link to be of great interest to explore how many developers have come to realise the problems of checked exceptions and how they make enterprise development harder, not easier - http://userstories.blogspot.com/2008/12/checked-exception-why-debat-is-not-over.html

    ReplyDelete
  4. Michel Ishizuka17 June 2010 at 12:06

    We have a sense of including closures to the Java language. However we have no sense of removing checked exceptions from the Java language.

    So, lone-throws approach is not a option. The project lambda approach is the only option.

    ReplyDelete
  5. @Stephen, I saw the point. The more calls you have between the exception and who has to manage it, the less checked exceptions are useful. But with a proper design, you could have each layer to deal with its exceptions, then move the ones that cannot be managed to the next level, possibly transforming them in a "MyApplicationException". This should help having both a finer management (at the single layer level), and a relatively clean interface.

    @Michel, I'm glad to hear that there are no plan to ditch checked exceptions. But still there should be a focus to avoid that Java become too much verbose and "strange", as it could scare some developers

    ReplyDelete
  6. Count me as one of the scared developers. Just looking at that syntax/exception handling makes me want to wretch.

    ReplyDelete
  7. The Project Lambda approach is the way to go, imho. Closures ought to declare their checked exceptions, just like simple methods do.

    ReplyDelete
  8. One thing if the closure having to declare exceptions, which I think is good.
    Another is having the library required to anticipate the type, declaring it 3 times.

    I would say that the Exception not expected by the library could be thrown as unchecked, as in any case it does not know what to do with it. The first code passing the closure, could be required to deal also with the real exceptions.

    But I understand that it is a sort of a mixed scenario.

    ReplyDelete
  9. Why cannot the compiler retain the exception information of the closure, but avoid to require that it is declared ?

    This way, the closure will be short to type, the compiler does the job of figuring out which ones it throws and require callers to declare it in the throws clause of methods if they are checked.

    Closure's throws clause will be implicit.

    ReplyDelete
  10. I think the "declaring it 3 times" argument is a bit off-base.

    In the following method, is the type "T" declared 3 times?

    public T last(Collection c)
    { return c.get(c.size()-1); }

    No, it is referred to in 3 places, but they have distinct meanings.
    One declares the generic (parameterised) type
    One declares the return type
    One declares the type of the first argument.

    So it is with the "throws E" clause. Yes, it occurs three times, but each one has a specific and distinct meaning.

    I really don't think it's a problem.

    ReplyDelete
  11. I would suggest we side step the issue of checked exceptions (mostly because I want closures now rather than MUCH later) and say closures may throw checked or unchecked exceptions (without declaration). BUT if they MAY throw a checked exceptions the compiler issues a warning:

    #String(File) block = @SuppressWarnings("checked") #(File file) {
    return file.getCanonicalPath(); //throws IOException
    };

    Then, if you attempt to catch a checked exception that can't be verified by the compiler you override the compiler with an annotation:

    try {
    paths = convert(list, block); // might throw IOException
    } catch ( @SuppressWarnings("checked") IOException ex) {
    // handle as normal - if you throw it, it is checked again
    }

    ReplyDelete
  12. @Peter: no offense, but have a look at how Project Lambda defines a closure:

    #String(File)(throws IOException) block = #(File file) {
    return file.getCanonicalPath();
    };

    & how you define a closure:

    #String(File) block = @SuppressWarnings("checked") #(File file) {
    return file.getCanonicalPath(); //throws IOException
    };

    Your code has less information in it, yet it is much more verbose!

    To me it 's clear that the Project Lambda approach of declaring a closure is the best one. As for the method signature of the library method, I 'm confident this can & will be improved (just like generics will be made simpler in Java 7).

    ReplyDelete
  13. Stephen Colebourne22 June 2010 at 16:19

    Lots of good discussion and options. It would appear that Oracle are determined to pursue the full-solution depite the terrible impact on code readability.

    ReplyDelete
  14. Java should just forget about checked exceptions. They suck. That's my bottom line.

    ReplyDelete