Thursday 26 June 2014

Iterable in Java SE 8

In the last few weeks I've finally had the chance to use lambdas and streams in Java SE 8 in anger. In doing so, I've found much to like, but some rough edges.

Iterable woes

The Iterable interface was added in Java SE 5 to enable the foreach loop to work. It was defined with a single method that returns an iterator:

 public interface Iterable<T> {
   Iterator<T> iterator();
 }

The simplicity of the interface was based around the primary use case, the foreach loop, which needed nothing more than a source of an iterator.

In Java SE 8, the interface changed, as it added two default methods:

 public interface Iterable<T> {
   Iterator<T> iterator();

   default void forEach(Consumer<? super T> action) {
     Objects.requireNonNull(action);
     for (T t : this) {
       action.accept(t);
     }
   }
   
   default Spliterator<T> spliterator() {
     return Spliterators.spliteratorUnknownSize(iterator(), 0);
   }
 }

Isn't this wonderful? Lots of new functionality for free! Er well, no.

Despite following the lambda-dev mailing lists, the problem I now have with Iterable isn't one I foresaw. Yet it took just a few days of active use of Java SE 8 to realise that Iterable is now a lot less useful than it was.

The problem as I see it, is that Iterable has been used for two different things in my code. The first, is as the parameter of a method, where it acted as a very abstract form of collection. The second is as a "common" interface on a domain object.

Receiving an Iterable parameter

The first case is where a method has been written to receive an Iterable parameter. In my code, I have defined many methods that take an Iterable, often alongside a varargs array:

   void addNames(Name... names);
   void addNames(Iterable<Name> names);

This pairing provides great flexibility to callers. If you have a set of actual name objects, you can use the varargs form. If you have an array of name objects, you can also use the varargs form. And if you have virtually any kind of collection, you can use the iterable form. (That abstract sense of just providing an iteration really broadens the kind of inputs allowed and blurs the line between collections and non-collections).

Code within the addName(Iterable) method will benefit to some degree from the new methods on Iterable. That is because the method receiving the Iterable knows nothing else about the input type other than it is an iterable.

Given this, the addition of the new forEach(Consumer) method is reasonable, as it provides additional looping options. But, in reality, there is little difference between these two:

   void addNames(Iterable<Name> names) {
     for (Name name : names) { ... }
   }
   void addNames(Iterable<Name> names) {
     names.forEach(() -> ...);
   }

Yes its a new method, but the actual new functionality is limited. Nevertheless, from this perspective it is a win.

By contrast, the addition of the new spliterator() method simply leads to frustration, as by itself the method has no value. What would have had value would have been a stream() method but there are good reasons why stream() was not added.

Instead, we have to write some rather ugly code to get the actually useful stream:

   void addNames(Iterable<Name> names) {
     StreamSupport.stream(names.spliterator(), false)
    .doMyWizzyStreamThing();
   }

As an aside, arrays manage this better:

   void addNames(Name[] names) {
     Stream.of(names)
    .doMyWizzyStreamThing();
   }

Now why oh why oh why is there no Stream.of(Iterable) method?

Actually, I know why, but that is a story for another day...

Streamable?

Perhaps you're thinking that the reason why Iterable isn't supported well is that there is a Streamable interface that you are supposed to use for this purpose instead?

Er, no. There was a Streamable interface, but it got removed during development. There is no JDK supplied abstraction for a type that provides streams.

Perhaps this is philosophical - you could just declare and call using a supplier:

   void addNames(Supplier<Stream<T>> streamSupplier) {
     streamSupplier.get()
    .doMyWizzyStreamThing();
   }
   Collection<T> coll = ...
   addNames(coll::stream));
   T[] array = ...
   addNames(Stream.of(array));

Definitely a different kind of use of the type system.

After much thought and playing with the feel of the options available, I decided to keep on declaring methods to take Iterable and varargs as that is friendliest to users of the API. Internally, the mapping has to be done from iterable to stream, which necessitates a convenience method in a helper for my sanity.

Iterable on domain objects

The second major way I use Iterable is where I had more difficulty.

I have found over the years that some domain objects are little more than a type-safe wrapper around a collection. Consider a FooSummary object that contains a list of FooItem. In many cases, I have made FooSummary implement Iterable<FooItem>:

 public class FooSummary implements Iterable<FooItem> {
   List<FooItem> items;
   @Override
   public Iterator<FooItem> iterator() {
     return items.iterator();
   }

Doing this allowed the object to be iterated over directly in a foreach loop.

 FooSummary summary = ...
 for (FooItem item : summary) { ... }

This approach is very useful for those domain objects that are primarily a collection of some other object, yet are distinct enough to have a class of their own.

However, this kind of approach relied on using Iterable as a kind of "common" interface, similar to Comparable or Serializable. These small "common" interfaces work on the basis that they provide a common language across a very broad spectrum of use cases. Comparable does nothing other than define the method needed to participate in comparisons. I used Iterable in exactly that sense - to accept the common language for things that can be iterated over, without any implication that they were actual collections.

In my opinion, usage of Iterable as a "common" interface has been all but destroyed by Java SE 8.

The FooSummary class is essentially a domain object. The methods it provides must make sense as a set in their own right. Adding the iterable() method was perfectly acceptable in exactly the same way as adding comparable() is. It is a well-known method with clearly defined properties and usage. However, in Java SE 8, the domain object has had two additional methods foisted upon it.

The issue is that from the perspective of a user of FooSummary, the two additional methods are net negative, rather than positive. While there is nothing wrong in theory with being able to call forEach(Consumer) or spliterator(), it needs to be a specific and separate decision to add them to the domain object.

In my cases, I definitely do not want those extra methods. As such, I have no choice but to stop treating Iterable as a "common" interface. It can now only be used by classes that really make sense to be treated as collections.

In practical terms, this change simply makes life for callers slightly more verbose:

 FooSummary summary = ...
 for (FooItem item : summary.getItems()) { ... }

Where the line between a collection and a non-collection could previously be blurred through Iterable, that option is not really present any more. As such, I feel an API design tool has been removed from my tool-box.

(Another example I played with was a Tuple interface abstracting over Pair and Triple. Again, while adding an iterator() method to Tuple is desirable, adding forEach(Consumer) and spliterator() was not. Here, none of the available options are very pleasing.)

Summary

The Iterable interface has had two default methods added in Java SE 8. These make reasonable sense when receiving an iterable. Unfortunately, the difficulty of obtaining an actual stream is rather annoying.

The more significant problem for me is that Iterable is no longer suitable for use as a "common" interface. The extra methods, and the threat of more in future releases mean that the use cases for Iterable have been reduced. It should now only be used for classes that really are collections, something that I find to be a significant reduction in usability.

Any thoughts on iterable, "common" interfaces and default methods? Has Iterable been ruined by the addition of default methods?

13 comments:

  1. i'm not sure i understood how those two additional methods affect users of your api

    ReplyDelete
  2. I'm with Oleg, what exactly is the problem with having these default methods defined? It's not as if they enable clients of your object to do anything they couldn't otherwise do. I guess maybe they clutter the interface?

    ReplyDelete
    Replies
    1. Yes, its about clutter. A FooSummary is not a real collection, its in that blury space where it can sometimes be thought of as a collection. It makes calling code less verbose to be able to call addItems(summary) rather than addItems(summary.getItems()). Its the difference between allowing FooSummary to operate nicely with collections and actually *being* a collection. As its the former, the forEach() and spliterator() methods just don't make sense.

      Delete
  3. I don't think I feel the same way.
    If there's little difference between forEach and the for(){...} construct, what's the harm in adding the default method ? As for spliterator, if Iterables are "things that can be iterated over", why wouldn't we want to use them as Streams ? Maybe that's because I still see streams as an improved way of iterating...

    Anyway, I fail to see how Iterable now imposes the property of "being a Collection". Stream contents could be generated on-the-fly, just like iterator contents.

    ReplyDelete
    Replies
    1. Iterable is two things - a super-type of collection and a type that produces an iterator. I didn't realise the distinction would matter until I started using Java 8, where only the former now works.

      As an analogy, imagine if default methods has been added to Comparable, say for isGreaterThan()/isLessThan(). Such a decision would have left date classes in a mess, as isBefore()/isAfter() are the correct terminology to use there. The point of the post is that these "common" interfaces exist to provide framework access to things (iteration/comparisons) and that Java 8 no longer has a "common" interface for iteration.

      Delete
  4. I also and still fail to understand.

    > In my cases, I definitely do not want those extra methods.

    Why not ?

    ReplyDelete
    Replies
    1. I don't want the domain class to *be* a collection, I just want to help it interact *with* collections.

      Delete
  5. I too make frequent use of Iterable as a common interface for domain objects, and share your criticism.

    As an experiment I created a dummy Iterator class and tried this for-each loop:

    for (final String s : SIterator::new)
    out.println(s);

    Ok, does not compile, but this does:

    for (final String s : (Iterable) SIterator::new)
    out.println(s);

    I was surprised the compiler did not work this out for itself.

    ReplyDelete
  6. I think I don't share your thoughts about "losing" the iteration behavior with the change introduced by Java 8. I use the Iterable interface as a common role to domain objects, as you do.

    The default methods are based only on abstract types and clients are not tightly coupled with any implementation classes, so they can decide to supply any Consumer concrete class or receive an SplitIterator abstract type.

    If you have an Iterable type acting as a method argument or as a role on your class, you implicitly have a sequence of objects behind it, so I think clients would be interested in consuming it or splitting it. Moreover, clients could add the behavior they want for consuming the objects in the sequence without managing the iteration explicitly.

    Another solution would be to extend the Iterable interface and add the new default methods on it. What do you think about it?

    ReplyDelete
    Replies
    1. If you are consuming an Iterable within a method then the extra default methods make perfect sense. It is the impact on classes which implement Iterable I'm annoyed by, because many of those are not collection classes, and do not warrant having the two extra methods. Now Java 8 is released, this is a permanent change.

      Delete
    2. I understand your criticism, but I think you have the same problem when implementing the Iterable interface on a domain class. It exposes the iterator() method, which means for clients that you have an "iteration" sense on your domain class. Clients can invoke the iterator() method as they'd like. In Java, It means you have an implicitly domain class backed by a Collection.

      The majority of clients can invoke:

      FooSummary summary = ...
      for (FooItem item : summary) {
      ...
      }

      but the Iterable interface allows other clients to do:

      Iterator<FooItem> iterator = summary.iterator();
      while(iterator.hasNext()) {
      FooItem item = iterator.next();
      }

      I think you have the same issue. What do you think about it?

      Delete
  7. I fail to get your point: as I see it, a Spliterator is just an Iterator for parallel traversal, so if you don't have anything against a client being able to do summary.iterator(), than why do you have something against a client's ability to do summary.spliterator()?
    And what's not to like about something as readable as:
    summary.forEach(item -> System.out::println);
    ?

    ReplyDelete
  8. Is the assertion here that these additions to Iterable violate the interface-segregation principle?

    ReplyDelete

Please be aware that by commenting you provide consent to associate your selected profile with your comment. Long comments or those with excessive links may be deleted by Blogger (not me!). All spam will be deleted.