Sunday, 29 May 2011

Beans and properties

A few words on beans and properties in Java, prompted by the JavaFX 2.0 API and my work on Joda-Beans.

Beans and Properties

I was prompted to write this by the new API of JavaFX 2.0. It has a new extension to JavaBeans that adds support for properties, so that the JavaFX GUI can bind one property to another. I've been writing property extensions to JavaBeans for over 12 years now, so I feel I should comment on the design.

The basic design approach is to have an object representing each property on a bean. This object allows meta-data attributes to be stored per property (useful for marking errors), property change listeners (for GUIs) and tool access without reflection (allowing XML input/output, XPath, JSON etc).

The basic design adds an additional method to the JavaBean pattern. Alongside getSurname() and setSurname() there would be the property method surname(). This would return a Property object, which supports querying and setting the value, checking whether it was read-only or read-write, finding its type and adding property change listeners.

// basic querying via JavaBeans or property
  String surname = person.getSurname();
  String surname = person.surname().get();
  
  // additional features available with properties
  Property<String> property1 = person.surname();
  Property<String> property2 = person.property("surname");
  Map<String, Property<?>> propertyMap = person.propertyMap();
  person.surname().setAttribute("error", "MANDATORY");
  person.surname().addListener(...);

All the versions of this I've written follow a pattern relatively similar to that above. And the JavaFX 2.0 design is remarkably similar to this. However, some implementations of the pattern are more successful than others.

Version 1 - Original Joda

My initial work in the area was private, but I wrote a variation in open source around 2000 (unmaintained from 2002). This was the first Joda project (before Joda-Time), and still lives on the web here (Javadoc). Here is the original Joda approach (shown with generics for clarity):

public interface Person {
    Property<String> surname();
    String getSurname();
    void setSurname(String surname);
  }

In that project, the class was created using a Proxy object. Clearly, this approach was rather limiting, as a factory had to be used to created an instance of a bean. For various reasons, including the factories, this version was abandoned.

Version 2 - Instantiated Property objects

This version existed at one of my old day jobs. It used code generation rather than Proxy, which was better. However, it used this approach to the beans:

public class Person extends Bean {
    private final Property<String> surname = new Property(String.class);  // object created when person created
 
    public Property<String> surname() { return surname; }
    public String getSurname() { return (Strung) surname.get(); }     // state held on the property
    public void setSurname(String surname) { surname.set(surname); }
  }

Note that every time a Person is created, a Property object is created. Now imagine what happens when you have 20 properties on the bean - thats 21 object creations for an empty Person! And 21 garbage collections! It turns out that this design works OK for client side work, where there is a single machine and processing is limited by what one human can do. However, on the server side it performs horribly. Thus, it got converted into version 3.

Version 3 - Data on the bean

The version 2 approach stored all the state on the Property, but this approach does not work well with the JVM (it could, if the JVM/Java had the ability to inline the state of immediate child objects into the state of the parent object). Thus, the next approach was to move all the state from the property to the bean:

public class Bean {
    private final Map<String, String> attributes;
  }
  public class Person extends Bean {
    private final String surname;
 
    // code generated:
    public Property surname() { return new Property<String>(this, "surname"); }
    public String getSurname() { return surname; }
    public void setSurname(String surname) { this.surname = surname; }
 
    protected Object propertyGet(String propertyName) {
      switch (propertyName.hashCode()) {
        case 36527358: return getSurname();  // number is "surname".hashCode()
      }
      return null;
    }
  }

This approach moves all the state from the property back to the bean itself. The superclass would store any common items, like the attribute map (for error code) or the listeners, while the main person class stores the data using normal fields. The code generation adds a method that connects up the field without reflection.

Since the property is esentially stateless, it can be created on demand, used and then discarded. Short lived objects like that are good for the garbage collector, so long as they are not used too frequently. This design was successful on the server side, and supported all the other goodies, like conversion to and from XML and XPath searching, all without reflection. I believe that a dynamic code generation approach would also work, but the static code generation was effective enough that we didn't go further.

Version 4 - Joda-Beans (the next generation...)

The new Joda-Beans project (Javadoc) is an open source version of the version 3 approach, written from the ground up, but focussed on the essential elements for simplicity. It is currently in use at OpenGamma.

Again, a code generator is used to create the getters, setters and property methods, plus the supporting methods like propertyGet() and propertySet(). This time however, there is a greater focus on the meta-property level, which is the representation of the surname property as a concept, rather than tied to a particular bean instance. Think of the relationship between a property and a meta-property as similar to the relationship between an object and a class. (Effectively, Joda-beans defines both type 1 and type 2 properties.)

Rather than show a code sample here, I encourage you to look at some example classes to see what the code generator produces: Person, Address and the subclass CompanyAddress. The developer only writes the part outside the "code generated" block. The block is generated and re-generated without destroying any additional methods that are hand-written, so you can treat it as a "normal" class in all other ways.

Once again, the property objects hold no long-lived state and are created on demand. After 12 years of working with properties, I'm convinced that is the only design that scales acceptably for the server side on the JVM.

While the Joda-Beans project is not at v1.0 yet, it does work and enhances coding in Java. I'd love to have some early adopters try it out!

JavaFX 2.0

The JavaFX 2.0 approach uses stateful properties (Javadoc - javafx.beans packages). As I've discussed above, such an approach is powerful and conceptually simple, but has performance issues on anything other than the client side. Some articles on the JavaFX 2.0 approach are here and here Clearly, a major concern of the design is listeners, but I still have concerns about this approach.

Other implementations

There are other projects too that have tackled this problem, including Proper properties, Bean-properties, Better beans binding, and Remi's compiler changes (I'll add anything in the comments to this list). No project has won a large mind share though, and most seem unmaintained. A related solution is the ugly JPA typesafe extension.

Language solutions

I've said this before and I'll say it again. The lack of properties in Java is the biggest missing langauge feature and the amount of wasted development time globally by developers writing beans or tools to access properties is gigantic. Moreover, I believe that adding properties to Java is more important than both generics and closures, and should have been prioritised accordingly. (Every enterprise developer knows how much they use the broken JavaBean approach.)

The new JVM languages all have properties (no surprise there), although not all have proper support for obtaining an object to represent the property. Maybe one day, Java itself will support properties as a language feature, it is being vaguely mentioned now, but I'm not holding my breath.

Summary

Properties are a hugely beneficial language feature and raise the abstraction level of coding. While the Java language continues to not have them, you can use or help the Joda-Beans project to add them to your codebase.

My experience suggests that the JVM is poor at handling stateful property objects. Unfortunately, this is the route that JavaFX 2.0 has chosen. It might work for them (because they are client side, and because JVMs are getting faster), but I would advise against using JavaFX 2.0 properties on the server side without lots of performance testing.

Feedback welcome...

22 comments:

  1. Steve,
    JavaFX 2.0 properties are created lazily on demand.

    And I think your hashCode approach in version 3 is broken, because two property names may have the same hashCode.

    Dirk.

    ReplyDelete
  2. The dependence on hashCode can cause problems, as Dirk suggested.

    1) Two property names may have the same hashCode. Perhaps this is a special case in your code generation, however; if you have two properties that possess the same hashCode, add an equality check as well.
    2) Non-existent properties (on this particular object) may respond with values.

    Problem #1 is potentially fatal, but ameliorated with a backing equality check.

    Problem #2 is *probably* more of a curiosity than anything else, though good luck to anyone debugging who encounters it.

    I imagine these problems have already been considered, and would like to hear the reasoning.

    ReplyDelete
  3. Stephen Colebourne29 May 2011 at 15:44

    @Dirk/James, yes I'm aware of the hashCode switch limitation. It hasn't caused a problem in about 8 years of using the approach, so its definitely a low probability issue, but strictly speaking it is a potential bug. With JDK 7, it can be changed to a string switch which works "correctly", but at lower performance.

    ReplyDelete
  4. What is your take on serialization? JavaFX's properties are not Serializable, and I guess this is purposeful because bindings and event listeners hooked into GUI components would often mean that an attempt to serialize your Person bean carries together the entire application model and UI tree into the stream... Joda-Beans looks similar too. We can argue that serialization (as done by Java) is an evil and failed idea, but at least EJB session beans and most web frameworks (for session data) are still very relevant and heavily dependent on serialization.

    ReplyDelete
  5. Stephen Colebourne29 May 2011 at 16:34

    @Osvaldo, the original Joda project had stateful properties not suited to serialization. However, the new Joda-Beans project only has lightweight properties which are semi-stateless. Thus there is no great need to serialize the Joda-Bean property (all the data is on the bean itself, so that serializes as per any other bean). That said, it would be perfectly possible to serialize the Joda-Bean property, as it would simply be a pointer to the "surname" on the "Person" of type "String".

    I should note that the new Joda-Beans project has no support for listeners ATM, as I have no need for that. If it were to be added, I might encourage it to hold its state completely separate from the bean, such as in a centralised manager.

    ReplyDelete
  6. Since Dirk raised the initial concern, I mostly wanted to point out that there were two cases, and point out that the more directly harmful case could be fixed at the time of code-generation.

    So if you include an equality check on the string iff the bean includes two property strings with the same hash code, you'll reduce the problem profile to isolated erroneous responses to non-existent property strings. You would pay a cost in the readability of the generated code only if there would otherwise be a problem.

    Looking at your code, since propertyGet is a protected method mostly called by equals, hashCode, and toString, this change might handle all of your expected behavior (though perhaps a brief warning in the protected method's javadoc would be merited).

    Anyway, the tradeoffs are naturally yours to make.

    On a separate note, I agree that Properties are a major item missing from the Java language spec, though I disagree that they're a bigger deal than generics (on closures, I must reserve judgement).

    ReplyDelete
  7. Hello,

    Not being an expert in terms of languages an compilers, could someone explain why the following would be a bad idea: the use of annotations:

    @ReadOnly
    String nino;

    The compiler would generate the getter only

    @ReadWrite
    etc

    Or it could be

    @Property(Prop.ReadOnly)

    If the code already has the getter and setter it does nothing, or could raise an error if one declares the property as read only and a setter is found...

    etc

    ReplyDelete
  8. I agree; properties are really important. Property objects are great, but unfortunately limit use/navigation to just 1-level, e.g.:

    bind(someUi, myEmployee.name);

    And don't allow:

    bind(someUi, myEmployee.employer.name);

    I think for the property problem to be "solved", especially if rolled in as a first-class language feature, I think the 2nd line needs to be supported just as much as the 1st.

    There are two ways I know of to do something like this, one is my bindgen (http://www.bindgen.org) project (which is an ugly but unfortunately necessary/functional hack for accomplishing its goals):

    bind(someUi, new EmployeeBinding(myEmployee).employer().name());

    Or Gosu's feature literals:

    bind(someUi, myEmployee#employer#name);

    http://guidewiredevelopment.wordpress.com/2011/03/03/feature-literals/

    Gosu, IMO, did their approach exactly right (for example, they support both stateful and stateless literals). Anyone looking to support properties as a first-class language concern (whether in Java or elsewhere), should heavily crib off Gosu's approach as I think how client code can access/bind the properties is just as important as the implementation code for the properties themselves.

    Note that technically Gosu's feature literals give you basic (listener-less) data binding without any property objects, so should be pretty light weight (you can think of it as just syntax sugar for caller-side inner classes). I'm not sure how their caller-side approach would integrate with callee-side stateful properties that supported listeners; it would be something to think about it.

    ReplyDelete
  9. Stephen Colebourne30 May 2011 at 20:23

    @James, I agree that code generation can make the hashCode approach more robust. If that was a big deal stopping use of JodaBeans, then I'd accept a github pull request to fix it!

    @CoolSteph, There are many ways to add properties to the language, and annotations are one. But its probably not the one I'd use.

    @Stephen, bindgen looks interesting, thanks for the link. I understand the nested properties point. As a language change goal it seems reasonable, but your approach may be the only possible way in libraries (and its not something I can ever remember needing).

    ReplyDelete
  10. I agree that Java needs properties, badly. Additionally, it needs member expressions (ala C#), so that APIs that perform configuration based on properties don't need to use string references to property names but instead can use actual references to the members themselves. This would be huge.

    ReplyDelete
  11. Followup - I just read @Stephen's comment on Gosu. It looks like its feature literals are basically the same thing as member expressions. It's good to see others appreciate the value of such a thing.

    ReplyDelete
  12. Actually, our implementation is currently using stateful properties, but it doesn't have to. Our property objects are getting "bean" and "name" properties, which means we could go to a GC-heavy stateless approach in the future, primarily when reflection is no longer needed (and we can use lambdas to do the job).

    ReplyDelete
  13. @CoolSteph:

    Gosu has a short hand syntax for exposing a field as a property:

    var _field1 as Field1
    var _field2 as readonly Field2

    @Stephen Colebourne:

    Totally agree that properties are a must have for a programming language, particularly for UI data-binding. Feature literals answer the reference vs. evaluation problem: you use the dot syntax to evaluate and the sharp syntax to reference. With respect to a first class listener implementation for feature literals, that is not supported because feature literals apply to any type and, therefore, every type would have to provide the needed listener infrastructure.

    *However* using enhancements and the implicit parameterization of feature literals, you could implement a listener infrastructure yourself, for your own classes.

    I'll put a post up on how to do that over at the guidewire dev blog later today.

    ReplyDelete
  14. OK, threw together a blog post here: http://guidewiredevelopment.wordpress.com/2011/06/05/feature-literals-enhancements-blocks-win/

    ReplyDelete
  15. Stephen Colebourne5 June 2011 at 22:20

    @Richard, I do think that method handles and lambdas will offer new options, I just find it frustrating trying to add a core feature outside the language.

    @Carson, thanks for writing about Gosu. The feature literal element looks good, and fits my view of how such things should work with # for the meta level. Unfortunately, Gosu declares variables as name : type, which makes it rather unappealing to me. Fix that, and I might take a real look at the language...

    ReplyDelete
  16. Yeah, the "name : Type" convention takes a bit to get used to. There are advantages to it, though. For example, generic functions can be declared and invoked much more naturally, since the return type appears after the method name, rather than before.

    It took me a while, but I switch between gosu and java quite a bit and don't notice it anymore. FWIW, it seems like most new statically typed languages are going with this convention.

    ReplyDelete
  17. A kind of "Generic Beans" is something I'm interested in for some time now.
    The JPA-Typesafe stuff is ugly how they did it but has one special feature which caught my attention. We use what i call a TypedKey approach to get some kind of type safety when storing mixed data in a Map (mostly for "tunneling" data through some generic code).

    public class TypedKey {...}
    static final TypedKey KEY_MY_INT1 = TypedKey.of("myint1");

    TypedKeyMap tkm = ...
    ...
    tkm.put(KEY_MY_INT1, 1234);
    ...
    Integer myint1 = tkm.get(KEY_MY_INT1);

    The point here is that the KEY carries the data type of the value as JPA does it.
    I took a look at the Person example.
    1. I wonder why you don't make use of typed keys? You "objectified" the value as a Property, why not take this a step further and use the generated meta constants as objectified keys?

    2. Avoiding 2 switches
    The generated meta code has direct access to the properties. So some kind of

    new Accessor() {
    String get() {
    return surname;
    }
    void set(String v) {
    surname = v;
    }
    }

    as part of the Metadata-Generation would provide direct access to the field without reflection.
    Using the meta data constants instead of Strings-Names would reduce the switches in propertyGet and propertySet to a delegate to the key.
    3. Make typed keys public
    I would also think about making the meta constants public for direct reference in other code. This way they could be used as keys to the property map providing some value type safety to avoid the " T get()" approach.

    Opinions?

    P.S.1
    Are you aware of lombok? (http://projectlombok.org/index.html)
    I didn't had the time to take a detailed look HOW they did it but sounds interesting generating getter/setter stuff on the fly using annotations.


    P.S.2
    Short note at DirectBean: I wonder why you just add the hashcodes instead of the "classical" 31 * oldHash + newHash leading to better distribution IMHO.

    ReplyDelete
  18. Stephen Colebourne27 June 2011 at 07:39

    @Carsten, Thanks for the detailed feedback. The TypedKey concept is interesting and I will take a further look. However, at first glance it looks similar to the Joda-Beans meta-bean property object (points 1 and 3). For the Person.surname() property, you can also get access to Person.meta().surname(), which is a MetaProperty. That can be used to access data directly without going back to a map, ie. metaProperty.get(bean), rather than bean.get(metaProperty) as with TypedKeyMap.

    A string key is useful to allow interaction with other framework/reflective systems that use a string as a key.

    The use of inner classes to access the data is possible, however it bloats the binary file size. It would remove the need for the string switch however and could be a generation option. That said, the right way for this code to go is to update it to Java 7 and method handles. With method handles, there will be no need for the string switch or the inner class. (I'd welcome a patch to add method handle support as an optional code generation)

    I am aware of Project Lombok, and perhaps this can be used by them at some point. Technically, they simply generate the simple getters and setters dynamically in the IDE.

    Finally, the DirectBean hash code and equals is now generated by code generation, so the superclass implementation isn't relevant any more. I think it was done that way originally to match the algorithm of a Set.

    ReplyDelete
  19. Hi Stephen, interesting project. Have you considered integrating with other bean generation tools like xjc? This would make a very interesting extension.

    ReplyDelete
    Replies
    1. I can't say I've ever used xjc. Happy to have github forks and pull requests though...

      Delete
  20. I know this is a very late follow-up, but Lombok seems to be fairly mature at this point and in my experience "just works," at least from my Maven builds. The IDE plugin is pretty transparent. When I can't use Groovy, it's a great tool to have around.

    They don't seem to generate the Property calls you have in your example, but otherwise the @Data annotation covers my (server-side-only) needs. (When I can't use @Value.) Adding an @Bean annotation to fill in the gap should be feasible.

    ReplyDelete
  21. Hello, it is very interesting article. Maybe will attract you a solution with a similar approach, where they are used static Keys on the place of the Property class. The basic features are described on the article:
    http://ujorm.org/sample/key-value.html

    ReplyDelete