Monday, 12 March 2007

Configuration in Java - It sure beats XML!

Is the Java community slowly remembering what's good about Java? I'm talking about static typing.

For a long time, Java developers have been forced by standards committees and framework writers to write (or generate) reams of XML. The argument for this was flexibility, simplicity, standards.

The reality is that most Java developers will tell you of XML hell. These are files where refactoring doesn't work. Files where auto-complete doesn't work. Files where errors occur at runtime instead of compile-time. Files that get so unwieldy that we ended up writing tools like XDoclet to generate them. Files whose contents are not type-safe.

In other words, everyone forgot that Java is a statically-typed language, and that is one of its key strengths.

Well, it seems to me that things are slowly changing. The change started with annotations, which are slowly feeding through to new ideas and new frameworks. What prompted me to write this blog was the release of Guice. This is a new dependency injection framework based around annotations and configuration using Java code. For example:

  bind(Service.class)
    .annotatedWith(Blue.class)
    .to(BlueService.class);

I've yet to use it, but I already like it ;-)

So, Guice allows us to replace reams of XML configuration in a tool like Spring with something a lot more compact, that can be refactored, thats type-safe, supports auto-complete and thats compile-time checked.

But lets not stop with module definitions. Many applications have hundreds or thousands of pieces of configuration stored in XML, text or properties files as arbitrary key-value pairs. Each has to be read in, parsed, validated before it can be used. And if you don't validate then an exception will occur.

My question is why don't we define this configuration in Java?

public class Config {
  public static <T> List<T> list(T... items) {
    return Arrays.asList(items);
  }
}
public class LogonConfig extends Config {
  public int userNameMinLength;
  public int userNameMaxLength;
  public int passwordMinLength;
  public int passwordMaxLength;
  public List<String> invalidPasswords;
}
public class LogonConfigInit {
  public void init(LogonConfig config) {
    config.userNameMinLength = 6;
    config.userNameMaxLength = 20;
    config.passwordMinLength = 6;
    config.passwordMaxLength = 20;
    config.invalidPasswords = list("password","mypass");
  }
}

So, we've got a common superclass, Config, with useful helper methods. A POJO, LogonConfig, without getters and setters (why are they needed here?). And the real class, LogonConfigInit, where the work is actually done. (By the way, please don't get hung up on the details of this example. Its late, and I can't think of anything better right now.)

So, is the Java class LogonConfigInit really any more complicated than an XML file? I don't think so, yet it has so many benefits in terms of speed, refactoring, validation, ... Heck we could even step through it with a debugger!

Whats more, its easy to have variations, or overrides. For example, if the XxxInit file was for database config, you could imagine decorating it with setups by machine, one for production, one for QA, etc.

So, what about reloading changes to configuration at runtime? Thats why we use text/xml files isn't it? Well once again, that needn't be true. Another of Java's key strengths is its ability to dynamically load classes, and to control the classloader.

All we need is a module that handles the 'reload configuration' button, invokes javac via the Java 6 API, compiles the new version of LogonConfigInit and runs it to load the new config. Apart from the plumbing, why is this fundamentally any different to reloading a text/xml file?

Well, there's my challenge to the Java community - how about a rock solid configuration framework with full class compilation and reloading and not a drop of XML in sight. Just compile-time checked, statically-typed, 100% pure Java. Who knows, maybe it already exists!

As always, your opinions on the issues raised are most welcome!

25 comments:

  1. The whole idea of having xml is not to recompile. we you have this inside of your ear file and want to change it then you need another build. This is not the case with xml since you just simple replace the xml file.
    If you are insisting on your idea then your framework should take care of this as well.

    ReplyDelete
  2. XML is a disease. Today's hip young programmers are falling over themselves to show how they can avoid compiling and writing code as if that were a good thing. Well, guess what - XML is code. It must be developed, maintained and deployed. When moving to XML you are substituting a great language with great tools (Java) for an inferior language with inferior tools. The quicker this illness passes, they better.

    ReplyDelete
  3. If the idea is so that you do not recompile, then I say simple text property files also meet that requirement.

    ReplyDelete
  4. Totally with you on the configuration framework idea, I have wanted that since the bloody beginning of enterprise Java. Sun is so bizarre. They made one of the biggest config disasters in all of history (EJB), then they have spun out JSRs on all manner of trivial matters, and when we want to change a property, we have to explode an ear, change it, then jar the mess back up and redeploy. As for your 'maybe it already exists,' well, it kind of does: JMX can do all this and more. You don't even need a restart. The newest JMX in JDK 6 finally delivers on some of the congenital midgetry that was originally baked in. Furthermore, instrumentation can be done now in very compelling and flexible ways: some annotations, some aspects, etc.

    As for your example, I know you were saying ignore it, but in fact, ironically, the first thing that came to mind was constraints: another gaping hole in the Java landscape, but like the other one: a hole that can now be patched. Things like mins and maxes should be done w/annotations DbC style. Checkout Contract4J for that. You would still need to write some code to use them, but that's not a problem.

    ReplyDelete
  5. Several years ago XML gurus told us that configuration/markup and business logic - everything should be in XML. Now we have an inverted situation. In my personal view both are wrong and the truth is in the middle (very close to Spring :)

    P.S. You can also use Java debugging hot-swap to change configuration parameters at runtime ;)

    ReplyDelete
  6. How about having .xml.java and/or .xml.class files next to the .xml files. They must be implementing an interface that outputs XML as text. Then:

    1) The framework can look for these files and generate the XML on the fly.
    2) You can have an Ant task generate the XML for frameworks that doesn't know about this yet.
    3) Have a running service that constantly reads the .xml.java/.xml.class files and generate the XML. This is for frameworks that both doesn't know about this new feature and needs for instance web.xml during the development (like Studio Creator).

    This mean that you can have type checked Java code that is your configuration, AND it would be child's play for all the frameworks to support it (which isn't even needed during a transition phase). Something like this for web.xml:

    in a file called "web.xml.java"
    addServlet(MyServlet.NAME, MyServlet.class, true);
    addServletMapping(MyServlet.NAME, "*.jsp");

    There can of course be a lot of nice feature included with this, like merging the new XML with what's in the original xml file and what not.

    How about it?

    Cheers,
    Mikael Grev

    ReplyDelete
  7. I also think that the right solution is in the middle... :-)

    1) I think the constant values that must be easily modified by a Sysadmin or a final user (host, port, log level, etc.) should reside in a config file, be it XML or ".properties". IMHO, it doesn't make sense to force the sysadmin or user to recompile and app just to change the port number, for instance.

    2) On the other hand, I'm against having a config file for the application architecture: which implementations of classes to use, how to connect them with each other, whether a class should be a singleton or not, ... I think changing those things deserve a recompilation and further testing , they're not so frequent and they don't need to be modified by a Sysadmin or a final user.

    In big apps, it's useful and maintainable to centralize the dependencies code (which belong to the app architecture), but I prefer to do it with code itself, not with XML files. So I wholly embrace the Guice solution, which allows to centralize the dependencies easily, but with code and more type safety.

    Regards,

    Xavi

    ReplyDelete
  8. I don't think there is anything wrong perse with using Java for configuration purposes, lots of languages, especially the more "script-like" ones do this all the time.

    You have to take care with deployment though. The configuration classes should not be part of the main body of code or otherwise an upgrade will wipe your settings. In fact I think they should be left out of the normal packages entirely.

    Maybe some structure where the default implementation (with the default values) is part of the normal source packages with some kind of generic mechanism to override that class with your own version.

    I do see a problem in situations where several modules might be using different versions of the same configuration class. With text/xml files it's easy to make them more robust so you can easily add and/or remove values from them without everything falling over. With classes that will not be possible I think.

    ReplyDelete
  9. For those who don't remember, EJB 1.0 did not have XML configuration files. Deployment descriptors were serializable Java classes.

    ReplyDelete
  10. To Rafael Chaves,
    Great example!

    To quintesse,
    Please, don't mix scripting languages with static typed/compiled ones. I've seen configuration.php many times and this is OK because the content is intuitive and does not require recompilation.

    P.S. JSP also allows dynamic configuration from Java in a PHP way, i.e.:
    <%
    Config.userNameMinLength = 6;
    Config.userNameMaxLength = 20
    %>

    ReplyDelete
  11. It is not correct to say that XML does not provide auto-completion. Any modern XML editor should be able to use a XSD to validate the XML on-the-fly and provide auto-completion. What is more, with XML and XSLT you can provide a nice, easy to read view of your configuration.

    Besides, anything can get unwieldy, Java interfaces uncluding. Configuration files are another kind of user interface that should be designed with care, for its use scenarios.

    ReplyDelete
  12. well, i think author gave a bad example and the commenters missed the point.
    The example given is not suitable for having it in the java class, it is suitablefor for an external "property" file. the idea shoud be replacing application logic related configurations (like Spring bean configurations, struts configurations, hibernate configurations ) that non programmer people cannot touch anyway. not the simple property file type parameters...

    ReplyDelete
  13. To afsina,
    Annotations perfectly solves the problem you desribe and they are supported by Spring, EJB3 and others quite a long time...
    Typical example is transaction attributes.

    ReplyDelete
  14. Jordan: [ When moving to XML you are substituting a great language with great tools (Java) for an inferior language with inferior tools. ]

    I'm not sure what you mean by that. With how I use Spring and IoC is that I remove hard coding into my classes a way of connecting to a database, or putting into my classes some JNDI lookup with a magic value to get that back. This forces you to code to interfaces which is what you should be doing anyway.

    How do you do IoC? Or do you? How do you unit test just the individual layers of MVC that you're working on? What alternative do you have?

    ReplyDelete
  15. @ejboy: mixing scripting and compiled languages? I didn't know I was suggesting such a thing? Do you mean I was confusing them? Either way I don't agree, it was just an example of languages that use their existing syntax to do configuration. I'm aware that you can't just do the same in Java but it might be worth considering.

    @Jordan: well, a possible (but undoubtedly silly) solution would be to use Spring to handle your configuration normally, maybe by having it create the Config objects (beans) and passing (inserting) them (in)to the objects that need them. But instead of using the Spring xml configuration files to set the Config bean's values you could have it create special Config Initializer beans that can be edited at any moment and recompiled and reloaded by the framework.

    ReplyDelete
  16. i also believe it is easier for the programmer to configure with the help of the IDE, eg. in Java, i mean thats what i do. If a sysadmin might need to override defaults then one can use XMLEncoder to export as a file, and XMLEncoder to import the config object, or use JAXB, or inject resources from a properties file, possibly driven by annotations.

    ReplyDelete
  17. To elaborate a bit more on my last remark. Imagine you have:

    class ConnectionConfig {
    public String host;
    public int port;
    }

    you could image that instead of having some kind of property or xml file you could have something like:

    class ConnectionConfigInitializer {
    public void configure(ConnectionConfig config) {
    config.host = "example.org";
    config.port = 1234;
    }
    }

    (although a better way might need to be found for complex configuration with sub-objects and lists/arrays)

    The problem I see is that even with good use of annotations on the Config bean to add things like validation and maybe hints as to the precise domain of the needed values I think it would be very difficult to make configuration editors that could easily edit that kind of files (lots of apps have configuration interfaces, they should be able to write the values back to the file).

    And of course configuration files are often edited by people who don't use IDEs (users or administrators) or in situations where you have no access to those tools (on production servers) so that's another advantage of this format that you won't be able to use often in practice.

    Dunno, the longer I think about it the less convinced I am that this could actually work well. Somebody had better come up with some pretty convincing use-cases and examples :-)

    ReplyDelete
  18. I have a class called Context. It gets passed to every object during construction, if some of its members are needed by the object. It also gets passed to every static method as the first parameter if the static method needs some of its members.

    It's an emulation of Lisp special variables, though I started using it before I knew what those were.

    Only in two places do I construct a Context; a) in the 'main' for the application, and b) at the start of every automated test that needs a Context.

    I don't need to provide a way for non-programmers to reconfigure my application - and if I did, I'd do it by reading a property file, without really minding what format is used.

    I think Lisp as data looks better than XML, even for non-programmers.

    ReplyDelete
  19. Blogreader asks: How do you do IoC? Or do you? How do you unit test just the individual layers of MVC that you're working on? What alternative do you have?

    I believe it is a very bad idea to specify coupling in a configuration file. The Spring/IoC approach is a major mistake from what I know about it. The current trends in architecture are very disturbing to me. Architects today are going out of their way to hide logic and obfuscate relationships. I want to see the coupling directly in the debugger. I don't want to have to wade through config files to figure it out. The demos I've seen where someone very proudly makes one change in an XML file and the behavior of the app changes scares me instead of impressing me.

    ReplyDelete
  20. "@Jordan: well, a possible (but undoubtedly silly) solution would be to use Spring to handle your configuration normally..."

    Spring's "dependency injection" seems heavy-handed and unnecessary to me. At the end of the day, it's just calling Class.forName and passing some parameters (sorry if this is a crude summary, I'm by no means a Spring expert).

    ReplyDelete
  21. Stephen Colebourne12 March 2007 23:16

    Thanks for all the great feedback! There's too much for individual replies, but I'd like to agree with the sentiment that "XML is code" and not very fun code to work with either.

    The point about needing to update the configuration using an online tool is well-made too, so anyone writing a configuration tool off the back of this blog would need to take this into account too.

    Finally, there were a couple of replies that missed the point of the blog. My argument is that a .java file *is* a text file capable of storing configuration. The only real differences are that you have to declare a class and method at the top/bottom (but you have headers and footers in XML too), and you have to compile the file before use. My point was that we need a configuration framework that can make the compilation and re-load part transparent. I want a tool where a .java file can be treated just like an xml file, such that it doesn't require a rebuild of the main jar/war/ear to pickup the changes.

    Anyway, its good to know that I'm not alone in finding XML frustrating.

    ReplyDelete
  22. "Anyway, its good to know that I'm not alone in finding XML frustrating"

    Well, personally I don't find XML frustrating at all because I still remember how (complex) configuration files were written before XML came along (see Apache). Basically eveybody made up his own format and wrote his own parsers, preferrably a different one for each project and I definitely don't want to go back to those times.

    But it doesn't mean I'm not open for better and more elegant solutions for specific problems if somebody can come up with them.

    ReplyDelete
  23. Petite already does this:
    http://jodd.sourceforge.net/doc/petite.html

    ReplyDelete
  24. "Petite already does this"

    Dunno, what it seems to do is automatic registration of beans, but I couldn't find anything related to initialization and the automatic recompilation of source files as suggested by Stephen.

    ReplyDelete
  25. Once I worked on a 'configuration service' for a Java distributed system. The system modules communicated with each other through a popular middleware (it wasn't SOA, the nature of those processes was different). Every module had several own config (xml) files (2 or 3, it depended on a module type). The system had to check its health at init stage before 'switching' into its production mode.

    XML config files in that case simply didn't work - ideally, I needed those files to be able to communicate to each other in order to check their corresponding values, exchange with their modules' test results, write the log, etc. Eventually I did it but with XML parsers/builders and XSD validation overhead. Those XML files had to be Java classes from the very beginning. And not just ordinary value holders but able to encapsulate a required behaviour.

    About IoC and text files: yeah, the note above makes sense. But why not to keep Java classes in files, let's say, with *.javat extension (java-text) as text files without compiling until they are requested by the runtime? A typical webapp does that for JSPs (off cause, unless they have not been compiled beforehand). If such an extension added to javac/jre, xml config files will have to go...

    About XML gurus: I've already seen a position of XML Architect advertised somewhere! XML does become more complicated. But will it ever have more power than OO language anyway?

    About non-programmers changing config XMLs: never seen it before. If it's the case, how many programmers would be on-call before, during and after the change made? :)

    ReplyDelete