Tuesday 29 November 2011

Real life Scala feedback from Yammer

Following up on my recent Scala posts, some commented that because I haven't used Scala seriously, my opinion is of little value. I responded by noting that writing the FCM closures proposal for Java, altering the javac compiler, and talking at conferences about language design might qualify me to have an opinion.

Having said all that, I do believe that the most valuable feedback comes from those that have tried Scala, and found it wanting.

Today there was a good example of this, the most important kind of feedback. Yammer, the Enterprise Social Network, announced provided an indication that they are moving away from Scala, and back to Java. And whats more, they provided a very detailed rationale. The original gist disappeared, so for certainty I've copied the current version inline here (my bold highlights):

Update: Twitter user coda says "@jodastephen I'm saying you're misrepresenting it. It was a personal email which did not represent the views of my employer. In the same sense your Twitter account isn't an official publication of OpenGamma, neither is my personal email.". The main point appears to be that this isn't official Yammer policy. Personally, I think the contents are more interesting than whether its an official announcement or not, but YMMV... For the record, here is the original Hacker News thread.

Update 2011-11-30: Coda has explained how this document (originally a private email) came to be public. For the record, I got the story off Twitter, then Hacker News, and did not receive anything privately, but wanted to ensure the quality of this feedback reached a wider audience (especially given the original gist was deleted). The text of the explanation is also in a comment below.

Update 2011-12-01: A formal response has now arrived from Yammer. If you read the text below, you should also read the response linked above to get the fuller picture. Very little in tech is black and white, and, despite what some readers may think, I understand that very well.


Hi Donald (and Martin),

Thanks for pinging me; it's nice to know Typesafe is keeping tabs on this, and I appreciate the tone. This is a Yegge-long response, but given that you and Martin are the two people best-situated to do anything about this, I'd rather err on the side of giving you too much to think about. I realize I'm being very critical of something in which you've invested a great deal (both financially and professionally) and I want to be explicit about my intentions: I think the world could benefit from a better Scala, and I'd like to see that work out even if it doesn't change what we're doing here.

Right now at Yammer we're moving our basic infrastructure stack over to Java, and keeping Scala support around in the form of fa├žades and legacy libraries. It's not a hurried process and we're just starting out on it, but it's been a long time coming. The essence of it is that the friction and complexity that comes with using Scala instead of Java isn't offset by enough productivity benefit or reduction of maintenance burden for it to make sense as our default language. We'll still have Scala in production, probably in perpetuity, but going forward our main development target will be Java.


Scala, as a language, has some profoundly interesting ideas in it. That's one of the things which attracted me to it in the first place. But it's also a very complex language. The number of concepts I had to explain to new members of our team for even the simplest usage of a collection was surprising: implicit parameters, builder typeclasses, "operator overloading", return type inference, etc. etc. Then the particulars: what's a Traversable vs. a TraversableOnce? GenTraversable? Iterable? IterableLike? Should they be choosing the most general type for parameters, and if so what was that? What was a =:= and where could they get one from?

A lot of this has been waved away as something only library authors really need to know about, but when an library's API bubbles all of this up to the top (and since most of these features resolve specifics at the call site, they do), engineers need to have an accurate mental model of how these libraries work or they shift into cargo-culting snippets of code as magic talismans of functionality.

In addition to the concepts and specific implementations that Scala introduces, there is also a cultural layer of what it means to write idiomatic Scala. The most vocal — and thus most visible — members of the Scala community at large seem to tend either towards the comic buffoonery of attempting to compile their Haskell using scalac or towards vigorously and enthusiastically reinventing the wheel as a way of exercising concepts they'd been struggling with or curious about. As my team navigated these waters, they would occasionally ask things like: "So this one guy says the only way to do this is with a bijective map on a semi-algebra, whatever the hell that is, and this other guy says to use a library which doesn't have docs and didn't exist until last week and that he wrote. The first guy and the second guy seem to hate each other. What's the Scala way of sending an HTTP request to a server?" We had some patchwork code where idioms which had been heartily recommended and then hotly criticized on Stack Overflow threads were tried out, but at some point a best practice emerged: ignore the community entirely.

Not being able to rely on a strong community presence meant we had to fend for ourselves in figuring out what "good" Scala was. In hindsight, I definitely underestimated both the difficulty and importance of learning (and teaching) Scala. Because it's effectively impossible to hire people with prior Scala experience (of the hundreds of people we've interviewed perhaps three had Scala experience, of those three we hired one), this matters much more than it might otherwise. If we take even the strongest of JVM engineers and rush them into writing Scala, we increase our maintenance burden with their funky code; if we invest heavily in teaching new hires Scala they won't be writing production code for a while, increasing our time-to-market. Contrast this with the default for the JVM ecosystem: if new hires write Java, they're productive as soon as we can get them a keyboard.

Even once our team members got up to speed on Scala, the development story was never as easy as I'd thought it would be. Because one never writes pure Scala in an industrial setting, we found ourselves having to superimpose four different levels of mental model — the Scala we wrote, the Java we didn't write, the bytecode it all compiles into, and the actual problem we were writing code to solve. It wasn't until I wrote some pure Java that I realized how much extra burden that had been, and I've heard similar comments from other team members. Even with services that only used Scala libraries, the choice was never between Java and Scala; it was between Java and Scala-and-Java.

Adding to the unease in development were issues with the build toolchain. We started with SBT 0.7, which offered a pleasant interface to some rather dubious internals, but by the time SBT 0.10 came out, we'd had endless issues trying to debug or extend SBT. We looked at using 0.10, but we found it to have the exact same problems managing dependencies (read: Ivy), two new, different flavors of inpenetrable, undocumented, symbol-heavy API, and an implementation which can only be described as an idioglossia. The fact that SBT plugin authors had to discover what "best practices" are in order to avoid making two plugins accidentally incompatible should have been a red flag for any tool which includes typesafety as a selling point. (The fact that I tried to write a plugin to replace SBT's usage of Ivy with Maven's Aether library should have been a red flag for me.)

We ended up moving to Maven, which isn't pretty but works. We jettisoned all of the SBT plugins I wrote to duplicate Maven functionality, our IDE integration worked properly, and the rest of our release toolchain (CI, deployment, etc.) no longer needed custom shims to work. But using Maven really highlighted the second-class status assigned to it in the Scala ecosystem. In addition to the "enterprisey" cat-calls and disbelief from the community, we found out that pointing out scalac's incremental compilation bugs had gotten that feature removed outright. Even the deprecation warning for -make: suggests using SBT or an IDE. This emphasis on SBT being the one true way has meant the marginalization of Maven and Ant -- the two main build tools in the Java ecosystem.

Cross-building is also crazy-making. I don't have any good solutions for backwards compatibility, but each major Scala release being incompatible with the previous one biases Scala developers towards newer libraries and promotes wheel-reinventing in the general ecosystem. Most Scala releases contain improvements in day-to-day programming (including compilation speed), but an application developer has to wait until all their dependencies are upgraded before they themselves can upgrade. If they can't wait, they have to take on the maintenance burden of that library indefinitely. In order to reduce their maintenance overhead, they naturally look for another, roughly equivalent library with a more responsive author. Even if the older library is better-tested, better-documented, and better-featured it will still lose out over time as developers jump ship for something that works with Scala 2.next sooner. (It's also worth noting that most companies using Scala at scale or in mission-critical capacities will not immediately upgrade; the library authors they employ will likely be similarly conservative, and the benefit their experience brings to their code will benefit the community less and less over time. As far as I've found, we're the only big startup in SF using 2.9.)

Once in production, Scala's runtime characteristics were the least subtle problem. At one point, half the team was working on a distributed database, and given the write fanout for our large networks some parts of the code could be called 10-20M times per write. Via profiling and examining the bytecode we managed to get a 100x improvement by adopting some simple rules:

1. Don't ever use a for-loop. Creating a new object for the loop closure, passing it to the iterable, etc., ends up being a forest of invokevirtual calls, even for the simple case of iterating over an array. Writing the same code as a while-loop or tail recursive call brings it back to simple field access and gotos. While I'm sure Scala will be have better optimizations in the future, we had to mutilate a fair portion of our code in order to actually ship it. (In another service, we got away with just using the ScalaCL compiler plugin and copying things to and from arrays instead of using immutable collections.)

2. Don't ever use scala.collection.mutable. Replacing a scala.collection.mutable.HashMap with a java.util.HashMap in a wrapper produced an order-of-magnitude performance benefit for one of these loops. Again, this led to some heinous code as any of its methods which took a Builder or CanBuildFrom would immediately land us with a mutable.HashMap. (We ended up using explicit external iterators and a while-loop, too.)

3. Don't ever use scala.collection.immutable. Replacing a scala.collection.immutable.HashMap with a java.util.concurrent.ConcurrentHashMap in a wrapper also produced a large performance benefit for a strictly read-only workload. Replacing a small Set with an array for lookups was another big win, performance-wise.

4. Always use private[this]. Doing so avoids turning simple field access into an invokevirtual on generated getters and setters. Generally HotSpot would end up inlining these, but inside our inner serialization loop this made a huge difference.

5. Avoid closures. Ditching Specs2 for my little JUnit wrapper meant that the main test class for one of our projects (~600-700 lines) no longer took three minutes to compile or produced 6MB of .class files. It did this by not capturing everything as closures. At some point, we stopped seeing lambdas as free and started seeing them as syntactic sugar on top of anonymous classes and thus acquired the same distaste for them as we did anonymous classes.

Now, every language has its performance issues, and the best a standard library can hope to do is to hit 80% of use cases. But what we found were pervasive issues — we could replace all of our own usages of s.c.i.HashMap, but it's a class which is extensively used throughout the standard library. It being slower than j.u.HashMap means groupBy is slower, as is a lot of other collections functionality I like.

At some point, I wondered if the positive aspects of our development experience owed less to Scala and more to the set of libraries we use, so I spent a few days and roughly ported a medium-sized service to pure Java. I broached this issue with the team, demo'd the two codebases, and was actually surprised by the rather immediate consensus on switching. There's definitely aspects of Scala we'll miss, but it's not enough to keep us around.

Already I've moved our base web service stack to Java, with Scala support as a separate module. New services are already being written on it, and given the results from our Hack Day at the beginning of this week it hasn't slowed our ability to quickly ship complex code. I'm keeping a close eye on the effects of this change, but I'm optimistic, and the team seems excited. We'll see.


I've tried hard here not to offer you advice. Some of these problems could easily be specific to our team and our workload; some of them won't make a difference in how your company does; some of them aren't even your problems to solve, really. But they're still the problems we've encountered over the past two years, and they compose the bulk of what's motivating this change.

Despite the fact that we're moving away from Scala, I still think it's one of the most interesting, innovative, and exciting languages I've used, and I hope this giant wall of opinion helps you in some way to see it succeed. If there's anything here I can clarify for you, please let me know.

Its not for me to add any more to this, Yammer's opinion. There is a gold mine of information about Scala in there. And in my opinion, everyone thinking of adopting Scala should read it - in detail.

Update 2011-12-01: Just a quick reminder to read the responses from Coda/Yammer: immediate and formal.

Thursday 24 November 2011

Scala EJB 2 feedback

My Scala/EJB post generated plenty of attention, as I expected. I left the comments there for others to discuss the post - this is my reply to a few of the points.

EJB 2 comparison

A number of comments arose about the comparison, especially those feeling it didn't make sense. Basically, EJB 1/2 was one of those technologies that appeared at first glance to have a lot of promise, targetting known pain points. But over time everyone figured out that the approach was needlessly complicated and created more problems than it solved. Collectively, developers who lived through the era wonder how on earth it got adopted - in hindsight it seems obviously bad.

As I indicated in the blog, I was trained in EJB, but it was obvious to me that the technology was deeply flawed. So, I argued against its adoption, and never had to use it in anger. I see Scala as equally deeply flawed, thus am arguing against its adoption, and endeavouring to avoid using it in anger.

Since I had that same reaction in my gut as I did with EJB, I used the analogy - a high level analogy, not a low level one. To me, Scala really does feel as bad an idea as EJB 2 did.


The trouble with today's tweet based soundbites is that it is difficult to have a slightly subtle position on something, and I'd say my position on Fantom is subtle. I think Fantom is hugely interesting because it shows what happens when you challenge your preconceptions about what a static type system should do (and also because of its ability to turn shared mutable state into a compile error).

The subtlety is that I don't see any evidence that the majority of the static typing community of developers (ie. Java) are willing to take the radical step that Fantom offers (paring down static typing to the bare minimum). For me, I think Ceylon and Kotlin are both being seduced into adding more to the type system than developers really need.

In my Devoxx talk, and in the evaluating Fantom blog post, I made the point about the type system. I also suggested that Fantom might well appeal to those from the dynamic side of the fence who have been bitten by an absence of static typing (like Ruby).

Thus, Fantom makes a good counterpoint to Scala. They are pretty much polar opposites in the static typing space. And I find it interesting and worth noting that Fantom spends its language complexity budget on things I care about, whereas Scala (over)spends its complexity budget on things I don't care about.

Thus, while it may seem like I'm saying "use Fantom, use Fantom, please use Fantom", I'm really just using it as an effective counterpoint. Pointing out something that in my opinion has better answers to the hard questions is not the same as saying go and adopt it. Linked yes, but not the same.

The other points

A number of comments from the Scala side noted that modules (of the type I was referring to) were a problem. I will also willingly acknowledge the heritage of the word module in other contexts.

On concurrency, some got the message and some didn't. My point is that you can design a programming language such that shared mutable state does not compile. Scala talks a good game in this area, but in forensic analysis it doesn't match up.

On the type system, some feel the strength of Scala's approach is valuable, while some like me see it as way too far beyond the point of sensible returns. I also maintain that if I add a string to a list of integers I should get a compile error, not a list of Any. With type inference and implicits, there is far too much potential for things that should be errors during maintenance/refactoring to not be spotted for my taste.

On syntax I was primarily driving at the open and flexible nature of the design. With optional method invocation dot, optional brackets, optional semicolons, symbols for methods/operators and implicits thrown in, it will necessarily be harder than many languages to work out what any individual piece of code does. And there are consequences. That flexibility leaves ample room for mailing list discussion about the "right way" to do something. It also makes it very difficult for IDEs and compilers to figure out what the code means - which is the reason for the slow compile speeds mentioned in a number of comments. Personally, I find the goal of the open and flexible syntax (arbitrary DSLs) to be not worth the pain. There are other neater ways to think about DSLs.

FUD, criticism and my goals

I was accused of spreading FUD. No big surprise there. My view is that if that was my goal I would have done a better job in showing the more complex interactions of the feature set, or just flat out lied. No, I think my goal was a bit more interesting than just FUD.

Basically, the key goal with the blog was to provide reassurance to others who feel as I do that Scala just isn't right. I opened my blog talking about the Scala community not liking dissent, and I stand by that. A number of reactions actually praised my bravery for being willing to stand up to the "Scala cult". I don't think its my imagination to suggest that Scala's enthusiasts have managed to stifle criticism and given the impression that you'd be crazy not to use Scala. If I have inspired confidence in others to speak out, or question what they've heard, then that is a Good Thing in my book.

Beyond that, the long term theme of this blog has been that we should look again at just how much Java threw away from C, and judge new languages as much on what they threw away as what they added. For me, Scala didn't throw enough away and added too much - a lethal combination.

In the end, as Dick Wall suggested, individuals should try it for themselves. I just ask those that do to think deeply before adopting it, and as I said in a tweet - make sure you get both the positives and negatives before deciding.

My personal favourite responses

These are selected because I found them funny, or the point they made was interesting to my biased eyes. I'll let you figure out which are which!

Colebourne is a sad, old twat.
Anonymous comment

He sounds like a naughty schoolboy who misses being spanked..

I developed with Scala for about a year. I learn a lot and got to work with some smart people. However, then I had to maintain code and, oh boy, it was like being forced onto a diet of live insects.
Anonymous comment

I've only evaluated Scala at the surface, but simply want to back up Stephen in agreeing with the subjective scary feel of the language. I can think of no other languages where I've scanned examples and documentation, and wanted to run away as I have with Scala.
Casper Bang comment

I'm sure the type system is genius, but I'd prefer if it was sensible genius instead of mad genius. I like my sanity.
mcv comment

Years ago, I was pretty excited about it. I saw lots of things Java couldn't do, and that would make my life easier. But then more features. And more. But I think that even before I got totally turned off by the language, I got turned off by the community first. It felt very elitist and unwelcoming to people who may be just interested in a language to 'get things done' instead of one you can put on your resume and earn an immediate 'smart like hell' badge with.
chillenious comment

What can be worse than a likeness to EJB 2? A likeness to WS-*
Paul Sandoz tweet

You know who else compared Scala with EJB? Hitler.
Runar Oli tweet (which I took in good humour)

According to @jodastephen "Scala feels like EJB 2" http://blog.joda.org/2011/11/scala-feels-like-ejb-2-and-other.html ... will the next article explain how #Scala will cause world war 3?
Mario Fusco tweet

FWIW, I've used Scala for two years, written tens of thousands of lines of code in it, and find your criticism incisive.
Coda Hale tweet

Not that I really should comment on Scala, but I feel that 50% of it would be better than 100% of it -- "too much of everything"
Tatu Saloranta tweet

i used scala for a few months. it sounded very promising, java without the verbosity. but in the end i decided to stop using it.
The biggest problem for me was readability. Scala is the first language that i've learned where at first i couldn't just read code and immediately guess what it does.

My personal experience in wrapping non-trivial Java libraries in Scala and Clojure is that with Clojure it usually just works and it works quickly. In Scala I am usually reduced to an extra hour or two of adding manifests to signatures until the compiler accepts it.
I am disappointed with Scala and having lived through EJB 1, EJB 2 and then onto Spring and EJB 3, I agree with Steve it makes me feel exactly the same as I felt about EJB 1 and EJB 2 - that is I am being sold overcomplicated technology as a panacea.
Tim Clark

I have similar feelings about Scala. It's a bit like C++. The difference is: I found a subset of C++ I liked.
Glyn Normington tweet

Scala sucks, and i'm blessed to know that i'm not alone feeling that way.
Evgeny Shepelyuk tweet

And finally

I don't see myself writing a post in quite the same way about Groovy, Ceylon, Kotlin, Xtend, Clojure,... I may critique them (all do or will have flaws), but I don't see myself ripping into them in the same way. There is just something about Scala...

My final thought is that it is OK to look at Scala and decide against. You're not alone.

Tuesday 22 November 2011

Scala feels like EJB 2, and other thoughts

At Devoxx last week I used the phrase "Scala feels like EJB 2 to me". What was on my mind?


For a number of years on this blog I've been mentioning a desire to write a post about Scala. Writing such a post is not easy, because anyone who has been paying attention to anti-Scala blog posts will know that writing one is a sure fire way of getting flamed. The Scala community is not tolerant of dissent.

But ultimately, I felt that it was important for me to speak out and express my opinions. As I said in my talk, if it was just me that had a poor opinion of Scala I would probably keep quiet (or try to figure out why I was out of step). But I perceive considerable uneasiness amongst many that have tried or looked at the language - something that reinforces my concerns.

Before I start I should mention that although I like the Fantom programming language, I also see merit in aspects of other languages - Groovy, Kotlin, Ceylon, Gosu, Xtend and many others. I also respect Clojure. By comparison, I really struggle to find positive feelings for Scala, and what positive feelings I have had have reduced over time. But why is that?

(For those not at my Devoxx talk, I tried to do two things - firstly to show Fantom off and explain how most simple comparisons to Scala rather miss the point, and secondly to point out some of the difficulties I have with Scala. To be clear, I'm not bashing Scala to promote Fantom. I'm bashing Scala because I think its entirely the wrong direction for the future.)

For this blog post Ive picked out a few key areas for discussion. I probably could have written a post 3 times as long as this one, and this one is very long as it is. There is lots to say about Scala, and very little is good.


Scala does not have a module system. By that I mean a deeply integrated system of modules that treats the basic program unit as something larger than a class, with versioning and dependencies. A key test is whether the new language compile modules or classes.

One of the greatest issues with Java is the lack of a module system. This absence has over time cause the platform to gain cruft (like CORBA) and struggle to shed it. The multi-year effort of modularising the JDK is evidence of how complicated this work is to do if not done from the start. And of course the Java platform will always have to support code not written in modules. Beyond the core JDK, most experienced Java developers have encountered the "Jar hell" scenario, where different versions of Jar files are required by different libraries and the ease with which it becomes impossible to assemble the whole application.

So, I have a clear sense that proper modularistion is a Good Thing, with all the versioned goodness that comes with it. (Managing change of a large application over time remains one of the largest problems faced by most large development shops, and one I don't see Scala tackling.) Over time, Java has evolved the Maven, Ivy and OSGi approaches to modules. Each has some benefits, but none are integrated into the platform itself, which is a significant disadvantage.

Yet, in a recent thread on modules, Scala aficionados claimed that Scala does have modules. In fact the opinion was clear - "see the object keyword", "Scala objects and path dependent types encode ML-style modules", "Also see http://www.mpi-sws.org/~rossberg/" (an academic paper). On further prompting, the ML view (standard source code can be used to express modules) was expanded on, before eventually the Scala approved way of using Maven/Ivy and the sbt tool was finally explained. There wasn't any real sense that this was a problem for Scala - so long as it integrated with the Java solutions that was fine.

I claim that integrating with Maven/Ivy is not fine. It misses huge opportunities to make life better on a topic where developers face real productivity issues in the field. Hence I commented that "Scala focuses on the wrong issues".

I also noted that backwards compatibility has been a constant problem of the Scala libraries. Modules are a tool for managing versioning and compatibility issues and would almost certainly have helped Scala evolve.

Finally, I noted that modules allow an application to find all the classes in the classpath/modulepath. This allows an application to find all the classes that implement an interface or annotation easily, which allows applications to be easily assembled from their parts. Java and Scala can achieve this, but only via complex and slow classpath scanning tools, like scannotation.


Scala makes a big deal about concurrency. About how the functional approach will aid the creation of safe multi-threaded code.

Except its really a bit of smoke and mirrors.

The big problem in concurrency is shared mutable state. It turns out that us developers are pretty bad at reasoning about it and using the tools at our disposal (synchronized, locks, java.util.concurrent) to manage that state. You'd expect that Scala would have tackled the concurrency problem at source - the shared mutable state - but it doesn't.

Scala (the language) does not know whether a class is immutable or not, nor does it provide a way to check is an object is immutable (Scala's libraries might help, but the language doesn't). As a result, it is perfectly possible to have a "static" (shared-thread) variable, or a "static" value of a mutable object. Its also possible to pass a mutable object to an actor and shared mutable state that way.

object Foo {
  var bar = "Hello"        // this is shared mutable state
  val baz = new Mutable()  // so is this

Tackling shared mutable state is not easy in language design. It involves designing the language to know about immutability, to track it, and to only allow immutable objects to be passed by reference to another thread/actor (mutable objects can be passed by copy). Done right, it eliminates the potential for concurrency issues from shared mutable state.

Scala relies on library design and discplined behaviour from developers to get this right (whereas Fantom builds this into the language, such that code equivalent to the example above will not compile). This is of course part of Scala's design approach - to give developers the power and trust that they will not abuse it. For me, this is simply another case of Scala failing to tackle the root cause of a big developer productivity issue.


Scala has a loud and vocal community, especially amongst those on forums and twitter. Some of these developers have gone on to create libraries based on Scala, in all manner of areas, from web frameworks to database access. This can have the effect of making Scala appear to be the "upcoming destination" where other developers think they should invest their time.

Unfortunately there are some aspects of the community that are much more negative. Scala appears to have attracted developers who are very comfortable with type theory, hard-core functional programming and the mathematical end of programming. Frequently, there is the sense of a brainiac competition, and an awfully large amount of argument of whether solution A, B, C or D is the right one when in reality they all do exactly the same thing (Scala typically offers many ways to achieve the same end result, something that Java sought to avoid, and something that tends to create more heat than light in debates).

There is also a sense that many in the Scala community struggle to understand how other developers cannot grasp Scala/Type/FP concepts which seem simple to them. This sometimes leads Scala aficionados to castigate those that don't understand as lazy or poor quality developers, as being second class citizens. This can easily lead into derogatory comments about "Java Joe" or worse.

My experience is that most developers are perfectly clever people, and perfectly capable of understanding many things if they are explained correctly. The classic example is variance in Java generics, where ? extends is needed. I find that it is perfectly possible to explain the issues to a developer, who will pretty quickly grasp why a List of Integer cannot just be assigned to a List of Number. However, what I find is that once the discussion is complete, and the developer solves their immediate problem, the explanation will tend to slip away. The problem is not that the developer isn't smart enough, its that the complexities of the type system isn't important enough to care about. Understanding the issue at hand, management priorities, the problem domain and the architecture/design of the large system they are working on are much more significant issues.

The Scala community is also infected with modern societies desire for more, more, more without considering the consequences (more gadgets, faster car, bigger TV, more money, more spending, yet bankrupt people and countries). Scala goes all the way with its language features - everything is about maximum power. And the community revels in that power, finding and exploiting every corner case that the power grants, without truly considering the harm it does.

Type system

Every time I look at Scala it feels rather like the type system fits the phrase "if all you have is a hammer, everything looks like a nail". Whatever the problem, the type system is bound to be part of the solution.

The trouble is that a big type system is inevitably a complex type system. The concepts added to support the type system have their own terminology which is instantly inaccessible without significant learning, from high kinds to type constructors to dependent types... Its all just a baffling mess of type theory that provides no meaningful connection to actual work that needs doing.

The trouble is that despite the pleading of aficionados, method signatures like this abound:

 def ++ [B >: A, That] (that: TraversableOnce[B])(implicit bf: CanBuildFrom[List[A], B, That]) : That

If you don't know Scala, you wouldn't have a hope of attempting to understand the code. In fact this is the equivalent to Java's addAll() on a list. (Its so complex that Scala tries to hide it from you in the documentation by giving you a simpler form instead.)

Oh, and by the way, I do get the idea that the humongous type system is there to prevent compile errors and pre-check your code. But applying that logic in the opposite direction would imply that no-one gets any real work done in languages with dynamic type systems. For me, Scala's type system is way way beyond the point of sensible returns for a language feature.

Steve Yegge's analysis was perhaps the most fun:

The... the the the... the language spec... oh, my god. I've gotta blog about this. It's, like, ninety percent [about the type system]. It's the biggest type system you've ever seen in your life, by 5x. Not by an order of magnitude, but man! There are type types, and type type types; there's complexity...

They have this concept called complexity complexity<T> Meaning it's not just complexity; it's not just complexity-complexity: it's parameterized complexity-complexity. ... I mean, this thing has types on its types on its types. It's gnarly

Steve uses Scala to argue for dynamic type systems. I disagree, and consider a static type system to be useful for documentation, communicating intent in a team and for basic error checking. But I don't need the world's most complicated type system to do that - I just need something simple and effective.

In essence, Scala's type system is giving static typing in general a bad name.


Scala's syntax is very wide open. It is the case that given a small piece of code (the kind that developers look at all day long) it is frequently difficult to reason about what that code does.

Scala has a big focus on flexible syntax with the aim of allowing a user to create a DSL in almost any form without having to write any parsing code. Instead, the developer just has to write a "normal" API, and use the language's syntactic flexibility to enable the ultimate end user to write in the desired style.

Take implicits, a technique that seems perfectly sensible to allow type conversions and object enhancements in a type safe way. Well, it may be type safe, but its also silent and very deadly. You can look a piece of code and not have any idea what is being converted. Unless you understand every import, every active implicit, their scope, their priorties and much more, you really don't have a clue what your code is doing. But thats OK, you didn't want to be able to understand Scala code did you???

Or take the fold operators and placeholder _, which produce delightful code like this:


Thats practically the very definition of line noise. (And I've not even shown any scalaz examples, or similar unicode weirdness)

By the way, if you're looking at Scala, you may come across conference presentations, blog posts and forums that show small snippets of code in Java and Scala and show how much less code you have to write in Scala, and how much simpler it appears to be. This is a clever trick. The real complexity occurs when you start mixing each of the seemingly simple features together in a large codebase, on a big team, over a period of time, where code is read far more than it is written. That is when you start to appreciate that some of Java's restrictions are there for a reason.


When evaluating a language, its important to get a sense of the quality of the implementation. This is useful for determining how easy it will be to maintain the current language and extend it in the future.

For this, I turn to the analysis by the core Scala committer, Paul Phillips, in a Scala podcast in June 2011 (selected elements):

The compiler is, and the libraries and language as a whole, its awesome but the number of places where features interact is astronomical. In order to really get the lid on that many feature intersections we need a massively comprehensive test suite, that we simply don't have.
[Question:] Its been suggested that you are skeptical of community involvement because there is no test suite? You're afraid that if anyone touches it but you the whole world will break.
[Answer:] I, unfortunately, continue to be bitten by extremely subtle bugs that come out because of the inadequecy of our test suite.
[Question:] Where do you want the test suites?
[Answer:] Collections
[Question:] Anywhere in particular?
[Answer:] All of them. There's no reason that many many many of the bugs we've seen in the collections over the last couple of years should ever have happened because they should be exhaustively shown not to exist by virtue of the tests that we have, but don't have yet.
[Question:] And what about the compiler?
[Answer:] An exhaustive test suite for the pattern matcher would certainly aid me in the process of finally really fixing it. It would be very very helpful actually.

An incredibly complicated language with very few tests? Sounds like a poor foundation to build real world applications on to me.

Specifically, note this line - "the number of places where features interact is astronomical". This is a key aspect of Scala. That each language feature is orthogonal and flexible. Implicits mean that code can be inserted almost anywhere (which slows the compiler, listen to the podcast). The ability to drop method invocation dots and brackets for parameters (to achieve DSLs) makes the meaning of code non-obvious, and leaves no spare syntax space for future enhancements. And these things combine to make a good IDE a very difficult challenge.

(If you're evaluating Scala for adoption by your team, I strongly recommend listening to the whole 40 minutes of the podcast. It will help you understand just what the real issues are with Scala, the quality of the implementation, and how difficult the language is to evolve.)


The EJB 2 spec was in many ways the nadir of Java EE, where huge amounts of boilerplate, XML and general complexity were foisted onto the Java industry. The spec was widely adopted, but adoption was followed by criticism. Developers found that while EJB 2 sought to reduce complexity in building an enterprise application through an abstracted higher level API, in reality it added more complexity without providing the expected gains. Documentation, best practices and tooling failed to solve the basic design issue. Spring was launched as a greatly simplified alternative, and eventually the much simpler EJB 3 was launched, a spec that had little to do with EJB 2.

As a data point, I attended a formal weeks training course in EJB. At the end of the course I knew that this was a very bad technology and that I would recommend against its use at every opportunity. Scala has exactly that same feel to me.

So, at Devoxx I said that "Scala feels like EJB 2 to me". The language is a well-meaning attempt to create something with a higher abstraction level. But what got created is a language that has huge inherent complexity and doesn't really address the true issues that developers face today, yet is being pitched as being a suitable replacement for Java, something which I find to be bonkers.

At the moment, Scala is at the stage of thinking that better documentation, best practices and tooling will make a huge difference. None of these helped EJB 2.

In fact one might argue that Java's biggest flaw down the years has been the architectural over-engineering of solutions, when something simpler would have done the job. Again, Scala feels very much in the mold of that strand of the Java community, over-engineered rather than YAGNI or 80/20.

Of course, neither Spring nor EJB 3 are perfect, but the core concept of injection appears to be easy to grasp and the basic mechanism of linking them simple. In particular, having easily cut and pasted sections of documentation proved very valuable. Having code that is easy to grasp where problems can be tracked down without needing a PhD in type theory is a Good Thing, not a bad one. Having code where you can work out what it does without needing to know every last detail of the "astronomical" number of language feature intersections is a Good Thing, not a bad one.

Of course the upside for Scala of my EJB 2 comparison is that EJB 3 is a lot better. Thus, it is conceivable that Scala could reinvent itself. But I would point out that EJB 2 and 3 are essentially utterly different approaches, happening to share a common name. I would say that Scala would need a similar reinvention from scratch to solve its problems.


I don't like Scala. And that dislike is increasing. Specifically, I do not want to spend any of my future working life having to write code in it.

Had Scala stayed as a remote language, for highly specialist use cases (like Haskell or Erlang) then I would have far less of an issue. But it is being sold as the solution for mainstream development, and for that it is as utterly unsuited as EJB 2 was.

Update 2011-11-24: Rather than respond to all the comments inline, I penned a response blog.

Friday 18 November 2011

Guide to evaluating Fantom

Yesterday at Devoxx I spoke about the Fantom programming language.

Evaluating Fantom

This blog post provides some links to help you evaluate Fantom, and could be useful if you attended the session yesterday, or if you're generally looking at languages.

While the "hello world" example is useful when evaluating a language, its important to look deeper. Key areas to look at include the type system, paradigm (OO vs functional), immutability, concurrency, standard library, modularity and composability. Instead of hello world, here is a word counting program:

class WordCount {
  Void main(Str[] args) {
    if (args.size != 1) {
      echo("usage: Wordcount <file>")

    // Set up an empty map to count each word, setting default for each value to zero
    wordCounts := Str:Int[:] { def = 0 }

    // Open the file, read each line in order
    file := Uri(args[0]).toFile
    file.eachLine |line| {
      // skip empty lines
      if (line.trim.isEmpty) return

      // split and trim on whitespace into words
      words := line.split

      // count each one
      words.each |word| { wordCounts[word] += 1 }

    // Show each word found, with its count, in alphabetical order
    wordCounts.keys.sort.each |key| {
      echo("$key ${wordCounts[key]}")

Even without the comments, a key Fantom feature is that the code is instantly readable. It is very much in the mainstream of syntax and there is a high chance that developers can read the code without learning the language.

Underneath, I think Fantom is more radical in some ways than many other new JVM languages. Fantom is effectively its own platform on top of the JVM. It compiles to its own bytecode format, fcode, with JVM bytecode emitted on demand. It can also compile to JavaScript, and people have investigated .NET and LLVM compilation as well. And, although you can call Java classes, most of the time you work with Fantom's API, not Java's API.

The biggest difference to other languages is the type system. Static typing is valuable for communication, documentation and basic error checking. But most static type systems are a straight-jacket of rules that effectively require you the developer to prove your application to the compiler, something which Java generics has shown can be very painful. Fantom's approach is radically different - it uses the type system to support you, rather than restrict you. This is what Brian Frank, the language author had to say:

Kotlin seems to be following in the same basic philosophical path of Java, Scala, and most all other statically typed languages: the job of a static type system is to always prove that something is type safe at compile time. I believe Fantom is unique in our approach of type systems: we allow things which might work to be caught at runtime versus compile time. This is a pragmatic trade-off which years of use seems to validate: you can use a weaker but much simpler type system and still catch most errors at compile time. Fantom proves that there can be a sweet spot between static and dynamic type systems.

Whether you can accept this approach to the type system will define your evaluation of Fantom. If you can appreciate the idea that you don't need to add more generics to fix the problems in Java, then you will appreciate Fantom. If you've used and liked the freedom of dynamic langauges, like Ruby, but understand that a little static typing could be useful, then you will appreciate Fantom.

Fantom also does something which every other new language should do, but doesn't. Everything is a module, which provides big knock on benefits to reflection and dynamic behaviour, plus a good way to assemble code.

Finally, Fantom tackles concurrency properly by actively preventing shared mutable state (something that Scala fails to do). Immutability is deeply built in. Every object, list, map and function knows if it is immutable, and lists/maps can be converted to be immutable. At the static level, there are no static variables, and constants must be immutable. The only way to access another thread is via the actor framework, and that only allows you to send an immutable object or a copy of a mutable one (via Fantom's built in serialization).


When Java was created it took C and C++ and threw lots of cruft away to create something radically simpler, yet more powerful overall. Fantom has done the same to Java, throwing lots of cruft away to create something radically simpler, yet more powerful overall.

Hopefully the links and this quick feature guide will help those looking at the language.

Monday 7 November 2011

The future is in the JEPs

If you want to know the most likely contents of Java 8 and beyond, the JEP process is the best place to look.

JDK Enhancement Proposals

JDK Enhancement Proposals (JEPs) are the simple descriptions of the tasks that are being considered for Java 8. Here is Brian Goetz's take:

JEP stands for "JDK Enhancement Proposal", and is part of our process for building a technical roadmap for the Java platform. Filing a JEP (what we used to called a "one pager") is the first step towards inclusion of a proposed feature in the JDK.

The JEP process document linked below outlines the states a JEP can go through, from Draft (we're just talking) to Submitted (I think this is good enough to review) to Candidate (Group/Area leads think the idea has enough merit to not toss out) to Funded (someone has actually committed resources to making it happen.) The last transition -- to Funded -- is the one at which this goes from being an idea to being part of the plan for some JDK release.
What JEP is not: it is not a "suggestion box" for drive-by requests. The JEP process is open to JDK *committers*, and it is unlikely a JEP will gain funding if the author is not prepared to contribute significant effort to the project's implementation or stewardship.

And the formal process document says:

The primary goal of this process is to produce a regularly-updated list of proposals to serve as the long-term Roadmap for JDK Release Projects and related efforts. The Roadmap should extend at least three years into the future so as to allow sufficient time for the most complex proposals to be investigated, defined, and implemented.
This process is open to every OpenJDK Committer. Decisions about specific proposals will be made in a transparent manner but are ultimately up to the OpenJDK Lead.

This process does not in any way supplant the Java Community Process. The JCP remains the governing body for all standard Java SE APIs and related interfaces. If a proposal accepted into this process intends to revise existing standard interfaces, or to define new ones, then a parallel effort to design, review, and approve those changes must be undertaken in the JCP, either as part of a Maintenance Review of an existing JSR or in the context of a new JSR.
That a particular JEP appears in the Roadmap means only that it is the proposal of record from a technical perspective. There is no guarantee that anyone will work on it, much less that its end result will appear in any JDK Release Project.

At the time of writing, there are 26 JEPs, plus 2 "meta" JEPs.

JEPTitle and LinkMy comments
101Generalized Target-Type InferenceMore type inference
102Process API UpdatesManaging OS processes
103Parallel Array SortingFrom the concurrency group
104Annotations on Java TypesAnnotations everywhere (JSR-305/308)
105DocTree APICompile AST access to javadoc
106Add Javadoc to javax.toolsProgramatic access to javadoc tool
107Bulk Data Operations for CollectionsAdd lambda aware methods to collections
108Collections Enhancements from Third-Party LibrariesNew collection methods and collection classes
109Enhance Core Libraries with LambdaAdd lambda aware methods to everything apart from collections
110New HTTP ClientReplace HttpURLConnection
111Additional Unicode Constructs for Regular ExpressionsEnhance RegEx
112Charset Implementation ImprovementsEnhance CharSet
113MS-SFU Kerberos 5 ExtensionsSecurity
114TLS Server Name Indication (SNI) ExtensionSecurity
115AEAD CipherSuitesSecurity
116Extended Validation CertificatesSecurity
117Remove the Annotation-Processing Tool (apt)Replaced by JSR-269
118Access to Parameter Names at RuntimeReflective access to parameter names
119javax.lang.model Implementation Backed by Core ReflectionEnhance low level compiler model
120Repeating AnnotationsAllow multiple annotations of the same type
121Stronger Algorithms for Password-Based EncryptionSecurity
122Remove the Permanent GenerationMajor change to JVM memory usage
123Configurable Secure Random-Number GenerationBetter random numbers (Security)
124Enhance the Certificate Revocation-Checking APISecurity
125Network Interface Aliases, Events, and DefaultsSuch as listening to wifi and mobile networks
126Lambda Expressions and Virtual Extension MethodsMain language change of Project lambda

Overall a good set of items so far, with access to parameter names and collection enhacements being of particular interest to me.

Since Oracle isn't accepting "drive by" enhancement requests, feel free to post your thoughts in the comments!

Tuesday 1 November 2011

My Google Reader fix

Google Reader's bland new look is a huge step backwards. So bad that I've installed GreaseMonkey for the first time.

Clearly, I'm a GreaseMonkey script newbie, but based on this script which does most of the hard work I've reached something that makes the product usable again. (The base script reduced the whitespace, while my extensions add back colour.)

// ==UserScript==
// @name           Fix New Google Reader Layout
// @namespace      http://joda.org
// @description    Combined fix for Google Reader
// @include        http*://www.google.*/reader/*
// @version        2.1
// ==/UserScript==
// based on http://userscripts.org/scripts/show/116850

var overrideCSS = " \
#top-bar { height:45px !important; } \
#search { padding:8px 0 !important; } \
#viewer-header { height:45px !important; } \
#lhn-add-subscription-section { height:45px !important; } \
#lhn-add-subscription, #viewer-top-controls-container \
{ margin-top:-15px !important; } \
#entries { padding:0 !important; background-color: #e2e7f0 } \
#title-and-status-holder { padding:0.3ex 0 0 0.5em !important; } \
.collapsed { line-height:2.2ex !important; padding:2px 0 !important; } \
.entry-icons { top:0 !important } \
.entry-source-title { top:2px !important } \
.entry-secondary { top:2px !important } \
.entry-main .entry-original { top:4px !important } \
.section-minimize { left: 0px !important } \
#overview-selector, #lhn-selectors .selector, .folder .name.name-d-0, \
#sub-tree-header \
{ padding-left: 15px !important; } \
.folder .folder .folder-toggle { margin-left:13px !important } \
.folder .sub-icon, .folder .folder>a>.icon { margin-left:27px !important } \
.folder .folder>ul .icon { margin-left:34px !important } \
.folder .folder .name-text { max-width:160px; !important } \
#reading-list-selector .label { display:inline !important } \
#entries.list .entry { background-color: #e2e7f0 !important; } \
#entries.list .entry .collapsed { border-bottom:2px solid #e2e7f0; background-color: #c7d0e2 !important } \
#entries.list .entry.read { background-color: #e2e7f0 !important; } \
#entries.list .entry.read .collapsed { background-color: #e2e7f0 !important; } \
#entries.list .entry-container { border-left:2px solid #6a7893 !important; border-right:2px solid #6a7893 !important; border-bottom:2px solid #6a7893 !important } \

(Save as a file ending in ".user.js", then drag it into FireFox to load it. You may find Reader needs some refreshes before it is picked up. BTW, you only need this script - you don't need the base script.)

I suspect others will make a better script, but this satisfies me for now.

Update 2011-11-03: On my other computer, I'm trying Stylish instead. The following is my customization of this style and this style. This style is closer to the old reader than the GreaseMonkey one above. Stylish is also safer to use for general web surfers than GreaseMonkey.

@namespace url(http://www.w3.org/1999/xhtml);

@-moz-document regexp("https?://www.google.com/reader/.*")   {

#top-bar {
    height: 40px !important;
#logo {
    height: 27px !important;
    margin: -13px 0 0 11px !important;
#search {
    margin-left: 157px !important;
    padding: 9px 0 !important;
#search-input {
    border: 1px solid #B2B2B2 !important;
    margin: 0 !important;
    padding: 3px 2px !important;
    width: 200px !important;
#viewer-container {
    background: none repeat scroll 0 0 #EBEFF9 !important;
#viewer-top-controls-container {
    margin-top: -12px !important;
.jfk-textinput {
    height: 17px !important;

#viewer-header-container {
    background: none repeat scroll 0 0 #C2CFF1 !important;
#viewer-header {
    background: none repeat scroll 0 0 #C2CFF1 !important;
    height: 30px !important;
.goog-button-base,.jfk-button,.goog-flat-menu-button {
    font-size: 1em !important;
    font-weight: normal !important;
.goog-menu-button .goog-button-base-content {
    padding: 7px !important;
.goog-button-base,.jfk-button,.goog-inline-block,.goog-flat-menu-button {
    cursor: pointer !important;
.goog-button-base-content {
    padding: 3px 0.461em 0 !important;
    position: relative !important;
    text-align: center !important;
.goog-button-tight .goog-button-base-content {
    line-height: 1em !important;
.goog-button-base-inner-box {
    background: none repeat scroll 0 0 #F9F9F9 !important;
    height: 20px !important;
.goog-button-base-outer-box {
    border-bottom: 1px solid #AAAAAA !important;
    border-top: 1px solid #BBBBBB !important;

.goog-button-base-inner-box {
    background: none repeat scroll 0 0 #F9F9F9 !important;
    border-left: 1px solid #BBBBBB !important;
    border-right: 1px solid #AAAAAA !important;
    margin: 0 -1px !important;
.goog-menu-button:active .goog-button-base-outer-box, .goog-menu-button:active .goog-button-base-inner-box, .goog-combobox-active .goog-button-base-outer-box, .goog-combobox-active .goog-button-base-inner-box, .goog-menu-button.goog-button-base-open .goog-button-base-outer-box, .goog-menu-button.goog-button-base-open .goog-button-base-inner-box {
    background: none repeat scroll 0 0 #F9F9F9 !important;
    border-color: #888888 !important;
.goog-button-base:hover .goog-button-base-outer-box, .goog-button-base:hover .goog-button-base-inner-box, .goog-button-base:focus .goog-button-base-outer-box, .goog-button-base:focus .goog-button-base-inner-box {
    border-color: #888888 !important;
.jfk-button-standard.jfk-button-hover, .jfk-button-standard.jfk-button-clear-outline.jfk-button-hover {
    border-color: #888888 !important;
.goog-flat-menu-button.goog-flat-menu-button-hover {
    border-color: #888888 !important;
.goog-menu-button:active .goog-button-base-outer-box, .goog-menu-button:active .goog-button-base-inner-box, .goog-combobox-active .goog-button-base-outer-box, .goog-combobox-active .goog-button-base-inner-box, .goog-menu-button.goog-button-base-open .goog-button-base-outer-box, .goog-menu-button.goog-button-base-open .goog-button-base-inner-box {
    background-color: #777777 !important;
    border-color: #888888 !important;
    border: 1px solid #AAAAAA !important;
    background: none repeat scroll 0 0 #F9F9F9 !important;
.jfk-button {
    height: 20px !important;
    line-height: 20px !important;
.goog-flat-menu-button {
    line-height: 20px !important;
    border: 1px solid #AAAAAA !important;
.goog-flat-menu-button-dropdown {
    top: 8px !important;
.goog-menu-button .goog-menu-button-dropdown {
    top: 8px !important;
.goog-menu-button .goog-button-base-content {
    padding: 4px !important;

#title-and-status-holder {
    padding: 0.1ex 0 0.1ex 0.5em !important;

#entries {
    padding: 0 !important;
    border-top: 1px solid #C2CFF1 !important;
#entries.list .entry .collapsed {
    background: none repeat scroll 0 0 #FFFFFF !important;
    border: 2px solid #FFFFFF !important;
    cursor: pointer !important;
    margin: 0 !important;
    overflow: hidden !important;
    padding: 0 !important;
    position: relative !important;
    width: auto !important;

#entries.list .read .collapsed {
    background: none repeat scroll 0 0 #F3F5FC !important;
    border: 2px solid #F3F5FC !important;
#entries.list .collapsed .entry-icons {
    top: 1px !important;
#entries.list .collapsed .entry-secondary,#entries.list .collapsed .entry-main .entry-source-title {
    top: 1px !important;
#entries.list .collapsed .entry-main .entry-original {
    top: 6px !important;

#current-entry .entry-container .entry-title a, #current-entry .entry-container a.entry-source-title, #current-entry .entry-container .entry-body a, #current-entry .entry-container a.entry-post-author-name {
    color: #2244BB !important;
#entries.list #current-entry .collapsed {
    border-color: #6688EE !important;
#entries.list .expanded .collapsed {
    border-bottom-width: 0 !important;
#entries.list #current-entry.expanded {
    border-color: #6688EE;
    border-style: solid !important;
    border-width: 0 2px !important;
#entries.list #current-entry.expanded .collapsed {
    border-color: #6688EE -moz-use-text-color;
    border-left: medium none !important;
    border-right: medium none !important;
#entries.list #current-entry.expanded .entry-actions {
    border-bottom-color: #6688EE !important;
#entries.list .entry .entry-actions {
    background-color: #EBEFF9 !important;
    border-top: 1px solid #C2CFF1 !important;
    color: #333333 !important;
.entry .entry-actions a, .entry .entry-actions .link {
    color: #2244BB !important;

/*This changes the folder icon. Play around with alpha settings to change opacity */

/*.folder-icon {
 background-position: 0 0 !important;
 background: url(https://ssl.gstatic.com/docs/doclist/images/collectionsprite_1.png) no-repeat !important;

.folder-icon, .tag-icon {
opacity: .1 !important;
filter: alpha(opacity=10) !important;


/* change colour of subscribe button */
.jfk-button-primary {
 background: #f2f2f2 url(none) !important;
 border: 1px solid #DCDCDC !important;
 color: black !important;


To use this style, install Stylish, go to AddOns manager, and click 'Write a New Style', pasting in the code above.