The IntelliJ IDEA Bomb

In case you missed the announcement from JetBrains yesterday, the popular IntelliJ IDEA has gone free and open source. Sort of. There is now a community edition, that is FOSS, and an enterprise edition that you must pay for. Why is this a big deal? Read on.

IntelliJ really revolutionized Java development. It wasn't the first IDE to allow for code completion, or even the first Java IDE to do this. However, it was definitely a pioneer in code refactoring. And this was huge. It took many of the code improvement ideas catalogued by Martin Fowler, and turned them into simple commands that any programmer could use. This also allowed for things like code navigation, where you could go from the usage of a class or a method to the implementation (or at least the declaration, if the usage only referred to the interface.) This was only the beginning. IntelliJ was a pioneer of bringing in the Java ecosystem. I remember how much easier IntelliJ made it to use Struts, for example.

This reminds me of my own personal use of IntelliJ over the years. When I first started working in enterprise Java development, I was working for a startup that built EJB applications targeted for WebLogic and WebSphere. For WebSphere, we used Visual Age for Java. For WebLogic, we just used Kawa (a bare bones editor.) I hated Visual Age with a passion. It scarred me badly. For years afterwards, I stuck to simple text editors.

Several year later (2003), I joined a fresh startup, KeepMedia (later renamed to MyWire). Prior to KeepMedia, I had a short stint with a .NET startup, Iteration Software (later renamed to Istante, and sold to Oracle.) There I used Visual Studio quite a bit, and appreciated the productivity gains it provided. So at KeepMedia, I took a look at various Java IDEs: JBuilder, JDeveloper, and IntelliJ. I was one of three programmers at KeepMedia, and one of my colleagues was really in love with Struts. He worked on the front of KeepMedia, but I worked mostly on our back end system that integrated magazine content from publishers. I didn't have to use Struts for that obviously, but I did occasionally help with the front end (we had three programmers after all.) I hated Struts, but IntelliJ made it more tolerable. The only negative about it was that it was flaky on Linux, but pretty much everything was flaky on Linux back then.

After I worked at KeepMedia, I worked at a consulting company, LavaStorm Engineering (2004). I had been using IntelliJ for a long time, and was convinced that everything else was crap. One of my colleagues introduced me to Eclipse. I was horrified. Most of Eclipse's codebase came from an all-Java rewrite of Visual Age (the version I had used was written in Smalltalk.) So even though it shared no code with the beast that I had hated, it shared a lot of the look-and-feel. For example, take a look at the outline view in Eclipse (any version, even the most recent.) This was completely taken from Visual Age. Even the icons are exactly the same. There was no way I was going to give up my IntelliJ for that thing...

A couple of years later, I was at another startup, Sharefare (which later changed its name to Ludi Labs.) Being a startup, there was no standardization arounds tools. However, everyone used Eclipse. I had warmed up to Eclipse (though I still preferred IntelliJ), and I had started writing Eclipse related articles for IBM. So I went with Eclipse there, as there were advantages to everyone using the same IDE (being able to share .project/.classpath, plugins, etc.)

After Ludi Labs went down in early 2007, I joined eBay. To say we're a major Eclipse shop, would be putting it mildly...

Eclipse @eBay 2009

View more presentations from michael.galpin.

We're not the only major Java shop that has invested heavily in Eclipse plugins. If you're doing GWT, App Engine, or Android development, then you're probably using Eclipse too. I know I am. I didn't really start using IntelliJ again, until the past year. The reason was simple: Scala. It is the best IDE for doing Scala development currently. Actually IntelliJ's Scala offering vs. the competition, is similar to anything else. It's not that it necessarily has a lot more features, but it has similar and most importantly, it is of higher quality. Here's a great visualization of this key difference:

This is why the open sourcing of IntelliJ is important. It's not that we didn't already have great open source IDEs for Java. No, it's more about what does this mean for IntelliJ. Will the attention to detail go down hill? Will the stable plugin ecosystem be disrupted?

Of course, for most current and would-be users of IntelliJ, the availability of a free IntelliJ is really not big news. The free version is really quite limited. You're not going to build web apps, or apps that connect to databases, use web services, etc. Not with the community version, unless in-the-wild plugins became available that enable this. IntelliJ has awesome support for all of these things, but those parts of IntelliJ remain behind a price wall.

There is one notable exception here: Scala. Right now, IntelliJ has the best Scala support. It has more features and less bugs than NetBeans, and it is much more stable than Eclipse. Scala support is one of the features supported in the community edition. That means a lot of newbie Scala developers can just use IntelliJ. This is great news. I would not give NetBeans or Eclipse to Scala newbies, for the simple reason that both of them report syntax errors inconsistently. In other words, both are guilty of either not identifying an error, or identifying a false error (or both.) That's anathema for somebody learning a language. I'm able to put up with it, simply because I know just enough Scala to say "you're wrong IDE" at times. I've rarely seen this happen with IntelliJ.

A Tipping Point for Scala

This past week's BASE meeting was all about IDE support for Scala. You can read my notes, posted to the Scala tools mailing list. I was very surprised by this meeting. Not by the findings, if you will, as I have used all three IDEs at various times in the last few months. What I was surprised by was the feedback from the group, and the logical conclusion of this discussion: Scala is near a tipping point, but IDE support is holding it back.

First off, there was a large turnout for the BASE meeting. I would say it was the second largest BASE meeting, only bested by the June meeting where Martin Odersky spoke. It is funny, because I think our esteemed organizer, Dick Wall, had been intending this topic to be like "well if we have nothing else to talk about it, we'll talk about IDEs." If there had been an alternative topic brought up, I don't think people would have objected. After all, developers and their attitude towards IDEs are contradictory. Most developers I know would tell you that IDE support for a language is very important, but they would also act indifferent about IDEs when it came to them personally. It's like "all of those other developers really need IDEs, but I would be ok without them." We all know our APIs so well, that we don't need code completion, right? And we don't write bugs, so a debugger is of limited use, right? However, I am sure that if the meeting had not been about IDEs, then there would have been less people in attendance.

So why so much interest? Like it or not, but Scala's primary audience right now are Java developers. Yes, I know Scala appeals to some dynamic language folks, and to some functional programming folks, and that its .NET implementation is being updated, but you could sum up all of the Scala developers from those disciplines and it would be dwarfed by the Java contingency. Scala has a lot of appeal on its own merits, but it is always going to be framed against Java. Scala's most (only?) likely path to mass appeal is as "the long term replacement for java."

So when you talk about developers choosing to use Scala, you are really talking about Java developers choosing to use Scala instead of Java. This is not the only use case, but not only is it the most common use case, it is arguably the only use case that matters. Without this use case, Scala will at most be a marginal language, a la Haskell, or OCaml, or Groovy for that matter.

Back to my point... Java developers need great IDEs. This is not because they "need" help from their IDE because of some lack of skill. No, it's because they have had great IDEs for a long time now, and thus it has become a requirement. I remember when I joined Ludi Labs (it was still called Sharefare at the time) several years ago, we had a Java programming quiz. Candidates were given a clean install of Eclipse to use for writing their programs. We could have given them Vi or Emacs and a command line, but that would have been asinine and foolish. IDEs are an integral part of Java development.

I knew all of the above before the BASE meeting, but what I did not know was how many development organizations were at a critical juncture when it comes to Scala. For many folks, Scala, the language, has won the arguments. Whatever perceived extra complexity that it has, has been judged as worth it. Whatever challenges there may be in hiring people to develop in Scala can be mitigated. Legacy code is not even a factor, as integration with existing Java code is trivial. Maybe it's bleak future of Java, or maybe it's the high profile use of Scala at Twitter. Who knows, but Scala is poised to take a big piece of the Java pie.

Thus the missing piece is IDE support. Development orgs can't switch to Scala without IDE support, and the support is not there yet. That's the bad news. The good news is that Scala is ready to explode once the IDE support is there. There are a lot of folks out there ready to adopt Scala simply as a "better Java." They just need an IDE that is on par with Java IDEs. That is the standard.

All of that being said, there is a lot of concern around the IDEs. Many people expressed to me that they are worried that IDE progress is being coupled to the release of Scala 2.8. That seems reasonable at first, but what happens if 2.8 is not released until 2010 sometime? Will Scala lose its momentum and window of opportunity?

Scala Puzzler and IntelliJ

Today I received a serial number for the highly regarded IntelliJ. The license was courtesy of the San Fran JUG, as a thank you for speaking there last month. I used to use IntelliJ many years ago, when I worked in consulting. Back then its refactoring was much more powerful than Eclipse and NetBeans was a joke. I haven't used it too much since then, as I became quite the Eclipse convert. Every once in awhile I will take a look at it. Today I thought I should give it more of a try, since I had this license for it. So during my lunch, I set it up, got its Scala plugin, and solved a Facebook puzzle. Actually I shouldn't say that I solved the puzzle, my solution is a naive, inefficient solution. Anyways, here's the code

import java.io._
import scala.collection.mutable._
import scala.io._

object PeakTraffic {
    def main(args:Array[String]){
        val users = HashSet.empty[User]
        val stream = if (args.length == 1) new FileInputStream(args(0))
            else Thread.currentThread.getContextClassLoader.getResourceAsStream("traffic.txt")
        val src = Source.fromInputStream(stream)
        src.getLines.foreach((str) => {
            val emails = str.split(" ").filter(_.contains("@")).map(_.trim)
            val sendingUser = User(emails(0))
            val receivingUser = User(emails(1))
            users(sendingUser) match {
                case true => sendingUser = users.find(_ == sendingUser).get
                case false => users += sendingUser
            }
            users(receivingUser) match {
                case true => receivingUser = users.find(_ == receivingUser).get
                case false => users += receivingUser
            }
            sendingUser <==> receivingUser
        })
        makeClusters(users).foreach(println(_))
    }

    def makeClusters(users:Collection[User])={
        val clusters = new ArrayBuffer[List[User]]
        users.foreach(_.friends.foreach((friend) =>{
            clusters += friend :: friend.friends.filter(friend.isFriend(_))
        }))
        clusters.filter(_.size >= 3).map(_.sort(_ < _)).toList.removeDuplicates.map(_.mkString(", ")).sort(_ < _)
    }
}
case class User(email:String) extends Ordered[User]{
    private val friendSet = HashSet.empty[User]
    private val sentSet   = HashSet.empty[User]
    private val receivedSet = HashSet.empty[User]

    def compare(that: User) = email compareTo that.email
    override def toString = email
    def friends = friendSet.toList
    def isFriend(friend:User) = friendSet(friend)
    def <==> (user:User) = {
      this >> user
      user << this
    }
    private
    def >>(user:User) = {
        sentSet(user) match {
            case false => {
                receivedSet(user) match {
                    case true => {
                        friendSet += user
                        receivedSet -= user
                    }
                    case false => sentSet += user
                }
            }
        }
    }
    def <<(user:User) = {
        receivedSet(user) match {
            case false => {
                sentSet(user) match {
                    case true => {
                        friendSet += user
                        sentSet -= user
                    }
                    case false => receivedSet += user
                }
            }
        }
    }
}

A couple of things about IntelliJ... Code completion seemed much slower than on NetBeans or Eclipse. Similarly most errors weren't flagged until the code was compiled (maybe it's this way with Java, too?) Debugging was good. Some refactoring is supported. I think IntelliJ is unique in this regard. However, I couldn't rename a method to an operator symbol (>> in this case.) Also, method extraction did not seem to work, which is probably the refactoring technique I use the most when I code. The indentation was also flaky. Like I could create a class, hit return (indented) create a val, hit return (indented even more), etc. I think the lack of semicolons might have been causing it problems, but that's just a guess. Overall, I think the NetBeans plugin is better, but at least my experience wasn't as bad as David's.