A Tipping Point for Scala

This past week's BASE meeting was all about IDE support for Scala. You can read my notes, posted to the Scala tools mailing list. I was very surprised by this meeting. Not by the findings, if you will, as I have used all three IDEs at various times in the last few months. What I was surprised by was the feedback from the group, and the logical conclusion of this discussion: Scala is near a tipping point, but IDE support is holding it back.

First off, there was a large turnout for the BASE meeting. I would say it was the second largest BASE meeting, only bested by the June meeting where Martin Odersky spoke. It is funny, because I think our esteemed organizer, Dick Wall, had been intending this topic to be like "well if we have nothing else to talk about it, we'll talk about IDEs." If there had been an alternative topic brought up, I don't think people would have objected. After all, developers and their attitude towards IDEs are contradictory. Most developers I know would tell you that IDE support for a language is very important, but they would also act indifferent about IDEs when it came to them personally. It's like "all of those other developers really need IDEs, but I would be ok without them." We all know our APIs so well, that we don't need code completion, right? And we don't write bugs, so a debugger is of limited use, right? However, I am sure that if the meeting had not been about IDEs, then there would have been less people in attendance.

So why so much interest? Like it or not, but Scala's primary audience right now are Java developers. Yes, I know Scala appeals to some dynamic language folks, and to some functional programming folks, and that its .NET implementation is being updated, but you could sum up all of the Scala developers from those disciplines and it would be dwarfed by the Java contingency. Scala has a lot of appeal on its own merits, but it is always going to be framed against Java. Scala's most (only?) likely path to mass appeal is as "the long term replacement for java."

So when you talk about developers choosing to use Scala, you are really talking about Java developers choosing to use Scala instead of Java. This is not the only use case, but not only is it the most common use case, it is arguably the only use case that matters. Without this use case, Scala will at most be a marginal language, a la Haskell, or OCaml, or Groovy for that matter.

Back to my point... Java developers need great IDEs. This is not because they "need" help from their IDE because of some lack of skill. No, it's because they have had great IDEs for a long time now, and thus it has become a requirement. I remember when I joined Ludi Labs (it was still called Sharefare at the time) several years ago, we had a Java programming quiz. Candidates were given a clean install of Eclipse to use for writing their programs. We could have given them Vi or Emacs and a command line, but that would have been asinine and foolish. IDEs are an integral part of Java development.

I knew all of the above before the BASE meeting, but what I did not know was how many development organizations were at a critical juncture when it comes to Scala. For many folks, Scala, the language, has won the arguments. Whatever perceived extra complexity that it has, has been judged as worth it. Whatever challenges there may be in hiring people to develop in Scala can be mitigated. Legacy code is not even a factor, as integration with existing Java code is trivial. Maybe it's bleak future of Java, or maybe it's the high profile use of Scala at Twitter. Who knows, but Scala is poised to take a big piece of the Java pie.

Thus the missing piece is IDE support. Development orgs can't switch to Scala without IDE support, and the support is not there yet. That's the bad news. The good news is that Scala is ready to explode once the IDE support is there. There are a lot of folks out there ready to adopt Scala simply as a "better Java." They just need an IDE that is on par with Java IDEs. That is the standard.

All of that being said, there is a lot of concern around the IDEs. Many people expressed to me that they are worried that IDE progress is being coupled to the release of Scala 2.8. That seems reasonable at first, but what happens if 2.8 is not released until 2010 sometime? Will Scala lose its momentum and window of opportunity?

IntelliJ Fail

Upgraded IntelliJ to 8.1 this morning. Launched it and got this:

That lovely message box is 2891 pixels across. So wide, that you cannot even see the control buttons in the bottom right. It also has a bunch of random HTML at the top of the box, as if it was meant to be displayed in a web page instead of this lovely dialog box.

Imagine if Microsoft or Adobe had software like this! Imagine the amount of ridicule. When you pay for software, you expect something a little more ... polished? Professional? Debugged? I don't know, but this is not it. Anyways, I've heard good things about 8.1 and the updated Scala plugin for it, so I hope this is just an aberration.

Scala Puzzler and IntelliJ

Today I received a serial number for the highly regarded IntelliJ. The license was courtesy of the San Fran JUG, as a thank you for speaking there last month. I used to use IntelliJ many years ago, when I worked in consulting. Back then its refactoring was much more powerful than Eclipse and NetBeans was a joke. I haven't used it too much since then, as I became quite the Eclipse convert. Every once in awhile I will take a look at it. Today I thought I should give it more of a try, since I had this license for it. So during my lunch, I set it up, got its Scala plugin, and solved a Facebook puzzle. Actually I shouldn't say that I solved the puzzle, my solution is a naive, inefficient solution. Anyways, here's the code

import java.io._
import scala.collection.mutable._
import scala.io._

object PeakTraffic {
    def main(args:Array[String]){
        val users = HashSet.empty[User]
        val stream = if (args.length == 1) new FileInputStream(args(0))
            else Thread.currentThread.getContextClassLoader.getResourceAsStream("traffic.txt")
        val src = Source.fromInputStream(stream)
        src.getLines.foreach((str) => {
            val emails = str.split(" ").filter(_.contains("@")).map(_.trim)
            val sendingUser = User(emails(0))
            val receivingUser = User(emails(1))
            users(sendingUser) match {
                case true => sendingUser = users.find(_ == sendingUser).get
                case false => users += sendingUser
            }
            users(receivingUser) match {
                case true => receivingUser = users.find(_ == receivingUser).get
                case false => users += receivingUser
            }
            sendingUser <==> receivingUser
        })
        makeClusters(users).foreach(println(_))
    }

    def makeClusters(users:Collection[User])={
        val clusters = new ArrayBuffer[List[User]]
        users.foreach(_.friends.foreach((friend) =>{
            clusters += friend :: friend.friends.filter(friend.isFriend(_))
        }))
        clusters.filter(_.size >= 3).map(_.sort(_ < _)).toList.removeDuplicates.map(_.mkString(", ")).sort(_ < _)
    }
}
case class User(email:String) extends Ordered[User]{
    private val friendSet = HashSet.empty[User]
    private val sentSet   = HashSet.empty[User]
    private val receivedSet = HashSet.empty[User]

    def compare(that: User) = email compareTo that.email
    override def toString = email
    def friends = friendSet.toList
    def isFriend(friend:User) = friendSet(friend)
    def <==> (user:User) = {
      this >> user
      user << this
    }
    private
    def >>(user:User) = {
        sentSet(user) match {
            case false => {
                receivedSet(user) match {
                    case true => {
                        friendSet += user
                        receivedSet -= user
                    }
                    case false => sentSet += user
                }
            }
        }
    }
    def <<(user:User) = {
        receivedSet(user) match {
            case false => {
                sentSet(user) match {
                    case true => {
                        friendSet += user
                        sentSet -= user
                    }
                    case false => receivedSet += user
                }
            }
        }
    }
}

A couple of things about IntelliJ... Code completion seemed much slower than on NetBeans or Eclipse. Similarly most errors weren't flagged until the code was compiled (maybe it's this way with Java, too?) Debugging was good. Some refactoring is supported. I think IntelliJ is unique in this regard. However, I couldn't rename a method to an operator symbol (>> in this case.) Also, method extraction did not seem to work, which is probably the refactoring technique I use the most when I code. The indentation was also flaky. Like I could create a class, hit return (indented) create a val, hit return (indented even more), etc. I think the lack of semicolons might have been causing it problems, but that's just a guess. Overall, I think the NetBeans plugin is better, but at least my experience wasn't as bad as David's.