Saturday, August 15, 2009

A Tipping Point for Scala

This past week's BASE meeting was all about IDE support for Scala. You can read my notes, posted to the Scala tools mailing list. I was very surprised by this meeting. Not by the findings, if you will, as I have used all three IDEs at various times in the last few months. What I was surprised by was the feedback from the group, and the logical conclusion of this discussion: Scala is near a tipping point, but IDE support is holding it back.

First off, there was a large turnout for the BASE meeting. I would say it was the second largest BASE meeting, only bested by the June meeting where Martin Odersky spoke. It is funny, because I think our esteemed organizer, Dick Wall, had been intending this topic to be like "well if we have nothing else to talk about it, we'll talk about IDEs." If there had been an alternative topic brought up, I don't think people would have objected. After all, developers and their attitude towards IDEs are contradictory. Most developers I know would tell you that IDE support for a language is very important, but they would also act indifferent about IDEs when it came to them personally. It's like "all of those other developers really need IDEs, but I would be ok without them." We all know our APIs so well, that we don't need code completion, right? And we don't write bugs, so a debugger is of limited use, right? However, I am sure that if the meeting had not been about IDEs, then there would have been less people in attendance.

So why so much interest? Like it or not, but Scala's primary audience right now are Java developers. Yes, I know Scala appeals to some dynamic language folks, and to some functional programming folks, and that its .NET implementation is being updated, but you could sum up all of the Scala developers from those disciplines and it would be dwarfed by the Java contingency. Scala has a lot of appeal on its own merits, but it is always going to be framed against Java. Scala's most (only?) likely path to mass appeal is as "the long term replacement for java."

So when you talk about developers choosing to use Scala, you are really talking about Java developers choosing to use Scala instead of Java. This is not the only use case, but not only is it the most common use case, it is arguably the only use case that matters. Without this use case, Scala will at most be a marginal language, a la Haskell, or OCaml, or Groovy for that matter.

Back to my point... Java developers need great IDEs. This is not because they "need" help from their IDE because of some lack of skill. No, it's because they have had great IDEs for a long time now, and thus it has become a requirement. I remember when I joined Ludi Labs (it was still called Sharefare at the time) several years ago, we had a Java programming quiz. Candidates were given a clean install of Eclipse to use for writing their programs. We could have given them Vi or Emacs and a command line, but that would have been asinine and foolish. IDEs are an integral part of Java development.

I knew all of the above before the BASE meeting, but what I did not know was how many development organizations were at a critical juncture when it comes to Scala. For many folks, Scala, the language, has won the arguments. Whatever perceived extra complexity that it has, has been judged as worth it. Whatever challenges there may be in hiring people to develop in Scala can be mitigated. Legacy code is not even a factor, as integration with existing Java code is trivial. Maybe it's bleak future of Java, or maybe it's the high profile use of Scala at Twitter. Who knows, but Scala is poised to take a big piece of the Java pie.

Thus the missing piece is IDE support. Development orgs can't switch to Scala without IDE support, and the support is not there yet. That's the bad news. The good news is that Scala is ready to explode once the IDE support is there. There are a lot of folks out there ready to adopt Scala simply as a "better Java." They just need an IDE that is on par with Java IDEs. That is the standard.

All of that being said, there is a lot of concern around the IDEs. Many people expressed to me that they are worried that IDE progress is being coupled to the release of Scala 2.8. That seems reasonable at first, but what happens if 2.8 is not released until 2010 sometime? Will Scala lose its momentum and window of opportunity?


Anonymous said...

I can tell you that for our organization, the IDE support is the only stopping me from putting scala in our project and starting to head down that path.

I can easily say if someone waved a magic wand and eclipse support was magically on par with java, we'd have scala in production within a month.

Anonymous said...

... ditto.

martin said...

The first beta of 2.8 is slated to come out end of September/beginning of October. The rewritten Eclipse plugin will be released with it. Now, we can't promise that either one of them will be rock-solid and bug free from the start, but they should be good enough for serious use and meaningful user feedback.

Anonymous said...

... same for our group. Eclipse support (or issues with that) is the thing that is putting people off.

Anonymous said...

One has to marvel at organizations with programmers so tender that they can not use a language which just has editor support.

Have all the less tender programmers left to do Rails work?

Anonymous said...

Same here. I am a director of engineering at a small software company. Love Scala, been following it for a while, but in order to have my developers start using it, better IDE support (Eclipse specifically) is crucial.

Channing Walton said...
This comment has been removed by the author.
Channing Walton said...

I suggest these organisations have bigger problems if their developers rely on an IDE so much.

IDE's are great, but their absence is not an obstacle to good developers.

PaulG said...

No IDE may be OK for good or should I say above average developers. But in large organisations with 00's of developers you will have on average just average or just below average developers. Can you imagine trying to run a team of 30 developers using a language with no 1st class IDE support? Lets get real no IDE may be ok for your hobby projects but not for large organisations.

Anonymous said...

Re: IDE supprt required and all less tender moving to Rails...

I have done java work and I have done rails work. One of the best developers I've ever worked with hates java and ruby, and loves PHP. There's no accouting for taste. There are different cultures in software development and different learning curves.

Do good developers NEED an IDE? No. Have good java developers mastered use of a IDE to where they are more productive because of it? Yes, of course. If someone writes a ton of code on a team where most of the team has mastery of the IDE, they're going to balk. If you told me some optional features in a new version of any language didn't have vi support (go along with me here) and only had emacs support, I'd be really pissed and say "Why'd you put this crap in here? I can't use vi". Same thing.

I know where you're coming from, but why do rails people have to be such jerks?

Anonymous said...

Channing - we're not talking about a developer, we're talking about a large team.

Channing Walton said...

I know we are talking about a large team. After 15 years of development I am tired of seeing large teams of below average developers making a huge mess that eventually needs to be cleaned up by a smaller more competent team.

The use of an IDE does not make poor developers produce better software. Often they don't use the features of the IDE anyway.

If a team needs an IDE to program in scala then the team is not ready for scala.

Erik said...

I know my vi and emacs as well as the next guy (used them all through uni and still am) but this constant harping about 'good developers don't need an IDE' reminds me of elementary school. IDEs save tons of time even for mundane tasks such as reading the javadoc for some class/method, finding your way around unknown APIs with autocompletion, etc. Of course developers who can't do without one suck. Let it go already. If you wanted to be cool you should have become a fighter pilot. Anyway, good IDE support for scala would be... totally sweet!

Stephan.Schmidt said...

As head of development here, my biggest block towards Scala is also the sub-par IDE support (IDEs are huge time safers, contrary to all the nay sayers. API auto-completion, FindBugs/Inspections, Debugging, JavaDoc, Navigation, Auto-Maven-Import, ...)


Anonymous said...

Same here. My team is waiting on an IDE on par with Eclipse/IDEA for Java. I've used the latest of these two, they're not quite ready for large scale projects where an IDE is essential. I could edit bytecode directly, if I wanted. Intead, I choose to use higher level tools for productivity reasons.

Olaf Bergner said...

After 10+ years of doing professional software development I'm tired of all those would be programming wizards who don't need no stinkin' IDE, no, that's for pussies, but only good ol' VI/Emacs/Sed for producing what these dinosaurs have the audacity to describe as high quality software. More specifically, I'm tired of

- having to refactor their code just because their editor of choice does not support constant, deterministic refactoring,
- having to eliminate potential bugs just because their editor of choice does not support real-time static code analysis,
- having to correct their immature usage of standard APIs just because their editor of choice does not support API exploration via code completion, leaving those "wizards" prone to confining themselves to the same small subset of available functionality in any given API.

A modern IDE is first and foremost about improving the quality of your code, improved productivity is just an added bonus. What using merely an editor - even such a brilliant editor as Emacs which I'm very fond of - for, say, Java development instills in all but the most brilliant developers is an unhealthy reluctance to change and restructure their code, all too often leaving behind as a "finished" product what should have been only their first attempt. A good IDE in the hands of a competent programmer all of a sudden turns that monolithic rock of code into a malleable lump of clay, ready to be bent into its final, much more polished form.

These observations notwithstanding, it remains true that a fool with a tool is still a fool, but that's a given, and of course Emacs et al. also qualify as tools.

So if Scala aspires to considerably expand its user base, a modern IDE roughly on par with Eclipse's JDT is indeed crucial.

Anonymous said...

Same here, waiting for the IDE support.

> One has to marvel at organizations with programmers so tender that they can not use a language which just has editor support.


Even when Scala *is* better than Java,
Scala + no IDE is by no means better than Java + great IDE. I'd rather work with Java with Eclipse than Scala with a text editor; I'd be more productive with the former. Absolutely. This is a fact. IDE adds a huge boost to my productivity.


Anonymous said...

NetBeans 6.7.1 scala plugin is getting close, too, right?

I'm fine in textpad, but I sure can't see any harm in being able to set a breakpoint / watch a field / step over code.

Cédric Vidal said...

Same here. Waiting for IDE support to move on to Scala.

Carl said...

Ditto here.

I can program Java in vi or Notepad and have, on occasion. But syntax coloring, error tagging, auto-completion and so forth offload work from my brain so I can concentrate on other things.

Specifically with respect to Scala: I'm just learning the language, and it would be MOST helpful to have continuous syntax checking there to help me. Completion, ditto. Maybe programming with a reference book in one hand would "put hair on my chest," but I'd be resentful of the process.

So for now I've put experimental programming in Scala on the back burner until 2.8 . Still, many thanks to the hard working folks from the team who are making it possible!

Ralph said...

Real programmer use 'ed'...


Jessica Jewel Joyce said...

The limp phase that the IT sector and the software industry is going through in the present times can be gauged from the fact that the pay of the employees in these two fields all over the world no longer remain lucrative. In many countries, software companies are also chucking out employees, especially those employees who have been “sitting on the bench”. This was an unheard of concept a few years back. There are chances that the software companies take more such radical steps to fight the damp phase.

Stein Kåre Skytteren said...

Would love to have a good eclipse plugin for Scala. It makes all the difference in the world to me as well. I just hope that we can get that in the near future.

Anonymous said...

sorry, but i have to reply. i hear a lot of complaints about the current Scala IDE and how crucial it is to productivity and sure, i agree. That's great, but you're not adding anything to the conversation about what exactly you need from a Scala IDE. have you use the existing Scala Eclipse plugin? what didn't you like about it? what did you like about it? keeping in mind that there might not be a Java-rich set of static code analysis tools and such for Scala, what features from the Java eclipse plugin would you want to see for Scala? For example, the existing 2.8 plugin does autocompletion pretty nicely, so you can remove that from the list of things it doesn't do. what else? contribute something other than complaints if you want to see it in your workplace. sorry,

Ajay said...

I'm trying to get our programmers to adopt Scala. I agree totally that a good IDE can significantly improve productivity and can help new adopters of the language as well as any API ride the learning curve a lot quicker. People that insist that IDE support is not needed by "good" or "real" programmers, are fairly disengaged from reality, or don't work in "real" enterprise environments, and think code quality is restricted to their biased view of code elegance rather than quantified static analysis.

Regarding the features that the current plugin lacks (in Helios Eclipse):

1. Organize imports
2. Auto coplete is erratic at best
3. open or go to declaration works only some of the times
4. Better support is definitely needed for referencing Java classes

These are some features lacking as yet

salu said...

Pictures, prints and onto canvas created from your own photos, framed and ready to hang. For the best quality canvas prints at the lowest online prices guaranteed, with fast delivery.
Pictures onto canvas

Anonymous said...

It has every significant news so once I interpret this site that I encompass be incredibly useful as well as I think it awfully necessary all nation.
Pakistan Portal

salu said...

free lots of games, this is Rocket games, play today

Rocket Games

Anonymous said...

It has every of the spirit news bulletin so when I take to indicate this site that I have been extremely helpful and I take for granted it very obligatory all assembly of people.
Water Games