The JavaScript assembler

Earlier this week, the third part in a series of articles I wrote was published on developerWorks. The series is on CoffeeScript, a language that I first learned of about a year ago. Last summer I read CoffeeScript by Trevor Burnham over the course of a week while on a cruise ship. Later that summer I was at the Strange Loop conference, and serendipitously wound up at a dinner table with several of the guys who develop Clojure. A lot of the conversation that evening was on ClojureScript.

As a  long time web developer I am not surprised by the growing interest in programming languages that compile into JavaScript. The browser has never been more important, and thus JavaScript has never been more important. Most programmers that I know find that JavaScript leaves a lot to be desired. This is not just a matter of taste, JS is fucking weird. In fact for years, a lot of the more "serious" programmers I knew would turn up their noses at JS. They were too good for that trash, so they kept to the back-end. It's become tougher and tougher to avoid JS, so ... use a different programming language that compiles to JS. Brilliant!

This idea is hardly new. Personally, I can't help but think of Google Web Toolkit. For years GWT has let your write in Java and compile to JS. So why bother with CoffeeScript, ClojureScript, and probably some other awesomes that I'm leaving out? One obvious thing is that Java is an existing language and is statically typed. So it has its own baggage (which includes excellent tooling, but still) and of course many developers do not like statically typed languages. In fact, many argue that web development with a statically typed language is an impedance mismatch of the highest order. Personally I think GWT was also hurt by initially defaulting to producing highly obfuscated "production ready" JS. I always thought that was developer unfriendly. I think that may have changed over the years, but sometimes it's hard to overcome an initial impression. I think one of CoffeeScript's great strengths is that it produces very developer friendly JS by default. I've never built any huge apps in either language, but in my limited experience I found that debugging was easier in CoffeeScript. I could usually figure out what I had done wrong just by looking at the error message.

Any discussion like this would be amiss without mentioning Dart. It gives you dynamic and static aspects, in a way that reminds me of ActionScript. Personally I think optional typing works best when use variable:type style declarations, like ActionScript and Scala, but that's a minor nit. Of course when you consider that "native Dart" is supported in a version of the world's most popular browser, the possibilities start to get really interesting. I attended a Dart meetup recently where the Chrome developer tools allowed breakpoints and the like directly in the Dart script that was running. The SourceMap proposal would allow this kind of debugging of other higher level languages like CoffeeScript and ClojureScript.

The future seems bright for languages targeting a JavaScript runtime. I haven't even mentioned node.js and how that makes these languages viable for both client and server applications. To me CoffeeScript, Dart, and ClojureScript are all promising. Each seems to have a large target audience. If you like dynamic languages like Python and Ruby, CoffeeScript fits right in. If you prefer statically typed languages like Java and C++ and all of the nice tools that come with them, then Dart is attractive. If you like pure functional languages like Haskell and Lisp, then ClojureScript is a fun way to get your lambda calculus kicks in the browser. Of course there is a lot of anti-Lisp bias out there, so maybe there's room for another functional language that targets a JS runtime, something like a F#-script or ScalaScript.

It's 2010, where's my HTML5?

This morning, one of my co-workers related to me that there had been mention of HTML 5 on an NPR broadcast that he heard. He joked that this morning there were probably lots of advertisers asking their designers about using HTML 5 for their next whiz-bang ad campaign. So obviously we are still in the early part of a hype cycle on HTML 5. So what's the state of the technology and where is it heading?

In terms of pure technology, there is a lot of notable progress. Google did the right thing recently and axed Gears. They had some nice innovations in Gears that have since been rolled up into the HTML 5 standard. Now it makes perfect sense to abandon their proprietary implementation, and embrace the standard. Gears is important because it is (or was) part of Chrome. Thus there are essentially five browsers that now implement a significant subset of HTML 5: IE8, Firefox 3.5, Safari 4, Chrome 4, Opera 10. According to the latest browser share numbers, that means that 53% of users out there are using a browser that supports a lot of cool features, like local storage and selectors. That's more than half! It's tempting to extrapolate from that, but don't get too carried away.

The #1 browser of the HTML 5 browsers, is IE8 -- which does not support a lot of the features that the other browsers support like canvas, video tag, and web workers. For those would be HTML 5 advertisers out there, you better scale back your expectations. For desktop browsers, a lot of the most compelling HTML 5 capabilities are far from being widely supported. How long until IE9 comes out? How much more of HTML 5 will it support? How long until there is high adoption of it? That's too many questions to be comfortable with. Ask me again this time next year.

However, it's fun to pretend sometimes. Let's pretend that IE 9 did come out soon and that it supported everything supported by Firefox and Chrome. Let's even pretend that Microsoft was able to awaken some of their old monopolist mojo and somehow force people to upgrade. So we have a world full of canvas. Will the next great annoying ad use this? If you read my last post on Flash hate, you probably realize the answer is no. As Terry pointed out in the comments there, the Flash authoring tools really empower designers over developers. So before we see an explosion of canvas magic, there will have to be huge strides made in tooling. Now what about video? Even in our dream world, there are issues known as codecs. There is no common video format that will run on both Firefox and Chrome right now, and who knows what Microsoft will do here (WMA FTW!) So we are going to need some technical consolidation here. Let's summarize all of the things that we need to see happen to open up the golden age of HTML 5.

1.) New version of Internet Explorer that supports much more of the HTML 5 specification.
2.) This new version of IE needs to become the dominant flavor of IE.
3.) Outstanding tooling developed to enable designers to leverage HTML 5 capabilities.
4.) Standardization on video codecs by browser vendors.

Don't get too bummed out about all of this. I actually think there is real progress happening on #1 and #3. #4 probably has to wait for #1 to happen. #2 is the most problematic. Since we're dreaming, maybe if IE 9 came out with H.264 support, Google could dramatically drop support for anything else on YouTube, and thus nudge all IE users to upgrade?

If you want to drink the HTML 5 Kool-Aid, and this is bumming you out, I'm sorry. You might feel better to know that the HTML 5 situation is significantly better on mobile devices. According to AdMob, 81% of internet usage in North America was either on iPhones or Android devices. Now, most of those Android devices are running Android 1.5 or 1.6, whose browsers (can we call it Mobile Chrome please?) still use Gears -- not HTML 5. Let's  hope that most of those are upgraded (by their carriers) to Android 2.1 soon, and then we're in business. We've got all kinds of HTML 5 goodness available to 80%+ of the users out there.

The bad news is that the mobile market is fundamentally different than the desktop (though, that may be changing.) Here mobile websites compete directly against native applications. Desktop users rarely face the question, should I use an app or a website to do XYZ? Normally XYZ dictates one or the other. So for a company, there may be a lot of cool stuff you can do with HTML 5 on a mobile browser, but there are even more cool things you can do in a native app. Not only are the native apps more powerful, they are generally easier to build because of superior tooling and cohesive architecture. Maybe JavaScript vs. Objective-C vs. Java is a matter of taste, but HTML 5 has nothing even resembling the application models presented by Cocoa and Android. Instead it has the DOM APIs in JavaScript. Now throw on the differences in tooling, and you can see why for many companies it is cheaper and easier to build an iPhone app and an Android app instead of building one HTML 5 app. That's not even considering the hypothesis that the "app store" model is more appealing to end users than the browser model.

The Flash Haters

The iPad announcement had lots of subtleties to it. One of them was that the iPad browser, presumably Safari, does not include a Flash plugin. This has lead to a lot of people from Adobe getting upset. What has amazed me through all of this, is that there is even more hate for Flash than ever before. As somebody who has done a fair amount of Flash development in my career, this has really struck me as odd. Why all of the hate for Flash?

In some ways, I can actually understand why Apple would hate Flash. The Flash plugin on OSX is nowhere near as good as the Flash plugin on Windows. Now Adobe will counter they spend a disproportionate amount of money on Flash for OSX, i.e. they spend more money per OSX user than they do per Windows user. So what? There are a lot of other technologies out there that work equally well on both Windows and OSX. Just focussing on web technologies, Firefox and Chrome on the Mac are very comparable to their Windows versions. So is the Java plugin. Some other products, like Opera and Skype, tend to trail in terms of features on OSX, but not in terms of quality. If these other companies can do this, why can't Adobe? It's obviously either lack of ability or prioritization. My guess is the latter, and hence you must expect hate from Mac users. And from Apple. So it should not be so surprising that Flash is left off the iPad, iPhone, etc. If Adobe really wanted to get Flash on those devices, an obvious first step would be to make Flash on OSX as good as Flash on Windows. That is something that Adobe can do all by themselves. Instead it seems like they would rather publicly bemoan their exclusion on the iPad. Now is there a guarantee that if they improved Flash on OSX, Apple would care? No, of course not. However, there is no way they will get on those devices without this first step, IMO.

Ok, so Mac user hate for Flash is understandable. Along the same lines, I would guess that Linux user hate is also understandable. That is, if there were actually any Linux users out there. I digress. I see a lot of hate from developers. What about that? That one is not so easy for me to understand. As a web developer, Flash has been opening doorways for me for a long time. Need to cache 30 KB of data on the client? For a long time this was impossible unless you used Flash (or maybe Java.) Need to make a cross-domain call? That is still generally impossible from the browser, unless you use Flash. Need to show 2D/3D graphs and visualizations? Again, this is still generally impossible, except through Flash. Oh and video... The only way to show a video that can be played on all of the top five web browsers (IE8, IE7, Firefox 3.5, IE6, Firefox 3.0) is to use Flash.

But, but, but, what about HTML 5? Many people think it will kill/replace/maim Flash. Apparently this list includes a certain Steve Jobs. As I have said before, I hate web standards. I mean, I love them, too. But I hate them mostly. Standards are the opposite of innovation, and perhaps its enemy. However, they are the engineer's best friend. Flash has provided a lot of innovation, that HTML 5 will simply copy and standardize. That is the best possible function of standards. However, the story is not so simple. Some things, like local storage,  are already supported by IE8, Firefox 3.5, and Safari 4. This is like Flash's SharedObjects, though slightly less powerful (but good enough.) Not everything is so rosy; canvas and video playback immediately come to mind. Canvas is particularly a hot topic. You can do some amazing things with it, but it requires payment in blood. I cannot imagine what it would be like to program Farmville in JavaScript using Canvas. Maybe somebody will come up with tools for that. In fact, I would be surprised if Adobe did not do this. Of course, we still need to hope and pray that canvas is supported in IE9...

So back to the original point. Why do developers hate Flash? It certainly enables them to do more stuff in web applications. Even if that list of extra features is being shrunk by the HTML 5 monster, it's still there. So shouldn't they love it? Well, you would think so, but they don't. Here are some guesses as to why:

1. Expensive tools -- Most of the tools used by a lot of developers are free. The Flash authoring tools are far from it. That can be annoying. You have to throw down a lot of money just learn something. That's not cool.
2. Bad tools -- At least from a developer perspective. The Flash authoring tools have historically been aimed at designers. That changed with Flex Builder/Flash Builder, but the damage was done. Also, those tools were aimed more at application development, which largely seemed like a waste of Flash.
3. Back to the Mac  -- A lot of developers use Macs, and we all know about Flash on OSX...
4. Accessibility -- Unfortunately this doesn't mean much to most developers. But it should. I must admit, that a year ago I probably wouldn't have listed this. However, as I have learned more, I have learned how Flash is such a huge problem for accessibility. This is actually one area where HTML 5 really shines, with the inclusion of the ARIA standards.
5. Design envy -- This kind of goes back to #2. Flash has had all the awesome capabilities for years, but they have largely gone untapped in web applications. Maybe by playing to designers all of those years, Flash has earned enemies. Imagine a developer being asked to scope some radical web feature. They give a typically large scope for it. His manager turns around and gets a designer, a freakin' designer, to do it in half the time.

What have I missed? Why do web developers hate Flash so much?

Stop the IE6 Hate

For a long time now, it has been common for developers to go on and on about how they hate IE6 and how users needed to be discouraged from using. It's gotten old. It's trite. It's time to move on. 

IE6 has become a non-factor for most sites. It trails far behind IE7 and IE8, not to mention Firefox. And of course, its numbers are always shrinking. I know that at a certain popular website that I happen to work for, we no longer test for IE6. It's been that way for us for 6+ months. It's over.

So now that it's over, there are some things that I want to see. First, I want to see all of those JavaScript libraries that everyone uses, drop their IE6 code branches. Shouldn't jQuery, Prototype, ExtJS, Dojo, etc. all be able to shave off a huge amount of their code? Shouldn't these libraries now progress even faster since they don't have to worry about IE6? Similarly, GWT and similar libraries should benefit as well. Websites should be able to remove some CSS hacks as well, though not all of them... 

Next, I want to see the same passionate disdain directed towards IE7. Now IE7 is not the bug ridden beast that IE6 was. Instead, IE7 is basically a properly patched IE6. As a developer, you would definitely prefer to deal with IE7. It works differently from all other browsers, but it is relatively consistent in the way it works. You could not say this about IE6. However, IE8 is much closer to the rest of the world. If IE7 were to drop off to obscurity like IE6 has done, then imagine the benefits. There would be far less need to write things in a different way for IE. There would be far less need for conditional CSS. 

So stop hating on IE6. Move on. Start taking advantage of a world where IE6 is irrelevant.  And if you still want to hate, start hating on IE7. With a passion.

Web Developers Are Stupid and Arrogant

The past couple of days has seen an amusing rant by PPK, that then turned into a retraction and call to action. The original rant included a condemnation of iPhone developers as being stupid and arrogant. Others have adequately refuted PPK, so I won't bother with any of that. His post made me realize that it is in fact web developers who are mostly commonly guilty of stupidity and arrogance. Here's what I mean.

It's easy to look at the world today and say that web applications have won. This is web developer arrogance. Stupidity is to think that web applications have won because web applications are superior to desktop applications. Smarter, but probably still arrogant developers would point to web applications as disruptive technologies. This involves admitting that web applications are inferior, but good enough, and present enough other "cheaper" advantages to compensate for their inferiority.

To understand why the "web apps have won" claim is dubious. There are definitely a lot of awesome web applications out there. Many of them were created back in the mid/late 90's, The "features" of these applications were the key to applications, not the user interface. Now these days, most of these web applications offer APIs/web services/RESTful interfaces/whatever you want to call them. In many cases it is possible to build desktop applications that tap into the same features as these web applications. However, this was certainly not the case 10-15 years ago.

So if APIs make it possible to build desktop apps that offer the same features as popular web applications, why haven't people switched? The first most obvious answer is inertia. If you are used to accessing Google or Amazon on the web, that's probably the way that you will always use it. Something else to consider is that for many web applications, it does not make sense for them to offer all of their features through APIs because it hurts their core business. This is most obviously true for advertising based companies like Google, Yahoo, and Facebook. Their web applications not only provide very useful features to end users, but they serve ads that make money for the companies. If all of their users switched to using desktop applications that only offered the features with no ads, then the companies would lose their revenue streams. Their business is connected at the hip with their UI, so it is in their best interest to make sure people use their UI -- which is a web application.

However, there are other very successful web applications whose main revenue does not come from ads. Their business is distinct from their UI. E-commerce companies like Amazon and my employer, eBay are obvious examples. For example, eBay offers trading APIs that provide almost all of the trading features of eBay. This is particularly true for selling on eBay. This makes sense, as eBay does not need a seller to use the eBay UI to sell something, as the UI is not what makes money for eBay. As a result, around 50% of all items for sale on eBay come through 3rd party applications built on top of the eBay trading APIs. The vast majority of these (especially the popular ones) are desktop applications. Give people a choice, and a lot of people choose desktop applications.

For another interesting example, just look at Twitter. This is a company that came into an existence after web APIs had become the norm. So Twitter has provided a comprehensive set of APIs since early in its existence. Further, they have not pursued an advertising model that would marry their web based UI to any revenue streams. So they have kept their APIs in sync with their features. For example, they recently added list and retweet features to their site, and added them to the APIs at the same time. As a result, there are a huge number of desktop applications for accessing Twitter. Indeed, Twitter says that 80% of their traffic is from APIs -- either desktop or mobile applications. For most Twitter users, there have always been feature-equivalent desktop alternatives to Twitter's web based UI, so many users chose desktop applications over the web.

Finally, let's look at one more example: existing desktop apps. There has been an incredible amount of money spent on creating web applications that provide similar functionality to traditional desktop applications: email, word processing, etc. Heck, Google has spent a lot in this space just by itself. These are useful applications, but it is rare for people to choose these apps over their desktop equivalents. In most cases these apps try to go the disruptive route, i.e. don't try to be as good, but good enough and cheaper. They have had little success so far. Of course, inertia is a valid argument here, too. The one case where there has been success is GMail. In my opinion its success is not because people like it's web UI over a desktop UI, or even that the web UI is "good enough" and cheaper. No, it's success is because it has offered innovative features over other web and desktop based alternatives: fast search, stars/labels, threaded conversations, etc. Even give all of that, many people still choose to use desktop clients to access their GMail (I'm definitely not one of them.) Anyways, once again it's the features, not the UI.

I am not going to sit here and claim that desktop wins over web all the time. I'm not arrogant or stupid enough to make such claims. However, be wary of claims that web apps win over desktop apps. One could argue that with the preponderance of APIs (especially spurned on by mobile apps) and the popping of the advertising based web 2.0 bubble, that the future will hold even more opportunities for desktop alternatives to web applications. Maybe web applications have jumped the shark. So don't put up with web developers who insist that web applications have won (especially if they try to extrapolate this flawed argument to the mobile world). They can go on and on about technology, standards, interoperability, etc. Just remind them that it's the users who matter, and when given a choice, the users do not always choose web applications. Time to polish off your MFC and Cocoa skills!

Don't Dream It's Over

Sometimes you need some 80's music. While you listen to that song, I want you to think about something: the Opera browser. As an OG geek, I used to use Opera -- I even paid for it. It was so much better than IE and that was back in the day when there was just the Mozilla Suite Monster, no Firefox. Sure, there were sites that didn't work well in it, or that actively discriminated against it (including my current employer...) That was ok. It was so much faster than anything else out there, that it didn't matter.

As the world has turned over the years, history has proved Opera right. How fast your browser is does matter. How secure your browser is does matter. There is plenty of room for innovation in the browser space: tabs, download managers, speed dial, magic wand, etc. So many of Opera's ideas have been taken by the Firefoxes, Safaris, and most lately, the Chromes of the world, and touted as innovations -- that were then copied by IE. Meanwhile, what's happened to Opera? It doesn't always pay to be right.

I got news for you, Opera is still kicking butt. There has been a renewed focus on browser technology, and in particular what is collectively known as HTML 5. Apple, Google, Mozilla, and even Microsoft all like to talk about how awesomely they implement the HTML 5 specifications. Turns out they are still way behind Opera. Don't believe me? Take a look at @ppk's HTML 5 browser comparison. Or perhaps you are more visual...

Code (courtesy of a colleague):

<form> 
    <datalist id="mylist"> 
        <option label="Mr" value="Mr"> 
        <option label="Ms" value="Ms"> 
        <option label="Professor"value="Prof"> 
    </datalist> 
    <div class="entry"> 
        <label for="form-1">Name (required) </label> 
        <input id="form-1" name="name" type="text" autofocus required> 
        ← autofocus here </div> 
    <div class="entry"> 
        <label for="form-2">Title</label> 
        <input id="form-2" name="title" list="mylist" type="text"> 
    </div> 
    <div class="entry"> 
        <label for="form-4">Age</label> 
        <input id="form-4" name="age" type="number" min="18" max="25"> 
    </div> 
    <div class="entry"> 
        <label for="form-5">Email (required)</label> 
        <input id="form-5" name="email" type="email" required> 
    </div> 
    <div class="entry"> 
        <label for="form-6">Blogs</label> 
        <input id="form-6" name="url" type="url"> 
    </div> 
    <div class="entry"> 
        <label for="form-7">Date of Birth</label> 
        <input id="form-7" name="dob" type="date"> 
    </div> 
    <div class="entry"> 
        <label for="form-8">Attractiveness </label> 
        <input id="form-8" name="a" type="range" step="0.5" min="1" max="10" value="5"> 
        <output name="result" onforminput="value=a.value">5</output> 
    </div> 
    <div class="button"> 
        <button type=submit>Submit</button> 
    </div> 
</form>

Opera:

Latest Chromium Nightly:

As you can see, Opera does a great job of implementing many of the new markups in HTML 5, while Chrome .... not so much. Big deal, right? You can do all of these things with JavaScript you say. Yeah, but not only will that be a lot of heavy JS, but it will lose semantics. Don't think that matters? Well, not only do those semantics matter to folks who use screen readers, but it also matters to the most important user of the Internet: googlebot.

Anyways, the point is that once again Opera is leading the way, not the folks who talk the loudest about pushing browser technology.

Note: This blog post created with Opera.

To Suggest or Not To Suggest

Ajax has been one of the most hyped technologies in recent memory. However, much of the hype is deserved. It really has changed the way we (web developers) build applications and the expectation of users. One of the archetypes of Ajax is the auto-suggest text box. I don't know who first came up with, but I first remember seeing it on Google. I think it was once a "special" version of the Google home page (Google Suggest?), but not it is standard:

It makes for a natural option on search text boxes. Here it is on some of the other search engines out there. Yahoo:

And on the search hotness, Bing:

Of course pure search engines are hardly the only sites that have search and thus have auto-suggest text boxes. It's pretty useful for ecommerce sites too...

As you can see, some sites have gotten creative with their suggestions. Here is another great example of that from Apple:

All of these examples are for search text boxes. If what you want is not suggested, you can still type it and the application will perform a search on it. A little more interesting example is on Facebook:

Here there is a "closed list" behind the suggest box: your friends. However, it is still a search box. If you type something that is not in the list, it will still perform a search that will return results. Of course, really all of the suggest boxes have a closed list behind them as well, but that list is probably much bigger than your list of friends on Facebook (unless you are Robert Scoble.) So the theme continues: there is a finite set of predetermined suggestions, but if you type in something not in that set, the application can still process your input.
Recently, I saw a different use for auto-suggest boxes: as a drop-in replacement for select/option boxes (a.k.a. drop-down box or combo box). This is fundamentally different than any of the examples above. It would be like the Facebook example, but with the limitation that your friends were the only valid input into the search box. In fact, Facebook has a scenario that is similar to this: tagging people in photos/videos:

However, even in this case, you can type the name of somebody who is not one of your friends. This is valid input. After all, maybe not everybody that you take pictures of has an account on Facebook. My kids are growing up fast, but my five year old son is not yet on Facebook...
I imagine that this pattern -- using auto-suggest box to replace a select/option box -- is used on websites out there. It seems like a reasonable thing to do to avoid a drop-down with a large number of choices, or a drop-down that is expensive to calculate and is often not used. However, it seems awkward, too. What do you do when a user types in something that is not in the box? I like to type, and I'm both reasonably fast at it and make enough spelling mistakes that this could be a common scenario from me.
Now I am no user experience expert. In fact, as a web developer, I would say that I am perhaps the least qualified person when it comes to judging user experience on the web. My knowledge makes me very forgiving of things that might be confusing to others, while it also makes me critical of things that others would not notice. So I am curious about other's people's experiences and opinions. Are there sites out there that use this pattern? Is it useful or awkward?

My HTML in Scala Wishlist

One of my favorite features in Scala is its native support for XML(link warning, shameless plug.) I love the "projections DSL" (for lack of a better term) that is the scala.xml package. It is great for slicing up XML, and is almost like having native XPath and XQuery built into the language. Using XML in Scala for HTML generation is not as universally appealing to people. Some people think it encourages mixing presentation and business logic and is not "clean." I have even heard folks say that this is a critical flaw in Lift. I completely disagree. Of course I also think MVC is an anti-pattern, but anyways... HTML support in Scala is great in my opinion, but could be even better. Here is my wishlist.

• Better understanding HTML semantics. For example, it is great that the compiler won't let me do something like this <div>Hello world</di>. However, I would not want it to even let me do <di>Hello world</di>. Similarly I should not be able to stick a thead outside of a table. If I have an anchor tag with an href attribute, then I'd like the compiler to do some validation on the value. Am I asking for too much? Well, that's I'm supposed to do, right? Is some/all of this possible with the help of XML schema support? Maybe so.
• CSS. HTML without CSS ceased to exist a long time ago. With all of the DSL possibilities in Scala, is there some way to allow CSS literals and create some type safety for me. For example, I if I fat finger the name of a style on a class attribute, the compiler should catch this. Also I can say from experience that introducing an object-oriented model is really useful in CSS. Finally, the XPath/XQuery-like support in Scala XML makes me think that something similar could be done for CSS selectors.
• JavaScript. Honestly I don't know what I would want here. I like JavaScript, but I know the idea of being able to write your web application in a single language is very appealing to a lot of people. If nothing else, it would once again be nice if the compiler would prevent me from doing something like <button onclick="foo()" ...> if there as no function called "foo" in scope.
• Tooling. Most modern IDEs support things like syntax highlighting, code completion, error checking, code formatting for HTML and XML+schema. I expect at least this much for XML/HTML literals in Scala. I'd also like to be able to step through and debug literals. Refactoring on subtrees of XML (extract method for example) would be sweet. Oh, and large XML literals should not overwhelm my IDE :-)

Safari Flash Fail

This happens to me a lot:

Take a look at the stack trace that gets sent to Apple:

So I think this is being caused because I set the storage allowed by the Flash player to 0 KB. The default is 100 KB. Why do I have it set at 0 KB? It's not because I am some privacy fanatic, just the opposite in fact. It was because I was debugging some PayPal code a few months back that required that Flash local storage be unavailable. Most sites then ask to set it back to the default 100 KB. However, if the SWF in question is too small to show the settings dialog, then it can't show the dialog to ask the user to set the amount of local storage for the site. You can right-click any SWF to bring this setting up, well any SWF that is biggest enough to display the settings manager. This is on a per site basis, but you can set things globally as well. Dealing with the error that is thrown when setting data fails is what I helped PayPal with. Obviously there are a lot of sites that do not deal with this error, and that seems to bubble up in Safari and cause it to crash.

Lift 1.0

The Lift web framework hit its 1.0 release today. Major congratulations goes to the Lift team. I have been lucky enough to get to know Lift founder David Pollak and Lift committer Jorge Ortiz. They are both very bright engineers and the high quality of Lift reflects this. I was very happy to give a quote about Lift for the 1.0 launch. In my quote I talked about how innovative Lift is. I would like to expand on this.

First, let me say that I realize that Lift has borrowed some ideas from other frameworks. Heck, David gives credit where its due and on the front page of Lift lists Seaside, Ruby on Rails, Django, and Wicket as sources of inspiration. Given that here are the things that I find innovative about Lift.

View-first design. Everybody loves MVC, but it is no panacea. I have seen a lot of apps that used Struts or Spring MVC that wound up putting a lot of code in the actions or controllers. If it wasn't there, then it often leaked out into the view code. Even if the view code did not contain enormous scriptlets, there would be tons of control flow tags (if, switch, for-loops, etc.) The view-first approach is to go to the view first, and pull in logic bits (snippets) as needed. If a snippet needs some data, it gets it. If it creates a form, then it handles the submission... sometimes with Ajax. Snippets use Scala's XML support for creating view code (XHTML.) Some people think this mixes view code into "back-end" code, but that is bit of a robotic response. What is really great is that it is all statically typed. You can't reference a variable that doesn't exist or a non-existent method/property (like you would in JSP, etc.) You would get a compile error. Plus your XHTML has to be well-formed -- or it won't compile. As tooling improves around Scala, you will be able to do static analysis and refactoring to snippet code.

Record/Mapper (ORM). Lift's ORM may not be quite as sophisticated as JPA, but its syntax is beautiful. Now some of this is thanks to Scala, but Lift really makes good use of Scala's type system. By using parameterized traits, you get class that look very declarative, i.e. like they are just defining metadata about the mapping between class and database table, but there is actually a lot of concrete code being reused. It is as elegant as the metaprogramming that you see in Rails or Grails (and thus more elegant than JPA) but its more concrete nature not only make it less mysterious, but makes it easier to debug. I would guess that it also helps with performance. Of course it would be hard to measure this, since Scala outperforms Ruby and Groovy.

Comet. One of the things that Rails has had going for it is that it seemed design for Ajax from the get-go. This is definitely true of Lift, too. I would say that Lift is even more designed for Ajax, but that is a little subjective. What is more objective is that Lift is designed for Comet. Once again it leverages Scala brilliantly, by using Scala's Actors.

That's just a few things, but hopefully you see the trend here. Not only does Lift improve on existing ideas, but in some cases it really breaks new ground.

EclipseWorld

This week I have been speaking at EclipseWorld in Reston, VA. I have been talking about Ruby on Rails mostly, and a little session on iPhone (web) development. Most of the developer here are Java developers, so they look at Rails as a way to make their jobs easier. It's a grim economy, and they are being given the classic task of doing more with less. They look at Ruby on Rails as a way to do just that. More features. Less code. Happier developers!

EclipseWorld has been a great conference to speak at. What has been very cool for me is interacting with developers outside of Silicon Valley. Now don't get me wrong, if you are a developer, especially a web developer, then Silicon Valley is the place you want to be. I would compare it to working on Wall Street if you are in finance, or working in Hollywood if you are in show business. It's not for everybody, but it presents the chance to prove that you've got what it takes, and, if you are lucky, a chance to make a lot of money.

However, in the Valley it is easy to forget that most web development is not about creating The Next Big Thing. In the Valley, @tychay will rip you up for using Rails because it fails at web scale. On the east coast, I have met a lot of developers creating internal web applications, or maybe customer service applications. These are applications that can run just fine a single multi-core box, even with the database running on the same machine. They aren't stressing out over page weight, database partitioning, or terabytes of cache. They are creating sophisticated applications, and always have way more feature requests than they have time.

These are the people who are most empowered by tools, especially Eclipse. They don't have some huge team with specialists for doing the CSS or tuning the database. They do it all themselves, and Eclipse makes that a whole lot easier. I've written a lot about things you can do with Eclipse, but this experience has really put things into better perspective.

Finally Grails

It sure took me long enough, but I finally got around to writing about Grails. This article gives some introduction to Grails, but also talks about using Grails with Geronimo. The most valuable info in there might be the Geronimo deployment plan. This shows which packages to exclude from Geronimo's classpath when deploying a Grails application. This prevents classpath implosion.

ACM Ajax Seminar

Today I taught a seminar on Ajax to the Bay Area chapter of the ACM. It was a lot of fun. The audience was great, asking lots and lots of excellent questions. The level of enthusiasm was very rewarding. I promised the folks there that I would provide download links. So here they are:

The Slides (OpenOffice format)
The Slides (One big PDF)
All Demo Code

The demo code also includes a Scriptaculous demo that I did not have time for. It uses Ruby on Rails. The Prototype demo uses PHP, and the other ones all use Java. A couple of the demos uses a database, and for those I used MySQL.

Finally I wanted to also thank Sang Shin and Doug Crockford for allowing me to use their material for the seminar.

Web 3x and Web Lite

Earlier today, I read this interesting article about the growth of web pages. In short, average web page size has tripled in the last five years.

Yes this swell in page size corresponds very nicely with everyone's favorite cliche, Web 2.0. It is obviously not a coincidence. More features and interactivity is going to be mean more initial download, which is the very coarse metric being used here. What may have been four web pages may now be one Ajax-ified one, whose weight is three times as much. More interesting tidbits.

• Average page weight: 310 KB
  
• Average total JS: 68 KB
  
• Average # of external JS files: 7
  
• Average # of unique external JS files: 6 (gotta love duplicates!)
  
• Average total CSS: 15 KB
  
• Average total image pixels: 49,144
  
• Percentage of HTTP requests dedicated to images: 75%
  

That last stat is not really as bad as it seems. Images are (can be) loaded in parallel. There is a limit of connections per domain, however (generally two.) A common trick is if you have to load 20 images all from the same server, trick the browser. Make the first two images from images0.mydomain.com, the next two from images1.mydomain.com, etc. Obviously images0 and images1 point to the same IP address, but you get the gist. The browser will load all of the images in parallel (or close to it.) The one disadvantage is you can lose some browser caching. The browser will think that http://images0.mydomain.com/foo.gif and http://images1.mydomain.com/foo.gif are different images.

The other interesting thing talked about in the article, is that broadband speed has more than kept up with page bloat, err Web 2.0. However, not everybody has broadband, and you are basically suffering now more than ever if you do not. These studies are only talking about home users, what about mobile phones? If you are an iPhone user on EDGE, how do you like 310 KB pages?

It seems likely that we will start seeing "lite" versions of websites in the future. We kind of see this for mobile devices, i.e. m.mydomain.com or mobile.mydomain.com. At some point, will we start directing dial-up users to the lite sites? If you were a dial-up user, how would you feel about that? I'm from the South, so it kind of feels like segregation to me: "equal but separate." 

AJAX, the REST Killer

I was at a demo today that was showing off an application that was designed to be a reference implementation / blueprint app. Of course it had some AJAX based features in it. One of the AJAX calls allowed an existing object to be edited, and another would delete said object. I noticed that the AJAX was using HTTP GET. I pointed out to the developers that since this was supposed to be a blueprint app, it should get everything right and use POST for both of these. Of course it should really use a PUT for the update and DELETE for the delete, but browsers just don't support that.

After the presentation, one of my colleagues pointed out that a lot of times we are forced into GETs because of cross-domain calls. He was right, and made me realize how the hack-that-is-AJAX is always rearing its ugly head. Cross domain calls are part of life for any large, distributed site. In an AJAX world you either have to use JSONP style calls, i.e. using a Script tag to make your call, or you have to use a server proxy. Of course going directly to the appropriate domain is much more efficient, so JSONP is going to usually win. There's no way to do an HTTP POST on a Script tag, so there you go. Forget all about respecting the REST protocol.

Of course Flash and Silverlight (gotta include it now!) both allow for declarative cross domain security. No need for JSONP hacks. Now if only they both supported HTTP PUT and DELETE...

Galpin on Rails on IBM

A couple of pieces that I wrote on Rails appeared on IBM at the end of last year. I didn't even realize it until today when somebody emailed me about one of the articles.

The first article is part one of a four part series on using Rails, XForms, and DB2's PUREXML tables together. I also wrote part two, but another writer wrote parts three and four. Only part one is up so far.

The second article is the third tutorial I wrote on using Eclipse as a web development platform. Part one was about using Eclipse for Java web development, and part two was on using it for PHP development. The last tutorial is on using Eclispe (via RadRails) for Rails development. Note, the link for part two shows the intro page to part one. If you login, it correctly gives you part two. Clearly some kind of technical glitch for IBM!

FUD by Doug Crockford

I like some of Yahoo's Doug Crockford's ideas on JavaScript. So I watched this talk by him on the State of Ajax. It has a great buildup on the history of computing and comparing pre-Ajax web apps to the IBM 3270. He's also a clearly opinionated guy, but he really lets this influence his statements. Examples...

He claims that Java applets failed primarily because of "write once, run anywhere" was a broken promise. That's ridiculous. First off Java applets only needed to succeed on Windows, so write once, run anywhere was irrelevant for a desktop technology. He does correctly state that Java UI (AWT) was no good. He commits a lie of omission, by failing to mention Microsoft's JVM infringements. He then says that Java succeeded on the server because (wait for it) write once, run anywhere is not an issue on the server. Not only is this wrong, it's the opposite of the truth. When I first started writing Java server apps in 2000, I was writing it on Windows and deploying it to Solaris.

Next he heaps praise on JScript and says that its "bugs" were just reproductions of Netscape's bugs. There is some truth in this, but it certainly shows his bias. He states that JScript inveted XMLHttpRequest and that there no innovations for five years after that until Jesse James Garret "discovered" Ajax.

He completely ignores Flash. Earlier he states that write once, run anywhere is really difficult and anybody that can pull it off deserves credit. Well Flash pulls it off. He claims that JavaScript pulls it off, but that's just plain wrong. I don't know how somebody who knows JavaScript so well can state this. He's smart enough to know all about doing browser sniffing and conditional logic based on the results of that sniff.

GWT 1.4

I recently finished writing a series of four articles for IBM on GWT. You might recall that earlier this year I wrote a two-part tutorial on GWT. The new articles for IBM aren't as introductory, though they don't require a lot of prior GWT knowledge. They're definitely more advanced though. I wrote the articles using the 1.4 release candidates, and then last week 1.4 went final. So I figured it was time to look at how GWT has progressed since the last time I wrote about it.

• Cross Platform Problems -- Hard for me to say how much things progressed. I did almost everything on my MacBook, and things had always worked well on OSX. I did do a little on Windows. I had to do a little more configuration to get it to correctly run in hosted mode than I did on OSX.
• Obfuscation -- Still a problem. Of course you can turn it off, which is good. Still, I pity the fool trying to debug GWT JS in Firebug.
• No Java 5 -- Still a problem. GWT will choke on a List, but String[] is ok. Also, any annotations will choke it as well. So no returning @Entity classes from your RPC services.
• Lots of boilerplate code to write -- Still a problem. You still have to write two interfaces and one class to create an RPC service. Inserting GWT into an existing page is easier, with just a single JS file to include.
• Better build support needed -- This has a improved a little. The ant script that is generated is slightly more useful than the previous version. Still frustrating that there's no build script generated for creating a WAR file. You don't have to use GWT with a Java application server, but obviously a lot of people are. The RPC services require it as well.
• RPC is bad -- Of course that can't change! RPC is still bad. I like how in the documentation, the GWT team mentions two different design philosophies for RPC services. One is to create RPC's specifically for the web application. The other is to have general purpose ones that can be re-used by many clients. Good luck on that last one. That would mean your general purpose services can only accept and return classes from the subset of Java 1.4 that GWT supports.

So that addresses my old pain points. Not much progress has been made. There are some new good points.

• Performance improvements -- Ok, they've really done a good job here. Things are snappy even on a page where you use lots of generated JS to create your UI.
• More Widgets -- Steady progress has been made here. You can get pretty far just using their widgets. I haven't tried out any of the commercial WYSIWYGs built for GWT, but these seem more viable and useful with more widgets. Of course, their widgets are very limited. They definitely aren't components. Compare a GWT widget to a Flex component, for example, and it's not even funny. GWT's widgets are pretty lightweight, but are low in functionality compared to similar abstractions found in similar frameworks.
• Image Bundles -- Very cool new feature. I like how they use CSS to get nice layout of the images from the bundle. It seems like a great way to optimize the static assets in your app. Seems like it might play tricks with a user who wants to right-click on one of your images and do save-as, but oh well.
• Localization -- This has always been there, but it seems better documented and exposed now. Plus there is a command line tool for it. It's pretty nice, though I question the logic behind doing your localization in code that is going to be converted to JavaScript and executed on the client.
  

So all in all, GWT has made some nice improvements and added some new features. They have not addressed the things that are their most blaring weaknesses (IMO), but then again those are probably the hardest things to address. I was surprised though. From my last big write up, I got a lot of feedback from people talking about how these exact issues were being addressed, i.e. that the work was already in progress. Was that really the case?

Powerset on Rails

Like a lot of folks, I've been curious about Powerset, the company promising to revolutionize search via natural language queries. I got an interesting email from them today. Here's an excerpt. All emphasis (bolds and italics) is from the original email:

I'm happy to announce that we are, in fact, launching our front-end on Ruby.

Cool, huh? For everyone's sanity (and in avoidance of some of the flame wars to ensue), do note that we are going to be using Ruby (the language) but not necessarily Ruby on Rails (the web framework).

Why Ruby?

1. We've already got the brains

2. Ruby is already being used throughout the company

3. We're not worried about scaling

This is obviously an unusual broadcast. Do most potential users of Powerset care what programming language its front-end is written in? So why send out this email. It all comes down to reason #3. We're not worried about scaling. Heh heh.

Why all the fuss? Rails can scale, right? The email goes on to outline how they've talked to the Twitter guys who told them not to worry about the problems they had and go for Ruby. They also explain how their system is simpler and they won't have the same problems that Twitter had.

This is just too funny. Basically they have a bunch of Ruby developers (see reasons #1 and #2) who want to do the front-end in Ruby. I'll go as far as to guess they really want to do it using Rails, but they haven't been able to win that argument yet (hence the disclaimer about it.) Yet, their management is so uncomfortable with this that they had to go and do a broadcast email apologizing-- err I mean explaining why it's ok for them to use Ruby.

I can't imagine a worse endorsement about a language than this! Seriously, if they had decided to write their front end in Java, PHP, C#, Perl, Python, etc. would they have felt the need to publicly defend their decision? Are they going to send out emails about their choices of databases, operating systems, web servers, etc.? I think not.

It's a sad day for Powerset, but an even sadder day for Ruby and Rails.

The Pain of Silverlight

I'm starting to write the Stocks application with Silverlight. I installed Visual Studio Orcas. That took forever. I installed the Silverlight extension to VS and the Silverlight 1.1 runtime. That was a lot to install! But that's the hard part, right? Microsoft development tools are great, right?

Uhh, no. Creating a Silverlight project was easy enough. I was surprised to see that VS did not provide any kind of visual designer for Silverlight. It was basically just an XML text editor for creating XAML. I was able to quickly throw together Hello World. I hit the Run/Debug button, and hit my first problem. VS had a problem passing the URL for my Hello World page to Firefox (the default browser on my system.) So I changed my default browser to IE, and tried again. Hey that worked.

I played around with the IntelliSense in the Silverlight/XAML editor. I only saw graphical options. No form elements. I needed a way for a user to input a Stock symbol for my Stock app! So what to do?

There is a visual designer for Silverlight, Expression Blend. So I downloaded that (yet another download) and cranked it up. I created Silverlight project in it. I looked for a text input box in the palette, but didn't see anything I consulted the Help. It indicated that there should be something in the palette for this:


Further exploring Help reinforced this:



The icon I see there sure looks like the Icon in the first screen, near the blue circled 4. But when I use Expression, I only get the TextBlock option. That's also the only option I got when I used IntelliSense in Visual Studio. I'm getting the impression that Silverlight does not currently support any kind of text input!

It appears that I am not alone in this frustration. Luckily that led me to some text controls that somebody else made...