The JavaScript assembler

Earlier this week, the third part in a series of articles I wrote was published on developerWorks. The series is on CoffeeScript, a language that I first learned of about a year ago. Last summer I read CoffeeScript by Trevor Burnham over the course of a week while on a cruise ship. Later that summer I was at the Strange Loop conference, and serendipitously wound up at a dinner table with several of the guys who develop Clojure. A lot of the conversation that evening was on ClojureScript.

As a  long time web developer I am not surprised by the growing interest in programming languages that compile into JavaScript. The browser has never been more important, and thus JavaScript has never been more important. Most programmers that I know find that JavaScript leaves a lot to be desired. This is not just a matter of taste, JS is fucking weird. In fact for years, a lot of the more "serious" programmers I knew would turn up their noses at JS. They were too good for that trash, so they kept to the back-end. It's become tougher and tougher to avoid JS, so ... use a different programming language that compiles to JS. Brilliant!

This idea is hardly new. Personally, I can't help but think of Google Web Toolkit. For years GWT has let your write in Java and compile to JS. So why bother with CoffeeScript, ClojureScript, and probably some other awesomes that I'm leaving out? One obvious thing is that Java is an existing language and is statically typed. So it has its own baggage (which includes excellent tooling, but still) and of course many developers do not like statically typed languages. In fact, many argue that web development with a statically typed language is an impedance mismatch of the highest order. Personally I think GWT was also hurt by initially defaulting to producing highly obfuscated "production ready" JS. I always thought that was developer unfriendly. I think that may have changed over the years, but sometimes it's hard to overcome an initial impression. I think one of CoffeeScript's great strengths is that it produces very developer friendly JS by default. I've never built any huge apps in either language, but in my limited experience I found that debugging was easier in CoffeeScript. I could usually figure out what I had done wrong just by looking at the error message.

Any discussion like this would be amiss without mentioning Dart. It gives you dynamic and static aspects, in a way that reminds me of ActionScript. Personally I think optional typing works best when use variable:type style declarations, like ActionScript and Scala, but that's a minor nit. Of course when you consider that "native Dart" is supported in a version of the world's most popular browser, the possibilities start to get really interesting. I attended a Dart meetup recently where the Chrome developer tools allowed breakpoints and the like directly in the Dart script that was running. The SourceMap proposal would allow this kind of debugging of other higher level languages like CoffeeScript and ClojureScript.

The future seems bright for languages targeting a JavaScript runtime. I haven't even mentioned node.js and how that makes these languages viable for both client and server applications. To me CoffeeScript, Dart, and ClojureScript are all promising. Each seems to have a large target audience. If you like dynamic languages like Python and Ruby, CoffeeScript fits right in. If you prefer statically typed languages like Java and C++ and all of the nice tools that come with them, then Dart is attractive. If you like pure functional languages like Haskell and Lisp, then ClojureScript is a fun way to get your lambda calculus kicks in the browser. Of course there is a lot of anti-Lisp bias out there, so maybe there's room for another functional language that targets a JS runtime, something like a F#-script or ScalaScript.

Testing Geolocation for Mobile Web Apps

Using geolocation in a web application is really cool. The JavaScript API is quite simple. Testing it, can be a different story. Here are some useful tips:

If you only need to get the user's location once, testing is not too bad at all. First, you can actually use Firefox 3.5+. However, make sure that you are using wi-fi, or otherwise you will never get a location. Your app will just seem to hang. Anyways, being able to use Firefox for testing is great because it is a full desktop browser with awesome developers tools like Firebug. Often you will have some pretty complex JavaScript in a web application that uses geolocation, so being able to debug with Firefox/Firebug is invaluable.

Of course you will also want to test on mobile browsers. You can definitely use the iPhone simulator that is part of the iPhone SDK. Geolocation works great on this, but it gives you a bogus location -- the location of Apple's HQ in Cupertino, CA. If you actually live near Cupertino, this can be really misleading. Anyways, it will only give you this location, never anything else. So it is somewhat limited.

Testing on an Android emulator is another possibility. However, in my experience, it is currently broken. On an Android device, when a web page wants your location, the browser brings up a window to prompt you for permission. On the emulator's browser, I never see this permission window and the app just hangs and never returns a location. That's a bummer, since the Android SDK makes it possible to send mock GPS coordinates to an Android emulator. It would be great if web developers could take advantage of this.

Testing on an actual devices is of course a possibility. Your web application needs to be reachable by the device. This could mean using wi-fi on your device, so you are in the same intranet as your development sever, or it could mean deploying your app somewhere public. I like using the Google App Engine for the latter, and then test on the assortment of devices that I have laying around.

If you need to test location updates, i.e. when the user moves you want your web application to know about it and respond in some way, then things get trickier. The iPhone simulator is out of the question. An Android emulator still suffers from the same problems. So now you are down to devices. There is no way to send mock GPS to an iPhone, so to test on an iPhone, you need to actually change your physical location. Coding and driving FTW!

However, it should be possible to test on an Android device. The Android SDK theoretically allows you to send mock GPS to a real device. The command that should work is "adb -s XXX shell geo fix AAA BBB" where XXX is the serial number of your device (get it by doing "adb devices"), and AAA/BBB are the fake latitude/longitude. Sounds good, right? I tried this on my Nexus One, and got a big fat permission denied. Turns out you need to be root to do this on a real device. Ok, so I rooted my Nexus One. Then I tried it. Result? "gps: not found." Suddenly the gps command was not recognized. So ... coding and driving FTW!

It's 2010, where's my HTML5?

This morning, one of my co-workers related to me that there had been mention of HTML 5 on an NPR broadcast that he heard. He joked that this morning there were probably lots of advertisers asking their designers about using HTML 5 for their next whiz-bang ad campaign. So obviously we are still in the early part of a hype cycle on HTML 5. So what's the state of the technology and where is it heading?

In terms of pure technology, there is a lot of notable progress. Google did the right thing recently and axed Gears. They had some nice innovations in Gears that have since been rolled up into the HTML 5 standard. Now it makes perfect sense to abandon their proprietary implementation, and embrace the standard. Gears is important because it is (or was) part of Chrome. Thus there are essentially five browsers that now implement a significant subset of HTML 5: IE8, Firefox 3.5, Safari 4, Chrome 4, Opera 10. According to the latest browser share numbers, that means that 53% of users out there are using a browser that supports a lot of cool features, like local storage and selectors. That's more than half! It's tempting to extrapolate from that, but don't get too carried away.

The #1 browser of the HTML 5 browsers, is IE8 -- which does not support a lot of the features that the other browsers support like canvas, video tag, and web workers. For those would be HTML 5 advertisers out there, you better scale back your expectations. For desktop browsers, a lot of the most compelling HTML 5 capabilities are far from being widely supported. How long until IE9 comes out? How much more of HTML 5 will it support? How long until there is high adoption of it? That's too many questions to be comfortable with. Ask me again this time next year.

However, it's fun to pretend sometimes. Let's pretend that IE 9 did come out soon and that it supported everything supported by Firefox and Chrome. Let's even pretend that Microsoft was able to awaken some of their old monopolist mojo and somehow force people to upgrade. So we have a world full of canvas. Will the next great annoying ad use this? If you read my last post on Flash hate, you probably realize the answer is no. As Terry pointed out in the comments there, the Flash authoring tools really empower designers over developers. So before we see an explosion of canvas magic, there will have to be huge strides made in tooling. Now what about video? Even in our dream world, there are issues known as codecs. There is no common video format that will run on both Firefox and Chrome right now, and who knows what Microsoft will do here (WMA FTW!) So we are going to need some technical consolidation here. Let's summarize all of the things that we need to see happen to open up the golden age of HTML 5.

1.) New version of Internet Explorer that supports much more of the HTML 5 specification.
2.) This new version of IE needs to become the dominant flavor of IE.
3.) Outstanding tooling developed to enable designers to leverage HTML 5 capabilities.
4.) Standardization on video codecs by browser vendors.

Don't get too bummed out about all of this. I actually think there is real progress happening on #1 and #3. #4 probably has to wait for #1 to happen. #2 is the most problematic. Since we're dreaming, maybe if IE 9 came out with H.264 support, Google could dramatically drop support for anything else on YouTube, and thus nudge all IE users to upgrade?

If you want to drink the HTML 5 Kool-Aid, and this is bumming you out, I'm sorry. You might feel better to know that the HTML 5 situation is significantly better on mobile devices. According to AdMob, 81% of internet usage in North America was either on iPhones or Android devices. Now, most of those Android devices are running Android 1.5 or 1.6, whose browsers (can we call it Mobile Chrome please?) still use Gears -- not HTML 5. Let's  hope that most of those are upgraded (by their carriers) to Android 2.1 soon, and then we're in business. We've got all kinds of HTML 5 goodness available to 80%+ of the users out there.

The bad news is that the mobile market is fundamentally different than the desktop (though, that may be changing.) Here mobile websites compete directly against native applications. Desktop users rarely face the question, should I use an app or a website to do XYZ? Normally XYZ dictates one or the other. So for a company, there may be a lot of cool stuff you can do with HTML 5 on a mobile browser, but there are even more cool things you can do in a native app. Not only are the native apps more powerful, they are generally easier to build because of superior tooling and cohesive architecture. Maybe JavaScript vs. Objective-C vs. Java is a matter of taste, but HTML 5 has nothing even resembling the application models presented by Cocoa and Android. Instead it has the DOM APIs in JavaScript. Now throw on the differences in tooling, and you can see why for many companies it is cheaper and easier to build an iPhone app and an Android app instead of building one HTML 5 app. That's not even considering the hypothesis that the "app store" model is more appealing to end users than the browser model.

The Flash Haters

The iPad announcement had lots of subtleties to it. One of them was that the iPad browser, presumably Safari, does not include a Flash plugin. This has lead to a lot of people from Adobe getting upset. What has amazed me through all of this, is that there is even more hate for Flash than ever before. As somebody who has done a fair amount of Flash development in my career, this has really struck me as odd. Why all of the hate for Flash?

In some ways, I can actually understand why Apple would hate Flash. The Flash plugin on OSX is nowhere near as good as the Flash plugin on Windows. Now Adobe will counter they spend a disproportionate amount of money on Flash for OSX, i.e. they spend more money per OSX user than they do per Windows user. So what? There are a lot of other technologies out there that work equally well on both Windows and OSX. Just focussing on web technologies, Firefox and Chrome on the Mac are very comparable to their Windows versions. So is the Java plugin. Some other products, like Opera and Skype, tend to trail in terms of features on OSX, but not in terms of quality. If these other companies can do this, why can't Adobe? It's obviously either lack of ability or prioritization. My guess is the latter, and hence you must expect hate from Mac users. And from Apple. So it should not be so surprising that Flash is left off the iPad, iPhone, etc. If Adobe really wanted to get Flash on those devices, an obvious first step would be to make Flash on OSX as good as Flash on Windows. That is something that Adobe can do all by themselves. Instead it seems like they would rather publicly bemoan their exclusion on the iPad. Now is there a guarantee that if they improved Flash on OSX, Apple would care? No, of course not. However, there is no way they will get on those devices without this first step, IMO.

Ok, so Mac user hate for Flash is understandable. Along the same lines, I would guess that Linux user hate is also understandable. That is, if there were actually any Linux users out there. I digress. I see a lot of hate from developers. What about that? That one is not so easy for me to understand. As a web developer, Flash has been opening doorways for me for a long time. Need to cache 30 KB of data on the client? For a long time this was impossible unless you used Flash (or maybe Java.) Need to make a cross-domain call? That is still generally impossible from the browser, unless you use Flash. Need to show 2D/3D graphs and visualizations? Again, this is still generally impossible, except through Flash. Oh and video... The only way to show a video that can be played on all of the top five web browsers (IE8, IE7, Firefox 3.5, IE6, Firefox 3.0) is to use Flash.

But, but, but, what about HTML 5? Many people think it will kill/replace/maim Flash. Apparently this list includes a certain Steve Jobs. As I have said before, I hate web standards. I mean, I love them, too. But I hate them mostly. Standards are the opposite of innovation, and perhaps its enemy. However, they are the engineer's best friend. Flash has provided a lot of innovation, that HTML 5 will simply copy and standardize. That is the best possible function of standards. However, the story is not so simple. Some things, like local storage,  are already supported by IE8, Firefox 3.5, and Safari 4. This is like Flash's SharedObjects, though slightly less powerful (but good enough.) Not everything is so rosy; canvas and video playback immediately come to mind. Canvas is particularly a hot topic. You can do some amazing things with it, but it requires payment in blood. I cannot imagine what it would be like to program Farmville in JavaScript using Canvas. Maybe somebody will come up with tools for that. In fact, I would be surprised if Adobe did not do this. Of course, we still need to hope and pray that canvas is supported in IE9...

So back to the original point. Why do developers hate Flash? It certainly enables them to do more stuff in web applications. Even if that list of extra features is being shrunk by the HTML 5 monster, it's still there. So shouldn't they love it? Well, you would think so, but they don't. Here are some guesses as to why:

1. Expensive tools -- Most of the tools used by a lot of developers are free. The Flash authoring tools are far from it. That can be annoying. You have to throw down a lot of money just learn something. That's not cool.
2. Bad tools -- At least from a developer perspective. The Flash authoring tools have historically been aimed at designers. That changed with Flex Builder/Flash Builder, but the damage was done. Also, those tools were aimed more at application development, which largely seemed like a waste of Flash.
3. Back to the Mac  -- A lot of developers use Macs, and we all know about Flash on OSX...
4. Accessibility -- Unfortunately this doesn't mean much to most developers. But it should. I must admit, that a year ago I probably wouldn't have listed this. However, as I have learned more, I have learned how Flash is such a huge problem for accessibility. This is actually one area where HTML 5 really shines, with the inclusion of the ARIA standards.
5. Design envy -- This kind of goes back to #2. Flash has had all the awesome capabilities for years, but they have largely gone untapped in web applications. Maybe by playing to designers all of those years, Flash has earned enemies. Imagine a developer being asked to scope some radical web feature. They give a typically large scope for it. His manager turns around and gets a designer, a freakin' designer, to do it in half the time.

What have I missed? Why do web developers hate Flash so much?

Stop the IE6 Hate

For a long time now, it has been common for developers to go on and on about how they hate IE6 and how users needed to be discouraged from using. It's gotten old. It's trite. It's time to move on. 

IE6 has become a non-factor for most sites. It trails far behind IE7 and IE8, not to mention Firefox. And of course, its numbers are always shrinking. I know that at a certain popular website that I happen to work for, we no longer test for IE6. It's been that way for us for 6+ months. It's over.

So now that it's over, there are some things that I want to see. First, I want to see all of those JavaScript libraries that everyone uses, drop their IE6 code branches. Shouldn't jQuery, Prototype, ExtJS, Dojo, etc. all be able to shave off a huge amount of their code? Shouldn't these libraries now progress even faster since they don't have to worry about IE6? Similarly, GWT and similar libraries should benefit as well. Websites should be able to remove some CSS hacks as well, though not all of them... 

Next, I want to see the same passionate disdain directed towards IE7. Now IE7 is not the bug ridden beast that IE6 was. Instead, IE7 is basically a properly patched IE6. As a developer, you would definitely prefer to deal with IE7. It works differently from all other browsers, but it is relatively consistent in the way it works. You could not say this about IE6. However, IE8 is much closer to the rest of the world. If IE7 were to drop off to obscurity like IE6 has done, then imagine the benefits. There would be far less need to write things in a different way for IE. There would be far less need for conditional CSS. 

So stop hating on IE6. Move on. Start taking advantage of a world where IE6 is irrelevant.  And if you still want to hate, start hating on IE7. With a passion.

How Google Makes The Net Suck

Some people like to compare developers to artists. When it comes to web development, some people say there's always a man behind the curtain. Whether you agree or not, there are definitely certain freedoms that web developers enjoy. As a web developer, what are the greatest limitations and obstacles in your way? Once it may have been browser quirks. Now maybe it's all those annoying users who still use IE6. However, I think the greatest obstacle to progress is Google.

Now Google would have you believe just the opposite. I do not think they are disingenuous. In a large organization, it's all too easy for different groups to have different motivations. But ask yourself this, how much money does Google from Chrome? What does Google make money from? That's easy: advertising on search. And that is what is hold us all back.

If you have endured my purple prose to this point, I will finally cut to the chase. One of the most important aspects of any web page is how its PageRank. If your web page is all about deep sea diving, where does it surface when somebody searches Google for deep sea diving? The black art of making your page get a higher PageRank has given birth to an entire cottage industry known as Search Engine Optimization (SEO.)

As a developer I have never given much thought to SEO. I always thought that SEO was about the content of the page, and web developers are not responsible for the content. We are responsible for retrieving/generating that content from all kinds of sources, as well as creating applications that are easy and intuitive for the user to interact with a meaningful way. But, if we go back to the deep sea diving example, we're not responsible for providing information about deep sea diving. Heck you are lucky if most developers even known how to swim, but I digress.

But I was wrong. SEO is not just about content. It is about structure. If you want a good PageRank, then quality content about deep sea diving will lead to other people linking to your page and that will increase your PageRank. But there are much more instantly gratifying things you can do. For example, your page should a title and it better contain the term deep sea diving. No big deal, right? The title is really just part of the template outside of the main contents of the page. Its value has little effect on anything, besides PageRank that is. However, it gets worse.

To maximize your PageRank, then immediately after your page's body tag you should have an H1 tag whose contents should contain the term deep sea diving. Oh maybe you put the phrase on the page, but you put it in a div that styled quite nicely? Not good enough. It needs to be in an H1 tag. Maybe you used some JavaScript to create the H1 tag? That is no good at all. Why? Because The All-Mighty Googlebot does not understand how the page looks to a user. It only understand basic HTML constructs. That's right, it's time to party like it's 1999.

Oh, maybe your organization hired an artist who created a killer deep sea diving logo and you load it on to the page as an image? Not good enough. If you put deep sea diving as the alt text, that will win you some bonus points from the Googlebot, but it is still dwarfed by the rewards you could receive by busting out the H1. Nothing compares to the mighty H1 tag. And don't just put that H1 tag anywhere on the page. Heck you might even get penalized for having more than one! Nope only one, and it better appear (in the HTML source code) as close to the body tag as possible.

Ok, so maybe you give in and put the catch phrase in an H1 tag. That wasn't too bad, right? Now back to your regularly scheduled hacking? Not so fast. Do you have some hierarchical information on the page? Sections, headings, menus, etc? How are you going to do those? Again you better not even think about using things like JavaScript to create them dynamically. Nope, they have got to be static on the page. Back to divs, spans (maybe a table or two), along with some oh-so-clever CSS? Forget about it. Let me introduce you to H1's other friends: H2, H3, H4, H5, H6. That's right, if you want that damn Googlebot to "understand" the hierarchy of concepts on your page, then you better put away your divs and spans.

Maybe you think that's going overboard, but it's not. Do you have a section on your deep sea diving page called "Gear" ? Then if you want to show up on a search for deep sea diving gear, you better have the term Gear wrapped in an H2 or an H3 tag.

What about RIA technologies? Again, if you dynamically create things with JavaScript, it will get picked up, but it is non-optimal. You have to do things the way that the Googlebot wants it done to get best results. What about Flash or Silverlight or JavaFX? Flash will get you screwed on about the same level as JavaScript. Silverlight or Java might as well be black holes. Whatever is in there, is never getting out.

There are tricks you can employ like progressive enhancement. There you do things the way that the Googlebot wants them, then dynamically obliterate that garbage and replace it with rich content that your users actually want. This can backfire. If the Googlebot figures out that you are tricking it, then it will banish you to purgatory.

What if you just make a great web application that users will love and don't bother to worry about the Googlebot? That's fine of course, it just means that people will not find your application by searching for it. Is your business model and marketing efforts robust enough to not need SEO? Yeah, I didn't think so.

So now do you understand? The Googlebot ties your hands, or at the very least makes you jump through all kinds of hoops. There are all these great technologies you could use to make your site as interactive as any desktop application, but The Googlebot does not like this. You've got to play his game whether you like it or not.

Syntax Matters: JavaFX Script

Last night I finished writing an article on JavaFX. It was a lot of fun, and it has convinced me that JavaFX has a strong future. A lot of its promise comes from its syntax. It is easy to get carried away with Rich Media! and Animation! and FPS! but then you forget that JavaFX is a new programming language on the JVM. Here are the highlights of its syntax.

The declarative constructors: When I first started looking at JavaFX, it looked like MXML but in a JSON format. Looks are deceiving. Essentially JavaFX supports name parameters in its constructors. The naming convention is foo : bar, where foo is the name of the parameter and bar is the value. You can put multiple parameters on one line by using a comma to separate, like in most other languages, or you can put each on its own line and eschew the commas. This leads to the JSON-ish syntax. It is probably more accurate to describe it as JavaScript-ish. This becomes even more apparent when you start nesting constructed objects. It really looks a lot like object literals in JavaScript. What is great is that this is not just something you only use for UI elements, like MXML. It is part of the language syntax, so you can use this syntax anywhere you want. Imagine being able to write an MXML expression directly inside a block of ActionScript code...

Speaking of JavaScript: A lot of the syntax of JavaFX looks like JavaScript's more sophisticated cousin, ActionScript 3.0. You use var to declare local variables. You put the type after the variable name, separated with a colon. You use function to define methods. The major difference is using def to denote a constant. In ActionScript, you use const, like you would in C. This is actually kind of bad in my opinion. The def keyword is used in many other languages, like Ruby and Scala, to denote a function. There is no obvious connection between def and a constant. I think they should have used const instead of def. It would have made more sense to do that and maybe use def instead of function.

Functional Programming: A lot of folks are disappointed that closures are not being added to Java, at least not in Java 7. JavaFX does have closures. You can define a var to have a function type and can specify the signature of that function. The syntax is pretty nice. For example, you could do var join:function(String[], String):String to define a variable called join that is a function that takes in an array of strings and a string as input parameters and returns a string. I would like to see this is in ActionScript. JavaFX also has support for list comprehensions. You could do var squares = for (i in [1..100]) { i*i} to get an array of the perfect sqaures up to 10,000. However, JavaFX does not make as much use of closures. You would think that its sequences would have common functional mehtods like filter, map, fold, etc. For filter and map, there are alternatives. For example, let's say you wanted the square of all of the odd integers less than 100. In Scala you would do something like val os = (1 until 100).filter(_ % 2 == 1).map(_^2) . In JavaFX it would be val os = for (x in [1..10][i | i % 2 == 1]){ x*x }. It's a case of syntax over API. The second set of curly braces is like a select clause. I want to like it because it uses mathematics insired symbols.

Other: There are a few other JavaFX syntax bits worth mentioning. First is bind and bound. These are for data binding expressions. These can be very powerful. You can bind variables together, so that the one changes when the other changes. Better is that you can bind to functions and list comprehensions. The other interesting syntax in JavaFX involve the keywords delete and insert. These give LINQ-ish syntax for sequences. In fact if you combine the mathematical style select syntax with insert/delete and with the declarative constructors, you get expressiveness that is pretty on-par with LINQ in my opinion. When you see everything working together, it kind of makes sense but it does seem kind of random at first.

JavaFX Performance

Recently I did a comparison of JavaFX performance vs. Scala. I did this mostly for kicks, and because some people thought that Mr. JavaFX was picking on other, non-Java languages that run on the VM. James Iry duly pointed out that JavaFX should be benchmarked against the likes of Ruby, Groovy, or Scala. It is meant to be a client-side technology, so it should go up against client-side technologies. So I re-did the little performance comparison to match JavaFX against JavaScript, ActionScript (Flash), and C# (Silverlight).

A comparison like this really becomes a comparison of Virtual Machines. For JavaScript there are a lot of choices. I decided to go with some that are supposed to be fast: Google Chrome, Firefox 3.1 (beta 2), and Safari 4 (developer preview.) Because I wanted Chrome involved, I had to go Windows. So I ran everything under Windows XP, running under Parallels on my MacBook. Here is the pretty graph:

I was just a little surprised by these results. JavaFX is indeed the fastest, but just barely. I was somewhat shocked to see Chrome's V8 JS engine right behind. In fact the difference is negligible for small iterations (shown above.) At larger iterations, JavaFX maintained 20-40% margin. As you can see from the graph, Flash and Silverlight were kneck-and-kneck as well, but was always about 7-10x slower than Chrome/JavaFX. Safari and Firefox were very underwhelming.

Of course this was just a micro-benchmark. The code just does a series of recursive calls. So what were are really measuring is the ability of the VMs to unwind these recursive calls. It is not suprising that HotSpot handles this easily. Actually, the same code in straight Java is much faster than the JavaFX version. It is surprising to see how well V8 handles this.

Now does the ability to unwind recursion translate into performance that a web application user would notice? Maybe. It certainly points to JavaFX's or V8's ability to make optimizations to application code. It is probably a more meaningful test than some raw number crunching.

AjaxWorld 2008

AjaxWorld was this week, and it was interesting. I think the down economy is having an affect on everyone, but there were still a lot of interesting things to learn about. On Monday, I did a talk on a favorite topic of mine, networked applications. The talk was a lot of fun, hopefully the audience would agree with that assessment. Overall though, I would say there were a couple of major themes at AjaxWorld this year.

1.) Comet. There were a lot of talks about some form of data push from the server to the browser. Kevin Nilson did a nice job of differentiating Ajax (infinite loop of XHR polls) vs. Comet (long poll.) The folks at ICEFaces have built some nice abstractions on top of Comet. There was also a lot of interest around WebSockets, especially the work by the folks at Kaazing. A duplexed socket connection to the server sounds great on paper. I think there will be some very interesting technologies that grow around that.

2.) Don't make me learn JavaScript! There seemed to be a lot of folks advocating the "only know one language" approach to the web. In most cases that language was NOT JavaScript (even though the Jaxer guys say it can be.) Vendors liked Oracle and ICEFaces preached abstractions that shielded the developer from JavaScript. Of course the GWT folks say do everything in Java. Microsoft says use Silverlight so you can do everything in C#. Of course one of the best sessions was by Doug Crockford who told everyone to man up and learn JavaScript. I tend to agree with Crockford, even though I prefer ActionScript...

JavaScript Faster than Flash

This is the last benchmark for awhile. Well, at least for today. I converted the JS benchmarks to ActionScript and tested them. The result were surprising, as JavaScript in Safari 4 and Firefox 3.x edged out Flash:


A few notes. I could not convert all of the tests, as two of them (the DOM and Ajax tests) were predicated on browser specific code. I could have done 'equivalent' functionality in ActionScript, but it did not appropriate for comparison. Otherwise the code was translated as is ... for the most part. I did add static type information where possible. There were also a few APIs (on Date and Array) that had be tweaked slightly. I tested similar changes to the JavaScript. The only test where there was any effect was the Date test. The JavaScript used Date.parse, which does not exist in ActionScript. The Date constructor does the same thing. If I switched to using the Date constructor in JavaScript, it was just slightly slower.

It certainly seems that much of the performance advantage enjoyed by Flash upon arrival of Flash Player 9 has been erased. Flash had a strong advantage still in more mathematical calculations (dates, integer and floating point arithmetic) as well as string manipulation. It did very poorly with arrays and regular expressions. I would guess that as the JITs for JavaScript get better, the string advantages will disappear. Flash will probably maintain an advantage in more mathematical computations, especially given its vector graphics features. Hopefully advances in JavaScript will spurn Flash VM progress.

Notes
1.) Tested on both Flash 9 and 10 RC2 on both OSX and Windows. Negligible performance differences in any of the permutations.
2.) Also tested with Silverlight, but only on Windows. It was slower than everything except IE7. However, that was because it was terribly slow at regular expressions and error handling. It clearly had the best JIT as it was able to reduce some of the tests to 0 after a couple of executions.

JavaScript Benchmarks, now with Chrome

As promised yesterday, I did the JS benchmarks again on a Windows machine so I could include Google Chrome. I tried to be pretty inclusive, adding in IE7, IE8 beta 2, Firefox 3.0.1 (current release), Firefox 3.1 with and without JIT, Safari 3.1 (current release), Safari 4 beta, Opera 9.5 and Chrome. This was all run on my workstation, a 4-core, 3.2 GHz box with 8 GB of RAM. Any add-ons, extensions were disabled. Here is the pretty picture.


Once again Safari is the kind. Safari 3.1 beats everything except for Safari 4 beta, which crushes even Safari 3.1. Opera was a little slower than Safari. Chrome was generally comparable to the various Firefox browsers, but overall slightly slower. Like Firefox 3.1+JIT, it was very on error handling! Of course IE was the slowest by far, but at least IE8 is faster than IE7. Maybe IE8 is shipping with debug symbols included (as Microsoft has often done in the past) and the release candidates will be much faster than the betas. Or not.

Anyways, Chrome, and its V8 engine, does well, but does not seem to be ahead of Firefox and is certainly behind Safari and Opera. Maybe they can do better on the Mac!

More JavaScript Benchmarking

My old boss sent me this link about Google Chrome performance. It's a good read. It includes a link to an interesting JavaScript micro-benchmark. It included some interesting findings on Chrome vs. Firefox 3, Safari 3.1, and the new IE 8 beta 2. I was curious about some other browsers, namely Firefox 3.1 beta with and without JIT, Safari 4 beta, and Opera 9.5. Of course I made a nice picture of my results.

Interesting results. First off, FF 3.1 with JIT did not crash. It crashed so many times on me yesterday, that I was sure it would crash on this. Even though it did not crash, it was barely faster than FF 3.1 no JIT or FF 3.0.1. In fact, it was really only faster at error handling and the same on everything else. Apparently errors are easy to JIT for TraceMonkey!

Next, Safari 4 beta is fast. If you look at the link above, Safari 3.1 was already the fastest thing out there, so I guess this should not be a surprise. It crushed everything and it did it on the kind of tasks that real developers do a lot: array and string manipulation, regular expressions, and DOM manipulation (technically not part of your JS engine, but practically the most important test.) I am not used to seeing Opera lose when it comes to any kind of benchmark. If you throw out the array manipulation, it and Safari are pretty close.

I will have to boot up Parallels and try out Chrome vs. Safari 4 beta vs. FF 3.1 beta on Windows.

Firefox 3.1: Bring on the JIT

Web developers everywhere are excited about Firefox 3.1. Part of that is because of CSS improvements, but the big reason is because of TraceMonkey. This a JavaScript engine with a JIT that uses trace trees, a pretty clever technique to turn interpreted JavaScript (read slow) into compiled native (read fast.) JIT is a big part of why VMs like the Java VM and the CLR are very fast, in general much faster than VMs that do not JIT like in Python, Ruby, or (until now) JavaScript. It is why JRuby is faster than Ruby. Thus the prospect of making JavaScript much faster is very exciting.

Recently I had done some micro-benchmarking of JavaScript performance vs. ActionScript/Flash performance. This concentrated on XML parsing only. Now the ActionScript VM is a JIT VM. In fact, Adobe donated it to Mozilla and it is known as Tamarin. It has been Mozilla's intention of using this for JavaScript in Firefox for awhile, as JavaScript is essentially a subset of ActionScript. TraceMonkey is based on Tamarin, but it adds the trace tree algorithm for picking what to JIT. The trace tree approach allows for smaller chunks of code to be JIT'd. For example if you had a large function, like say a single script that runs when the page loads, then with a traditional JIT you either JIT the whole function or not at all. Now what if that function has a loop that runs dozens of times, maybe populating a data table for example. With a trace JIT you can JIT just that one critical loop, but not the whole giant function. So it should be an improvement over Tamarin and thus ActionScript. Of course there is only one way to tell...

So I repeated the same XML parsing tests that I did for Firefox 3.0 and Safari 4 (beta). First, I had to enable JIT in Firefox. One of the links above describes how to do this (open about:config in FF 3.1, look for the jit.content option and set it to true.) I restarted FF 3.1 just to make sure this took effect. I then ran the tests. The results? Not much difference between FF 3.0 and 3.1b+JIT. FF 3.1b+JIT was about 4% faster, which is probably statistically negligible. It was still 6x slower than ActionScript and almost 3x slower than Safari 4.

So what went wrong? Not sure. Here is the code that gets executed in my test:

function load(){
 var parser = new DOMParser();
 var xml = {};
 var start = 0;
 var end = 0;
 var msg = "";
 var results = document.getElementById("result");
 var li = document.createElement("li");
 initReq();
 req.open("GET", "LargeDataSet?size=50", true);
 req.setRequestHeader("Connection", "close");
 // use a closure for the response handler
 req.onreadystatechange = function(){
  if (req.readyState == 4 && req.status == 200){
   msg = "XML Size=" + req.responseText.length;
   start = (new Date()).getTime();
   xml = parser.parseFromString(req.responseText, "text/xml");
   end = (new Date()).getTime();
   msg += " Parsing took: " + (end-start) + " ms";
   li.appendChild(document.createTextNode(msg));
   results.appendChild(li);
  }
 };
 req.send(null);
}

Pretty simple code. I manually execute it 20 times. It would sure seem like it could be JIT'd. What gets timed is just the parser.parseFromString(...) call, where parser is a DOMParser. Maybe that object cannot be JIT'd? Maybe there is a bug with the JIT that will be resolved in the future? It does seem to suggest that TraceMonkey may not always be the slam dunk everyone expects.

I was surprised by the results. I thought that FF3.1 would be faster than FF3. I didn't think it would be faster than ActionScript in this case, but I thought that it might be close. In many other cases, I expect ActionScript to still be much faster than TraceMonkey. Why? Well there is one other ingredient in VMs like the JVM and CLR that make them fast: static typing. This allows the VM to make a lot of other optimizations that work in combination with JIT'ing. For example, knowing that a particular variable is a number or a string allows the VM to inline references to that variable. This can eliminate branches in logic (if-else statements, where maybe the else is not possible.) The JIT can then take place on the simplified, inlined code, and be about as fast as possible.

If you read about some of the techniques used in TraceMonkey, it tries to do a lot of the above via type inference. So in some cases TraceMonkey and the AVM2 (ActionScript VM) may be able to do the same level of optimizations. In fact, given its tracing approach, TraceMonkey may be able to do better. But I am guessing that there will be a lot of situations where AVM2 will be able to do more optimizations just because of the extra information it has at its disposal in the form of static typing.

Parsing XML on the Client: JavaScrpt vs. ActionScript

Developer: Wow the JavaScript interpreter on Firefox 3 is awesome, but the one on the new Safari is even better. It is a great time to be a JavaScript developer!

Me: It is still a lot slower than ActionScript.

Developer: Oh don't quote me old numbers, the new browsers are so much faster.

Me: Still slower than ActionScript.

Developer: Maybe slower at doing useless things like calculating prime numbers. Who would do that in a browser anyways? The new browsers are fast at doing realistic things.

Me: But still slower than ActionScript.

Developer: Show me some proof on something realistic, like parsing XML coming back from an Ajax call.

Me: [Whips together a servlet for producing huge chunks of XML and some JS and a SWF for calling it and doing a DOM parse.] Alright let's see the results of these tests...

Everything is O(N), as you would expect, and can verify by doing a linear regression of XML document size vs. parse time. Safari 4 is much faster than Firefox 3, the ratio of their slopes (FF3/S4) = 2.95. But they both lose badly to ActionScript 3 (running Flash Player 10, RC2), (FF3/AS3) = 6.36 and (S4/AS3) = 2.16. Maybe IE can do better, should we give it a try?

Developer: Now you are being a jerk.

It's Called Script-A-Cu-Lus

Not script-a-licious! I wrote an article on script.acu.lous and it is now online at developerWorks.

ACM Ajax Seminar

Today I taught a seminar on Ajax to the Bay Area chapter of the ACM. It was a lot of fun. The audience was great, asking lots and lots of excellent questions. The level of enthusiasm was very rewarding. I promised the folks there that I would provide download links. So here they are:

The Slides (OpenOffice format)
The Slides (One big PDF)
All Demo Code

The demo code also includes a Scriptaculous demo that I did not have time for. It uses Ruby on Rails. The Prototype demo uses PHP, and the other ones all use Java. A couple of the demos uses a database, and for those I used MySQL.

Finally I wanted to also thank Sang Shin and Doug Crockford for allowing me to use their material for the seminar.

Gears and Flash

Tonight I launched up Firefox 3, and it informed there was an update to Gears ready to install. I went for it, and sure enough Gears was now working with Firefox 3! I didn't find any official announcement about this, but I didn't let that stop me from finishing an idea I had last week: getting Gears and Flash to work together.

This was not too hard. It is just a matter of using ExternalInterface to create a bridge between the two worlds. I wrote a quick POC, using a simple DB table for storing name/value pairs. I think it would be fun to re-implement the flash.data package that is normally only available in AIR apps...

Here is the JavaScript code:

function dbCall(sql, args){

    var db = google.gears.factory.create('beta.database');
    var rs = null;
    try{

        db.open('database-test');
        db.execute('create table if not exists little_table (key varchar(20), value varchar(100))');
        rs = args ? db.execute(sql, args) : db.execute(sql);
        data = [];
        while (rs.isValidRow()){

            row = {};
            for (var i=0;i<rs.fieldCount();i++){

                var name = rs.fieldName(i);
                row[name] = rs.field(i);
            }

            data.push(row);
            rs.next();
        }
        return data;
    } finally {

        rs.close();
    }
}

Here is the ActionScript code:

import flash.external.ExternalInterface;
 

[Bindable]
private var rs:Array = [];
 
private function load():void{

    rs = ExternalInterface.call('dbCall', 'select * from little_table');

}
private function save():void{
    var key:String = keyIn.text;
    var value:String = valueIn.text;
    ExternalInterface.call('dbCall', 'insert into little_table values(?,?)', [key, value]);
    keyIn.text = "";
    valueIn.text = "";
    load();

}
private function clearData():void{
    ExternalInterface.call('dbCall', 'delete from little_table');
    load();

}

And here is the MXML I used to test it out:

<?xml version="1.0" encoding="utf-8"?>

<mx:Application xmlns:mx="http://www.adobe.com/2006/mxml" layout="absolute" initialize="load()">
    <mx:DataGrid x="202" y="59" id="grid" dataProvider="{rs}">

        <mx:columns>
            <mx:DataGridColumn headerText="Key" dataField="key"/>
            <mx:DataGridColumn headerText="Value" dataField="value"/>

        </mx:columns>
    </mx:DataGrid>
    <mx:Label x="10" y="251" text="Key"/>

    <mx:TextInput x="74" y="249" id="keyIn"/>
    <mx:Label x="242" y="251" text="Value"/>

    <mx:TextInput x="285" y="249" id="valueIn"/>
    <mx:Button x="460" y="249" label="Save" click="save()"/>

    <mx:Button x="243" y="298" label="Clear" click="clearData()"/>

</mx:Application>

IE-Hell Freezing Over

Even after I heard the rumors, I still never thought I'd see it with my own eyes:

My enthusiasm was quickly curbed when I tried to use IE8 to post this blog entry. Not only did the Blogger interface look crazy, but the upload image link did not work. I tried switching to "Emulate IE7". To do that, you must restart the browser. That's not very practical, and just means that most people will have to default to the IE7 emulator. Oh well, I guess Blogger just needs to fix up their non-standard JS? Maybe I will open up a debugger to figure out what is breaking. Probably some kind of browser sniffing code that picks different JS syntax depending on if it is IE or not. Maybe what is needed is an IE8 add-on that fakes the user-agent so that most sites think IE8 surfers are actually Firefox surfers.

Flash Hacks: Call Arbitrary JavaScript!

In the past I've been annoyed by the lack of HTTP header access in Flash. Turns out there are some hacks you can do to get around it. Let's say you want the URL of the web page that loaded the SWF. This would be the referrer of the HTTP request that loaded the SWF. Flash should provide access to that, but it doesn't because some browsers don't provide it this info. That's ok, you can get to it:

var hostPageUrl:String = ExternalInterface.call("window.location.href.toString");

That's right, you can "call" a JavaScript expression, not just a function defined on the page. Of course the key is that everything in JavaScript (and in ActionScript too) is a function. So any JavaScript expression is a function and thus invokable via ExternalInterface. You can see where this is going! Let's say you want the query string of the host page's URL:

var queryString:String = ExternalInterface.call("window.location.search.toString");

This will give you something like "?param=value&foo=bar..." What about the referrer of that page? 2EZ:

var referrer:String = ExternalInterface.call("document.referrer");

This is a classic case of Too Much Information, really. You are able to execute arbitrary JavaScript, ExternalInterface acts like an eval(). So you can do arbitrary badness like modify the DOM. You can also access cookies (document.cookie).

Of course the key to all of this is script access. The SWF needs to have it. If the SWF is from the same domain as the web page, it gets this by default. If not, then you need to set allowScriptAccess="always". That gives the SWF the keys to the kingdom, if you will. Of course you could use an IFrame for the SWF and sandbox it in.

Cute EcmaScript

The following is a nifty trick that works in JavaScript, but you should be able to tweak it slightly to work in ActionScript as it leverages EcmaScript features.

function Train(){
	this.name = "Choo Choo";
	this.show = function(){
	    return this.name;
	};
};
var engine = new Train;
engine.name = "Orient Express";
function go(){
	alert(engine["show"].call(engine));
}

What is cool is that you can access a function just like any other property of the object. So the go function could take a parameter that would be the name of the function to call on the engine object. It's like reflection, only better.