Interesting JVM Stuff

I attended a lecture. today by the tech lead of IBM's JVM team,  Trent Gray-Donald. It was a fantastic lecture. One of the interesting topics he spoke about was the Gencon garbage collector in their JVM. It's obviously a generational garbage collector. One of his interesting points was that one of the assumptions of generational collectors is that most objects are short lived, and people often misunderstand what "short lived" means. The example he gave was a SIP. 

A telephone session lasts as long as a phone conversation, and so if you have a Java app behind the SIP, it's going to have a lot of objects who lifecycle is tied to the length of a phone conversation. Thus these objects will survive a lot of minor collections in a generational collector and be promoted to an old(er) generation. 

This causes two big problems. First, most collectors are copy collectors, so they copy the survivors of the collection. If you have a lot of survivors, then you have a lot of copying and this is really slow. The next problem is even more insidious. Lots of objects wind up in older generations that are collected infrequently. So you run out of space, wind up with failed minor collections, and trigger full stop-the-world collections.

So generational collectors are poor choices for SIP systems, so what? Well the S in SIP is for session, and their concept of sessions is quite similar to a web session. So you could see the same behavior in a web application that stores a lot of data in a session. Anything that lives in "human" time is going to considered long lived in JVM time. So if you have a web application with a lot of data in session, it could be a pretty poor fit to a generational collector.

Now there are a lot of truly short lived objects in a web application as well. A lot of things are only around for the duration of a request. So that would make a web application a much better fit for a generational collector. Basically the less you keep in session, then your web app will be an even better fit for a generational collector.

Tetris and the iPhone

I really shouldn't write about the iPhone. There's way too much being written about it. But I was sitting outside today, waiting for somebody to call me back on my cell phone. I was a little bored so I went to Blackberry's portal site. I browsed some games, and noticed Tetris. So I downloaded the trial version to my phone and started playing it.

That's when I realized that such a thing would not be possible on the iPhone and that just seemed silly. Oh, that's right somebody just needs to do an AJAXy Tetris web application that doesn't use Java or Flash... Ok, so somebody has already done that and maybe it will play nice on the iPhone. But you get my point. The limitations of not being able to install software on a device that is supposedly running a full blown operating system (OSX) is just ridiculous.

And in case Steve Jobs asks, Tetris did not cause my Blackberry to crash and it did not bring down the Verizon Wireless network.

Rasputin's

Here's another sign that I'm old... I went to Rasputin's recently, with the intention of buying the new White Stripes CD, Icky Thump. I used to really like going to Rasputin's. I would spend lots of time digging through used CDs and listening to obscure recommendations posted around the store.

This time was different. I was immediately annoyed by the smell. Then I was annoyed by the music playing. It was not very good and it was very loud. Then I was annoyed not to find Icky Thump in the White Stripes section. I did find it on a display at the end of the aisle. I was annoyed that it was "on sale" for $12.99.

I had no inclination to browse. I just wanted to get my CD and get out of there. So I got in line to check out. There were three cashiers, each with a customer, but nobody else in line. I was optimistic, but that was a mistake. I waited forever before one of the customers finally finished their transaction and left. I immediately went to the open cashier, who then promptly walked away to go and converse with one of the customers at the adjacent cashier.

That was it. I had had enough. I put down my CD, and walked out of the store. I felt like shouting to the people "What is wrong with you idiots! Do you not realize that you are on borrowed time? Your business model is failing. It is just a matter of time before you disappear like Tower Records! In the time I waited in line here, I could have downloaded your CD from iTunes or off Bit Torrent!" I doubt that I will be back. Oh well.

Tonight I bought Icky Thump on eBay for $11 total with shipping. I like going to the store and buying a CD. I like being able to go back to my car, pop in my new CD, and listen to it. I think CDs are generally still the best way to buy music. I like being able to rip them to high quality MP3s for listening to on my computer, iPod, and car stereo. But I cannot tolerate Rasputin's.

Hacking my 8830

Ok, so not really. I've had my Blackberry for a little over a week, and I continue to be more impressed with it. Here are some useful things.

1. Enable mass storage. You get a cryptic message for this, but totally want to do to this. It let's you access your Blackberry's file system. On my Macbook it makes the Blackberry look like a mounted drive, a la a DMG file or something, only its read/write not read-only. This is the easiest way to put pics or MP3s on there.
2. MP3 Ringtones. Forget buying ringtones. Just load some MP3s on there. Go to your song by going to Media -> Music. Pick your favorite song and the pick Menu -> Set as Phone Tone. Now that MP3 is your default ring tone. Want a special ring tone for different people? Go to the person in your address book. Click Menu -> Edit. Then click Menu -> Add Custom Phone Tune. Click enter. This will bring up a list of pre-set ringtones. You don't want any of this garbage. But scroll to the top of the list and click Browse. Then click Menu -> Explore. I went to Media Card/MY_MP3S. That gave me all of my MP3s. I just picked whatever I wanted from there.
3. Google Apps. Google obviously loves Blackberry users. GMail and Google Maps both rock for the Blackberry. Google Talk from RIM also rocks. If you open up Google Maps and pick a location, you can get directions. You'll notice that the "from here" is disabled. That's because GPS is not enabled for Google Maps by default, which brings me to my next tip.
4. Enable GPS. A former co-worker of got an 8850 and thought that Verizon had disabled the GPS. That would be a very Verizon thing to do, but it turns it's not the case. Go to Options -> Advanced -> GPS -> GPS Services and switch it to Location On. It's set to "911 Only" by default. Open Google Maps and you'll have to grant Google Maps permission to use the GPS. Now you've got the full GPS enabled Google Maps functionality. I found that I needed to be outside to get a good signal on it.

These tips are for Mac users only.

1. PocketMac. The Blackberry version is free and is very good.
2. Google Calendar. I sync iCal to my Google Calendars, and then sync iCal to my Blackberry via PocketMac.
3. Plaxo. I use Plaxo as a central repository for my contacts. I then installed the Plaxo plugin for Address Book. It becomes a cool read/write system. I update contacts in Address Book. That syncs things to Plaxo where they are persisted and available to me at work, etc. They are also copied to my Blackberry via PocketMac.

Other stuff...

1. I downloaded Opera Mini. It works ok. There are some web pages I can view in it that I cannot view in the Blackberry browser, such as ESPN Fantasy Baseball. Entering text into Opera is a pain though. It has its own interface for this that seems awkward. I still think it will come in handy in the long run.
2. I haven't tried syncing my Safari bookmarks to the Blackberry. That's a cool feature offered by PocketMac. I would need to clean up my Safari bookmarks first though.
3. Facebook Mobile is really well done. You can do most things that you'd want to do on a mobile device there. A lot of folks prescribe to the iPod/iTunes model when it comes to mobile apps, i.e. read-only on the device, do all your writes from your computer. Facebook does a good job of giving the right "writes" on their mobile version.

Update: An old co-worker of mine and an anonymous commenter both pointed out that even though you can supposedly enable GPS, Google Maps and other similar programs are not able to use it. My friend claims that Verizon has disabled the GPS in some way. My experience has been the same, so maybe this is true. I don't know why Verizon would do this...

Update #2: Ah ha, now I know why Verizon has disabled the GPS on the 8830. There's got to be a hack though...

New Bike

Earlier this week I was biking and almost got into an accident. I was getting on to the 87 Bike Trail. The entrance is in a bad place. It's at the north end of Santa Teresa, with entrances/exits onto to the 87 and 85 freeways. I was crossing an intersection and there was a car getting off of northbound 85. The car wanted to turn right, which would put him on the on-ramp to 87 north. If you know this area, you know that this makes no sense. There is a direct ramp from 85 north to 87 north, so why get off the freeway to accomplish the same thing? The only answer I could come up with was that it was early in the morning and the metering lights were enabled on the on-ramp in question. So maybe this guy wanted to avoid the back-up from the metering lights by getting on the 85 freeway on to Santa Teresa and then getting on the 87.

Anyways, the point is that this guy didn't stop and made right turn off the 85 on to the 87 ramp. He would have probably hit me, but I saw him and swerved. The swerve caused me to miss this entrance to the bike trail and hit a high curb. I caught some air, but didn't fall off the bike. My bike took the worst of it though. It knocked the chain off and bent my back wheel. My bike already had some other issues -- the suspension was cranky and the brakes were in bad need of replacement. So Crystal wisely advised me to just get a new bike. So I did.

It's a San Rafael by Marin Bike. I bought it from Bike Connection in Palo Alto.

Powerset on Rails

Like a lot of folks, I've been curious about Powerset, the company promising to revolutionize search via natural language queries. I got an interesting email from them today. Here's an excerpt. All emphasis (bolds and italics) is from the original email:

I'm happy to announce that we are, in fact, launching our front-end on Ruby.

Cool, huh? For everyone's sanity (and in avoidance of some of the flame wars to ensue), do note that we are going to be using Ruby (the language) but not necessarily Ruby on Rails (the web framework).

Why Ruby?

1. We've already got the brains

2. Ruby is already being used throughout the company

3. We're not worried about scaling

This is obviously an unusual broadcast. Do most potential users of Powerset care what programming language its front-end is written in? So why send out this email. It all comes down to reason #3. We're not worried about scaling. Heh heh.

Why all the fuss? Rails can scale, right? The email goes on to outline how they've talked to the Twitter guys who told them not to worry about the problems they had and go for Ruby. They also explain how their system is simpler and they won't have the same problems that Twitter had.

This is just too funny. Basically they have a bunch of Ruby developers (see reasons #1 and #2) who want to do the front-end in Ruby. I'll go as far as to guess they really want to do it using Rails, but they haven't been able to win that argument yet (hence the disclaimer about it.) Yet, their management is so uncomfortable with this that they had to go and do a broadcast email apologizing-- err I mean explaining why it's ok for them to use Ruby.

I can't imagine a worse endorsement about a language than this! Seriously, if they had decided to write their front end in Java, PHP, C#, Perl, Python, etc. would they have felt the need to publicly defend their decision? Are they going to send out emails about their choices of databases, operating systems, web servers, etc.? I think not.

It's a sad day for Powerset, but an even sadder day for Ruby and Rails.

Ning

Last week I went to yet another Lunch 2.0. This one was at Ning. As I mentioned previously, Ning is of special interest to me. What they are doing is very similar to what we were doing at Ludi Labs. Actually I'd say that if you took a little bit of what Facebook has done with their markup and query languages, and added that to Ning, you get really darn close to what Ludi Labs was doing.

After listening to the folks at Ning and watching their demo, I was really amazed. I went out and created a Ning network for my fellow Ludi Labs survivors. Of course the thing I am most interested in doing is hacking the source code, even though it is in PHP. I've requested it from Ning.

Their content store is also very interesting. I'm curious to see just how extensible it is. They seem to claim they have a large selection of "common" data types. I'm skeptical. Why? Because that was always a thorny issue for us at Ludi Labs. We did the same thing. Actually I'd say that it was one of our mistakes. We created too many data types and each type was too flexible.

Even if they have the too large catalog like we did at Ludi, it's not going to be enough. What if I want to create a social network for Blackberry 8830 users? I create a widget of all the mobile apps that you can install on the 8830. I want people to be able to add to it and to search it. Clearly I'm going to have a data type that Ning does not support. They might have some kind of generic object type that can take user defined attributes, but it's going to have problems when I start busting out searches against these user defined attributes.

This may sound contrived, but trust me it is not. It is something that Ning must solve to be a viable platform. Marc Andreessen did a great write-up about the Facebook platform. One his great points was about how if you write a great Facebook app, you'll be doomed just like iLike. That's because you must host the app on your own server (farm.) He was clearly hinting out how this problem is solved by Ning, where your app is hosted by Ning. But if my little Blackberry app I just described is super slow on Ning once we get a few million entries into it and a few million users running searches against it, then it's no better than the Facebook Catch-22.

Father's Day Baseball Game

I took Michael, Jr. to his first baseball game on Saturday night.


We saw the A's host the Cardinals. Unfortunately the Cardinals blew out the A's 15-6. Here's a picture of Albert Pujols batting:


Michale made it into the 6th inning, which is pretty darn good for a three year old. He could've gone longer actually, but I think he would have fallen asleep. So we headed home. It was just another fun part of my Father's Day weekend.

Blackberry 8830

Well, well, well Happy Father's Day to me! My wife and kids bought me a Blackberry 8830. It works on both Verizon's CDMA network here and GSM networks in Europe. Should come in handy when we go to Rome.

I'm happy to report that the 8830 syncs beautifully using Pocket Mac, which is free for Blackberry owners, and runs GMail, Google Maps, and Opera Mini flawlessly. Forget the iPhone!

More iPhone Thoughts

Lots has been written about the lack of an SDK for the iPhone:

Valleywag had a pretty good aggregation of opinions. This included a great write-up by John Gruber that echoes the opinions of many developers. Dave Winer chimed in on Apple's overall dislike of developers and then gave an alternate theory that it's AT&T's fear of Skype that is the real key.

I have to agree more with Dave's first point and with John Gruber. Apple has never treated their developers well. Apple has always treated developers very poorly, unlike Microsoft.



Focusing on more positive things, Eugene Ciurana on TheServerSide looks at the iPhone and its lack of SDK as an opportunity for Java web developers. This is some truth here, but there are way more people online already than will ever be using the iPhone, so that opportunity must be put in context. Also, I have to admit finding it amusing that Eugene thinks that:

Most mission-critical and scalable websites run on Java, and most serious Web 2.0 frameworks are implemented in Java.

Playing on this point, I did notice the new add from Apple playing up the web app capabilities of the iPhone.

(More?) ESPN Fantasy Baseball Problems

My ESPN fantasy baseball teams have been doing very well this year. I always check them in the mornings. This morning I noticed something odd with the statistics and standings. Here's a picture of the standings sorted by WHIP:



Quick note if you're not familiar with the WHIP statistic in baseball. It stands for Walks + Hits per Inning Pitched. It's basically how many base runners a pitcher allows each inning on average. Thus a lower value is better.

So you'd expect the team with the lowest WHIP (VENEZUELA ANATOMY with a 1.143) to be at the top of the list when teams are sorted by WHIP. Having the best WHIP, they should also get 10 points contributed to the point totals used for overall standings. Instead, VENEZUELA is in third place and the team with the fourth best WHIP is in first.

Do we have another case of ESPN having problems? Well, sort of. You might have noticed that I was using the Windows Safari browser. There have been a lot of bugs with WinSafari. So it was only prudent for me to load the same page in Firefox:


Now everything looks exactly as it should! Notice that not only have the display orders changed, but the point totals as well. You might notice the my team (Campbell HGH in bold) even has two more points in Firefox than in Safari (89 vs. 87.) I tried the same page in IE7 and Opera, and it displayed correctly (just like Firefox) in both of those as well. I'll give it a try on MacSafari later, too.

So how could using Safari cause such a bizarre "bug" with Safari? Well doing a view source on the ESPN page revealed a lot. Here's a snippet:

This Sortable JavaScript class is the key. It looks like ESPN just passes in the raw data (each team plus it's cumulative statis in each category) to this JavaScript class and then uses JavaScript to do not only the sorting, but also the point calculations (since it is based on the sorting) and overall rankings.

It seems like a reasonable way to render such a page. All sorting and calculations are done client-side. JavaScript is certainly more than capable of handling such tasks. Well, at least it should be. These screen shots suggest that there is a bug with Safari's JavaScript. It would seem like a fairly severe bug, too, since it seems to be related to how it is comparing floating point values. That's just a guess. It could be a lot more subtle than that.

Let Them Eat Safari

I can't complain about Apple's WWDC being too disappointing, not with it coming fresh on the heels of true disappointment the night before. Still, there wasn't much there. In fact there was only thing of note:

Apple doesn't want to open the iPhone up as an application development platform. Instead they want everyone to build AJAXed web apps (oh by they way, screw Flash, Silverlight, and JavaFX.) To help developers with this, they are pushing out Safari for Windows, so you can easily test your web app against the "same" rendering engine that will be used on the iPhone. In essence, they are trying to push Safari as the standard for HTML/CSS/JavaScript rendering ... Which isn't so bad, since Safari passes the Acid2 test, but I digress.

I know we all keep hearing about how Apple doesn't third party apps causing the iPhone to crash or whatever, but that is just lame and obviously far from honest. I think it is more likely that they have some technical limitations that are preventing them from supporting JavaME or some other standardized runtime, and they are not ready to roll out their own.

So I guess if I want to do map search on the iPhone, I better like using Google Maps. If I want push email to it, I better be using Yahoo Mail. I don't guess I'll be pulling down videos from YouTube or ESPN to watch on the phone.

Of course having a "full" browser on the phone is huge. Apple has to be totally committed to this at this point. Theoretically I can go to GMail directly on the iPhone and get the full AJAX interface we all know and love. Maybe there is some way to have it run in the background so I can get the email pushed to me. Of course I still can't go to Yahoo Maps, since it uses Flash ... Ditto with YouTube... Also, Outlook Web Access works ok on Safari, though it's not going to provide push email like the IE version would (or like you would get with a Blackberry.)

So does any of this matter? The most significant thing is that one can expect that support for Safari just became more important web developers everywhere. Not a lot more important, but at least a little. I don't think Microsoft or Mozilla have to worry too much about IE/Firefox users switching to Safari. It's big draw is speed, and Opera was already way faster than IE or Firefox. Plus it's got a lot of issues...

Google Gears

This is actually the second time I've written about Gears. The other time, Flock ate my post. I wasn't too upset, since I was using a daily build of "Sulfur". It's much better (in terms of feature set) than the current 0.7 version of Flock, at least when it comes to blogging on Blogger because it supports Blogger labels. Anyways, back to Google Gears.

I gave it a try (via Google Reader of course), and was not too impressed for one simple reason. I have to tell it to "go offline." So I guess the idea is that I know I'll be offline later, so I will "go offline" now, thus downloading content to read later. Ok, that's fine, but not very useful. It requires too much planning, in my opinion.

It's really not that big of a deal for Google Reader. But clearly Gears has the most potential for other Google Apps that have more direct "offline" analogies: GMail and Google Docs. The choosing to go offline thing would be silly for either of these.

With a Gears enhanced GMail, I would expect to always have a copy of (at least) my most recent email. I'd expect to be able to search it, too, but that's another story for another day. For Google Docs, I would at least expect to have an offline version of all my "active" docs. For both of these apps, I would expect to have these things all the time, seamlessly. Anything else is of very limited value in my opinion.

So I am left to assume that Google will enable this kind of seamless enhancement via Gears, but they just haven't figured it out, yet. I think that makes the most sense. If they had figured it out, I'm sure they would have enabled it with Reader, even though it's not quite as essential for Reader. In fact I'll wager the reason they launched with Reader was exactly because they didn't have the above use cases solved, yet.

One other Gear related thing. I was fascinated that Adobe seemed to be hanging around for the launch of Gears. It's interesting in a lot of ways. First, Adobe already has something similar, but far less powerful than Gears: Flash Shared Objects. Gears and its database make SO's look silly. Second, doesn't a Gears enabled Flex application provide everything that an Apollo application could promise? Sure, an Apollo app could be "heavier", have a fancier UI maybe, do more with local files, etc. Finally, I got the feeling that there was a Adobe+Gears announcement that didn't happen. That something that was supposed to be ready for Google Developer Day, wasn't ready. That's why Adobe was there.

Installing Subversion on Linux

Last year I installed a Subversion server on a Windows server. It was really quite easy. This week I needed to do the same thing on a Linux server. It turned out to be almost as easy. Almost.

The server in question was a server from Rackspace. It had Red Hat Enterprise Linux 4 running on it. One nice thing about Rackspace is that their servers are pretty bare bones. In this case, the typical LAMP stack was needed, so all that was on there, but that was about it.

My first (notice I said first) approach was to install a Subversion RPM. Here is what I got when I tried that:

[root@123038-www1 XXX]# rpm -ivh subversion-1.4.3-1.i386.rpm
error: Failed dependencies:
apr >= 0.9.7 is needed by subversion-1.4.3-1.i386
apr-util >= 0.9.7 is needed by subversion-1.4.3-1.i386
libneon.so.24 is needed by subversion-1.4.3-1.i386
neon >= 0.24.7 is needed by subversion-1.4.3-1.i386
Suggested resolutions:
neon-0.24.7-4.i386.rpm

Nice! Luckily I found the exact neon RPM suggested, so I tried that:

[root@123038-www1 XXX]# rpm -ivh neon-0.24.7-4.i386.rpm
warning: neon-0.24.7-4.i386.rpm: V3 DSA signature: NOKEY, key ID e01260f1
error: Failed dependencies:
libcrypto.so.0.9.7 is needed by neon-0.24.7-4.i386
libssl.so.0.9.7 is needed by neon-0.24.7-4.i386

That's just not cool. What to do? Build it myself, of course. What's nice is that Collab has both the source for Subversion, and the source for its dependencies. So I built the dependencies, and then built Subversion , and then ... it worked! Not too bad.

The nicest part came next. Configuring it was just as easy as it was on Windows. I followed the same guide I had found a year ago, and everything worked flawlessly.

One interesting thing is that in this case I already had Apache 2.2 running on the server. For the Windows install, I had installed Apache specifically for Subversion. I ran Subversion in a different directory of that server, but on the same Apache instance and same port obviously. There's probably some potential security or performance issues with this. Maybe I should have run a second instance of Apache listening on a different port?

One final nice thing. The Apache instance already had DAV enabled. This was convenient. I guess this is standard for Rackspace? It's not standard for Apache.

Writing a Facebook App

I've been writing a Facebook app. They offer a Java client library (as well as several other non-official ones for other languages) but I'm using the PHP one. I really like the library they include. It has been incredibly easy to use, even for a PHP hacker like me.

One of the best parts of their library is the FaceRestClient call_method function. Only problem is that it does not work with photo uploads. Of course that happened to be one of the things I wanted to do. It doesn't work because you need to set a multi-part mime header for a photo upload and format the binary data per that content-type. Obviously all this can be done by hand, but luckily I found something better: the photos lib. This makes it trivial to upload an image. Here's the code I wrote for it:

 include_once 'facebookapi_php5_restlib.php';
 include_once 'facebook_php5_photoslib.php';

 // get your own api_key and secret!
 $facebook = new FacebookPhotosRestClient($api_key, $secret);
        
 // user logged in prior to uploading pic, so sesion key passed as param
 $session_key = $_REQUEST['session_key'];
 $facebook->session_key = $session_key;

 $tmpLocation = $_FILES['uploadedfile']['tmp_name'];

 if (is_uploaded_file($tmpLocation)){
  $aid = $_REQUEST['album'];
  $caption = $_REQUEST['caption'];
  $res = $facebook->photos_upload($tmpLocation, $aid, $caption);
  echo "Here is a link to your pic, make sure you approve it first:".$res['link'];
 } else {
  echo "Ye gods! Something went wrong!";
 }
?>

Lunch 2.0 Pics

Not only did I somehow wind up in some pics from Lunch 2.0 @ NetGear, somebody even tagged those pics with my name.