Thursday, July 31, 2008

Jython, It ain't no JRuby

All of my recent excursions into Python convinced me it was time to try out Jython. I have had a great experiences with JRuby. It is definitely faster than CRuby for long running processes (obviously slower for short scripts, because of the JVM startup overhead.) In my experiences, which range from doing mathematical algorithms to web applications using Rails, there is extremely good interoperability. I have had no code blow up in JRuby that ran fine in CRuby. Even IDE support has been on par. So my expectations were high for Jython.

Maybe that was the first problem, unrealistic expectations. Let's not jump into the shortcomings, just yet. First off, Jython installation is nice. Well nice as in "there is a gui." The installer only seemed to copy files into the installation directory, nothing more. That is fine, but makes me wonder I bothered with an installer at all. I at least expected into put the jython executable on my path, but it did not. No big deal.

Running a script is painless. It was odd to see a lot of activity the first time I ran a script, but the messaging (or should I say logging) was good enough to give me a good idea about what was going on and it only happens once. It also seemed like Jython was going out of its way to help performance, and JRuby had caused me to expect a nice performance boost from Jython.

So I tried out a script I wrote to solve a Project Euler problem, in particular Problem #43. The performance on Python is not that great, as it takes about 58 seconds to solve on my Macbook. Here is the code. It has been slightly optimized by Gilly.

def combinations(items, n):
if n == 0:
yield []
else:
i = 0
l = len(items)
while i < l:
for combo in combinations(items[:i] + items[i+1:], n-1):
yield [items[i]] + combo
i += 1

def permutations(items):
return combinations(items, len(items))

def to_int(seq):
return reduce(lambda x,y: 10*x + y, seq, 0)

def main():
primes = [2, 3, 5, 7, 11, 13, 17]
digits = range(0, 10)
sum = 0
for p in permutations(digits):
if p[5] == 5 and p[0] != 0:
j = 0
trait = True
while trait:
if j == 7:
y = to_int(p)
sum += y
print y
break
x = 100*p[j+1] + 10*p[j+2] + p[j+3]
trait = not x % primes[j]
j += 1
print "sum = " + str(sum)

if __name__ == '__main__':
main()

Lots of brute force, with a little bit of clever use of Python features. I was ready to crank this up with Jython, but when I tried to run it, I got this error message:

Traceback (innermost last):
(no code object) at line 0
File "euler43.py", line 5
yield []
^
SyntaxError: invalid syntax

Ouch. This actually made me feel stupid. I should have noticed that the current version of Jython is numbered 2.2.1 and that this obviously corresponds to Python 2.2.1 (that is obvious, right?) I had used a Python feature that did not exist in 2.2.1. Luckily there is an alpha version of Jython that is numbered 2.5. What about 2.3 or 2.4, you say? Uhh...

Anyways, with the alpha version of Jython 2.5 used instead everything worked. However, the performance was not what I expected. The same script ran in 171 seconds! It took three times longer than CPython. Wow.

I wrote a similar algorithm in Ruby and it was horribly slow. Perhaps this is why JRuby is faster than CRuby, CRuby is just so slow. Perhaps not. The nice thing about this is that it forced me to optimize the code more. Here is the optimized Ruby code.

def calc(seq)
seq.inject(0) {|x,y| x = 10*x + y}
end

def test(seq)
primes = [2,3,5,7,11,13,17]
j = 1
trait = true
while j < 8 && trait
num = calc(seq[j,3])
trait = (num % primes[j-1] == 0)
j += 1
end
trait
end

sum = 0
digits = (0..4).to_a + (6..9).to_a
for p in digits.permutation
if p[0] != 0
x = p[0,5] + [5] + p[5,4]
trait = test(x)
if trait
y = calc(x)
sum += y
puts "found one " + y.to_s
end
end
end
print "sum= " + sum.to_s

A couple of things to note. This uses Array#permutation, a new feature of Ruby 1.8.7. This is written in C, so you would think it would be super fast. You would be wrong. The latest JRuby is 1.1.3 and does not implement all of the 1.8.7 features, including Array#permutation. So this code will not run in JRuby. It winds up being much faster than the Python code, but only because the algorithm is so much better. It only deals with 9! numbers instead of 10!. Without the modification, Ruby was so slow that I did not have the patience to let it finsh. We're talking 10+ minutes for something that only took 1 minute in Python. With the improved algorithm it took about 40s to solve the problem in Ruby. When I ported the change over to Python, it dropped Python down to around 9s. When I tried the ported code in Jython, it would not run at all... I haven't tracked down that problem yet.

Wednesday, July 30, 2008

Python Threads

I finished Core Python. My last post on it hits on a few random things towards the end of the book. The most interesting is threading in Python and begs the question, is Python more or less multi-threaded than it claims?

Sunday, July 27, 2008

Unreal Guitar Hero

I love playing Guitar Hero. Whenever I feel like I am pretty good at the game, I just play with my wife Crystal and she puts me in my place. If that was not enough, I could always watch this video of Chris Chike, the greatest Guitar Hero player. Ever.

Friday, July 25, 2008

JRuby 1.1.3 Performance

It's been awhile since I did a post with pretty graphs and meaningless micro-benchmarks. I am using the latest version of JRuby, 1.1.3 on a new project. I knew from some correspondence with Charlie Nutter that the performance bug that I had encountered previously were believed to be solved. I said believed because he had not had a chance to test on the IBM J9 JVM that (along with some other JVMs) exhibited the performance problem. Actually this was fixed in 1.1.2, and I should have tested things then, but didn't get around to it until now. I decided to throw in the latest beta of JDK 1.6 update 10, the magical JDK for The Rest of Us. Here is the purrty chart:


The good news is that even with the J9 JVM, JRuby outperforms native Ruby, at least on this mathematical algorithm. This was not the case prior, so the JRuby fixes have really helped. In fact, if I look at the last data point (x=100), the code executed in less than half the time it took previously with the only change being the JRuby 1.1.3 instead of 1.1. I could describe the source of the bug, but you are much better off reading Charlie's description.

Thursday, July 24, 2008

The City of San Jose

The Man has been keeping me down lately. Time for a laundry list of complaints about the city I live and work in.

Licensure -- Last year I got a letter from The City telling me that I owed them money, and lots of it. They said that I had a business in San Jose, but that I had not been paying my Business License Fee. Not only did I owe money for the License, but I owed it for the past couple of years plus penalties for not paying it. This was because of money I make for writing for IBM. Of course I had to also pay this License fee again this year...

Crime -- As I mentioned a few months ago, my wife's mini-van got broken into and the DVD player was stolen. I filed a report with The City. The City's response? That would be no response. That is what I should expect, right?

Police Harassment -- Last month, we were parked in downtown San Jose, taking the kids to get ice cream at Ben & Jerry's. The spot was a metered spot. We went past the time limit by 10 minutes and of course we got a parking ticket for $28.

Trespassing -- This is the worst. On Monday, I was walking to my car about to go to work. I notice a piece of paper on my wife's mini-van. I look, and it is a ticket. The $35 ticket was for blocking the sidewalk. Our van was parked in our driveway, but the rear of the van did indeed protrude on to said sidewalk. It did not prevent it from being used, and there was more driveway past the sidewalk, so there was plenty of room to walk by my house without having to use the street or walk in our yard/driveway. Whatever. It was very aggravating to me that I could have our car parked at our house and a policeman could come on to my property to give me a ticket.

All in all I feel that The City of San Jose is doing a lot to take money from me, but doing very little in return. I guess this can be said of any government. A lot of my fellow libertarian-thinkers like to favor local (state, county, city) over federal, but this is just a gentle reminder that even the smaller governments will screw you over as much as possible.

Wednesday, July 23, 2008

I ♥ Slots

The title makes me want to go to Vegas ... No gambling, just a discussion on slotted classes in Python and in particular how to pickle and sort them.

Tuesday, July 22, 2008

Get Lifted

Yeah, I'm a John Legend fan, but that's not the reference here. The reference is to Lift, the web application for Scala. Today IBM published an article I wrote about Lift. This was an article that I pitched to IBM and I was very excited about writing. Lift itself takes a different approach to web development than the typical MVC approaches. It is also in Scala, and takes great advantage of that languages features, especially its native support for XML.

One of the most surprising features of Lift was the ORM that it includes. Lift's creator, David Pollak, commented that he would use JPA for complex schemas and not Lift's ORM. I must admit, it was the part of Lift that took me the longest to get my head around. I consider myself quite the veteran of ORMs, but Lift's is definitely unique. I think it needs some tooling around it, as it felt like some boilerplate-ish code was present (like creating a model class and singleton factory for the model class.) However, I really liked its use of generics.

I didn't have time in the article to get into Comet with Lift and Scala's Actors. That is an awesome feature. Hmm, perhaps that should be another IBM article..

Finally, of Lift and Scala related news... check out Graceless Failure by some of the developer at Twitter. It seems to imply that Scala and maybe Lift or at least "in the mix" at Twitter. I wonder if it is replacing Ruby and/or Rails in some places. Certainly Actors would seem like an obvious way to handle new updates on Twitter.

Monday, July 21, 2008

Movies

I see 1 or maybe 2 movies a year. By seeing a movie, I mean going to a traditional movie theater to see a movie. This year I had to miss the eBay summer movie. That's where eBay takes over a local cineplex, and watch a movie. This year it was the new Indiana Jones movie. Last year, it was The Simpsons.

So by missing the eBay movie, it seemed likely that I would see no movies this summer. Instead, I have seen three. Amazing. Being such an obvious connoisseur, I thought I would share my opinions.

The Hulk -- I was a little skeptical of Ed Norton, and very skeptical of Liv Tyler ... but it was pretty good. Norton was good. Tyler was not convincing as a scientist, sorry. Even worse, she wasn't really that attractive in the movie either. Was her acting good? I'm no good at making such determinations. Still, pretty good.

Iron Man -- I love comic book movies, obviously. The Hulk was good, but Iron Man was much better. Its faithfulness to the mythology was excellent. Obviously Robert Downey, Jr. was the perfect person to play Tony Stark. I really hope that future installments tap into the Stark the Drunk + Rhodey as Iron Man, as it seemed to hint at during the movie.

Wall E -- I took my kids to see this. I was a little nervous about my youngest son, Raymond. He is 2 1/2 years old, and very lively, so I wasn't sure if he would make it through the movie. I will never know if my worries were warranted to or not, as he fell asleep about 20 minutes into the movie. My 4 year old, Michael, Jr. absolutely loved it, and so did I. It is easily the best Pixar movie, and that is really saying a lot.

Tuesday, July 15, 2008

Eclipse Day Video

Couldn't make it to Eclipse Day? You're in luck. Since it was at Google, it was recorded and is now on YouTube. Enjoy!

Sunday, July 13, 2008

Google App Engine, Python, and The Gator

Today I finished the part three of a series of articles on the Google App Engine. Of course right now, GAE only supports Python. So I've been polishing my Python at the same time by reading Core Python. The end result of all this is not only the three part series, but a FriendFeed clone called the AggroGator. Oh, and I've formed some opinions as well!

First, GAE. From a developer standpoint, it is excellent. The SDK works well. It is easy to get develop, test, and deploy. I used Eclipse and PyDev, and I was very impressed with it. PyDev does a good job of providing code assist and syntax checking, something greatly appreciated by a statically typed language guy like myself.

Now once you get beyond the development phase, GAE has had some notable problems. I have been subscribed to the mailing list, and it is very active with people having problems. Personally, I had problems with urlfetch returning HTTP 503's when I could go to the same URL and get a nice little 200. This didi give me the pleasure of using GAE's logging feature to identify these errors, but there was nothing to be done about them. The 503's have since gone away, but I was just glad that this wasn't a "real" app that my job depended on.

A big part of the fun of using GAE can accredited to Python. It is a nice language. It is definitely not as elegant as Ruby or Scala. It does read nicely and has a lot of very practical sugar to it. I think it was a good choice by Google to use for GAE. I don't think they use it much for their high profile applications (I think Google Code is written in Python, but not certain.) Python is known to have better performance than Ruby, but I don't think it has nearly the popular as the other common P in LAMP (PHP.)

Wednesday, July 09, 2008

Farrv-ruuh

Oh the drama in the Green Bay... So Brett Favre wants to play again next year. Green Bay does not want him to play. They are all too happy to move on and start the Aaron Rodgers era. Is Green Bay being foolish? Is Favre just going to embarrass himself trying to come out of retirement? Let's look at some facts.

Favre had a 95.7 passer rating last year. That placed him #2 in the NFC, behind Tony Romo. He was also #2 touchdown passes (28) and #3 in passing yards (4155.) So it sure seems like Favre is still playing at not just a "good" level, but an elite level. But...
In the two previous seasons, Favre had passer ratings of 70.9 and 72.7. He threw 9 more interceptions than touchdown passes during those two years. This is definitely below average for an NFL quarterback, especially for a QB in a passer-stat friendly system like they run in Green Bay.

So what Favre will show up in 2008? Everybody wants to villify Green Bay for disrespecting one the most beloved players in NFL history. I wonder what John Madden would say about it...

I digress... Anyways, you can't just say the Packers are being crazy jerks. There are some good reasons to think that Favre may not play well this season: past performance and age. Clearly they must think highly of Aaron Rodgers. Personally, I've heard a lot of sound bites from this guy, and he seems like a real jerk. But so what? He may still be a good QB, though he seems more similar to Pac-10 QBs like Cade McNown or at his best a Matt Leinart as opposed to a Carson Palmer.

So what should the Packers do? Trade Favre. However, they must trade him to a team they won't play and preferably to a team in the AFC. They need to get something back for him that will appease the fans. Luckily for the Packers, I've got the solution. They need to trade Favre for Lilly:


Who would not be happy about getting Lilly? Miami was terrible last year, so it is kind of a punishment for Favre for allowing his indecisiveness cause such problems. There is no chance that they would have to face Favre next year as well. For Miami, well losing Lilly would hurt. However, Favre would be a big upgrade. If things don't go well ... well it will still be better than last year, right? It would certainly get fans excited and at the games. It is win-win.

Tuesday, July 08, 2008

Functional Programming

Inspired by Python, here are a few tastes.

Government Interference

Gilly had some fun making this Friedman video:

Finally Grails

It sure took me long enough, but I finally got around to writing about Grails. This article gives some introduction to Grails, but also talks about using Grails with Geronimo. The most valuable info in there might be the Geronimo deployment plan. This shows which packages to exclude from Geronimo's classpath when deploying a Grails application. This prevents classpath implosion.