satine.org

Smaller bars are better.

Wow.

I compared WebKit’s new SquirrelFish bytecode JavaScript interpreter against Tamarin, the JIT JavaScript engine currently in Flash 9 and in development for Firefox.

On the SunSpider JavaScript benchmark, SquirrelFish is at least 1.9 times faster than Tamarin with tracing turned on; 1.8 times faster than vanilla Tamarin, the same engine used in Flash 9. When you add type annotations to Tamarin (SquirrelFish doesn’t support them), Tamarin gets a little faster, but is still slower than SquirrelFish on these benchmarks.

Keep in mind, SquirrelFish’s reported times include source compilation, while Tamarin is executing pre-compiled bytecode outside the test.

First Impressions

1. I can imagine the “performance per watt” power consumption for SquirrelFish is also much lower. Good for my iPhone’s battery life.
2. I think I’m wrong about the importance of JITs. At least until SquirrelFish has gone through a few rounds of optimizations.
3. ECMAScript 4 type annotations can clearly help in performance critical code, but most of the web executes in untyped, vanilla JavaScript.

Notes on my tests: Running SunSpider, tamarin-tracing, tamarin-central, and SquirrelFish all pulled from source on 2008-06-02. I ran the test on a 2.4 Ghz iMac. Raw numbers from the comparison runs are available here.

Running your own version of the tests

To reproduce these tests, build WebKit, build tamarin-tracing and tamarin-central from Mozilla.org, and install the Flex SDK for a bytecode compiler. Then,

1. Set up your environment as follows: DYLDFRAMEWORKPATH="~/jsbench/WebKit/WebKitBuild/Release" ASC="~/jsbench/asc.jar" GLOBALABC="~/jsbench/tamarin-tracing/core/builtin_full.abc"
2. Create a “timer.js” file that adds getTimer to the SunSpider benchmark: function getTimer() { return new Date(); }
3. Create a “jscore” script that will run the original SunSpider AS source files instead of the ABC bytecode files: #!/bin/sh ~/jsbench/WebKit/WebKitBuild/Release/testkjs -f ~/jsbench/timer.js -f echo $1 | sed s/abc/as/
4. And call the runtests.py script in the SunSpider test/performance sub-directory: ./runtests.py --avm ~/research/jsbench/jscore --avm2 ~/research/jsbench/tracing-avmshell ./runtests.py --avm ~/research/jsbench/jscore --avm2 ~/research/jsbench/central-avmshell

Technorati Tags: SquirrelFish, Tamarin, WebKit, Flash, ActionScript, JavaScript