perlbal: Livejournal's Loadbalancer

Writing about web page http://www.danga.com/perlbal/

I blogged a while back about testing out memcached, the in-memory TCP-based object cacheing application developed by Danga as part of Livejournal. (Fortunately for us, we haven't actually needed to use it yet, but it's sitting there waiting in case we do).

This weekend I've been playing around with another nice bit of kit from the LJ stables; perlbal. Perlbal is a load-balancer, similar to apache's mod_jk, but with some handy extra features:

• It's not tied to AJP, tomcat's proprietary protocol. It runs over plain HTTP, meaning you can balance any kind of web service. If you want to, say, stick a squid in between the balancer and the app server, it's no problem.


• Its balancing algorithm seems smarter. It can autodetect whether it's issuing a connection to the actual back-end server, or just to the listen queue in the OS, and it doesn't seem to be quite so keen to farm requests onto bogged-down servers (need to test this a bit more though)


• The internal redirect thing looks way cool for fast file-serving. Perlbal takes a request, hands off to one service that authenticates the request and returns a resource location via an X-Reproxy HTTP header, then perlbal serves the resource. Clients can't request those resources directly so it's secure. It means you don't tie up your DB app serving 100MB files down slow modems.


• It's remotely configurable. You can telnet into the balancer and reconfigure it at runtime. Note to self: Need to find out how to secure this!


• It's got lots of status information in the management interface.

Installing it wasn't too hard. It needs perl, one or other of the AIO modules ( I couldn't get Linux::AIO to install, but IO::AIO was fine on both my linux test box and my mac) and Danga::Socket ; other than that you just pull it down, either from CPAN or download it directly. I don't think it should be too hard to get working on the Sun boxes either.

I'm not quite sure whereabouts in our architecture we'd put such a beast, but I quite like the idea of having apache simply using mod_rewrite, and delegating to perlbal based balancers to spray requests between cluster nodes.

Chris May : 26 Feb 2006 13:28 |  Comments (0) |  Report a problem