Tuesday, December 19, 2006

Scaling & Pretty Printing

I've been spending a fair bit of time trying to get AllTheCode able to scale to handle more than just a few users, and I'm finding bottlenecks in the usual places, plus a few interesting ones.

Pretty printing is one of those things which has been around for a long time and most people just kind of ignore. However, for all the code, I have to show people the results, and rather than just provide them with a download link [as I first wanted to do], most of my competitors seem to have decided to provided a pretty printed HTML version of the code. I can totally understand why, for one thing it is more convenient since if you just want to look at some of the results quickly you don't want to download all of them. As luck would have it, vim can dump its wonderful pretty printing to HTML and Ruby on Rails has a built in method to highlight the relevant terms in the output [yay!]. The only problem is, that vim2html is a surprisingly cpu intensive. Since there will probably not be too many users, I'm going to keep it for now and perhaps replace it with Kate eventually, but I still find it humorous that my next projected bottleneck is pretty printing :)

The more traditional bottlenecks of the DB [despite some early locking issues], seems to have been mostly solved. Things are easy for me since there are plenty of places for me to cheat [like handling updates, pshaw. Updates are for people aren't me :)]. Despite all of the "ruby on rails" is hard to deploy, so far its only marginally more difficult than deploying a traditional php script. I'll admit, a single server deployment has a lot of unnecessary hassle, but when doing a multi-server deployment, the things that people complain about you have to deal with regardless of ruby on rails, or django, or perl or really anything.

1 comment:

Anonymous said...

I suspect you can scale the pretty printing by either pre-processing it all or using some kind of cache.

However, in general RoR has very poor scalability characteristics, so that may in fact turn out to be your achilles' heel.

Free Blog Counter