Holden's Blog

One of the things that a lot of people have asked for is plugins for their favourite development environment. So far we've got plugins for Eclipse and Emacs mostly finished, and we've got an opensearch specification done. For the eclipse plugin bug 120610 makes me a little sad, since our plugin (naturally) depends on org.eclipse.jdt.apt.core . I'd like to add support for a few more IDEs before the next version, and I'd like some idea of what people want, so I've added a poll to the side of this blog. Personally I'm leaning towards netbeans, but thats just for personal bias.

My daily check of the news sites has gotten a whole lot quicker to do thanks to aideRSS and I've managed to find some more interesting things. I'll talk more about aideRSS once I've had the time to poke at it enough. In my semi-daily perusal I ran across this lovely piece about the importance making urls bookmark friendly . For the most part, All The Code has been relatively free of any of the problems mentioned (we don't over use POST or encode session information into the URLs), but it got me thinking about one possible problem. When All The Code does an index re-build the file IDs for a given program can change, and the links to the code viewer is based on the file ID. Now, index re-builds don't happen all that often with the public version of All The Code, but with the upcoming "enterprise" version index re-builds happen on a more regular basis. With that in mind I quickly whipped up the code to make the links less brittle. If you have a application ...

Developing for two worlds (Internet and appliance, or perhaps more accurately public and corporate) has its benefits and its downsides. A lot of my time is spent implementing features which only really are going to be used by the Internet version, or the appliance version. Or, more commonly, after implementing a feature for one of them, I have to do a significant amount of re-tooling the feature to work for the other world. For example, adding tagging to the appliance version of All The Code was able to be done in only a few minutes, however for the Internet version I need to keep in mind the problems of database synchronization, possible untrusted users. On the other hand, developing for two very different types of users has its occasional benefit. Traditionally with appliance software, and most software which is for internal company use, getting beta testers is much harder than with a public application. With all that said and done, there are still things which are almost exactly the...

Amazon recently discontinued AWSP, which is kind of unfortunate given that All The Code relies on it for raw data for the public version. That being said, the new interface is seems much cleaner and has the potential to make my life a whole lot easier. Fortunately, this coincides with my need to test the subversion integration of All The Code, so I'm in the process of pulling down as lot of code from various subversion repositories. I'm looking to truly stress test the core before I hand it off to the first round of beta testing, so I'm also pulling down code from a few other sources. The next public version of ATC should see the index increase in size by a factor of about 20 [and this will hopefully be ready for the end of next week, but no promises :)]. There are quite a few sources of potential problems related to growing the amount of code indexed by so much, so I'm really hoping to flush out some bugs. If there is time, I will try to roll out tagging and possibly v...

Krugle is apparently launching its enterprise search beta soon. One of the benefits of subscribing to my competitions mailing lists is I get to see the wonderful research that they put in to trying to sell there product. For example they site a study which says that 25% of software developers time is spent looking for technical information. Now I can use the same study without having to pay to have one done for me. Go savings :) Now, to watch for press surrounding Krugle and look for opportunities for follow ups.

I'm in the process of getting All The Code integrated with more than just AWSP and I was strongly considering replacing the crawling back end entirely with something more modular and just today I got an e-mail from Amazon stating that AWSP is going to be transition to a new (lower cost) platform based on top of S3+EC2. I love it when things line up at just the right times :) In related news, I'm starting to look around for small to medium sized programming companies in Ontario that are interested in beta testing the new version in either late May or early June.

All The Code has a new toll-free (within North America) phone # 1-866-500-1777 .

My proposal to create scheme bindings for subversion for Google Summer of Code has been accepted, which is awesome. I'm looking forward to a summer of largely functional programming (scheme & ml). If time permits I'm going to try and pick up haskell and make haskell bindings as well. The awesome thing about this is that one of the things I need to do for All The Code is create a subversion interface, so the work is related to what I was going to be doing anyways :)

I signed the forms on Monday and All The Code now has its first bit of funding, in the form of a grant. Its not much but given that I didn't have to take on debt or give up equity I'm happy.

So All The Code recently suffered some downtime and I figured I should probably talk about it. The problem came from the reverse proxy software that I'm using to act as a load balancer and (ironically) avoid nodes that are down. Even then, the reverse proxy going down shouldn't be the end of the world since there are two reverse proxy boxes and ZoneEdit is supposed to failover to the second server in the event the first one goes down. During the brief stint on slashdot, I disabled the second reverse proxy and started using it as just another node to keep up with the load. Sadly, I forgot to switch it back to its old configuration after the load went back to normal, so the failover system didn't work out as planned :) I've got a full schedule for the next little while, sadly not enough of it involves fun coding, but such is life. This Friday I'm giving a presentation to a small group (3 people) looking at the business aspects of All The Code. I'm also going to b...

Being on slashdot, reddit and other news sites has its ups & downs. On the positive side, you get a lot of viewers, and for the most part people seemed fairly positive. I also got a lot of useful bug reports about such wonderful things as cross sight scripting vulnerability, places where things should have been escaped, and a few feature requests. Since All The Code has a fairly small database, since it only indexes .java files and does not look inside cvs/svn/git or even zip files for the time being, a lot of searches didn't return many results. The solution to this is to allow substring matching IE (you search for "btree" it finds "tree" and gives you tree). This has the wonderful effect of giving you lots of results, but depending on what you are searching for this stemming can lead to much sadness, giving not so good results at time. My present solution involves some magic determining when to use stemming and when not to use stemming. After being on slas...

So, after All The Code was up for a few hours I made one small change to the production system, breaking codeviewing for the better part of half an hour. Anyways, I came up with a way to improve results that shouldn't take more than a hundred lines or so to implement, so as soon as I get some free time I'm going to give that a shot (not on the production version though, I've learnt my lesson there). On a less than totally random note, newsforge put up a small blurb on it for awesome.

All The Code launched today, with a bit of a rockety start [namely () and similar broke the engine]. Right now its on reddit . Its kind of interesting looking at what people are searching for, namely one person searched for "lang:c [somestuff]", so I'm thinking about adding an error message for that. I'll probably write some more later, but I've got to be ready for tomorrow.

So I was starting the post of as an explanation of yet another delay for AllTheCode. I decided I'd take a quick 30 minutes to take a look at the state of things. Those 30 minutes quickly turned into two hours, but now All The Code seems to work properly enough to be ready for release. The nameservers have been swapped over, and its just a matter of swapping some files on the 31st and launching the instance, and hoping nothing else breaks :). Sure, it still has a lot of "features" of the suspiciously like a bug variety, but its usable enough. Now back to the work I should have been doing :)

The past few weeks have been fairly hectic which have been keeping me from working on All The Code . Another DB build did go through [thanks largely to amazon increasing my EC2 instance limit :) ], but I haven't had time to fully verify it yet. Early next week I'm giving another demo of All The Code so I really should get around to that soon. I've started the process to take the four months starting in may to work solely on All The Code. The way things are structured, there is a lot of writing and presenting that I'm going to have to do before I can get final approval, but even if that does not work out I've got a back up plan. I'm hoping I can find some time in February to really pull things together, since I am going to be giving a public demo of it on the 26th of February. While the demo will probably not in front of a large number of people, but it will be recorded and I would much rather not have a "oops it crashed" moment on tape [or even worse l...

So, the database built again. There seems to be a bug in the code where by certain cases involving weird class inheritance isn't being properly handled [whereas it use to be handled properly], so I'm going to have to look into that and probably do another database build. I have a meeting on Wednesday with the person at UW responsible for students wishing to do business on their Co-Op term. From what I recall, his background is much more manufacturing / physical oriented, so I might have a bit of difficulty explaining exactly what I do. I may also try talking to the people I talked to about eight months ago, but I'm not sure if thats a good idea or not. Once the final alpha 0.1 database is ready, I will probably give a demo to the csclub , which will hopefully go better than my previous demo :). The present pre-alpha todo list is: 1) fix class bug 2) rebuild DB 3) fix up HTML so it works on most popular browsers (presently breaks in IE6,Konquerer, and probably a few more). ...

The University of Waterloo is celebrating its 50th anniversary, and they have chosen the slogan to be "the spirit of why not?". Right now I'm in my 3A academic term, and I'm trying to figure out what I should do with the Co-Op term between 3A and 3B. I've been going back and forth between applying only to awesome jobs, and not applying to any jobs and working on my own projects (like AllTheCode ). I figure, why not?

So, there is now a completed DB build for the production system, however there seem to be some serious differences between the production system and the development/testing system which have resulted in such fun things as the code viewer erroring out quite frequently. Things are getting quite busy with compilers, so fixing it is going to be challenging to get done. I'm hoping that everything should be ready for the end of the month, since after that point I won't have any serious amount of time to put into All The Code.

Not particularly surprisingly, the DB build encountered some not so nice errors, its still trundling along but at its present rate it would take ~ 20 days to build [which is totally not cool]. My normal solution to a problem like this is to through inexpensive hardware at it, but sadly the last stage DB build is still only handled in a monolithic single threaded manner, so it does not look like it will be done in time. The solution is to [hopefully one last time] delay the full product launch until the 8th, which will hopefully give me enough time to either track down and fix the bottle neck, or write some code to split the task up and get Amazon to increase my EC2 limit, or both :)