Monday, November 13, 2006

Amazon WebServices, or why I might get a chance to sleep after all

I wasn't expecting such a fast response from amazon with regards to my problems using AWS, but I must say I'm pleasantry surprised. Monday afternoon my time (Monday morning there time) I got a conference call, and we went over how I was trying to use the Alexa Websearch service, and they explained the reason for all my frustration. Namely, that it times out with very large start values, which is why I couldn't get more than a few hundred records. They didn't place a hard limit on the start values, since when the problems will start to occur is apparently dependant on a large number of factors.

On the upside, it looks like I won't have to write my own crawler since they also enabled my Amazon Web Search Platform account, which should not have these problems [AWSP is designed for people trying to build search engines, and has a different model than the regular AWS system]. The next seven days are going to killer finish my demo, but it should be possible :-)

No comments:

Free Blog Counter