I wasn't expecting such a fast response from amazon with regards to my problems using AWS, but I must say I'm pleasantry surprised. Monday afternoon my time (Monday morning there time) I got a conference call, and we went over how I was trying to use the Alexa Websearch service, and they explained the reason for all my frustration. Namely, that it times out with very large start values, which is why I couldn't get more than a few hundred records. They didn't place a hard limit on the start values, since when the problems will start to occur is apparently dependant on a large number of factors.
On the upside, it looks like I won't have to write my own crawler since they also enabled my Amazon Web Search Platform account, which should not have these problems [AWSP is designed for people trying to build search engines, and has a different model than the regular AWS system]. The next seven days are going to killer finish my demo, but it should be possible :-)
Monday, November 13, 2006
Amazon WebServices, or why I might get a chance to sleep after all
Posted by
Holden Karau
at
8:24 PM
Labels: all the code, amazon, amazon web services, atc, codeing, life
Subscribe to:
Post Comments (Atom)


No comments:
Post a Comment