@Steve: I've used that method, too. I like to add an escape, so the program can never get caught in an infinite loop. For web crawling, I used a short delay inside the loop, and a counter to keep track of ow many times the delay was used. At some point, I gave it instructions to "time out."
Pete