The HarvestMan WebCrawler Robot - Web crawler in Python - for example for your own search engine