Wikipedia
PHP-Crawler
PHP-Crawler is an open source crawling script based on PHP and MySQL. Created to implement simple as possible local website search it became popular for small websites on shared hosting. Script set is distributed under the terms of BSD License.