peterm
committed
on 19 Nov 03
debugging the crawler and adding ability to exclude links