Best web crawler software?

[prod]

Hi all.

Currently we have a license for IDM and we're using IDM's site grabber for crawling web sites and obtaining data. IDM however, is just about the worst piece of software ever programmed, especially when it comes to web crawling. I'm now looking for a decent web crawler with customized settings and a scheduler,doesn't matter if I need to buy it. I've been doing some googling, but I thought that maybe theres somone who knows of a decent web crawler.

Thanks in advance

 

[Divestar]

SPAMMER!!! RAS! RAS! RAS!

 

[prod]

SPAMMER!!! RAS! RAS! RAS!

Thanks so much for your helpful input.

/shootshimself

 

[spiderz]

Here's a decent web crawler.
http://mybroadband.co.za/photos/showphoto.php/photo/13009/title/huge-spider-web/cat/500

 

[Divestar]

Here's a decent web crawler.
http://mybroadband.co.za/photos/showphoto.php/photo/13009/title/huge-spider-web/cat/500

ROFL

 

[Lino]

SPAMMER!!! RAS! RAS! RAS!

Here's a decent web crawler.
http://mybroadband.co.za/photos/showphoto.php/photo/13009/title/huge-spider-web/cat/500

No wonder this forum has gone down, with responses like this to important questions.

@prOd

A couple of options:

http://www.winwebcrawler.com/
http://sourceforge.net/projects/methabot/

 

[prod]

Thanks Lino.

Currently looking into Download Studio. Will check out what you suggested.