funny, i am requesting a tool on my forum, but i just can't come up with anything right now. i need something that crawls the website in the deepest way possible, it means it will 'click' any link it sees, and lists everything, but will not leave the site. it will stay on the site, including sub-domains.
I'm using acunetix and few others, but i want a simple crawler that would do only crawling in the deepest way possible.
oh yeah and i forgot to mention..something that would use my login. so i could crawl site's like facebook, orkut ..etc. otherwise i would use google as crawler, but it will not work on sites that needs login first.
Web Crawler with login ?
- Sethioz
- Admin
- Posts: 4764
- Joined: Fri Jul 27, 2007 5:11 pm
- Custom: Gaming YT > https://youtube.com/SethiozEntertainment
Game Hacking YT > https://youtube.com/sethioz
Game Hacks Store > https://sethioz.com/shopz - Location: unknown
- Contact: