Back

Tool: Gospider


 https://github.com/jaeles-project/gospider

1.1 Description
A fast web-spider which is able to perform web crawling, brute-forcing, parsing of default files like robots.txt or sitemap.xml, link-extraction from JavaScript files and gathering of old URLs based on archives like Wayback Machine or Common Crawl.


1.2 Installation

go get -u github.com/jaeles-project/gospider

1.3 Example

gospider -s "https://google.com/" -o output -c 10 -d 1