The following tutorial is a beginner guide on the Burpsuite web application spider feature, which is using to crawl the web application.
Burp Suite: Burp Suite is a Java-based Web Penetration Testing framework. It has become an industry-standard suite of tools used by information security professionals. Burp Suite helps you identify vulnerabilities and verify attack vectors that are affecting web applications.
You can Download BurpSuite New Edition here.
Requirements?
- BurpSuite
- Any Browser (We use chrome here)
- Network Proxy
- Demonstrate link testphp.vulnweb.com
First, ensure that Burp is correctly configured with your browser. If not then follow the steps.
To configure Burp Suite demonstrate.
First Go to your browser settings and in the search box type proxy then select open proxy settings > In connection tabs > Lan settings > Tick Use a proxy server for your LAN > (127.0.0.1 port number 8080) then Click ok .
Now open the Burp Suite.
In the Burp Proxy tab, ensure “Intercept is off” and visit the target web application, which you like testing in your browser.
As you can see in the screenshot, currently there is no targeted web application inside sitemap of the burp suite. To add your targeted web application inside it, you need to fetch the http request sent by the browser to the web application server using the intercept option of the proxy tab.
Also read- BurpSuite Run time file Payload Working
Click on the Proxy tab and turn on intercept for catching http request and refresh the web application.
Note: In the new version of Burp “send to Spider option” has changed to a “New scan“.
Now in Proxy tab intercept right click on the request to bring up the context menu and click “Scan”.
Select scan type “crawl” enter the “URLS to scan : testphp.vulnweb.com” which automatically starts web crawling, then click Ok.
Now choose Dashboard for a further step, here you will find task-based model option.
You can check the scanning details through view details.
You can also monitor the status of the scan when running, via the Control tab. Any newly discovered content will be added to the Target site map.
When scanning branch of the site map, Burp will carry out the following actions (depending on your settings):
- Request any unrequested URLs already present within the branch.
- Submit any discovered forms whose action URLs lay within the branch.
- Re-request any items in the branch that previously returned 304 status codes, to retrieve fresh (uncached) copies of the application’s responses.
- Parse all content retrieved to identify new URLs and forms.
- Recursively repeat these steps as new content is discovered.
- Continue spidering all in-scope areas until no new content is discovered.
Now click on preferred target site map further content which has been discovering by the spider will get added inside it.
You can see its dump all items of web site even by throwing request and response of the host.
