Recon — my way.

A detailed blog post on my reconnaissance processes for web applications security testing. I always wanted to write about this subject being asked by many friends, community members, etc. but I hooked wasting a lot of hours on a Meme Channel & The Big Bang Theory TV Series.

UPDATE: I created a GitHub repository with tools from this post and personal installation guide.

ehsahil/recon-my-way

yes, I did !! 🌋

Recently, some new members of the InfoSec community asked me to share my recon process. Hence, I decided to begin writing this blog and tried to include such tools and services which helps me a lot while testing and will help the readers too, for sure.

Summary

1. Introduction
2. A tool I modified.
3. Visual recon
4. More Assets — More findings — More win.
5. Data Storage Buckets.
6. Github for Recon.
7. Read Every JS
8. Archive
9. Continuous Recon
10. Extra points for recon.

1. Introduction

Whenever I get an invitation for a new program or I want to test a target, I start my recon process by using the Knockpy.

Why I use knockpy Initially? It provides me with a quick overlook of the subdomains with a response code.

Once, I found a subdomain takeover bug within 2 mins.

I ran knockpy on an old program’s in-scope asset with almost 150 bugs resolved on HackerOne. Quickly saw 404 page pointed to AWS S3 bucket and bucket were available to create. Hence, with no delay, I created the new AWS S3 bucket and uploaded a text file with the encoded filename and reported the bug and guess what? I got the bounty that too within 15 mins.

Lesson: knockpy = quick win

see 404 error in above screenshot == Quick win.

2. A Tool I Modified.

I use a customized tool for subdomain reconnaissance.

I used a resolver tool by Malvinsh and customized it.

melvinsh/subresolve

Malvinsh resolver tool runs as.

Malvinsh Tools Original output.

Malvinsh tool is doing two simple processes.

• Getting the IP address of the domain/subdomain from the wordlist using HOST.
• Performing the Nmap scan.

Using its logic I created two scripts.

• Subdomain.rb
• Recon.rb

Caution: Do not use these scripts on programs where automatic testing is forbidden and also on the targets on which you are not allowed to.

Subdomain.rb

Subdomain.rb is a lightweight script to automate tools for subdomain finding and it’s damn flexible — more tools can be added easily.

Subfinder and sublist3r results sometime overlap. I tried to run both tools separately on a particular asset and got some distinct results — that’s why I kept both tools in this script.

The script is using the following tools to get the subdomains data.

• Subfinder

Ice3man543/subfinder

• Censys subdomain finder.

christophetd/censys-subdomain-finder

• Knockpy

guelfoweb/knock

• Sublist3r

aboul3la/Sublist3r

• Aquatone

michenriksen/aquatone

subdomain.rb gist.

Usage: ruby subdomain.rb domain.com

Subdomain.rb demo run.

I, usually, create a file and add all subdomains from the above outputs. Then, I use the sort command to delete all duplicate/overlapped subdomains from the file.

sort wordlist | uniq

I pass the final unique subdomains file to the recon.rb

Resolve.rb

Recon.rb is another lightweight script and it is also flexible hence, more tools can be added easily.Tools included in recon.rb

• Host : Resolve the subdomain.
• Nmap : Perform Port scan on subdomain

Nmap: the Network Mapper — Free Security Scanner

• AWS CLI — In the script AWS, CLI used test subdomains are connected to AWS bucket or not also checks for list permission. (Can be customized to test for write permission files.)

AWS Command Line Interface

• Dirsearch — Search for directories with default wordlist and all (*) extensions.

maurosoria/dirsearch

recon.rb gist.

Usage: ruby recon.rb wordlist

Recon.rb demo run.

A tool with Above functionality in the organized fashion.

Lazyrecon

nahamsec/lazyrecon

Interesting blog posts

[BugBounty] Decoding a $😱,000.00 htpasswd bounty

Scanning the Alexa Top 1M for .DS_Store files

3. Visual Recon

I use the previously generated wordlist from subdomian.rb for visual recon.

I generally use the following two tools.

• WebScreenshot

maaaaz/webscreenshot

• Lazyshot

mdhama/lazyshot

Interesting blog posts

[Tools] Visual Recon — A beginners guide

4. More Assets — More findings — More win.

I go for this section after reporting 2–3 issues in a particular program and wait for the response, if I find program interesting then I try to collect as much information as I can about the target using the following services.

• Censys

Censys

Tool for Censys:

yamakira/censys-enumeration

• Shodan

Shodan

• ViewDNS — Reverse Whois Lookup.

ViewDNS.info — Your one source for DNS related tools!

Get the whois information of target using whois command or use an online tool.

whois domain.com

If the company is not using Domain Privacy Service,

you will find the host-masters email address then use that email to find other domains registered on same email address using Reverse Whois Lookup. Targets Registered legal name also can be used.

Result for hackeone inc query

• IP range Crawl

IP range for hackerone.com

• AltDNS

infosec-au/altdns

• Nmap Subdomain finding.
• Content-Security-Policy (CSP)

Tools

yamakira/domains-from-csp

Analyse your HTTP response headers

• More targets with Burp Suite by Jason Haddix

Jason Haddix Tweet about the method

• Domain Analyzer

eldraco/domain_analyzer

• Domain Profiler

jpf/domain-profiler

• VHost Scan

codingo/VHostScan

• ThreatCrowd

Threat Crowd | Threatcrowd.org Open Source Threat Intelligence

• Visual Site Mapper

Visual Site Mapper — Create a visual map of your site

• Certificate Transparency
• Google Transparency Report

Google Transparency Report

• Certsspotter

https://certspotter.com/api/v0/certs?domain=hackerone.com

• CertDB

CertDB — SSL certificates search engine

• Crt.sh —

https://crt.sh/?q=%25domain.com

• Facebook Certificate Transparency Monitoring Subscriptions.

Certificate Transparency Monitoring — Facebook for Developers

Facebook Crt transparency monitoring subscriptions.

Typical notification from Facebook when new asset on the same crt is available.

Interesting blog posts & tools taken from

Asset Discovery: Doing Reconnaissance the Hard Way

Subdomain Takeover: Thoughts on Risks

Subdomain Takeover: Proof Creation for Bug Bounties

Project Sonar: An Underrated Source of Internet-wide Data

EdOverflow/can-i-take-over-xyz

5. Data Storage Buckets.

Common Places to find Data Storage buckets.

• Github
• Javascript files
• CSP Headers
• Archive crawl
• Pastebin

Tip: If a bucket is returning access denied. Try to search it on Google. There are good chances that the team has recently changed the permissions of the bucket and specific files have been indexed by Google (which are available with read permission).

If the application has file uploading functionality, then try to capture the file uploading request and see where the file is being uploaded. Sometimes you will find AWS or other data storage buckets, which cannot be detected by any other method.

If you find the bucket like upload-user-content-target-prod — try to change the prod to dev, staging, sandbox, etc.

• AWS CLI — AWS CLI is useful for verifying or testing the permissions of the AWS S3 buckets, Creating Buckets and Read other buckets data. AWS Account needed to use CLI.

AWS Command Line Interface

• Bucket Finder — Great Tool for finding buckets using subdomains wordlist, can be integrated into recon.rb script but I don’t always use it.

Bucket Finder — DigiNinja

• LazyS3 — LazyS3 is an another tool which I use almost frequently to find the staging, sandboxed, dev and production buckets.

nahamsec/lazys3

• Slurp: Slurp great tool for AWS Buckets Recon. (Deleted After Microsoft Purchased Github I guess). Weird!

https://github.com/bbb31/slurp

• S3 Bucket Finder — Another good tool for AWS S3 buckets.

gwen001/s3-buckets-finder

Interesting blog post

A deep dive into AWS S3 access controls — taking full control over your assets

6. Github For Recon.

Github is extremely helpful in finding Sensitive information regarding the targets. Access-keys, password, open endings, s3 buckets, backup files, etc. can be found on public GitHub repositories.

Interesting blog post

GitHub for Bug Bounty Hunters

7. Read every JS.

Sometimes, Javascript files contain sensitive information including various secrets or hardcoded tokens. It’s always worth to examine JS files manually.

I have found the following things in Javascript.

• AWS or Other services Access keys
• AWS S3 buckets or other data storage buckets with read/write permissions.
• Open backup sql database endpoints
• Open endpoints of internal services.

The more Javascript files you read, the more chances will be of win.

Tools

I generally like to read the javascript code manually with the help of JSBeautifier.

Online JavaScript beautifier

The following tools are useful:

LinkFinder

GerbenJavado/LinkFinder

JSParser — Another great tool by Behrouz Sadeghipour

nahamsec/JSParser

Interesting blog post

Bug Bounty — Tips / Tricks / JS (JavaScript Files)

8. Archive

Searching for the targets webpages in waybackmachine, the following things can be found.

• Old and abandoned JS files.
• Old API endpoints.
• Abandoned CDN’s Endpoints.
• Abandoned Subdomains.
• Dev & staging endpoint with juicy info in source code comments.

If you are getting 403 on a page, you can also search that 403 pages of targets in way back machine sometimes, you will find them open with helpful information.

Tool: Waybackurl

9. Continuous Recon.

• The most essential thing in continuous recon is handling of the recon data for the future uses, for this, SecurityEscape created a tool known as Swiftness. I use Swiftness to hold all of my recon data for every target.

Swiftness- My personal websec checklist.

• I use the reminder to revisiting the targets and perform recon regularly. (Every month)

10. Extra points for recon.

Note: I have provided some data in gist because I regularly update them and those updates will be automatically available here.

Best of luck for all of your future infosec things.

If you have questions and anything about the post you want to ask me, Please contact me via twitter/fb. I’ll have my DM open.

Feedbacks and edits are welcome

Twitter Facebook

Until next time!

If you like my blog posts and my work, Please consider checking out my “Buy me a coffee” page

Buy Me A Coffee - Best Way for Creators to Receive Tips