Web App Sec RECON — Black Box Foundations and TTPs
A summary of civilian freelancer Theory and TTPs (Tactics, Techniques, Procedures) for Reconnaissance before the Vulnerability Assessment stage in a Web Application Penetration Test.
THEORY
THE DELIVERABLE IS A REPORT
In order to improve security, we are gathering information about weaknesses in a technology that can be exploited through corresponding vulnerabilities. We use vast open-source methodologies to map the attack surface. If the exploit can make a critical impact on the assets, then the risk should be mitigated and is worth reporting.
THE TESTING MODEL
Tester: Performs the testing activities
Tools and methodology: (tactics, techniques and procedures)
Application: The target black box to test
Specialization: Context + Language + Characters + Symbols
There exist our computing machines, the network with its computing machines, and the target computing machines.
The one who asks for a service is a client and the one who provides a service is a server. The service is a web application (website).
As the attacking tester, we are the client using the network of the public internet to attack and test the service on the target server. Recon can detect if the architecture or infrastructure has a weakness which makes the client-side or server-side vulnerable.
TARGET COMPUTING MACHINES
The target ARCHITECTURE is how it’s built. Focus is on the design and conceptual aspects that shape the system’s functionality and behaviour, the client-server architecture, APIs, microservices and implementation of specific security mechanisms.
The target INFRASTRUCTURE is what it’s built with. Focus is on the physical and technical elements of the IT system, the web servers, database servers, load balancers, firewalls, frontend and backend frameworks, third party services and locally hosted files.
FOUNDATION — TESTER DOMAIN DIMENSIONS
Information Technology (IT) Specialist — InfoSec — Security Operations
— Cyber Security — Network Security (NetSec) — Application Security (AppSec)
— — Secure Software Development: SSDLC/DevSecOps Testing Phase of CI/CD
— — — Offensive — Web Application Penetration Testing — Red Team
— — — — Open Source — OWASP — Black Box manual DAST
— — — — — Information Gathering — Passive Recon (Indirect/Direct); OSINT Offensive/Defensive
— — — — — — Stealth
— — — — — — — Character/Symbol/Signal Wizardry/Electricity
— — — — — — — — Fingerprint/Enumeration — Active Recon (automated DAST scanning)
— — — — — — — — — Vulnerability Assessment — Bug Bounty Hunting — Deliverables: Reports
LAYERED PENETRATION TESTING
Within popular Cyber Security frameworks which describe the progression of a cyberattack or the equivalent professional ethical hacking methodologies, Reconnaissance (Recon) is the first stage and involves researching the potential target. It is used to identify the scope, the technologies used and public information for building an understanding of the system.
To be effective, testers should know exactly what they are looking for and how the data will be used before collection starts. Using passive reconnaissance and limiting the amount of data collected minimizes the risk of being detected by the target.
Penetration testers or attackers generally follow a process of structured information gathering, moving from a broad business scope to specific data. An attacker will typically dedicate a majority of the overall work effort on recon.
Note: There exist various levels of ethical hackers at various layers who do minimal to no recon.
The Red Team only needs specific info, not the whole range. Testers and attackers have access to current public vulnerability lists and exploitation kits. The collected information is later weaponized to facilitate exploitation. This means the penetration tester can design specific types of attacks, exploits, and payloads that are suitable for the attack surface of the target.
METHODOLOGIES
GOAL ORIENTED
Define and work towards a goal.
CHECKLIST DRIVEN
The OWASP WSTG and Web Application Hacker’s Handbook.
INTUITIVE
Understand the life of a Web Application and its context.
TACTICS — TYPES OF RECON IN SEQUENCE
The usefulness of data gathered increases with the risk of detection.
PASSIVE RECON
INDIRECT PASSIVE RECON
In this instance there is no interaction with the target Web Application or organization.
DIRECT PASSIVE RECON
Involves the normal interactions that occur when an attacker expectedly interacts with the target.
DEFENSIVE OSINT
To understand the current threat landscape for the target: identify any leaks, employees, and products on social media, forums, and dark web marketplaces, other adversaries discussing the same target.
Information Leakage Monitoring, Dark Web, Security Breaches, Threat Intelligence Feeds, Public Records, Indicators of Compromise (IoC), Geopolitical Monitoring, Competitor Analysis.
OFFENSIVE OSINT
Involves actively seeking more sensitive information that can be directly used to exploit vulnerabilities: credentials, or personal data for social engineering attacks and access.
Scraping (social media, Shodan), Web Archives, Google Hacking Database.
ACTIVE RECON
PACKET INFORMATION GATHERING
Packet route tracing (hops).
MANUAL RECON
Involves direct queries or other interactions (ie one particular single port test of the target network).
DYNAMIC APPLICATION SECURITY TEST (DAST) SCAN
Scanning is a technique that’s used to discover live systems on a network, identify the open service ports on a system, vulnerabilities on host machines and operating system architecture.
More focused probing: Fingerprint internal and external infrastructure and hosts, port scans. List the directories, files and users.
ENUMERATION
Identify the services and applications that are operational on the target system. If possible, the attacker will want to know the service type, vendor, and version.
Actively probing the target system for specific details produces more useful information but interactions with the target system may be logged, triggering alarms by protective devices, such as firewalls, IDS/IPS and EDR systems.
SOCIAL ENGINEERING
Gather information about the web application by contacting employees or vendors and tricking them into revealing sensitive information.
TACTICS — RECON SEQUENCE
Business Logic Analysis: Content Valuation / Application Logic
FINGERPRINTING / ENUMERATION
Network Infrastructure Security: WAF / Load Balancer / Reverse Proxy / IDPS / DHCP
Network Infrastructure: Cloud: Content Delivery Network / Data Centre / Computers
Web Application Server Infrastructure: OS / Web Server Software / Web Application Framework / Databases / Resources / Dependencies
Domain Scope: IPs / Certificates / Subdomains / Directories / Filepaths / Files / Extensions / Spidering
Browsing (Functional / Explorative): Access Handling / Input Handling
TECHNIQUES — STEALTH ENTRY
To improve the effectiveness of active reconnaissance in providing detailed information. Going “stealth” in information security refers to the practice of conducting security testing in a covert or low-profile manner, without alerting the target of the activity. Attackers approach the target with a goal in mind and send the minimum number of packets needed to determine the objective. They will only test the ports that impact the kill chain they are following to their specific target while knowing the possible maximum about the attack surface scope.
First consider securing one’s own device by using a new dedicated computing machine. Get a new Purism laptop for every test and have all unnecessary services and programs deactivated.
WORKING WITH EVASTION TECHNIQUES
In the TCP/IP protocol stack, whenever a packet is sent from one device to another, the source IP address is included within the header of the packet. All address details must be included within all packets that need to traverse a network.
In this way, the testing attacker’s time and data stamps, the source IP address, and additional information can be identified by the target.
SOCK PUPPET
Penetration testers usually create a facke identity, a sock puppet, to mask their true identity when performing any type of information gathering.
ANONYMITY
To remain anonymous when interacting with a web application, follow basic online OPSEC: sock puppets on TAILS OS, TOR Browser (with resize, NoScript, and user agent spoofing), Proxy Chains, right Monero for transactions, VPS, OpenVPN, OpenDNS, MAC/IP Spoofing, HTTPS Obfuscation.
TOR: An open source implementation of free access to an anonymous proxy network.
Proxychains: Create a logical chain of connections between multiple proxy servers when sending traffic to a target to prevent your real IP addresses from being exposed to the target.
DECOYS
Trick the target into believing that the network scans are originating from multiple sources rather than a single source IP address.
SPOOFING THE MAC AND IP
Spoofing the MAC address simply allows the attacker to pretend to be someone else on the network, a network switch or even a router.
IGNORE COMMUNICATIONS PROTOCOLS
Alter the TCP/IP handshake to avoid connecting with target system and leaking data.
CAMOUFLAGE TOOL SIGNATURES
Using Scapy to craft TCP packets with specific flags set.
MODIFY PACKET PARAMETERS
Randomize or spoof the source IP, port, MAC address; adjust timing to slow the transmission of packets, change packet size by fragmenting or appending random data. Hide within legitimate traffic.
TECHNIQUES — OBSERVATIONS DURING RECON
What type of web app is the target: software as service, open source, paid etc.
First understand the Business logic:
- What app should do.
- What app does.
- How it does this.
- What can go wrong.
The attack surface is all the data/signal entry points and how users interact with each other.
Seek behaviour associated with vulnerabilities:
If site allows/has X then check for Y vulnerabilities.
Is there any reaction to application misuse? Are there any defense mechanisms?
PROCEDURES — BASIC EXAMPLES OF RECON
SPECIFIC TOOLSET
Methodology: Custom OWASP WSTG
Products: Firefox Developer Edition (with any extensions), ZED Attack Proxy (ZAP with any of its community extensions) and Kali Linux.
INDIRECT PASSIVE RECON
Kali Linux: Run theHarvester to collect email addresses, subdomains, IPs, and employee names from public sources:
theHarvester -d bla.com -l 500 -b urlscan
DIRECT PASSIVE RECON
Zed Attack Proxy: Configure ZAP to monitor traffic from a browser while browsing the target website. Analyze the gathered HTTP responses for security misconfigurations, comments in HTML, and exposed information.
DEFENSIVE OSINT
Kali Linux: Use SpiderFoot to search for the client’s domain and email addresses on the dark web.
spiderfoot -s bla.com -m all
OFFENSIVE OSINT
Kali Linux: Scrape social media and use Metagoofil to extract metadata from publicly available documents.
metagoofil -d bla.com -t doc,pdf,xls -l 200 -n 50
PACKET INFORMATION GATHERING
Wireshark: Capture network traffic to and from the target web server to identify protocols, services, and potential vulnerabilities. Filter the capture to focus on traffic related to the target.
MANUAL RECON
Firefox Developer Edition: Use the developer tools (F12) to manually inspect the HTML, JavaScript, and network traffic. Open the target website, use the Inspector tabto examine the HTML and JavaScript, use the Network tabtab to monitor and analyze network requests and responses.
DYNAMIC APPLICATION SECURITY TEST (DAST) SCAN
Zed Attack Proxy: Conduct an automated scan to identify common vulnerabilities in the web application: Open ZAP. Use the Quick Start tab to enter the target. Review the scan results in the Alerts tab.
ENUMERATION
Kali Linux: Enumerate directories and files on the target web server with Gobuster:
gobuster dir -u http://bla.com -w /usr/share/wordlists/dirbuster/directory-list-2.3-medium.txt
SOCIAL ENGINEERING
Kali Linux: Use the Social Engineering Toolkit (SET) to create a phishing campaign or clone a website to trick users into providing information: Open SET. Navigate through the menu to create a phishing campaign. Send the phishing emails and collect responses.
STEALTH
To check if port 443 (HTTPS) is open on a web application using the command line in TAILS OS while maintaining stealth, you can use the curl command with specific options. By specifying — resolve, you are instructing curl to directly connect to the IP address associated with bla.com on port 443 without performing DNS resolution. This will display only the HTTP status code returned.
curl — -resolve bla.com:443:1.2.3.4 https://bla.com -s -o /dev/null -w “%{http_code}”
AFTER RECON
When something of interest is found, stop and begin Testing, looking for an indication of a vulnerability:
- unexpected message returned
- delay in response time
- unsanitized input being returned
- serverside check being bypassed
Testing after recon can be performed on client or server vulnerabilities. These vulnerabilities typically exist because of a lack of skill in security in any particlar stage of design, coding, configuration or maintenance.
On the server side, attackers could typically perform the following list of attacks:
• Web application firewall evasion
• Injection attacks
• Remote code execution
• File inclusion — remote and local
• Directory path traversal
• Exploiting session management
• Exploiting the business logic of the system or application implementation
• Web services misconfiguration or excess authorization privileges
• Baiting the vulnerable services through shared infrastructure
• Identifying any relevant information that can help them to perform more dedicated attacks
FRAMEWORKS AND MODELLING
Knowledge bases like the MITRE ATT&CK framework can be used to identify gaps in a security program and help organizations to improve their defenses against potential threats. Models like the Lockheed Martin Cyber Kill Chain can used to describe the different stages of a cyberattack. Here it is in seven stages:
1. Reconnaissance: The tester gathers information about the target system and maps this attack surface to vulnerabilities which themselves include the following testing:
• Configuration and Deployment Management Testing
• Identity Management Testing
• Authentication Testing
• Authorization Testing
• Session Management Testing
• Input Validation Testing
• Error Handling
• Cryptography
• Business Logic Testing
• Client Side Testing
2. Weaponization
3. Delivery
4. Exploitation
5. Installation
6. Command and Control
7. Actions on Objectives
END NOTES AND FUTURE WORK
These are all Professional layer and Specialization layer notes.
To continue this Recon exploration, please see the “Aspie Hackers — Bug Whisperers” Discord server. Here is an example “Ethical Hackers — Aspie Bug Whisperers — SINGLE PAGE SUMMARY” Google Document available on the mentioned server. There are many other available documents there for collaboration on paid work and deeper technical procedural insights.
New cloud services are significantly impacting web application security testing and we will adapt to these changes by integrating new tools and methodologies suitable for cloud environments.
The high level Business layer would be using top technologies outside of the available public models. Secret layer would include aspects beyond popular contemporary science such as metaphysics and psychic phenomena. It is at least worth noting their existence because they ultimately impact everything we do at the Recon Specialization layer.
