Beginner’s Guide to Learn Hacking with CTFs

Kartik Gupta
GDSC VIT Vellore
Published in
12 min readAug 4, 2023

Getting Started With Security Capture the Flag Competitions

Why CTF?

They are one of the most effective methods for learning certain security techniques such as binary exploitation, web exploitation, or reverse engineering.

CTFs are also an excellent way to meet other security geeks because they are generally played in groups. There are several collegiate-level CTFs where you may compete against other students, and many practical security professionals participate in these CTFs as well.

Finally, they hone your hacker perseverance. The experience of becoming stuck in a challenge, persisting, and eventually discovering a solution mimics real-world hacking circumstances. When you’re trapped hacking, CTFs teach you to be patient and positive.

Types of CTF challenges:

JEOPARDY STYLE:

Jeopardy-style CTFs have a few tasks in a variety of areas. Web, forensics, cryptography, binary, and even OSINT. Each activity completed earns the squad the specified amount of points. More points are awarded for more difficult tasks. The next job in the series may only be started once one of the team members completes the previous challenge. When the playing duration exceeds the total number of digits, reveal the CTF winner.

ATTACK DEFENSE STYLE:

In the attack-defence style of CTF combat, each team is assigned its own network or host with vulnerable services. The teams are given a designated period to secure and fortify their own services while adding challenges. Once the preparation time is over, the contestants are introduced, and the battle begins! In this style, the objective is to both defend your own services from attacks and launch successful hacking attempts against your opponents’ services. It’s a dynamic and competitive mode where teams must balance their defensive and offensive strategies to emerge victorious.

MIXED STYLE:

Mixed competition formats are self explanatory. They may resemble a wargame, with specified timeframes for task-based features. CTF games frequently cover a wide range of information security topics, including cryptography, stenography, binary analysis, reverse arranging, mobile security, and others.

CTF skills:

There are two very important things that you’d have to learn to do in order to start participating in the CTF world: finding teams and learning new skills.

How to find teams:

Firstly, how do you find teams to enter CTF competitions?

If you are a high school or college student, see if your school has a cyber security club. These clubs often have already established CTF teams that you can join and compete with. On the other hand, if your school does not already have a club, try starting one and gathering like-minded people! Before you know it, you’d have a group of teammates who are passionate about hacking as well.

If you are not a student, social media is a great way to find teammates. Twitter is one of the best ways to reach out to people you want to collaborate with. Hacking forums and infosec discord channels are also good for this.

How to gain the required technical skills:

For beginner Jeopardy challenges, core security tools and technologies are often not required. After all, that is what you are trying to learn! However, it is good to have a basic understanding of how to use the command line and to have basic programming knowledge.

More advanced skills can be gained by completing easier challenges or by surfing the web. It is also helpful to keep in touch with the latest security news, as these challenges are often based on recently found vulnerabilities.

Challenge Types & Tools

In the exciting world of Capture the Flag (CTF) competitions, participants are faced with a wide range of challenges that test their skills in various domains. In this section, we will explore some of the common challenge types and the tools that can aid in solving them. From cryptography and steganography to web exploits, forensics, and reverse engineering, each category presents its own unique set of puzzles to unravel. Let’s delve into the diverse tools available for each challenge type.

Cryptography:-

In CTF cryptography challenges, the objective often revolves around cracking or deciphering cryptographic objects or algorithms to discover the flag. Here are some cryptographic tools that can assist in tackling these challenges:

· FeatherDuster: FeatherDuster is a powerful automated tool that assists in analysing and breaking cryptographic systems, making it an essential resource for cryptanalysis tasks in CTF competitions.

· Hash Extender :Hash Extender is a handy utility that allows for hash length extension attacks, which can be valuable when attempting to exploit vulnerabilities in hash-based authentication systems during CTF challenges.

. PkCrack: PkCrack is a specialised tool specifically designed to break PkZip encryption, making it an indispensable asset for cracking encrypted archives in CTF scenarios.

· RSATool : RSATool is a tool that simplifies the generation of private keys by utilising the values of p and q, essential components of RSA encryption. It proves useful when solving RSA-based cryptographic puzzles.

· XORTool: XORTool is a dedicated tool for analysing multi-byte XOR ciphers. It provides valuable insights and aids in decrypting XOR-encrypted messages, a commonly encountered cryptographic technique in CTF challenges.

Steganography:-

In the context of CTFs, steganography usually involves finding the hints or flags that have been hidden with steganography. Most commonly, a media file will be given as a task with no further instructions, and the participants have to be able to uncover the message that has been encoded in the media.

· Steghide: Steghide is a versatile tool that allows for data concealment within a wide range of image formats, making it an indispensable resource for steganography challenges in CTF competitions.

· Stegsolve: Stegsolve is a powerful tool that applies various steganography techniques to images, enabling participants to uncover hidden messages or flags embedded within the image data.

· Zsteg: Zsteg is a specialised tool specifically designed for the analysis of PNG and BMP images, enabling the detection and extraction of hidden information encoded within these image formats during steganography challenges.

· Exiftool: Exiftool is a highly versatile tool used to read and write meta information in files. It proves invaluable for extracting hidden data or clues embedded within the metadata of various file formats, including images, documents, and more.

· Pngtools : Pngtools is a comprehensive suite of tools specifically tailored for various analyses related to PNG files. It provides capabilities for manipulating, examining, and extracting information from PNG images, which can be crucial in solving steganography challenges involving this image format.

Web:-

Web challenges in CTF competitions usually involve the use of HTTP (or similar protocols) and technologies involved in information transfer and display over the internet like PHP, CMSs (e.g. Django), SQL, Javascript, and more.

· BurpSuite: BurpSuite is a widely-used graphical tool that aids in testing the security of websites. It provides a comprehensive set of features for analysing and identifying vulnerabilities, making it an essential tool for web security challenges in CTF competitions.

· Postman: Postman is a Chrome browser add-on that facilitates the debugging of network requests. It allows users to send, capture, and manipulate HTTP requests and responses, making it an invaluable tool for understanding and troubleshooting web-related challenges.

· Raccoon: Raccoon is a high-performance offensive security tool designed for reconnaissance and vulnerability scanning. It helps identify potential weaknesses in target systems and aids in gathering the information necessary for exploiting web-based CTF challenges.

· SQLMap: SQLMap is an automated tool specifically designed for SQL injection and database takeover. It streamlines the process of detecting and exploiting SQL injection vulnerabilities, making it a vital asset for web-related challenges involving database interactions.

· W3af: W3af, also known as the Web Application Attack and Audit Framework, is a comprehensive tool for performing web application security testing and auditing. It provides a wide range of features to identify vulnerabilities, test for common web attacks, and generate reports, making it an essential tool for web-based CTF challenges.

Forensics:-

In a CTF context, “Forensics” challenges can include file format analysis, steganography, memory dump analysis, or network packet capture analysis

· Audacity: Audacity is a powerful tool used to analyse various sound files, such as mp3 and m4a formats. It enables users to dissect audio files and uncover hidden information, making it invaluable for audio-based forensics challenges in CTF competitions.

· Bkhive and Samdump2: Bkhive and Samdump2 are tools that work in conjunction to dump SYSTEM and SAM files from Windows systems. These files contain essential information such as user account credentials and password hashes, making them crucial for forensics challenges involving Windows systems.

· CFF Explorer: CFF Explorer is a feature-rich PE (Portable Executable) editor. It allows users to inspect, analyse, and modify the inner workings of executable and DLL files, making it a valuable tool for reverse engineering and forensics tasks involving executable files.

· Creddump: Creddump is a tool specifically designed to dump Windows credentials. It aids in extracting login credentials stored on a Windows system, enabling investigators to uncover valuable information during forensic investigations.

· Foremost: Foremost is a versatile tool used to extract specific types of files based on their headers. It is particularly useful in forensic scenarios where the goal is to recover specific file types, such as images or documents, from a disk image or other storage media.

· NetworkMiner: NetworkMiner is a comprehensive network forensic analysis tool. It captures and analyzes network traffic to extract information such as host names, files, and images, making it an essential resource for investigating network-based attacks and analyzing packet captures in CTF challenges.

· Shellbags: Shellbags is a tool used to investigate NT_USER.dat files, which contain valuable information about a user’s folder navigation history in Windows. By analysing Shellbags, investigators can gain insights into a user’s activities and potentially uncover hidden artefacts.

· UsbForensics: UsbForensics is a collection of tools specifically geared towards USB forensics. It includes a range of utilities that aid in the analysis of USB devices, such as extracting device information, examining file systems, and uncovering evidence related to USB activity.

· Volatility: Volatility is a powerful tool used to investigate memory dumps. It allows forensic analysts to extract and analyse valuable information from memory images, such as running processes, network connections, and open files, providing essential insights for memory analysis in forensic challenges.

Reverse engineering:-

Reverse Engineering in a CTF is typically the process of taking a compiled (machine code, bytecode) program and converting it back into a more human-readable format.

· ApkTool: ApkTool is a powerful tool used for decompiling Android applications (APK files). It allows reverse engineers to extract resources, analyse code, and gain insights into the inner workings of Android apps.

· Barf: Barf is a comprehensive binary analysis and reverse engineering framework. It provides a wide range of features for analysing binary files, including disassembly, code emulation, and vulnerability detection, making it a valuable tool for reverse engineering challenges.

· Binary Ninja: Binary Ninja is a professional-grade binary analysis framework. It offers advanced features for analysing and reverse engineering binaries, including a powerful disassembler, customizable analysis plugins, and collaborative capabilities, making it a popular choice among reverse engineers.

· BinWalk: BinWalk is a versatile tool used to analyse, reverse engineer, and extract data from firmware images. It automatically detects and extracts embedded file systems, hidden code, and other valuable information within firmware images, making it an essential tool for firmware-related challenges.

· Boomerang: Boomerang is a specialised tool used to decompile x86 binaries into C code. It aids in converting machine code back into a more human-readable and understandable format, facilitating the analysis and understanding of binary executables during reverse engineering tasks.

· Frida: Frida is a dynamic code injection tool. It allows developers and reverse engineers to inject and modify code in real-time, enabling them to intercept function calls, modify behaviour, and perform runtime analysis, making it a powerful tool for debugging and analysing software.

· GDB: GDB (GNU Debugger) is a widely-used debugger that provides essential features for debugging and analysing binary executables. It allows users to set breakpoints, examine memory and registers, and step through code, making it an indispensable tool for reverse engineers.

· GEF: GEF is a popular GDB plugin that enhances the functionality of the GNU Debugger. It provides additional features and commands specifically tailored for reverse engineering tasks, such as advanced memory analysis, function analysis, and exploit development capabilities.

· IDA Pro: IDA Pro is one of the most widely used and powerful reversing software available. It offers a comprehensive set of features for static analysis of binaries, including disassembly, decompilation, and graph visualisation, making it a preferred tool for reverse engineers in both professional and CTF environments.

· Jadx: Jadx is a tool used for decompiling Android files, including APKs and DEX files. It allows reverse engineers to extract Java source code from compiled Android applications, enabling a deeper understanding of the app’s functionality and aiding in the analysis of Android-specific challenges.

Miscellaneous:-

Many challenges in CTFs will be completely random and unprecedented, requiring simple logic, knowledge, and patience to be solved. There is no sure-fire way to prepare for these, but as you compete in more CTFs you will be able to recognize them and hopefully have more clues on how to solve them.

YouTube Channels to follow:-

LiveOverflow

HackerSploit

IppSec

HackHappy

Practice

The best way to develop your skills is through consistent practice, you can practice by taking part in CTFs or practicing online using labs.

Competitions

Basics

PicoCTF: https://picoctf.com/

Intermediate to Advanced

HITCON CTF: https://ctf2019.hitcon.org/

Google CTF: https://capturetheflag.withgoogle.com/

Online Practice Labs

Basics

Hacker101: https://www.hacker101.com/

TryHackMe: https://tryhackme.com

Intermediate to Advanced

HackTheBox: https://www.hackthebox.eu/

OverTheWire: https://overthewire.org/wargames/

Natas: https://overthewire.org/wargames/natas/

Bandit: https://overthewire.org/wargames/bandit/

What to Follow?

Writeups

GitHub: https://github.com/ctfs

Reddit: https://www.reddit.com/r/securityCTF/

Medium: https://medium.com/ctf-writeups

CTF Calendar

CTFTime: https://ctftime.org/

Other Useful resources

GitHub: https://github.com/apsdehal/awesome-ctf

GitHub: https://ctfs.github.io/resources/

InfoSec Institute: https://resources.infosecinstitute.com/tools-of-trade-and-resources-to-prepare-in-a-hacker-ctf-competition-or-challenge/#gref

PicoCTF: https://picoctf.com/resources

https://kongwenbin.com/resources-for-capture-the-flag/

CTF Etiquette!

Before you go on to playing CTFs (and have the time of your life!), here are a few sacred rules of CTF participation that you should keep in mind.

First, absolutely do not post solutions and flags online! The purpose of CTFs is to help people become better hackers by brainstorming solutions to challenges. Giving solutions away is denying the chance for others to learn.

On the other hand, you should not try to search up solutions or ask for flags online. You can ask for help, discuss with others, or even collaborate on solving a challenge, but asking or searching for solutions takes away from the experience. Even if you try to understand the solution, it is not the same as working hard and finally finding the answer yourself!

Pro Tips / Words of Wisdom

1. PATIENCE IS THE KEY: Mastering CTF takes time and dedication. Overnight success is rare, so stay patient and keep learning and practising.

2. Do not expect someone will spoon feed you everything: CTFs are about problem-solving and self-learning. Don’t rely on others to give you all the answers. Embrace the challenge and strive to find solutions on your own.

3. Confidence: Believe in your abilities and approach each challenge with confidence. Trust in the skills and knowledge you have acquired

4. SKILL DEVELOPMENT AND EXPERIENCE ARE THE BIGGEST REWARDS: The journey of participating in CTFs is not just about winning. The real rewards come from the skills you develop and the experience you gain along the way.

5. Not always for rewards: Participate in CTFs not solely for the rewards but for the joy of learning, solving puzzles, and expanding your knowledge in the cybersecurity field.

6. Learn a lot: CTFs are a valuable learning experience. Take advantage of the challenges to gain new skills, discover new tools, and explore different areas of cybersecurity.

7. You may not be able to solve anything at the beginning: It’s common to face difficulties and struggle with challenges at the start. Remember, everyone goes through this phase. Don’t lose hope and keep pushing forward.

8. Stay focused: CTFs can be intense and time-sensitive. Stay focused on the task at hand and avoid distractions. Concentration and attention to detail are key.

9. Depend on yourself: While collaboration can be beneficial, ultimately, you need to rely on your own abilities and problem-solving skills. Trust in yourself and your capabilities to tackle challenges.

10. Stay updated with the infosec world: The cybersecurity landscape is constantly evolving. Stay updated with the latest trends, techniques, and tools in the infosec world to enhance your knowledge and stay competitive in CTFs.

--

--