Buffer Overflow – The First Attack I Ever Tried (and Defended)

It’s not for any particular reason, but I wanted to write about buffer overflow because it was the first type of attack I ever experimented with — and also the first one I learned to defend against.

Nowadays, memory is often managed for us by safer languages, so we don’t usually have to worry about it too much. But understanding the technique can help in other areas too.
Come to think of it, do embedded or IoT systems still rely heavily on C++? I’ve mostly worked in one domain, so I feel like I’m behind on what’s currently considered standard across industries. (wikipedia)

Scope of this post: this is a compact overview built around the smallest possible example. It does not include the step-by-step debugger session from my own first experiment.

1. What is a Buffer Overflow ?

A buffer overflow occurs when a program writes more data to a memory buffer than it can hold. This typically happens in languages like C or C++, where the programmer is responsible for memory management.

Attackers can exploit this to change the flow of a program or even execute malicious code, making it a serious security vulnerability.

2. Common Attack Types

1) Stack-based Overflow

• Overwrites the return address by overflowing a local variable buffer
• Injects shellcode and redirects execution flow

2) Heap-based Overflow

• Attacks dynamically allocated memory
• Can alter function pointers or object structures

3) Format String Attack

• Similar to overflow; abuses %s, %x, etc.
• Allows reading or writing arbitrary memory

3. Sample Code (in C)

#include <stdio.h>
#include <string.h>

void vulnerable_function() {
    char buffer[10];
    printf("Enter input: ");
    gets(buffer); // Dangerous! No bounds checking.
    printf("You entered: %s\n", buffer);
}

int main() {
    vulnerable_function();
    return 0;
}

Explanation

• The buffer holds only 10 bytes, but gets() doesn’t check the input size.
• If you enter more than 10 characters, you’ll overwrite adjacent memory — possibly the return address.
• In real attacks, this can lead to injected shellcode execution or program hijacking.

Why gets() is a bad idea: it was removed from the C standard in C11, and modern toolchains warn about it. The version below uses fgets(), which takes the buffer size as an argument.

#include <stdio.h>

void safe_function() {
    char buffer[10];
    printf("Enter input: ");
    if (fgets(buffer, sizeof(buffer), stdin) != NULL) { // reads at most 9 chars + '\0'
        printf("You entered: %s\n", buffer);
    }
}

int main() {
    safe_function();
    return 0;
}

fgets() stops after at most 9 characters plus the terminating '\0'. Longer input is cut off, and the unread characters stay in stdin for the next read.

What you will likely see when you run the vulnerable version: many Linux distributions build with gcc’s stack protector enabled by default, so a long input often ends with *** stack smashing detected *** instead of a hijacked return address. That is the canary described in the next section doing its job (behavior depends on your distribution and compiler flags).

4. How to Defend Against It

• Replace unsafe functions like gets() with fgets(), which takes the buffer size as an argument. The same idea applies to snprintf() versus sprintf().
• Use stack canaries to detect overflow: a guard value is placed before the return address and checked before the function returns; if it changed, the program aborts. In gcc this is -fstack-protector-strong.
• Enable ASLR (Address Space Layout Randomization): memory addresses change on each run, so an attacker cannot rely on fixed addresses. It is not a complete defense on its own, because leaked addresses can defeat it.
• Use NX (Non-Executable) memory protection: data areas such as the stack cannot be executed as code. Techniques like return-oriented programming (ROP) can get around it, so it is used together with the other defenses.

Final Thoughts

Buffer overflows may look like simple programming mistakes, but they can be devastating vulnerabilities that allow full system takeover.
If you’re working with languages or environments that handle memory for you, it might not be something you worry about daily.
But if you ever dive into lower-level programming or embedded systems, it’s a technique — and mindset — worth knowing.