Executable space protection is an exploit mitigation that marks regions of memory as either writable or executable, but not both, so that code injected into data areas such as the stack or the heap cannot be run. In OS security, it is one of the foundational built-in protections of desktop operating systems, known on Windows as DEP (Data Execution Prevention), enforced in hardware by the NX bit, and described by the policy name W^X, meaning write XOR execute.
The protection targets the classic buffer overflow attack, in which an attacker writes machine code into a buffer and redirects execution to it. With executable space protection in place, the processor refuses to fetch instructions from those pages and raises a fault instead.
How executable space protection works in hardware and software
Executable space protection works through page table permissions. Every page of virtual memory has attributes that the memory management unit checks on each access. Modern processors add a dedicated no-execute attribute: when it is set and the processor tries to fetch an instruction from that page, the hardware raises a fault and the operating system terminates the process.
The operating system decides which pages get the attribute. Program code sections are mapped readable and executable but not writable, while the stack, the heap and data sections are mapped readable and writable but not executable. The loader, the memory allocator and system calls such as mprotect on Unix-like systems or VirtualProtect on Windows maintain this separation.
Before hardware support was common, some systems implemented the protection in software by emulating the bit through segment limits or page tricks, which was less complete and could cost performance. Hardware enforcement made the protection effectively free at run time.
| Name | Layer | Origin or platform |
|---|---|---|
| NX bit | Processor page tables | AMD64 architecture |
| XD bit | Processor page tables | Intel 64 architecture |
| XN and PXN | Processor page tables | Arm architecture |
| DEP | Operating system policy | Windows |
| W^X | Memory mapping policy | OpenBSD and other Unix-like systems |
| PaX PAGEEXEC | Kernel patch | Hardened Linux kernels |
Operating system support and enforcement modes
All current desktop operating systems enable executable space protection by default for 64-bit processes. Differences remain in how strictly it is applied to 32-bit software and how applications can opt in or out. On Windows, DEP is governed by a system-wide policy with four settings.
| Policy | Scope | Typical use |
|---|---|---|
| OptIn | System components and opted-in programs | Default on client editions |
| OptOut | All programs except listed exceptions | Default on server editions |
| AlwaysOn | All processes, no exceptions | Hardened systems |
| AlwaysOff | No processes | Troubleshooting only |
Regardless of this policy, 64-bit Windows processes always run with DEP. On Linux, the kernel enforces non-executable mappings and stacks when the hardware supports it, and executable stacks are only granted when a binary explicitly requests them. macOS enforces the same separation and adds strict rules for just-in-time compilers through dedicated memory mapping flags.
Executable space protection and related mitigations
Executable space protection stops code injection but not code reuse. Attackers responded with techniques such as return-to-libc and return-oriented programming, which chain together existing executable fragments instead of introducing new code. For this reason the protection is always deployed alongside other mitigations that make reuse harder.
| Mitigation | What it prevents | Weakness on its own |
|---|---|---|
| Executable space protection | Running injected code from data pages | Bypassed by code reuse |
| ASLR | Predicting addresses of code and data | Defeated by information leaks |
| Stack canaries | Silent overwrite of return addresses | Do not protect all corruption |
| Control-flow integrity | Jumps to unintended targets | Coverage depends on the compiler |
| Shadow stacks | Tampered return addresses | Need recent hardware support |
The combination of executable space protection and ASLR is particularly strong: the attacker cannot inject code and also does not know where existing code lives, which forces them to find an additional memory disclosure vulnerability before an exploit can succeed.
Practical considerations for developers and administrators
Most software is unaffected by executable space protection, but programs that generate code at run time must handle memory carefully. Just-in-time compilers in browsers, script engines and emulators allocate memory, write instructions into it and then switch it to executable, ideally never holding both permissions at once.
| Situation | Correct handling | Reason |
|---|---|---|
| JIT compiler emits code | Write first, then switch to execute | Preserves W^X |
| Old program crashes under DEP | Update it or add a narrow exception | Avoids weakening the whole system |
| Linux binary requests executable stack | Rebuild without that flag | Removes an unnecessary risk |
| Server hardening baseline | Set DEP to AlwaysOn | No program can opt out |
Limitations and common misconceptions
A frequent misconception is that executable space protection prevents memory corruption. It does not: the overflow still happens, and attackers can still corrupt data or hijack control flow. The protection only removes one convenient way of turning corruption into arbitrary code execution.
- Code reuse attacks such as return-oriented programming bypass it by design.
- Writable and executable mappings created by careless JIT code reopen the gap.
- Disabling DEP for compatibility removes the protection for that process entirely.
- Kernel code needs its own enforcement, which modern kernels provide separately.
- Data-only attacks that change variables rather than code are unaffected.
In plain words
Think of a computer’s memory as a building with two kinds of rooms: offices where instructions are followed, and storage rooms where papers are kept. Executable space protection is the rule that notes in a storage room are never treated as orders.
An attacker who sneaks a harmful note into storage cannot make anyone follow it, because the building only takes instructions from the offices. Attackers can still try to rearrange existing instructions, so the rule works best together with other protections.
A real-world example
A company runs an old document viewer on its 64-bit Windows workstations. A malicious file exploits a buffer overflow in the viewer, writes attack code onto the stack and redirects execution to it.
Because the viewer runs as a 64-bit process, DEP is always enforced. The processor refuses to execute the stack page, Windows raises an access violation and terminates the viewer. The user sees the program close unexpectedly, and the attempt is recorded in the event log.
The security team identifies the vulnerable viewer from the crash report and replaces it with a patched version. The outcome is a failed exploit with no compromise, although the team notes that a more advanced attack using code reuse would have needed additional defenses such as ASLR and control-flow integrity.