Security Technologies: Stack Smashing Protection (StackGuard)

This article was originally published on the Red Hat Customer Portal. The information may no longer be current.

In our previous blog, we saw how arbitrary code execution resulting from stack-buffer overflows can be partly mitigated by marking segments of memory as non-executable, a technology known as Execshield. However stack-buffer overflow exploits can still effectively overwrite the function return address, which leads to several interesting exploitation techniques like ret2libc, ret2gets, and ret2plt. With all of these methods, the function return address is overwritten and attacker controlled code is executed when the program control transfers to overwritten address on the stack. 

In 1998 GCC introduced StackGuard, which was successfully used in conjunction with other security hardening technologies to rebuild the Red Hat Linux 7.3 distribution (GCC 2.96-113). The StackGuard patch was also applied to the source for GCC 3.2-7 used in the Red Hat Linux 8 distribution to rebuild both the compiler and GLIBC.

StackGuard basically works by inserting a small value known as a canary between the stack variables (buffers) and the function return address. When a stack-buffer overflows into the function return address, the canary is overwritten. During function return the canary value is checked and if the value has changed the program is terminated. Thus reducing code execution to a mere denial of service attack. The performance cost of inserting and checking the canary is very small for the benefit it brings, and can be reduced further if the compiler detects that no local buffer variables are used by the function so the canary can be safely omitted.

Canaries

There are currently three types of canaries which are supported by StackGuard:

Terminator canaries

Most buffer overflow attacks are based on certain string operations which end at string terminators. A terminator canary contains NULL(0x00), CR (0x0d), LF (0x0a), and EOF (0xff), four characters that should terminate most string operations, rendering the overflow attempt harmless. This prevents attacks using strcpy() and other methods that return upon copying a null character while the undesirable result is that the canary is known.

This type of protection can be bypassed by an attacker overwriting the canary with its known values and the return address with specially-crafted value resulting in a code execution. This can be when non-string functions are used to copy buffers and both the buffer contents and the length of the buffer are attacker controlled.

Random canaries

A random canary is chosen at random at the time the program execs. With this method, the attacker could not learn the canary value prior to the program start by searching the executable image. The random value is taken from /dev/urandom if available, and created by hashing the time of day if /dev/urandom is not supported. This randomness is sufficient to prevent most prediction attempts. If there is an information leak flaw in the application, which can be used to read the canary value, this kind of protection could be bypassed.

random XOR canaries

Random XOR canaries are random canaries that are XOR-scrambled using all or part of the control data (frame pointer + return address etc). In this way, once the canary or the control data is clobbered, the canary value is wrong and it will result in immediate program termination.

How does stackguard actually work

Compilers implement this feature by selecting appropriate functions, storing the stack canary during the function prologue, checking the value in the epilogue, and invoking a failure handler if it was changed. For example consider the following code:

void function1 (const char* str){
        char buffer[16];
        strcpy(buffer, str);
        }

StackGuard automatically converts this code to:

extern uintptr_t __stack_chk_guard;
noreturn void __stack_chk_fail(void);void function1(const char* str){
        uintptr_t canary = __stack_chk_guard;
        char buffer[16];
        strcpy(buffer, str);
        if ( (canary = canary ^ __stack_chk_guard) != 0 )
                __stack_chk_fail();}

Note how the secret value is stored in a global variable (initialized at program load time) and is copied into the stack frame and how it is safely erased from the stack as part of the check, preventing the value being leaked when this stack space is re-used. Since stacks grow downwards on many architectures, in this example the canary gets overwritten whenever input to strcpy is more than 16 characters.

The detection method works because it is impossible to get the correct value via trial and error. Since one incorrect canary value prevents further alterations, an attacker cannot keep trying until the correct value is found. In the example above, if the canary contained a zero byte, it would be impossible for a single strcpy to overwrite it correctly. This forces the attacker to either not attack, or be detected and be unable to alter the stack any further. This does not mean that the buffer cannot be exploited, but it makes exploitation much more difficult, often requiring multiple bugs to be used together. Also, __stack_chk_guard can be stored in various places; some architectures use TLS (Thread-local Storage) data for it.

One heuristic ordering often used, with the stack growing downwards, is first storing the canary, then buffers (that might overflow into each other), and finally all the small variables unaffected by overruns. This is based on the idea that it is generally less dangerous if arrays are modified, compared to variables that hold flags, pointers and function pointers, which much more seriously alter execution. Some compilers randomize the order of stack variables and randomize the stack frame layout, which further complicates determining the right input with the intended malicious effect.

Limitations of StackGuard:

Though StackGuard may be effective in preventing stack-buffer overflow attacks it has certain limitations as well:

• An information disclosure flaw in a different part of the program could disclose the global __stack_chk_guard value. This would allow an attacker to write the correct canary value and overwrite the function return address.

• Not all buffer overflows are on stack. StackGuard cannot prevent heap-based buffer overflows.

• While StackGuard effectively prevents most stack buffer overflows, some out-of-bounds write bugs can allow the attacker to write to the stack frame after the canary, without overwriting the canary value itself.

• If a function has multiple local data structures and pointers to functions, these are allocated on the stack as well, before the canary value. If there is a buffer overflow in any one of these structures, the attacker can use this to overwrite adjacent buffers/pointers which could result in arbitrary code execution. This really depends on the arrangement of data on the stack.

• On some architectures, multi-threaded programs store the reference canary __stack_chk_guard in Thread Local Storage, which is located a few kb after the end of the thread's stack. In these circumstances, a sufficiently large overflow can overwrite both canary and __stack_chk_guard to the same value, causing the detection to incorrectly fail.

• Lastly, for network applications which fork() child processes, there are techniques for brute forcing canary values. This only works in some limited cases though.

The GCC compiler provides various options to control StackGuard implementation during compilation. Despite of some limitations, StackGuard is quite effective in protecting binaries against runtime stack-buffer overflows.