New Tool Brings Fuzzing Approach to Memory System Security
Heap allocators manage one of the most common types of memory. Georgia Tech researchers have created an automated tool that reveals how exploitable they are, though.
Unlike other heap exploitation techniques that require considerable effort from the researcher, ArcHeap can autonomously explore the system.
“Many heap exploitation techniques have been discovered by researchers; however, this task always relies on manual efforts,” said School of Computer Science (SCS) Ph.D. student Insu Yun. “We wanted to automate this process.”
[RELATED CONTENT: Team IDs Real-world Vulnerabilities In Popular Browser During Premier Hackathon]
Heap exploitation techniques
Heap is dynamically allocated memory, or memory that’s size is determined during program execution. Heap allocators manage it efficiently, yet they are also very susceptible to attack.
Exploitation techniques abuse underlying heap allocator mechanisms to exploit vulnerabilities. Popular systems software is plagued by heap-related vulnerabilities. Microsoft said heap vulnerabilities led to more than half of their security problems in 2017. Heap vulnerability attacks have also been seen in popular software such as WhatsApp, VMware, and Eximail in 2019.
Since each exploit is specific to the allocator, previous research was specialized and manually done. This created a barrier to understanding security issues with various heap allocators and led to even more attacks.
How ArcHeap works
ArcHeap introduces operations and attack capabilities to see if these trigger an exploitation. The approach is comparable to fuzzing, an automatic software testing technique that inputs random data to expose vulnerabilities.
“We found that fuzzing is very useful for finding software vulnerabilities, so we extended this idea into discovering heap exploitation techniques,” Yun said. “However, classical fuzzing cannot be naively applied to this new problem, so ArcHeap employs several new ideas.”
The researchers determined heap allocators share common design components that allowed them to abstract enough so that the tool can be applied to any allocator. ArcHeap also synthesizes its finding as it moves through the allocator to reduce redundancies.
Researchers evaluated ArcHeap on 11 allocators and found five new exploitation techniques in Linux's default allocator, ptmalloc2. Despite decades of research in this area, ArcHeap successfully discovered heap exploitation techniques in ptmalloc2.
“Our results show that their manual security analysis was insufficient to cover a large space of heap exploitation techniques,” Yun said. “As a result of this insufficient manual testing, these allocators were actually not secure as their claims.”
They also found vulnerabilities in seven of the 10 other popular allocators.
Although right now ArcHeap can determine if a heap allocator is vulnerable or not, the researchers hope to put a quantitative value on that security in the future.
They presented the research in the paper, Automatic Techniques to Systematically Discover New Heap Exploitation Primitives, at Usenix’s 29th Security Symposium from August 12 to 14. Yun co-wrote the paper with SCS Associate Professor Taesoo Kim and Facebook’s Dhaval Kapil.