LLVM, Clang and getting a new libFuzzer

Security fuzzers

If you’re doing anything security sensitive with C and C++, you must have heard about the American Fuzzy Lop, the most famous of fuzzers. It uses a given corpus of data to feed your program, mutates it a little bit and sees which mutations result in a different code path. Given that your program has assertions compiled in, or if you’re using AddressSanitizer (-fsanitize=address), these mutations tend to quickly result in finding a vulnerable place in your code.

Well, the LLVM project develops its own, a slightly less featured but supposedly faster fuzzing system, the LibFuzzer. The premise is the same: you compile your code with -fsanitizer=fuzzer and it generates an executable, which, given a corpus, repeatedly calls your int LLVMFuzzerTestOneInput(const uint8_t *Data, size_t Size) function, steering it into a crash somehow. It also does it faster, since unlike AFL it does fuzzing of the input data all within a single process. This might come handy if you’re writing all kinds of parsers which don’t spend much processing time to finish a single cycle of processing the input data.

The LibFuzzer project is being developed rather rapidly, so the old instructions don’t work. For example, the runtime support for LibFuzzer is expected to be packaged as libFuzzer.a, libLLVMFuzzer.a, libcompiler-rt.a and some other other names, depending on clang version.

Compiling new clang + libFuzzer from scratch

The most sane way to utilize the LLVM’s fuzzer appears to be to build the LLVM/Clang project and its runtime support from scratch. Below is an automated procedure which worked for me on the most recent macOS Sierra, as of time of publication.

• It expects Homebrew is installed, as it needs to install or update cmake and ninja.
• It uses ninja to steer parallel compilation, because it builds faster and provides a somewhat better progress indication.

Here’s the script to build the recent fuzzer-capable clang:

#!/usr/bin/env bash
set -x
set -e
INSTALL_PREFIX=/usr/local/clang-devel
mkdir -p clang-src
cd clang-src
function brew_install() {
    local pkgname=$1
    echo Ensuring $pkgname is installed
    test -f .has-$pkgname && return
    brew install $pkgname || brew update $pkgname
    touch .has-$pkgname
}
brew_install ninja  # Faster make replacement
brew_install cmake
CLONE_DEPTH="--depth 1"
echo Cloning LLVM/Clang sources
test ! -d llvm && git clone ${CLONE_DEPTH} http://llvm.org/git/llvm.git
test ! -d llvm/tools/clang && \
    (cd llvm/tools && git clone ${CLONE_DEPTH} http://llvm.org/git/clang.git)
test ! -d llvm/projects/libcxx && \
    (cd llvm/projects && git clone ${CLONE_DEPTH} http://llvm.org/git/libcxx.git)
test ! -d compiler-rt && \
    (git clone ${CLONE_DEPTH} http://llvm.org/git/compiler-rt.git)
mkdir -p build
cd build
echo Configuring LLVM/CLang for ninja
cmake -G Ninja                                  \
    -DCMAKE_BUILD_TYPE=Release                  \
    -DCMAKE_INSTALL_PREFIX=${INSTALL_PREFIX}    \
    -DLLVM_TARGETS_TO_BUILD=X86                 \
    --build ../llvm
echo Building LLVM/Clang
ninja
cd ../compiler-rt
# Suppress x86_64h arch
sed -i '' -e 's/set(X86_64 x86_64 x86_64h)/set(X86_64 x86_64)/' cmake/config-ix.cmake
sed -i '' -e 's/set(X86_64 x86_64 x86_64h)/set(X86_64 x86_64)/' cmake/builtin-config-ix.cmake
sed -i '' -e 's/set(DARWIN_osx_ARCHS i386 x86_64 x86_64h)/set(DARWIN_osx_ARCHS i386 x86_64)/' cmake/builtin-config-ix.cmake
HOST_TARGET=$(../build/bin/llvm-config --host-target)
echo Configuring compiler-rt which contains fuzzer runtime.
PATH=../build/bin:$PATH cmake -G Ninja              \
    -DLLVM_CONFIG_PATH=../build/bin/llvm-config     \
    -DCMAKE_BUILD_TYPE=Release                      \
    -DCMAKE_C_COMPILER_TARGET=${HOST_TARGET}        \
    -DCMAKE_C_COMPILER=clang                        \
    -DCMAKE_CXX_FLAGS=-I$(pwd)/../build/include     \
    -DCMAKE_CXX_COMPILER=clang++                    \
    -DCMAKE_INSTALL_PREFIX=/usr/local/clang-devel/lib/clang/6.0.0   \
    -DCOMPILER_RT_ENABLE_IOS=OFF                    \
    -DCOMPILER_RT_DEFAULT_TARGET_ONLY=ON            \
    --build .
# Some weird recursive dependency
sed -i '' -E 's/CUSTOM_COMMAND incl.*\.h$/CUSTOM_COMMAND/' build.ninja
echo Building compiler-rt
ninja
cd ..
echo "Install clang into ${INSTALL_PREFIX}? Enter \"yes\":"
if read install && test "$install" == yes; then
    (cd build && sudo ninja install)
    (cd compiler-rt && sudo ninja install)
else
    echo "Not installing. Execute:"
    echo "   (cd clang-src/build && ninja install)"
    echo "   (cd clang-src/compiler-rt && ninja install)"
    echo "to install into ${INSTALL_PREFIX}"
fi

It might take a couple of hours to compile and finish the installation into /usr/local/bin/clang-devel. Make sure it is in your path if you want to use it right away:

t='export PATH=/usr/local/clang-devel/bin:$PATH'
eval "$t"
echo "$t" >> ~/.bash_profile

Fuzzing example

Now, let’s come up with a short yet working example of a broken program to test our newly compiled fuzzing instrumentation on:

#include <string.h>
int LLVMFuzzerTestOneInput(const uint8_t *Data, size_t Size) {
    /* Whatever broken logic you want to execute on Data... */
    return Data && strlen((const char *)Data) == Size;
}

Compile and run that straight to the crash:

/usr/local/clang-devel/bin/clang -o fuzz fuzz.c \
            -fsanitize=undefined,address,fuzzer
./fuzz

Where undefined and address are forms of sanitizing instrumentation (to figure out if something is wrong exactly when it happens, not when it crashes). And fuzzer is the main guest here, the fuzzer instrumentation itself. You can remove either or both undefined and address sanitizers, or even replace them with memory (which is normally not compatible with address sanitizer anyway).

The resulting crash is going to look somewhat like this:

Outstanding problems

Although this recipe appears to be working, I consider it a rather nasty solution anyway:

1. The clang-6.0 binary works fine with -fsanitize=fuzzer, whereas both clang and clang++, despite being symlinks, don’t support this option. They just fail as if this option is ignored. I couldn’t figure out, why yet. The full path works for /usr/local/clang-devel/bin/clang, but not for …clang++.
2. A useful memory sanitizer (-fsanitize=memory) doesn’t work with this recipe.
3. I failed to beat the compiler-rt library to compile the normal way, when it is expected to reside in the clang-src/llvm/projects directory. So I had to move it out and compile stand-alone.

Tell me if you have a shorter way to build it, or you can help with the most disappointing first item.