Android AHardwareBuffer Shared Memory over Unix Domain Sockets

This is a detailed write up on how to use the Android NDK AHardwareBuffer API (<android/hardware_buffer.h>) to share memory between two processes. This API was introduced in Android 8.0 (API 26) and little is documented about it or NDK use of Unix Domain Sockets and here is what I found after hacking at it for awhile. I have done this as a fun side project since I have been hacking at the Android NDK for 2 years now while being a university student and wanted to experiment with IPC between Android apps.

TL:DR Here is the code of an example that lets you send a buffer of data between two apps which is used to draw the ANativeWindow_Buffer

Demo of app changing color in other app

Unix Domain Sockets

If you aren’t familiar with Unix Sockets (also known as Berkeley Sockets) I would read more about them first here and here. Basically, a socket in a Unix OS is designed to allow an user to open a communication channel with another process, even on a different computer. To read this Medium article you need to call an HTTP request which uses a TCP socket connection underneath. There are various forms of socket connections and the Unix Socket is the fundamental base element regardless of how its used in the application. There are two commonly used “Unix Domains” from the socket(2) manual: AF_INET and AF_UNIX. The AF_INET is used to communicate with the IP protocol which is for all your internet based applications. But what if you want to send data to another process on the same machine? This is where AF_UNIX comes in.

Unix Domain Sockets on Android

At its core, the AF_UNIX socket creates a socket and returns a file descriptor. This file needs to be accessible to both processes, which in this case is both our apps. Android for security reasons does not let you just start writing and reading from any files and this poses restrictions. I tried using the public external storage directory, but kept getting errors trying to set up my socket on these files. The workaround for this is to use an abstracted namespace for the socket. This allows each process to point to the same file descriptor by using a matching name instead of an actual absolute file location on the device. To achieve this we need to set the first byte in the sun_path value to a null terminator.

// There is a 108 byte limit to sun_path length
#define SOCKET_NAME “SharedSocket”
struct sockaddr_un server;
server.sun_domain = AF_UNIX;
server.sun_path[0] = ‘\0`;
server.sun_path[1] = SOCKET_NAME;

Security Note!
If you plan on having a rooted phone then abstract namespace is a BAD IDEA. A great paper The Misuse of Android Unix Domain Sockets and Security Implications goes into depth how they are able to exploit rooted phones from applications using an abstracted namespace.

Setting Up a Client-Server Model

This is going in a quick overview of the typical Unix Socket client-server model. This might be all old news for some people, don’t worry the AHardwareBuffer detail is right after.

The first thing you need to do is setup a file descriptor reference for both the server and client with the socket() function. The example uses SOCK_STREAM which makes sure the data is sent with a TCP-like protocol (SOCK_DGRAM and SOCK_SEQPACKET are also other options). From here the server needs to bind() the socket to the type it is. This is where we set the AF_UNIX telling the server the socket is a Unix Domain socket and the file path to set it. The server now calls listen() with the number of back buffers to allocate for being able to receive incoming socket request. The server now calls accept() which will block the code until a client sends a request to connect. The accept() function returns the socket used with the AHardwareBuffer function calls. Also note that we run a separate thread for the server because the accept() function blocks the flow of execution. On the client side, after we create a socket() we need to setup the connect() call with the same AF_UNIX and file path as the server’s bind() call. After the a successful connect()/accept() combination, the two sides can send data with calls like write() and read() but we will be using the AHardwareBuffer methods instead.

Sending Shared Memory

The idea is pretty simple, you have a process to create some memory which can be accessed by another process. There are already ways of doing this normally using the shm_open() function and mmap() which involves a lot more code. The Android 8.0 release also gave us a Shared Memory API (<android/sharedmem.h>) which helps with this task since Android will not just let you start calling shm_open() for security risks. The Shared Memory API is designed (to my knowledge) for other NDK API like the NerualNetworks API as seen in this example from ggfan. So even if you use the Shared Memory API to set up some shared memory you will need to find a way to send it over to the other process. The issue is you can’t just send the file descriptor over since a file descriptor is nothing more then an integers that maps a process to a file. This means each process won’t have the same value. Luckily there is a way to use struct msghdr to wrap the file descriptor and send the file descriptor with the sendmsg() call. This is verbose and takes some knowledge to do right and error check, so lucky this where AHardwareBuffer comes in!

AHardwareBuffer to the Rescue

As mentioned above you can write your own code to send the file descriptor over your Unix domain socket, but the NDK team already did for you! This is the source code for AHardwareBuffer_sendHandleToUnixSocket() which we can use to send our shared memory buffer over. I want to point out this was designed with the GPU in mind as given from both the description of AHardwareBuffer and the source code using the internal NDK GraphicBuffer object and the android::hardward::graphics::mapper namespace. One future application was also proposed by Vulkan users who wanted an alternative to GL_TEXTURE_EXTERNAL_OES since you needed to read data from something like a camera to the CPU memory then copy it over to the GPU which a wasteful copy. The idea is to have a shared memory (since the CPU and GPU share memory anyway on mobile SoC) so applications like AR can get the extra speed advantage. The demo I made was designed as an easier way to show the use of AHardwareBuffer

Using AHardwareBuffer

The first thing we need is to allocate some shared memory which you use the AHardwareBuffer_Desc struct to describe the buffer you want. There are various enumerations and options to set it with. Once you have your description we can call

//*** Client App ***//
// AHardwareBuffer_Desc h_buffer_desc;
// AHardwareBuffer* h_buffer;
h_buffer_desc = { /* set struct */ };int ret = AHardwareBuffer_allocate(&h_buffer_desc, &h_buffer)
if (ret != 0) { /* handle error */ }

From here we have now acquired the AHardwareBuffer on creation. The next thing to do is write our data to it from a virtual mapping created with the AHardwareBuffer_lock() call. For this example, I generated a gradient pattern to set it with. For the example we are assuming a controlled access pattern so no need for a fence, but please read AHardwareBuffer_lock() for details about threading considerations. Also don’t forget to AHardwareBuffer_unlock() when done using the memory.

//*** Client App ***//
void* shared_buffer;
int ret = AHardwareBuffer_lock(h_buffer,
AHARDWAREBUFFER_USAGE_CPU_WRITE_MASK,
-1, // no fence in demo
NULL,
&shared_buffer);
if (ret != 0) { /* handle error */ }
// … set data …memcpy((char*)shared_buffer, &some_data, some_data_size);// … unlock …
ret = AHardwareBuffer_unlock(h_buffer, NULL);
if (ret != 0) { /* handle error */ }

Now we are able to use the file descriptor to the Unix socket returned from our connect() call to send the file descriptor over with AHardwareBuffer_sendHandleToUnixSocket()

//*** Client App ***//
int ret = AHardwareBuffer_sendHandleToUnixSocket(h_buffer, data_socket);
if (ret != 0) { /* handle error */ }

From here we switch over to the server side of the model receiving the data. We take the file descriptor from the accept() call in our Unix socket and call AHardwareBuffer_recvHandleFromUnixSocket()

//*** Server App ***//
int ret = AHardwareBuffer_recvHandleFromUnixSocket(data_socket, &h_buffer);
if (ret != 0) { /* handle error */ }

Now we have the AHardwareBuffer* but no knowledge of the data format, width, height, etc. This is where we now call AHardwareBuffer_describe() to get a AHardwareBuffer_Desc struct for the receiving side.

//*** Server App ***//
AHardwareBuffer_describe(h_buffer, &h_buffer_desc);

From here it is just simply a matter of locking the buffer to get the virtual mapping and using our shared buffer! For this example I take the shared memory buffer and directly memcpy() it to the ANativeWindow_Buffer which mapped to the screen being displayed. After it unlocks both buffer the server app (receiving end) will now display the gradient picture.

//*** Server App ***//
int ret = AHardwareBuffer_lock(h_buffer,
AHARDWAREBUFFER_USAGE_CPU_READ_MASK,
-1, // no fence in demo
NULL,
&shared_buffer);
if (ret != 0) { /* handle error */ }
// Copy data somewhere
memcpy(window_buffer, shared_buffer, h_buffer_desc.height * h_buffer_desc.stride * 4);
ret = AHardwareBuffer_unlock(h_buffer, NULL);
if (ret != 0) { /* handle error */ }

Not That Hard After All

I feel that once you see this example you will realize that AHardwareBuffer is actually a really nice API for the NDK. A lot of the hardship of using the NDK is the lack of examples and blogs like this so hopefully someone finds this useful and speeds up their development time. I am no professional at the moment and happy to make any corrections if there are any errors.

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store