Keeping File Descriptors (FD’s) in check on Android

Keeping it in limits

What is a FileDescriptor?

A file descriptor is an abstract indicator used to access a file or other input/output resource, such as a pipe or network socket.

Any I/O connection is channeled via a FileDescriptor object.

In Android, all I/O operations like Files/Sockets/Database connections/network connections/Handlers/Loopers/Resources open a FileDescriptor and keep it open until this connection is explicitly or implicitly closed.

Why should we be concerned?

Each Android app runs in its own sandboxed process. An app can spawn multiple processes, but they are also sandboxed, meaning, 2 processes spawned from the same app would need to use IPC to communicate with each other.

Android allows maximum 1024 FDs open for a single process. Some devices may even lower this threshold to 512. Any process exceeding this limit is immediately killed with:

java.lang.RuntimeException: Too many open files

If we keep file streams open, use too many sockets without closing them, create too many Handlers etc., some or the other day we might run into this problem.

The weird part is, this error can lead to many other seemingly unrelated errors. For example:

android.database.cursorwindowAllocationException:CursorWindow could not create cursorwindow from binder
java.lang.OutOfMemoryError: pthread_create (1040KB stack) failed: Try again

sqlite::Exception: unable to open database file

Many of these errors are non-intuitive and seem remotely impossible to relate/connect with Too many files open error but they are sometimes a direct result of too many FD’s being kept open.

These errors are hard to debug, specially when they are observed in production only. With crash reports only indicating an unrelated error (as shown above), the underlying cause might as well be too many open FDs. It is better to be safe in development stage itself rather than scratching our heads with errors in production.

What can we do?

For starters, let us make it a habit to always close any resource that we use. We should not be dependent on Garbage Collector to recycle/close open FD connections as it may not always work out. Why take any chance when simple habits can prevent it?

• Use Kotlin’s use function to ensure streams are closed after use.
• Use Java’s try-with-resources to ensure streams are closed properly.
• Always disconnect a HttpURLConnection object after network calls are completed.
• Use limited Handler/Looper threads. Each Looper.prepare() opens a FileDescriptor.
• Always keep number of running threads in check. Put an upper limit on threads spawned via ExecutorServices.
• Close WebSockets after use.
• Ensure SQLiteDatabaseHelper connections are singleton objects. Each helper object holds an open FD to the underlying sqlite db file.
• Ensure no memory leaks in any workflow of your app.

Simple process to follow

• Include a test case for open FD limits in your QA checklist. Follow checklists religiously.
• Use the app for some time, run through various work flows of your app.
• After some time, check for open FD count as described below.
• A count of 100–150 is acceptable, anything above 300 is definitely a resource/FD leak.
• Unfortunately, this check has to be done manually since we need root permissions to run shell commands on android device/emulator.

Open FileDescriptors can be checked via simple commands in adb shell

adb shell

su

ls -l /proc/${YOUR_APP_PID}/fd | wc -l

I would have loved to automate this in an automation suite of tests as a TestRule, but unfortunately running these shell commands from an app requires setting up a rooted device or emulator.

Keep those open FD’s in check and stay safe!