Making Android BLE work — part 1

Last year I learned how to develop Bluetooth Low Energy (BLE) applications for iOS and using BLE on iOS seemed very straightforward. Next up was porting it to Android…how difficult could it be?

I can now safely say that it was much harder than expected and it took considerable effort to get a stable working version for Android that worked well on the most popular Android phones. I used a lot of freely available information from the community out there; some of it turned out to be wrong but a lot of it was extremely useful and helped a lot! In this series of articles, I want to summarize my findings, so you don’t have to spend hours and hours googling for information like I did…

So why is BLE on Android so hard?

In hindsight, I would say that BLE on Android is hard because of the following root causes:

• Google’s documentation on BLE is very basic, and in some cases lacks crucial information or is partially out of date. The examples applications are also not exactly showing you how to do BLE properly. There are only a few sources that tell you how to do it properly. Stuart Kent’s presentation is definitely a good place to start if you are new to BLE on Android. For some advanced topics, the Nordic’s guide is also very good!
• The Android BLE api’s are a bit low-level on Android. Any real world app needs to have some layers on top of it so that it becomes easier to use, similar to what Apple has done with CoreBluetooth. Typically such layers take care of command queuing, bonding, connection management, bug workarounds, threading issues, and simply bringing it all to a higher abstraction level. On Android, the most well-known libraries are iDevices’ SweetBlue, Polidea’s RxAndroidBle and the Nordic’s BLE library. If you want start simple, take the Nordic one or the Blessed library I wrote ;-)
• Vendors make changes to the default Android BLE stack or have replaced it with their own implementations. There are some differences in behavior to take into account when developing your app. What works fine on one phone may not work on other phones! It is not all bad though since some manufacturers like Samsung actually made it better than Google’s own implementation in some aspects!
• There are several known (and unknown) bugs in the Android code that need to be handled where possible, especially in Android 4,5 & 6. Later versions are definitely a lot better but there are still issues that don’t have a proper workaround or solution. For example the mysterious occasional connection failures that return error 133! More about that later…

I am certainly not claiming to have solved all issues but I managed to get to an ‘acceptable level’. Let’s talk about what I learned….starting with the topic of scanning!

note that in this article we assume a minimal version of Android 6

Scanning for devices

In order to connect to a BLE device you must first scan for it. You do this with the BluetoothLeScanner object you can obtain from the BluetoothAdaptor. To start the scan you call startScan and you must provide it with the filters you want to use, the scanSettings you desire and a scanCallback object to receive found devices on.

BluetoothAdapter adapter = BluetoothAdapter.getDefaultAdapter();
BluetoothLeScanner scanner = adapter.getBluetoothLeScanner();

if (scanner != null) {
    scanner.startScan(filters, scanSettings, scanCallback);
    Log.d(TAG, "scan started");
}  else {
    Log.e(TAG, "could not get scanner object");
}

The scanner will now start looking for devices that match your filters and when it finds one, it will call your scanCallback

private final ScanCallback scanCallback = new ScanCallback() {
    @Override
    public void onScanResult(int callbackType, ScanResult result) {
        BluetoothDevice device = result.getDevice();
        // ...do whatever you want with this found device
    }

    @Override
    public void onBatchScanResults(List<ScanResult> results) {
        // Ignore for now
    }

    @Override
    public void onScanFailed(int errorCode) {
        // Ignore for now
    }
};

When you receive a ScanResult you can get the BluetoothDevice object from it. You can then use it to start a connection to the device. But before we move to connecting to devices lets discuss scanning a bit more.

Besides the BluetoothDevice object, the ScanResult object contains several other useful pieces of information about the device:

• Its advertisement data. A byte array with data relevant to the device. Sometime this even contains actual data like ‘temperature’. However, on most devices it just contains the ‘name’ and ‘service UUIDs’ of the device. If you scan for devices with a specific name or UUID, this is what the Android stack will look at to determine a match.
• The RSSI value (signal strength). This can be used to determine how nearby the device is.
• …and some more technical information about the device. See Google’s documentation for more info.

Perhaps needless to mention, but make sure you do all your Bluetooth stuff outside of an Activity. Activities get created and recreated many times by Android, so if you do your scanning in an Activity the scan may be started several times. Or worse, you connection may break because Android decided to recreate your Activity…you have been warned!

Setting up scan filters

The first thing you want to do is to set up some filters. If you don’t set up filters (simply pass a null value) you will receive all devices that are advertising around you. That might be sometimes what you want, but often you are looking for devices with a specific name or devices that advertise specific services.

Scanning for devices with a specific service UUID

Filtering on service UUIDs is useful if you are looking for devices from a ‘category’ of devices. For example: blood pressure monitors that use the standard ‘Blood Pressure Service’ with UUID 1810. When a device advertises itself it may advertise a service UUID that it has and which ‘explains’ what this device is primarily about. The device will have several other services as well but you will have to connect to it first to find out. Even though the device lists a specific service UUID in the advertisement data it is not guaranteed that the devices actually has that service! Most cases it will of course, but I have also seen some manufacturers that use fake service UUIDs to broadcast the state the device is in…creative use of service UUIDs!

Here is an example showing how to scan for a blood pressure service:

UUID BLP_SERVICE_UUID = UUID.fromString("00001810-0000-1000-8000-00805f9b34fb");
UUID[] serviceUUIDs = new UUID[]{BLP_SERVICE_UUID};
List<ScanFilter> filters = null;
if(serviceUUIDs != null) {
    filters = new ArrayList<>();
    for (UUID serviceUUID : serviceUUIDs) {
        ScanFilter filter = new ScanFilter.Builder()
                .setServiceUuid(new ParcelUuid(serviceUUID))
                .build();
        filters.add(filter);
    }
}
scanner.startScan(filters, scanSettings, scanCallback);

Note that you’ll often see short UUIDs like 1810, which is called the ‘16bit UUID’. This is equivalent to 00001810–0000–1000–8000–00805f9b34fb, which is the full ‘128bit UUID’. The 16bit UUID is simply only a part of the 128bit UUID. The 16bit UUID assumes a BASE_UUID of 00000000–0000–1000–8000–00805F9B34FB. See the Bluetooth specification for more information

Scanning by device name

Looking for a device by it’s name has 2 main use cases: to look for 1 specific device or to look for 1 specific device model. For example, my Polar H7 chest strap advertises itself as “Polar H7 391BB014”. The latter part (‘391BB014’) is a unique number (or serial number) and the first part is generic for all Polar H7 devices. This is very common practice. Unfortunately the device name scan filter can only be used to find specific devices as it does full string matching. If you want to find all Polar H7 devices you will need to do a ‘substring’ compare on ‘Polar H7', but you can’t do that with a filter. You just need to pass null as the filter and do the substring comparison yourself in onScanResult.

So here is an example of how to scan for devices by exact name:

String[] names = new String[]{"Polar H7 391BB014"};
List<ScanFilter> filters = null;
if(names != null) {
    filters = new ArrayList<>();
    for (String name : names) {
        ScanFilter filter = new ScanFilter.Builder()
                .setDeviceName(name)
                .build();
        filters.add(filter);
    }
}
scanner.startScan(filters, scanSettings, scanCallback);

Scanning by mac address

Filtering on mac address is a bit special. Normally, you don’t know the mac address of you device unless you already scanned for it before! But perhaps the mac address was printed on the box the device came in, which definitely happens in practice, especially for medical devices. However, I think it is safe to say that scanning by mac address is mainly used to reconnect to known devices. There is another way to reconnect to known devices but it some cases you will simply have to scan for the device again; for example when the Bluetooth cache is purged. More about that later.

String[] peripheralAddresses = new String[]{"01:0A:5C:7D:D0:1A"};
// Build filters list
List<ScanFilter> filters = null;
if (peripheralAddresses != null) {
    filters = new ArrayList<>();
    for (String address : peripheralAddresses) {
        ScanFilter filter = new ScanFilter.Builder()
                .setDeviceAddress(address)
                .build();
        filters.add(filter);
    }
}
scanner.startScan(filters, scanSettings, scanByServiceUUIDCallback);

As you probably figured out by now you can also combine UUID filters with address filters and name filters. Technically nice, but in practice I never found a need for it. But perhaps it is useful in your use case?

Defining your ScanSettings

The scan settings are there to tell Android what scanning behavior you desire. There are many settings you can set and here is an example that uses all possible settings. In this setting we want to use ‘low power’ but be very aggressive in finding devices. In other words, it will scan intermittently but when it scans, it matches a device at first sight:

ScanSettings scanSettings = new ScanSettings.Builder()
        .setScanMode(ScanSettings.SCAN_MODE_LOW_POWER)
        .setCallbackType(ScanSettings.CALLBACK_TYPE_ALL_MATCHES)
        .setMatchMode(ScanSettings.MATCH_MODE_AGGRESSIVE)
        .setNumOfMatches(ScanSettings.MATCH_NUM_ONE_ADVERTISEMENT)
        .setReportDelay(0L)
        .build();

Let’s see what they mean:

ScanMode

This is by far the most important setting. This controls when and how long the Bluetooth stack is actually searching for devices. Since scanning is a very power consuming feature you definitively want to have some control in order not to drain your phone’s battery too quickly.

There are 4 modes and this is what they mean according to Nordics guide:

1. SCAN_MODE_LOW_POWER. In this mode, Android scans for 0.5 sec and then pauses 4.5 sec. If you choose this mode, it may take relatively long before a device is found, depending on how often a device sends an advertisement packet. But the good thing of this mode is that is uses very low power so it is ideal for long scanning times.
2. SCAN_MODE_BALANCED. In this mode, Android scans for 2 sec and waits 3 seconds. This is the ‘compromise’ mode. Not sure how useful this is though…
3. SCAN_MODE_LOW_LATENCY. In this mode, Android scans continuously. Obviously this uses the most power, but it will also guarantee the best scanning results. So if you want to find a device very quickly you must use this mode. But don’t use it for long running scans.
4. SCAN_MODE_OPPORTUNISTIC. In this mode, you only get scan results if other apps are scanning! Basically, this means it is totally unpredictable when Android will find your device. It may even not find your device at all…Again, you probably never want to use this in your own apps. The Android stack uses this mode to ‘downgrade’ your scan if you scan too long. More about that later.

Callback type

This setting allows you to control how many times Android will tell you about a device that matches the filters. There are 3 possible settings:

1. CALLBACK_TYPE_ALL_MATCHES. With this setting you will get a callback every time an advertisement packet that matches is received. So in practice, you will get callbacks every 200–500ms depending on how often your device advertises.
2. CALLBACK_TYPE_FIRST_MATCH. With this setting you only get a callback once for device even though subsequent advertisements may be received.
3. CALLBACK_TYPE_MATCH_LOST. This one may sound a bit odd but with this setting you get a callback when no more advertisement packets are found after a first packet was found.

In practice you will typically use either CALLBACK_TYPE_ALL_MATCHES or CALLBACK_TYPE_FIRST_MATCH. There is no right one, it depends on your use case. If you don’t know what to choose take CALLBACK_TYPE_ALL_MATCHES since that will give you more control when receiving callbacks. If you stop the scan after receiving a result you are effectively mimicking CALLBACK_TYPE_FIRST_MATCH.

Match mode

This mode is about how Android will determine there is a ‘match’. Again there are several options to choose from:

1. MATCH_MODE_AGGRESSIVE. Rather than talking about the number of advertisements, this mode tells Android to be ‘aggressive’. This comes down to very few advertisements and even reporting devices with feeble signal strength.
2. MATCH_MODE_STICKY. This is the counterpart of ‘aggressive’ and tells Android to be ‘conservative’ and need more advertisements and higher signal strength.

I didn’t study these parameters in great detail but I mainly use MATCH_MODE_AGGRESSIVE. This will help find devices quickly and I never experience connection issues.

Number of matches

This controls how many advertisements are needed for a match.

1. MATCH_NUM_ONE_ADVERTISEMENT. One advertisement is enough for a match
2. MATCH_NUM_FEW_ADVERTISEMENT. A few advertisements are needed for a match
3. MATCH_NUM_MAX_ADVERTISEMENT. The maximum number of advertisements the hardware can handle per timeframe is needed for a match. Since we don’t know what the hardware limits are exactly we cannot tell how many are needed exactly.

Again, I don’t really see the need for so much low level control. In most cases you’ll want to find you device quickly and hence use one of the first two options.

Report delay

You can specify a delay in milliseconds. If the delay is >0, Android will collect all the scan results it finds and send them in a batch after the delay. So in this case you will not get a callback on onScanResult, but instead Android will call onBatchScanResults. The main use case here is a situation where you expect multiple devices of the same type and want to let the user choose the right one. The only issue there is what information you can supply the end user to make the right decision. It should be more than a mac address because otherwise the user still won’t know which device to choose!

Note that there is a known bug on the Samsung S6 and Samsung S6 Edge where all scan results have the same RSSI value when you use a delay >0.

Caching by the Android Bluetooth stack

Scanning not only gives you a list of BLE devices, but it also allows the bluetooth stack to ‘cache’ devices. The stack will store important information about the devices that were found like the name, mac address, address type (public/random), device type (Classic, Dual, BLE) and so on. Android needs this information in order to connect successfully to a device. It caches all devices it sees in a scan and not only the one you intend to connect to. Per device, a small file is written with the information about each device. When you want to connect to a device the Android stack will look for this file, read the contents and use it to connect. The key message here is that a mac address alone is not enough to connect successfully to a device!

Clearing the cache

Like any cache, it doesn’t live forever and there are certain moments when the cache is being cleared. On Android there are at least 3 moments when the cache is cleared:

1. When you switch Bluetooth off and back on
2. When you reboot/start your phone
3. When you manually clear your cache in your settings menu

This is actually quite a pain for you as a developer. Rebooting a phone happens quite a lot and also turning Bluetooth on/off happens regularly for good reasons like putting your phone in flight-mode. On top of that, there are some differences per device manufacturer to note. On some Samsung phones I tried the cache was not cleared by toggling Bluetooth on off.

This means that you cannot rely on device information being cached by Android. Luckily, it is possible to find out if your device is cached by Android or not! Suppose you try to get a BluetoothDevice object using it’s mac address because you want to reconnect to that device, you can then check the device type to see if it is DEVICE_TYPE_UNKNOWN. If so, it is not cached by Android.

// Get device object for a mac address
BluetoothDevice device = bluetoothAdapter.getRemoteDevice(peripheralAddress)
// Check if the peripheral is cached or not
int deviceType = device.getType();
if(deviceType == BluetoothDevice.DEVICE_TYPE_UNKNOWN) {
    // The peripheral is not cached
} else {
    // The peripheral is cached
}

This information is crucial if you want to connect to a device later without scanning for it. More about that later….

Scanning continuously?

In general, you should not scan continuously because it is a very power consuming operation. Your users value their battery life a lot! If you really need to do it, e.g. because you are always looking for BLE beacons, then choose a low power settings or try to limit when scanning is needed; like only scanning when you app is in the foreground instead of always, or use intermittent scanning.

Recently Google has been taking (undocumented) measures to limit continuous scanning. Here is an overview the recent changes that have been implemented:

• As of Android 8.1, scans without any filters are blocked when the screen is off. So if you are not using any ScanFilters, Android will pause your scan when the screen is off and resume it when the screen is on again. See also these commit comments from Google. Obviously this is another attempt at power saving from Google.
• As of Android 7, you can only scan for 30 min continuously and after that Android changes you ScanSettings to SCAN_MODE_OPPORTUNISTIC. The obvious workaround is of course to scan for periods < 30 mins, then stop the scan and start it again. See this commit in the Android source code.
• As of Android 7, starting and stopping scans more than 5 times in a window of 30 seconds will temporarily disable scanning.

Scanning continuously in the background

Google has made scanning in the foreground already a lot harder, but if you want to scan continuously in the background you will face even more challenges!

The main issue here is that in new versions of Android, Google has limited how long and when background services can run. Typically after 10 minutes, the background service is killed.

Here are some pointers to workarounds if you really want to do this:

• Discussion on Stackoverflow
• David Young’s blog post

Check your permissions and Bluetooth state

Before we end this article I need to mention some things about acquiring the appropriate permissions. Scanning only works if you have the following permission AND you have Location Service activated.

<uses-permission android:name="android.permission.BLUETOOTH" />
<uses-permission android:name="android.permission.BLUETOOTH_ADMIN" />
<uses-permission android:name="android.permission.ACCESS_COARSE_LOCATION" />

So make sure you check if all permission are granted. Just use the normal way to check permissions. If permission is not granted, prompt the user to give the permissions. The permission for ‘ACCESS_COARSE_LOCATION’ is considered ‘dangerous’ by Google and therefor you need the user’s consent:

private boolean hasPermissions() {
    if (Build.VERSION.SDK_INT >= Build.VERSION_CODES.M) {
        if (getApplicationContext().checkSelfPermission(Manifest.permission.ACCESS_COARSE_LOCATION) != PackageManager.PERMISSION_GRANTED) {
            requestPermissions(new String[] { Manifest.permission.ACCESS_COARSE_LOCATION }, ACCESS_COARSE_LOCATION_REQUEST);
            return false;
        }
    }
    return true;
}

Also check if Bluetooth is actually on! If it is not on, use an intent to ask the user to turn it on:

BluetoothAdapter bluetoothAdapter = BluetoothAdapter.getDefaultAdapter();
if (!bluetoothAdapter.isEnabled()) {
    Intent enableBtIntent = new Intent(BluetoothAdapter.ACTION_REQUEST_ENABLE);
    startActivityForResult(enableBtIntent, REQUEST_ENABLE_BT);
}

Next article: connecting and disconnecting

Now that we covered scanning, in the next article we will go in-depth on connecting and disconnecting with devices.

If you want to get started with BLE immediately, try my Blessed library for Android. It uses all the techniques described in this series of articles on BLE on Android and simplifies the use of BLE in your apps!