Author: Sukhman Singh

This post is the final post in the continuation of this blog series. In part 2, I discussed the theory behind Qaul BLE support for Linux and a walkthrough for creating a BLE GATT server, advertisement of services, and scanning of devices advertising qaul services. We also discussed the theory of message transfer through BLE GATT characteristics. Please refer to that blog before proceeding.

In this blog, we will explore the implementation and low-level design of the BLE support in greater depth.

Correction in the previous Blog

In the previous blog, while creating the GATT server, we defined two services:- “main_service” and “msg_service” with their respective characteristics. The issue here is, that multiple services lead to longer advertisement messages that are only supported by extended advertisements. So, for compatibility, we are using only “main_service“, which encapsulates both characteristics:- “mainChar“, and “msgChar” for normal advertisements. Rest all things are same.

Introduction

We will begin by configuring services and characteristics of the qaul device to send libqaul messages to other devices and spawn a new thread for receiving messages from all nearby devices. For byte-loss transfer, we are enclosing our data inside delimiters “$$” and then breaking it into byte arrays. On the spawned message listener, we receive messages from all nearby qaul devices and separate the messages into a map of type <qaul_id, message>. The message starts at “$$” and ends when ending “$$” is encountered. The message is then sent to libqaul for decoding.

Message receiver

The message is received by “msgChar” when some other device overwrites the value of this characteristic. While configuring msgChar, we created a msg_charcteristic_ctrl which emits an event, CharacteristicControlEvent::Write(write), io_write by characteristic. On accepting the write request into read_buffer which is then sent to ble_message_sender for further processing of bytes. Ble_message_reciever receives the byteArray in the set of 20 and maintains a map of qaul_id v/s message received till now. As soon as ending $$ is encountered, the message is sent to libqaul.

The whole above architecture works by spawning two new threads, one for receiving data from other devices and the other for manipulation of received bytes, for loss-less transfer of bytes.

Message Sender

On discovering a nearby qaul device emitting the same main_service and read_char as any qaul device, the libqaul starts to send routing messages at regular intervals for other devices to get updated on routing tables. All other public or personal messages are also transmitted similarly.

On receiving a “DirectSend” request from libqaul, ble_module adds delimiters(“$$”) to the start and end of the message. Then it breaks the message into arrays of 20 or fewer bytes each and pushes each into a queue. The Gatt client then connects to the specified server and tries to io_write the message into the value of the characteristic of the server which will eventually trigger the Write event of characteristic_ctrl.

Also, while writing data to another device, the GATT client also updates its list for last_found_time for out_of_range_checker.

Result :

We were successful in creating a reliable connection between Android 33+ and Linux devices with nearby device discovery and message transfer using ble_module.

Messages are Successfully sent and received by both devices.

Images for BLE Message from Android to Linux

Message sent by Android.

Message received by Linux.

Images for BLE Message from Linux to Android

Message sent by Linux.

Message received by Android.

Limitations:

The above implementation works well for any Linux device with Bluetooth Adapter version >= 5.0 and Android SDK version >= 33. The absence of any of the above conditions would lead to the loss of some random bytes from the message leading to the failure of this protocol.

Conclusion

The implementation of the Bluetooth Low Energy module using Bluez is limited in extensibility and reliability, especially within the planned architecture. The instability of Linux Bluetooth can cause message loss if conditions aren’t met, indicating a need for further optimization of the BLE module and the underlying protocol to enhance robustness and reliability.

While working on the Qaul project and implementing the ble module, I learned a lot about peer-to-peer communication, routing, and Bluetooth in general.

I would like to express my sincere gratitude to my mentors, Mathias Jud and Breno, for allowing me to participate in GSoC 2024 and for their invaluable guidance throughout this project. I am also grateful to Andi and all the Freifunk members involved with GSoC for making this project possible.

This marks the end of my GSoC’2024 project, but as I mentioned earlier, there is still work to be done. If you have any questions, please feel free to reach out. I hope you found this project as rewarding and enjoyable as I did!

You can refer to PR for reference.

This post is a continuation of my last one, which discussed the theory behind Qaul BLE support for Linux where I explained the theoretical concepts and jargon and high-level system design for Bluetooth low energy flow using GATT. Please refer to that blog before proceeding.

In this blog, we will explore the implementation and low-level design of the BLE support in greater depth.

Introduction

We will begin with creating a GATT server serving two Bluetooth services “main_service” and “msg_service” with their respective UUIDs and characteristics for our qaul device. Then we will use the adapter to advertise the “main_service_uuid” for another qaul device to connect to the GATT server. Afterward, we will create a GATT client by starting a scanning service to discover nearby advertisements. The last thing to do is to set a few functionalities and callbacks like read and write characteristic data methods and request_read and request_write callbacks for characteristics to set a basic GATT prototype with message transfer support.

GATT Server

If you remember the GATT data hierarchy diagram from the last blog shows the GATT server is made up of services and each service is made up of characteristics and each characteristic of descriptors. Here, we will dive into details of how to implement it using BlueR.

GATT Service: GATT services are a set of similar attributes in one common section of the GATT server. For Qaul GATT Server implementation, we have two services – mainService, and msgService and their UUIDs (99xxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxx23). Below is a struct for Service where you specify uuid, characteristics and whether the service is primary or secondary.

GATT Characteristics: Characteristics are the containers for user data. For the Qaul Gatt Server, we have one characteristic for each service ➖ mainChar, msgChar, and their UUIDs with respective configurations. You can specify the uuid, descriptors(if any), read permission, and callbacks under the read attribute, and write permissions and conditions under the write attribute.

Now, you can create a new session to get an instance of your Bluetooth adapter and start the service GATT application using the above configurations. Below is a sample example of a GATT server with read characteristics functionality enabled.

Advertisements and Device discovery

We will use the above instance of the adapter to start advertising the main_service_uuid to be discovered by other devices. (Refer to sample code).

We can start scanning for nearby devices advertising the same UUID for the presence of qaul. For this, BlueR gives us adapter.set_discovery_filter(get_filter()).await; a feature for filter-based UUID scanning. and stream the discovered device adapter.discover_devices().await.

Till now we have created a Ble GATT server and client, the next thing we will implement is confirming the discovered devices for a qaul node, maintaining a list of discovered devices to save the throughput of scanning devices again and again, and keeping track of devices out of range.

For Qaul ble architecture, we have stored the qaul_id in the main_service’s characteristics, Now, whenever any device discovers this node, it will connect to it, check for the presence of both services along with their characteristics, and mark that qaul_id as discoverable. If the conditions are satisfied, it adds the node and its properties to a lookup map. To keep track of the device’s availability after discovery, Qaul spawns a new task out_of_range_checker to ping recently discovered devices at regular intervals to confirm their presence.

Any message to be sent can be written into msg_service’s characteristics using CharacteristicWriteMethod::Io and receive any write request using CharacteristicControlEvent::Write(write). The message read and write process is a bit more complex than function calls. We will discuss it in more depth in the next blog.

So, in the current blog, we are able to create a GATT server and a GATT client, advertise our UUIDs, and scan for nearby BLE devices. We also learned to identify the correct device and maintain the lookup table for better connectivity and more stability.

Concluding Thoughts

The first half of the GSoC coding period had more code implementation for ble connectivity. The major part was into integrating the prototype to work with async rust on multiple threads. The next half will be implementing loss-less transmission of data and experimenting with bluer for better stability. If you want to know more about my project, please feel free to reach out. Thanks for reading!