Bluetooth Low Energy

---

Warning: The information on this page refers to version 1.1.0 (file format 1.7) onwards. Bluetooth Low Energy was not available on earlier versions of phyphox.

---

Phyphox can communicate with almost any Bluetooth Low Energy (BLE) device to receive and send data. It comes with several prepared experiment configurations, which you can use without any additional knowledge. On top you can add support for additional devices through our editor or by directly creating/editing a phyphox-file with a text editor. If you are just looking for devices with existing support, you can check out our Bluetooth device database instead.

This page describes how the Bluetooth Low Energy integration works in phyphox files, which applies to both, the editor and direct editing of the phyphox-file. It also explains in detail the syntax in the phyphox file and the protocol how a device can offer its own phyphox-file to the app, so you can for example program an Arduino so it can be used without the necessity to manually transfer a phyphox-file.

If this is the first time that you use the editor, you should start with our introduction to our editor and if you just started to learn about our file format (without using the editor), you should start on the main wiki page about our file format. Also, if you are new to Bluetooth Low Energy, you should look for a short introduction, so you know what the terms "GATT", "UUID", "service", "characteristic" and "notification" mean.

In order to write a phyphox-file for a BLE device, you will need to know how its communication works. If the device is supported by phyphox out of the box, you can open the existing phyphox-file in our editor to see how we communicate with it. If this is not the case and you try to communicate with the device for the first time, you can often use some documentation by the manufacturer or some similar source of information. Finally, you can try to figure out the communication yourself by using apps that list the services and characteristics and capturing the communication from other apps (please make sure that you are legally allowed to do so).

If you create support for a device which is of interest to a wider audience (or need help to do so), please contact us as we might want to include it with the next phyphox version (with concent by the manufacturer).

Also, if you want to add support for your own sensors (for example Arduino-based) you can find an example on the page on Micro-controller based sensors.

BLE integration in phyphox

Matching BLE devices

There are two methods to access a BLE device from the user interface.

The first one is the one that is probably most familiar to many users. If you tap the plus button on the main screen (bottom right corner on Android, top right on iOS) you can select the option "Bluetooth" which initiates a scan for such devices. This lists every device, but will show most of them in grey unless phyphox has a matching experiment in its collection. A matching experiment needs to be included with phyphox and cannot be achieved by transferring a phyphox-file. The only exception is if the device brings its own phyphox-file, which is a rather advanced method for your own hardware, described below. Also note that these experiments will only work with a single BLE device as you first scan for one device and pick matching experiments afterwards.

The other method works just like opening any other custom eperiment in phyphox. You transfer your file to the phone (as a .phyphox-file, in a zip file or through a QR code) or pick an experiment from the list on the main screen (after you have chosen to add the transfered file or an included BLE experiment to the collection). The experiment is opened in phyphox and it will try to connect to all devices required for the experiment, which in many cases is a single device but you could also use many different devices in the same experiment.

To find a matching device, the following criteria apply:

1. If the experiment configuration defines a specific Bluetooth adress, phyphox directly tries to connect to it without bothering the user. If this fails, you cannot start the experiment.
2. If the name (or a part of a name) of a devices is specified and a matching device has been paired with your phone, phyphox will also directly try to connect to it without asking. Again, if this fails, you cannot start the experiment.
3. If neither an address is specificied nor a name that matches a paired device, phyphox will start to scan for devices and list any device that has a matching name or that advertises a matching service UUID, if a name or service is specified in the phyphox-file

BLE as input or output

BLE devices can be either used as an input to phyphox or as an output to receive data acquired by phyphox.

When used as an input, the characteristics are read from the device, their data will be interpreted and then written to a phyphox buffer. When used as an output, the values from a phyphox buffer are converted to a binary representation and then written to a characteristic on the device.

Notifications and polling

Most BLE devices support notifications, which means that phyphox can subscribe to a characteristic and gets notified if the device offers a new value. This is the most efficient way to retrieve data and should be used when using a BLE device as input (it does not apply to writing to the device).

However, when the device does not support notifications or if you want values at a fixed rate (i.e. not having "gaps" if there is no new value for a long time or not collecting many data points if only few are sufficient), the mode of the input device can be switched to "poll". In this case you also need to define a rate at which values are being read from the device.

Phyphox also supports indications, which are similar to notifications, but most devices typically do not support them for simple data values.

Accessing characteristics as input, output or config

When a BLE device is used as input, it needs to define its outputs to the phyphox buffers. Therefore, a BLE device that is an input to phyphox, sets up buffers to which it outputs its data. Each of these output buffers needs to be associated with a characteristic and multiple output buffers may be associated with the same characteristic. Each time the characteristic is read (through polling or because of a notification), the same value is fed to all buffers, which then interpret the data differently according to a conversion function which needs to be set as well (the conversion functions are listed below). Many conversion functions offer to set an offset and/or a size, so they only interpret a part of the incoming data. This way different values within a single characteristic can be split to different buffers.

Output devices work similar, but instead of defining output buffers, they define input buffers, which also use slightly different conversion functions.

In case of an output buffer (for an input device) the conversion function can be left out if instead the "extra" field is set to "time", in which case the output buffer does not recieve values from the characteristic, but the time (in seconds since the start of the experiment) at which it has been read.

Both, input and output devices, can also define config settings for a characteristic. These do not associate the characteristic with a buffer but with a constant value. This value will be written to the device once after a connection has been established. This is usually used for configuration like enabling a sensor or setting a data acquisition rate.

BLE-related syntax in phyphox-files

BLE device matching

Whenever a BLE device is defined in phyphox, you will need to set up how it is matched against scan results. You can either define the address of a specific device (note that it then will only work with this single device), a text that is compared to the name of the device or the UUID of an advertised service. (See above)

... <bluetooth name="..." uuid="..." id="..."> ... </bluetooth> ...

name

If this name is contained in the name of a device found while scanning, the device matches the criterium.

uuid

This is the UUID of a service advertised by the device. Note that even if the device advertises a short 16-bit UUID according to the Bluetooth specification (for example 0x1812 for a Human Interface Device service), you need to specify the 128-bit version (in this case 00001812-0000-1000-8000-00805f9b34fb) using the base-UUID 0000xxxx-0000-1000-8000-00805f9b34fb.

id

The id entry combines two functions to manage multiple BLE devices in one experiment. It acts as a string shown to the user to identify the use of a device during scanning (for example, if two sensor boxes are used, the could be labeled as "wheel" and "distance" if one is attached to a wheel and the other one measures distance). For this purpose the text is translatable. The second function is that phyphox treats devices with the same ID as a single device, so multiple bluetooth entries (even input and output blocks) can refer to the same device without asking the user twice to pick it from a scan.

BLE as input

If data should be read from a BLE device and displayed in phyphox, the device should be defined within the input block of the phyphox file:

    ...
    <input>
        <bluetooth name="..." uuid="..." id="..." mode="..." rate="..." subscribeOnStart="...">
            ...
            <config ...>...</config>
            ...
            <output ...>...</output>
            ...
        </bluetooth>
    </input>
    ...

When using the "bluetooth" tag within an "input" tag, values are read from the device and written to buffers specified in output tags (more on these below). Additionally, "config" tags can be used to set up the device by writing configuration values to the device after connecting.

In addition to the device matching tags "name", "address" and "uuid" (see above), there are tags to define how and when data is read from the device:

mode

Mode can have on of three values:

poll

phyphox requests new values at a rate given by the attribute "rate".

notification

New values are received when the device notifies that there is a new value. The characteristic needs to support notifications. For most devices, this mode should give the best results.

indication

New values are received when the device indicates that there is a new value. The characteristic needs to support indication, which is very uncommon for simple values. In most cases you probably want to use notifications instead.

rate

This attribute is only used when the mode is set to "poll" and gives the rate in Hz at which new values are polled.

subscribeOnStart

Usually, phyphox subscribes to notifications as soon as a connection to the device has been established. This minimizes latencies when starting the measurement. However, some devices should react to a subscription by starting to send buffered data or triggering something in the experiment, so if this attribute is set to true, phyphox will subscribe to notifications only when the measurement is started. true or false, default: false

BLE as output

If data should be read from a BLE device and displayed in phyphox, the device should be defined within the output block of the phyphox file:

    ...
    <output>
        <bluetooth name="..." uuid="..." id="..." >
            ...
            <config ...>...</config>
            ...
            <input ...>...</output>
            ...
        </bluetooth>
    </output>
    ...

In contrast to using the tag "bluetooth" within an input tag, you cannot specify a mode or a poll rate. The values are read from buffers defined in input tags and written to the device when the analysis process of the experiment has finished. Config tags work as usual.

config, input and output block

There are three different tags within a bluetooth block:

config

The config tag writes a configuration value to the device after the connection has been established. This is usually used to enable a specific sensor, set a data acquisition rate or set a measurement mode. The config tag always takes a constant value.

input

An input tag can only be used in an output block. The name "output" of the outer block refers to phyphox as it outputs data from phyphox, while the name "input" of the inner block refers to the BLE device, as it is the input to the device. This defines a buffer from which data is written to the device.

output

Output tags can only be used in an input block. The name "input" of the outer block refers to phyphox as it represents a data input for phyphox, while the name "output" of the inner block refers to the BLE device, as it is the output of the device. This defines a buffer to which data is written from the device.

All three types have to be connected to the UUID of a characteristic on the BLE device through its UUID. This is defined through the "char" attribute. On top of this, a range of conversion functions are available, which need to be defined through the attribute "conversion". These are different for input, output and config tags and may imply additional attributes. They are discussed separately below.

Here are some examples:

This sets the a HID mouse to BIOS mode by writing 0x00 to the characteristic 0x2a4e after connecting to the device:

 <config char="00002A4e-0000-1000-8000-00805F9B34FB" conversion="hexadecimal">00</config>

These two outputs read from the same characteristic but use different conversion functions (or rather different parameters to the same function) to get x and y movement of the mouse and write it to the buffers dx and dy:

 <output char="00002A33-0000-1000-8000-00805F9B34FB" offset="1" length="1" conversion="singleByte">dx</output>
 <output char="00002A33-0000-1000-8000-00805F9B34FB" offset="2" length="1" conversion="singleByte">dy</output>

The following line defines buffer "char" as an input, so its values are written to the LED character output of a BBC Micro:bit.

 <input char="E95D93EE-251D-470A-A062-FA1922DFA9A8" conversion="singleByte">char</input>

config conversion functions

Config tags take a constant string and convert it to a binary value to be written to a characteristic. Currently, the following conversion functions are defined:

string

The string is interpreted as a string and written directly to the characteristic

hexadecimal

The string is a hexadecimal representation of the binary value that should be written to the characteristic. (Note that there is no trailing "0x", so 0x1f is simply written as 1f.)

singleByte

The string is parsed as a decimal number and encoded as a single byte value (same as uInt8)

int8

The string is parsed as a decimal number and encoded as a signed 8bit integer

uInt8

The string is parsed as a decimal number and encoded as an unsigned 8bit integer

int16LittleEndian

The string is parsed as a decimal number and encoded as a signed 16bit integer with little endian byte order (that is starting with the LSB)

uInt16LittleEndian

The string is parsed as a decimal number and encoded as an unsigned 16bit integer with little endian byte order (that is starting with the LSB)

int16BigEndian

The string is parsed as a decimal number and encoded as a signed 16bit integer with big endian byte order (that is starting with the MSB)

uInt16BigEndian

The string is parsed as a decimal number and encoded as an unsigned 16bit integer with big endian byte order (that is starting with the MSB)

int24LittleEndian

The string is parsed as a decimal number and encoded as a signed 24bit integer with little endian byte order (that is starting with the LSB)

uInt24LittleEndian

The string is parsed as a decimal number and encoded as an unsigned 24bit integer with little endian byte order (that is starting with the LSB)

int24BigEndian

The string is parsed as a decimal number and encoded as a signed 24bit integer with big endian byte order (that is starting with the MSB)

uInt24BigEndian

The string is parsed as a decimal number and encoded as an unsigned 24bit integer with big endian byte order (that is starting with the MSB)

int32LittleEndian

The string is parsed as a decimal number and encoded as a signed 32bit integer with little endian byte order (that is starting with the LSB)

uInt32LittleEndian

The string is parsed as a decimal number and encoded as an unsigned 32bit integer with little endian byte order (that is starting with the LSB)

int32BigEndian

The string is parsed as a decimal number and encoded as a signed 32bit integer with big endian byte order (that is starting with the MSB)

uInt32BigEndian

The string is parsed as a decimal number and encoded as an unsigned 32bit integer with big endian byte order (that is starting with the MSB)

float32LittleEndian

The string is parsed as a decimal number and encoded as a 32bit floating point number with little endian byte order (that is starting with the LSB)

float64LittleEndian

The string is parsed as a decimal number and encoded as a 64bit floating point number with little endian byte order (that is starting with the LSB)

float32BigEndian

The string is parsed as a decimal number and encoded as a 32bit floating point number with big endian byte order (that is starting with the MSB)

float64BigEndian

The string is parsed as a decimal number and encoded as a 64bit floating point number with big endian byte order (that is starting with the MSB)

output conversion functions

The input tags in an output block take a number from a phyphox buffer and convert it to a binary value to be written to a characteristic. Some functions can also take the whole content of the buffer instead of a single value. Currently, the following conversion functions are defined:

string

The value from the buffer is converted to a string representation of a decimal number

byteArray

All values from the buffer are taken, individually converted to bytes and then written as sequence of bytes to the characteristic

singleByte

The value from the buffer is converted to a decimal number and encoded as a single byte value (same as uInt8)

int8

The value from the buffer is converted to a decimal number and encoded as a signed 8bit integer

uInt8

The value from the buffer is converted to a decimal number and encoded as an unsigned 8bit integer

int16LittleEndian

The value from the buffer is converted to a decimal number and encoded as a signed 16bit integer with little endian byte order (that is starting with the LSB)

uInt16LittleEndian

The value from the buffer is converted to a decimal number and encoded as an unsigned 16bit integer with little endian byte order (that is starting with the LSB)

int16BigEndian

The value from the buffer is converted to a decimal number and encoded as a signed 16bit integer with big endian byte order (that is starting with the MSB)

uInt16BigEndian

The value from the buffer is converted to a decimal number and encoded as an unsigned 16bit integer with big endian byte order (that is starting with the MSB)

int24LittleEndian

The value from the buffer is converted to a decimal number and encoded as a signed 24bit integer with little endian byte order (that is starting with the LSB)

uInt24LittleEndian

The value from the buffer is converted to a decimal number and encoded as an unsigned 24bit integer with little endian byte order (that is starting with the LSB)

int24BigEndian

The value from the buffer is converted to a decimal number and encoded as a signed 24bit integer with big endian byte order (that is starting with the MSB)

uInt24BigEndian

The value from the buffer is converted to a decimal number and encoded as an unsigned 24bit integer with big endian byte order (that is starting with the MSB)

int32LittleEndian

The value from the buffer is converted to a decimal number and encoded as a signed 32bit integer with little endian byte order (that is starting with the LSB)

uInt32LittleEndian

The value from the buffer is converted to a decimal number and encoded as an unsigned 32bit integer with little endian byte order (that is starting with the LSB)

int32BigEndian

The value from the buffer is converted to a decimal number and encoded as a signed 32bit integer with big endian byte order (that is starting with the MSB)

uInt32BigEndian

The value from the buffer is converted to a decimal number and encoded as an unsigned 32bit integer with big endian byte order (that is starting with the MSB)

float32LittleEndian

The value from the buffer is converted to a decimal number and encoded as a 32bit floating point number with little endian byte order (that is starting with the LSB)

float64LittleEndian

The value from the buffer is converted to a decimal number and encoded as a 64bit floating point number with little endian byte order (that is starting with the LSB)

float32BigEndian

The value from the buffer is converted to a decimal number and encoded as a 32bit floating point number with big endian byte order (that is starting with the MSB)

float64BigEndian

The value from the buffer is converted to a decimal number and encoded as a 64bit floating point number with big endian byte order (that is starting with the MSB)

input conversion functions

The output tags in an input block take a byte sequence from the characteristic and convert it to a number to be written to a phyphox buffer. Most conversion functions (unless noted otherwise) can use the attributes "offset" and "size" to define a subsequence of bytes. The start is defined by "offset" (in bytes, starting at zero) and the length is set by "size" (in bytes).

Instead of defining a conversion function, you can also set the attribute "extra" to "time" (extra="time"). This means that the associated buffer does not receive the value from the characteristic, but the time at which the characteristic has been read in seconds since the beginning of the experiment.

Currently, the following conversion functions are defined:

string

The value from the characteristic parsed as a decimal number

formattedString

This function always parses the whole data sequence from the characteristic as a single string (i.e. there is no "offset" or "size" attribute) and tries to split it into several values at the sequence set by the "separator" attribute. Each output tag can then specify which chunk corresponds to the buffer by either setting the "index" attribute, which gives the n-th entry (starting at zero), or by setting the "label" attribute, which is another character sequence which prepends the entry. Each entry is then parsed as a decimal number (labels are stripped before that happens).

Examples:

A simple comma-separated value string

42;23

An output tag with separator=";" and index="0" returns 42 and a second output tag with separator=";" and index="1" returns 23.

A string with simple labels:

U42;I23

An output tag with separator=";" and label="U" returns 42, an output tag with separator=";" and label="I" returns 23.

A string from the characteristic with a line black and labels:

U = 42

I = 23

An output tag with separator="\n" and label="U = " returns 42 and an output tag with separator="\n" and label="I = " returns 23.

singleByte

The data from the characteristic is interpreted as a single byte value (same as uInt8)

int8

The data from the characteristic is interpreted as a signed 8bit integer

uInt8

The data from the characteristic is interpreted as an unsigned 8bit integer

int16LittleEndian

The data from the characteristic is interpreted as a signed 16bit integer with little endian byte order (that is starting with the LSB)

uInt16LittleEndian

The data from the characteristic is interpreted as an unsigned 16bit integer with little endian byte order (that is starting with the LSB)

int16BigEndian

The data from the characteristic is interpreted as a signed 16bit integer with big endian byte order (that is starting with the MSB)

uInt16BigEndian

The data from the characteristic is interpreted as an unsigned 16bit integer with big endian byte order (that is starting with the MSB)

int24LittleEndian

The data from the characteristic is interpreted as a signed 24bit integer with little endian byte order (that is starting with the LSB)

uInt24LittleEndian

The data from the characteristic is interpreted as an unsigned 24bit integer with little endian byte order (that is starting with the LSB)

int24BigEndian

The data from the characteristic is interpreted as a signed 24bit integer with big endian byte order (that is starting with the MSB)

uInt24BigEndian

The data from the characteristic is interpreted as an unsigned 24bit integer with big endian byte order (that is starting with the MSB)

int32LittleEndian

The data from the characteristic is interpreted as a signed 32bit integer with little endian byte order (that is starting with the LSB)

uInt32LittleEndian

The data from the characteristic is interpreted as an unsigned 32bit integer with little endian byte order (that is starting with the LSB)

int32BigEndian

The data from the characteristic is interpreted as a signed 32bit integer with big endian byte order (that is starting with the MSB)

uInt32BigEndian

The data from the characteristic is interpreted as an unsigned 32bit integer with big endian byte order (that is starting with the MSB)

float32LittleEndian

The data from the characteristic is interpreted as a 32bit floating point number with little endian byte order (that is starting with the LSB)

float64LittleEndian

The data from the characteristic is interpreted as a 64bit floating point number with little endian byte order (that is starting with the LSB)

float32BigEndian

The data from the characteristic is interpreted as a 32bit floating point number with big endian byte order (that is starting with the MSB)

float64BigEndian

The data from the characteristic is interpreted as a 64bit floating point number with big endian byte order (that is starting with the MSB)

Sending phyphox-files from a device

If you create your own device (for example an Arduino-based sensor with a BLE module), you can also store experiments on the device and send them to phyphox.

To do so, your device needs to advertise and implement the phyphox service, which uses the UUID cddf0001-30f7-4671-8b43-5e40ba53514a. When the user picks the Bluetooth scan from the "+" button on the main screen, phyphox will list devices that advertise this UUID in white like any other device for which it has an internal phyphox-file.

When connecting to this device, phyphox will look for a characteristic with the UUID cddf0002-30f7-4671-8b43-5e40ba53514a and subscribe to its notifications. Your device needs to react to this subscription by starting to send the experiment file (preferably in a zip file) through this characteristic.

Sending the experiment file starts with a header. So the first package you send needs to consist of (at least 15 bytes):

phyphox[size][crc32]

The first seven bytes need to contain the keyword "phyphox", followed by four bytes for an unsigned 32-bit big endian integer giving the size of the experiment file (not including this header) and the last four bytes must give the CRC32 check value of the whole experiment file (again, not including this header). Remaining bytes in the first message are ignored and can be set to zero.

The experiment is transmitted sequentially in subsequent messages. If there is space left in the final message, it can simply be filled with zeros as phyphox ignores any bytes after the size set in the header has been reached.