Network Connections

Since version 1.1.3 (December 2019) (file format 1.8), phyphox features a versatile interface for network connections. This is done on an experiment configuration level, so to use the network communication for your own project, you need create your own experiment configuration. In it, you can then define, how network services should be discovered, when and which data should be sent, where to put received data and which protocols to use for the communication.

This network interface is designed to be easily extended, so while at the time of this writing not many services (protocols) are supported, you can expect more to come in the future and if you think something relevant is missing, let us know, so we can implement it.

Note, that there is no network service provided by phyphox for your experiments. While we set up our own servers for project like our sensor data base using the interface described here, we can not offer a generic service. If you want to use these features, you need to set up your own server on the receiving end and you to know how to receive the data there (or know someone who knows this). Of course, we are happy to help you and give you insight into how we do this.

General implementation and syntax

How network connections work in phyphox

A network connection can define phyphox buffers from which data should be sent to a network service and buffers that should receive data from the network service. The communication can either be triggered by the user through a button press or automatically at a defined interval. However, due to the nature of network communication, a response does not need to come immediately and data may even be received independent of a prior request, if the protocol allows this. For example, some protocols only receive updated data as a response to a request (HTTP) while others might subscribe to updates upon connection and receive new data at random times.

In any case, the sending part of the network communication is only triggered between analysis runs of phyphox and any received data is only written to the phyphox buffers in-between analysis runs as well.

When an experiment using the network interface is loaded by the user, phyphox will always present the user with detailed information about the data that might be transmitted by the experiment. Specifically, phyphox will mention:

• If you submit a unique id which can identify subsequent requests by the user
• The submission of audio data
• The submission of location data
• The submission of other sensor data, including a list of sensors used
• The submission of device information
• The submission of technical information on the sensors, including a list of these sensors

You need to provide a URL that points to a privacy policy that tells the user how his/her data will be handled.

General syntax

The network connections are defined in a network block in the document root as follows:

<network>
   ...
   <connection id="..." privacy="..." service="..." address="..." conversion="..." discovery="..." discoveryAddress="..." autoConnect="true/false" interval="...">
       ...
       <send id="..." type="buffer/meta">...</send>
       ...
       <receive id="..." clear="true/false">...</receive>
       ...
   </connection>
   ...

</nowiki>

Let's start with the attributes of the network connection tag:

id

This id is not required, but it can be used to identify the connection from other elements. For example, a button view element can use this id in a trigger block to trigger the connection.

privacy

A URL to a privacy policy for your experiment.

service

A name from the list of Network services below, defining the service (protocol) for the connection

address

An address for the service. This is typically a fixed IP address or URL, but its meaning may vary depending on the service and it might work differently in combination with different discovery methods (see below)

conversion

A name from the list of response conversions below. This defines how the data stream from the network service is interpreted and converted into numbers that can be used in phyphox.

discovery

A method to discover possible network services, so they can be checked before connection or picked by the user. (See below)

discoveryAddress

The meaning of this address depends on the discoveryService. For example it could be a URL with wild cards an address from which to request a list of servers or filter for mDNS entries.

autoConnect

If set to true, phyphox will connect to the first viable service (address or from the discovery method). If set to false, options from the discovery method will be presented to the user for selection.

interval

If set to a value larger than 0, this defines an interval in seconds at which the connection is triggered periodically.

The send and receive tags within the connection tag define, which data should be send and received. Each entry has an id with varying meanings. For send tags, the meaning of the id depends on the network service and is usually used as a label that is attached to the data for the remote server. For receive tags, the id is interpreted by the response conversion function and determines which part of the converted response should be used.

Send tags usually contain a buffer name. Data from this buffer is send using the network service, but it depends on the network service whether only the last value or the entire buffer (array) is submitted. Alternatively, the attribute "type" can be used to switch from the default type="buffer" to type="meta". In this case, the name in the send tag is not a buffer, but a label that identifies a specific text string of metadata (see list below).

Receive tags can only have a buffer name inside them and can set the clear attribute. If set to true, the phyphox buffer will be cleared before the received data is written to it. If set to false (default), new data is simply appended.

Metadata

If type="meta" is set for a send-tag, the following identifiers may be used to select metadata to be send to the network service. Note, that device specific information is not available on all devices (especially not on Apple devices)

uniqueID

This is a md5 hash that is unique to the user and the address of the network service. It can be used to match subsequent submissions by a single user, but only as long as it was submitted to the same address. (You can not match users across different remote servers as different addresses!)

version

The version of phyphox

build

The build number of phyphox

fileFormat

The file format version of phyphox

deviceModel

The model id of the device

deviceBrand

The brand of the device

deviceBoard

An id for the board on which the device is based

deviceManufacturer

The manufacturer of the device

deviceBaseOS

The operating system on which the device software is based

deviceCodename

A codename identifying the device

deviceRelease

The version of the device

Additionally, you can get detailed metadata on the sensors supported by phyphox. These are not available on Apple devices and on Android devices, their meaning and accuracy can vary.

[sensor]Name

Usually the model of the sensor

[sensor]Vendor

Usually the manufacturer of the sensor

[sensor]Range

Should give the maximum range of the sensor (not very reliable)

[sensor]Resolution

Should give the resolution of the sensor (not very reliable)

[sensor]MinDelay

Typically the period of the maximum available acquisition rate (not very reliable)

[sensor]MaxDelay

The maximum delay value from Android

[sensor]Power

An estimate of the power consumption

[sensor]Version

A version of the sensor

In this list, [sensor] can be replaces by "accelerometer", "linear_acceleration", "gyroscope", "magnetic_field", "pressure", "temperature", "humidity", "light" and "proximity". Note that in some cases, phyphox will try to find a sensor by its name even though it is not officially designated to be a sensor of that type (for example vendor-specific temperature sensors).

Network Services (Protocols)

HTTP/GET

Attribute service="http/get"

Does not support array data

The HTTP/GET service makes a http request to a webserver. The GET version will do a GET request and encode the submitted data in the request URL. It only supports the last value in each buffer. The response may occur delayed and this service will give up on a request after the default timeout time of the device.

Meaning of id

The id attribute of the send tags is used as an identifier when encoding the buffer. For example, <send id="abc">buffer</send> will contribute "abc=42" to the URL if the last value in the buffer is 42. If you try to receive this data on a web server using PHP, you should be able to access this value via $_GET["abc"].

HTTP/POST

Attribute service="http/post"

Supports array data

The HTTP/POST service makes a http request to a webserver. The POST version will do a POST request and encode the submitted data as JSON. It will always encode the buffers as JSON arrays, even if they only contain a single value. Metadata is encoded as strings. The response may occur delayed and this service will give up on a request after the default timeout time of the device.

Note for PHP users: Encoding POST data as JSON will not fill $_POST by default. Instead, you will need to do the following first:

$json = file_get_contents('php://input');
$data = json_decode($json, true);

Then, if you submitted <send id="abc">buffer</send> you can access an array with the entire content of the buffer at $data["abc"] or its first value at $data["abc"][0].

Meaning of id

The ids of the send tags are used to label the JSON arrays and string within the JSON object. A buffer with id "abc" (<send id="abc">buffer</send>) and metadata with id "def" (<send id="def" type="meta">deviceBrand</send>) would be encoded as

{
  "abc": [0, 3, 42],
  "def": "Fancy Company"
}

Additional attributes to send

Since file format 1.10 (phyphox version version 1.1.6), you can additionally set the datatype attribute to determine whether a buffer should be encoded as an array (default) or number (last value only).

For example

<send id="abc" datatype="array">buffer</send>
<send id="def" datatype="number">buffer</send>

is encoded as

{
  "abc": [0, 3, 42],
  "def": 42
}

Response conversions

JSON

Attribute conversion="json"

Supports array data

The response is interpreted as a UTF8 encoded string, which is then parsed as a JSON object.

Meaning of id

The IDs of the receive tags represent a simple hierarchy within the JSON object, separated by dots. So, let's take the following JSON object as an example:

{
  "main": {
    "subentry": [2, 3, 23]
  },
  "other": 42,
  "yetanother": [1, 7, 42]
}

A receive tag with id="other" will receive a single value, 42. id="yetanother" will receive three values, 1, 7 and 42. To access something deeper within the hierarchy, id="main.subentry" will receive three values, 2, 3 and 23.

Discovery methods for network services

A discovery service is a protocol or method to get a list of possible network services. These can either be picked by the user (if "autoConnect" connect is set to false, see general syntax above) or phyphox will connect to the first one available (autoConnect set to true). The discovery method is set by the attribute "discovery" of the connection-block (see above). If left out, you need to set a fixed connection target depending on the network service you want to use (typically, you will have to set a fixed address like for HTTP requests).

Discovery methods like mDNS are not yet implemented, but will follow soon. For now, you can only work with fixed targets.