A generic payload protocol by KPN, also suitable for LoRa.
ThingsML provides a more compact way of sending measurements based on SenML by indexing commonly used measurements and allowing to only send the measurement index instead of the full name and unit. It is inspired by the way developers try to cope with the small number of bytes available when using LoRaWAN for data transmission. They often employ context-based encoding schemes, meaning you need to know a lot of specifics to be able to decode a message. ThingsML tries to find the middle ground between bloaty generic protocols and niche specific protocols.
The ThingsML specification only covers the JSON and CBOR data types.
In KPN Things we use ThingsML as a generic decoding and encoding service for the communication with Devices. SDKs from KPN Things are available to further help you implement ThingsML in your device.
KPN Things interprets the ThingsML sent by your Device and forwards SenML to your Application
Currently only the following use cases are supported in KPN Things:
Support for M2M and downlink message will follow. For M2M communication we currently use SenML, and for LoRa downlink messages only raw payload communication is available.
This section explains the way ThingsML works.
In SenML the type of measurement present in a record is expressed by its name and its unit. You will find almost all temperature measurements described with name temperature and unit Cel. In ThingsML every of these very common types of measurements is indexed in the measurement index list. This means every combination of SenML name and unit gets a unique number.
The measurement index value starts with -24, because this value is finetuned on the CBOR specification to allow for optimal usage of this protocol for LoRaWAN applications. In CBOR a number from -24 to 23 can be encoded without additional bytes. For more information read the Major types section in the CBOR specifications https://tools.ietf.org/html/rfc7049#section-2.1.
So now, if you want to put a measurement in SenML, you only need to add the measurement index to the record instead of the full name and unit, saving you a lot of bytes. This measurement index is stored in a new field called the measurement index field and it has the label name
If you want to send a measurement using ThingsML, but you want to send a measurement that is not in the measurement index list, you still can. Just add the name and unit to the record as you are used to, and a ThingsML decoder will accept it nonetheless. ThingsML is fully compatible with SenML, since it is a valid extension of the latter.
To experiment with ThingsML, please visit the online ThingsML Playground:
This section explains the fields that ThingsML introduces or uses.
Since the measurement index field is a custom field that is mandatory to understand to correctly process the pack, it must have a label name that ends with the "_" character [SenML, Section 12.2].
For every update of the measurement index list we will up the version number. The version number of the list you implemented can be added to the ThingsML pack in the bver field. This guarantees that your ThingsML pack will not be interpreted incorrectly.
We aim to only extend and not modify the measurement index list for backwards compatibility.
Also, we aim to keep the ThingsML Decoder in KPN Things up-to-date with the most recent version of the measurement index list. So, when using ThingsML in combination with KPN Things, you would not need to use the bver field.
This section explains more about ThingsML in JSON data format.
The difference between SenML and ThingsML in JSON is illustrated with the example below.
This section explains more about ThingsML in CBOR data format.
CBOR stands for Concise Binary Object Representation and is a standard data format: https://tools.ietf.org/html/rfc7049. It is the more compact counterpart of JSON. That means each JSON object has its equal in CBOR and the other way around. In the SenML specification additional translation steps from JSON to CBOR are proposed to further decrease the size of SenML packs in CBOR. More information about SenML in CBOR can be found here: https://tools.ietf.org/html/rfc8428#section-6.
ThingsML has an integer CBOR representation for the label of the measurement index field. Officially, only default SenML labels should have an integer as CBOR representation. But this would save us 2 extra bytes per measurement in the SenML record, giving is more compactness. Since the measurement index is a regular field, the CBOR representation should be a positive integer.
The example below shows a measurement list in both SenML and ThingsML in CBOR hexadecimal representation and CBOR diagnostic notation. The hexadecimal representation could be copy-pasted into the CBOR tool at http://cbor.me.
This example encodes all decimal numbers as double point float representations. Further payload size optimization can be achieved by using single point or half point float representations.
ThingsML for LoRa follows the following additional aspects:
- ThingsML for LoRa only supports CBOR data representation, since this is the most compact way of sending ThingsML.
- You may omit the base name and base time values from your measurement. The LoRaWAN network layer already provides this information to KPN Things, so your device doesn't have to.
If you have a measurement name and unit you think should be in the ThingsML measurement index list? Please contact the author for a request for change. Proposed records will be added if they are generic and unique in the list. For instance Temperature in Fahrenheit shall not be added to the list, since translation to Celsius can be trivially implemented in the device.
We plan to support use case specific measurement tables, enabling you to define your own measurement index list.