ThingsML
A generic payload protocol by KPN, also suitable for LoRa.
ThingsML is an extension of the SenML specification, in a way SenML is designed to be extended (as described in Section 4.4 of the SenML specification).
Introduction
ThingsML provides a more compact way of sending measurements based on SenML by indexing commonly used measurements and allowing to only send the measurement index instead of the full name and unit. It is inspired by the way developers try to cope with the small number of bytes available when using LoRaWAN for data transmission. They often employ context-based encoding schemes, meaning you need to know a lot of specifics to be able to decode a message. ThingsML tries to find the middle ground between bloaty generic protocols and niche specific protocols.
The ThingsML specification only covers the JSON and CBOR data types.
In KPN Things
Currently only the following use cases are supported in KPN Things:
ThingsML for LoRa for uplink messages from own LoRa devices.
Support for M2M and downlink message will follow. For M2M communication we currently use SenML, and for LoRa downlink messages only raw payload communication is available.
Additional sources
Device SDK
ThingsML is part of our Device SDK (written in C). With the Device SDK you can easily setup communication between your Device and KPN Things. Learn more about the Things Device SDK.
ThingsML playground
With the ThingsML playground you can compare SenML with ThingsML, as well as JSON with CBOR. Visit the ThingsML playground.
Working
This section explains the way ThingsML works.
Measurement index list
In SenML the type of measurement present in a record is expressed by its name and its unit. You will find almost all temperature measurements described with name temperature and unit Cel. In ThingsML every of these very common types of measurements is indexed in the measurement index list. This means every combination of SenML name and unit gets a unique number.
The measurement index value starts with -24, because this value is finetuned on the CBOR specification to allow for optimal usage of this protocol for LoRaWAN applications. In CBOR a number from -24 to 23 can be encoded without additional bytes. For more information read the Major types section in the CBOR specifications https://tools.ietf.org/html/rfc7049#section-2.1.
Measurement index field
So now, if you want to put a measurement in SenML, you only need to add the measurement index to the record instead of the full name and unit, saving you a lot of bytes. This measurement index is stored in a new field called the measurement index field and it has the label name i_
.
Compatible with SenML
If you want to send a measurement using ThingsML, but you want to send a measurement that is not in the measurement index list, you still can. Just add the name and unit to the record as you are used to, and a ThingsML decoder will accept it nonetheless. ThingsML is fully compatible with SenML, since it is a valid extension of the latter.
Online playground
To experiment with ThingsML, please visit the online ThingsML Playground:
Fields
This section explains the fields that ThingsML introduces or uses.
New field: measurement index field "i_"
Since the measurement index field is a custom field that is mandatory to understand to correctly process the pack, it must have a label name that ends with the "_" character [SenML, Section 12.2].
Used field: base version "bver"
For every update of the measurement index list we will up the version number. The version number of the list you implemented can be added to the ThingsML pack in the bver field. This guarantees that your ThingsML pack will not be interpreted incorrectly.
We aim to only extend and not modify the measurement index list for backwards compatibility.
Also, we aim to keep the ThingsML Decoder in KPN Things up-to-date with the most recent version of the measurement index list. So, when using ThingsML in combination with KPN Things, you would not need to use the bver field.
JSON
This section explains more about ThingsML in JSON data format.
Example
The difference between SenML and ThingsML in JSON is illustrated with the example below.
CBOR
This section explains more about ThingsML in CBOR data format.
About CBOR
CBOR stands for Concise Binary Object Representation and is a standard data format: https://tools.ietf.org/html/rfc7049. It is the more compact counterpart of JSON. That means each JSON object has its equal in CBOR and the other way around. In the SenML specification additional translation steps from JSON to CBOR are proposed to further decrease the size of SenML packs in CBOR. More information about SenML in CBOR can be found here: https://tools.ietf.org/html/rfc8428#section-6.
CBOR representation for measurement index field
ThingsML has an integer CBOR representation for the label of the measurement index field. Officially, only default SenML labels should have an integer as CBOR representation. But this would save us 2 extra bytes per measurement in the SenML record, giving is more compactness. Since the measurement index is a regular field, the CBOR representation should be a positive integer.
Example 1: fully encodable measurements
The example below shows a measurement list in both SenML and ThingsML in CBOR hexadecimal representation and CBOR diagnostic notation. The hexadecimal representation could be copy-pasted into the CBOR tool at http://cbor.me.
This example encodes all decimal numbers as double point float representations. Further payload size optimization can be achieved by using single point or half point float representations.
Example 2: some custom measurements
ThingsML for LoRa
ThingsML for LoRa follows the following additional aspects:
ThingsML for LoRa only supports CBOR data representation, since this is the most compact way of sending ThingsML.
You may omit the base name and base time values from your measurement. The LoRaWAN network layer already provides this information to KPN Things, so your device doesn't have to.
Measurement index list
This section gives the measurement index list and the way it is moderated. This list can be found on the common measurements page.
Request for change
If you have a measurement name and unit you think should be in the ThingsML measurement index list? Please contact the author for a request for change. Proposed records will be added if they are generic and unique in the list. For instance Temperature in Fahrenheit shall not be added to the list, since translation to Celsius can be trivially implemented in the device.
Custom measurement index list
We plan to support use case specific measurement tables, enabling you to define your own measurement index list.
Changelog
Specification by Paul Marcelis. Email: pjmarcelis@gmail.com
Last updated