LogoLogo
  • KPN THINGS USER GUIDE
    • Introduction
    • Getting started
      • Registration and setup
      • Device simulator app
      • Connect devices
        • SIM cards
        • Supported devices and KPN Devices
        • SODAQ R412M
        • Smartrak
        • Marvin development board
        • LoPy5
        • Arduino MKR WAN 1300/1310
        • Arduin MKR NB 1500
    • All videos
    • Tutorials
      • Configure a flow
      • Connect a HTTPs destination
      • Create a simple IoT web applicationPage
      • Send instruction to LoRa DevicePage
      • Send your data to Azure SQL
      • Technical overview
        • Management data model
        • Uplink communication
        • Downlink communication
        • Location data
      • Accounts and projects
        • Customer accounts and users
        • Projects
        • Customer management
        • Cost overview and contract
    • Frequently asked
      • Multi Factor Authentication
  • GENERAL FUNCTIONS
    • Quick reference videos
    • APIs
  • THE PORTAL EXPLAINED
    • Things Manager
      • Devices
        • Bulk operations
      • Data
        • Device data graph
      • Connectivity
        • LoRa connectivity configuration
      • Bulk reports
    • Things Creator
    • Support
      • News
      • Release notes
      • Real-time status
      • Service reports
      • Documents
      • Support tickets
      • Invoices
      • Contact info
    • Shop
  • BUILDING BLOCKS
    • Devices
      • Sensors
        • KPN Conditionsensor CO2 II
        • KPN Conditionsensor CO2 III
        • KPN Conditionsensor CO2 III motionPage 1
        • KPN FillTag I
        • KPN FillTag II
        • 1M2M ED1608
        • Elsys ERS
        • Device Simulator (Mobile Phone app)
        • Generic Device (Internet connected)
        • Generic LoRa device (preset connectivity)
        • Generic LoRa device (programmable)
        • Generic M2M device
        • Generic M2M device (with Internet connection)
      • Trackers
        • KPN Global Tracker Autonomous
        • KPN Global Tracker Wired
        • KPN LocationTag I
        • KPN LocationTag II
        • KPN LocationTag II v2
        • KPN LocationTag II v2 Expert Mode
        • KPN LocationTag II v3
        • KPN LocationTag III (with GPS)
        • KPN LocationTag III (without GPS)
        • KPN LocationTag III v2
        • KPN LocationTag III v2 Buffer
        • KPN LocationTag III v3
        • KPN LocationTag III v4
        • KPN LocationTag V
        • KPN LocationTag VI
        • KPN LocationTag WS
        • Streamline LoRa
        • Streamline M2M
        • Viloc
      • Routers
      • Gateways
        • KPN FSK Basestation
      • Device SDK
    • Connectivity
      • Connecting LoRa devices
        • LoRa connectivity configuration
      • Connecting M2M devices
      • Connecting Internet devices
      • Connecting MQTT Devices
    • Data Processing
      • ThingsML and SenML
        • ThingsML
        • SenML
          • Understanding and Interpreting SenML Data
          • Changes in KPN SenML
      • Device Twin
      • Decoders and encoders
        • Common measurements list
        • Decoder migrations
        • DIY Decoder
          • DIY decoder scripts
      • Merger
      • Device data
      • Devices
    • Destinations
      • HTTPS destination
      • MQTT broker
      • Cumulocity IoT Platform
      • Azure Event Hubs
      • Azure IoT Hub
      • Test Endpoint
      • ThingsBoard
      • Datacake
      • AWS Lambda
      • Google Cloud Run functions
  • IOT LINE UP
    • Freemium
      • Test SIM cards
    • Explorer
    • Modular
    • Tailored
    • Terms and Conditions
Powered by GitBook

© 2024 KPN - All rights reserved.

On this page
  • Introduction
  • In KPN Things
  • Additional sources
  • Device SDK
  • ThingsML playground
  • Working
  • Measurement index list
  • Measurement index field
  • Compatible with SenML
  • Online playground
  • Fields
  • New field: measurement index field "i_"
  • Used field: base version "bver"
  • JSON
  • Example
  • CBOR
  • About CBOR
  • CBOR representation for measurement index field
  • Example 1: fully encodable measurements
  • Example 2: some custom measurements
  • ThingsML for LoRa
  • Measurement index list
  • Request for change
  • Custom measurement index list
  • Changelog

Was this helpful?

Export as PDF
  1. BUILDING BLOCKS
  2. Data Processing
  3. ThingsML and SenML

ThingsML

A generic payload protocol by KPN, also suitable for LoRa

PreviousThingsML and SenMLNextSenML

Last updated 3 months ago

Was this helpful?

ThingsML is an extension of the SenML specification, in a way SenML is designed to be extended (as described in ).

Introduction

ThingsML provides a more compact way of sending measurements based on SenML by indexing commonly used measurements and allowing to only send the measurement index instead of the full name and unit. It is inspired by the way developers try to cope with the small number of bytes available when using LoRaWAN for data transmission. They often employ context-based encoding schemes, meaning you need to know a lot of specifics to be able to decode a message. ThingsML tries to find the middle ground between bloaty generic protocols and niche specific protocols.

The ThingsML specification only covers the JSON and CBOR data types.

  • JSON is a suitable data type for Internet and M2M communication.

  • CBOR is a suitable data type for LoRa communication.

In KPN Things

Currently only the following use cases are supported in KPN Things:

  • for uplink messages from own LoRa devices.

Support for M2M and downlink message will follow. For M2M communication we currently use SenML, and for LoRa downlink messages only raw payload communication is available.

Additional sources

Device SDK

ThingsML is part of our Device SDK (written in C). With the Device SDK you can easily setup communication between your Device and KPN Things. .

ThingsML playground

Working

This section explains the way ThingsML works.

Measurement index list

In SenML the type of measurement present in a record is expressed by its name and its unit. You will find almost all temperature measurements described with name temperature and unit Cel. In ThingsML every of these very common types of measurements is indexed in the measurement index list. This means every combination of SenML name and unit gets a unique number.

Measurement index field

So now, if you want to put a measurement in SenML, you only need to add the measurement index to the record instead of the full name and unit, saving you a lot of bytes. This measurement index is stored in a new field called the measurement index field and it has the label name i_.

Compatible with SenML

If you want to send a measurement using ThingsML, but you want to send a measurement that is not in the measurement index list, you still can. Just add the name and unit to the record as you are used to, and a ThingsML decoder will accept it nonetheless. ThingsML is fully compatible with SenML, since it is a valid extension of the latter.

Online playground

To experiment with ThingsML, please visit the online ThingsML Playground:

Fields

This section explains the fields that ThingsML introduces or uses.

New field: measurement index field "i_"

Since the measurement index field is a custom field that is mandatory to understand to correctly process the pack, it must have a label name that ends with the "_" character [SenML, Section 12.2].

Used field: base version "bver"

For every update of the measurement index list we will up the version number. The version number of the list you implemented can be added to the ThingsML pack in the bver field. This guarantees that your ThingsML pack will not be interpreted incorrectly.

We aim to only extend and not modify the measurement index list for backwards compatibility.

Also, we aim to keep the ThingsML Decoder in KPN Things up-to-date with the most recent version of the measurement index list. So, when using ThingsML in combination with KPN Things, you would not need to use the bver field.

JSON

This section explains more about ThingsML in JSON data format.

Example

The difference between SenML and ThingsML in JSON is illustrated with the example below.

SenML

277 bytes

ThingsML

203 bytes (-30%)

CBOR

This section explains more about ThingsML in CBOR data format.

About CBOR

CBOR representation for measurement index field

ThingsML has an integer CBOR representation for the label of the measurement index field. Officially, only default SenML labels should have an integer as CBOR representation. But this would save us 2 extra bytes per measurement in the SenML record, giving is more compactness. Since the measurement index is a regular field, the CBOR representation should be a positive integer.

Name

Label

CBOR Label

Measurement index

i_

23

Example 1: fully encodable measurements

SenML

142 bytes

ThingsML

66 bytes (-55%)

85a300686c6174697475646502fb4049f9ed288ce70401636c6174a3 00696c6f6e67697475646502fb40155165d3996fa801636c6f6ea300 6b74656d706572617475726502fb403dc00000000000016343656ca3 006e62617474657279566f6c7461676502fb400cb851eb851eb80161 56a3006b74656d706572617475726502fb40313851eb851eb8016343 656c

85a202fb4049f9ed288ce7041735a202fb40155165d3996fa81734a2 02fb403dc000000000001737a202fb400cb851eb851eb81706a202fb 40313851eb851eb81737

This example encodes all decimal numbers as double point float representations. Further payload size optimization can be achieved by using single point or half point float representations.

Example 2: some custom measurements

SenML

187 bytes

ThingsML

128 bytes (-30%)

87a221782075726e3a6465763a4445564555493a3031323334353637 38393031323334353a221a5cb35887a300686c6174697475646502fb 4049f9ed288ce70401636c6174a300696c6f6e67697475646502fb40 155165d3996fa801636c6f6ea3006b74656d706572617475726502fb 403dc00000000000016343656ca3006e62617474657279566f6c7461 676502fb400cb851eb851eb8016156a3006765787454656d7002fb40 313851eb851eb8016343656ca20062696f04f4

87a221782075726e3a6465763a4445564555493a3031323334353637 38393031323334353a221a5cb35887a202fb4049f9ed288ce7041735 a202fb40155165d3996fa81734a202fb403dc000000000001737a202 fb400cb851eb851eb81706a3006765787454656d7002fb40313851eb 851eb8016343656ca20062696f04f4

ThingsML for LoRa

ThingsML for LoRa follows the following additional aspects:

  • ThingsML for LoRa only supports CBOR data representation, since this is the most compact way of sending ThingsML.

  • You may omit the base name and base time values from your measurement. The LoRaWAN network layer already provides this information to KPN Things, so your device doesn't have to.

Measurement index list

Request for change

If you have a measurement name and unit you think should be in the ThingsML measurement index list? Please contact the author for a request for change. Proposed records will be added if they are generic and unique in the list. For instance Temperature in Fahrenheit shall not be added to the list, since translation to Celsius can be trivially implemented in the device.

Custom measurement index list

We plan to support use case specific measurement tables, enabling you to define your own measurement index list.

Changelog

Date

Description

26-11-2019

First version

8-1-2020

Updated text

27-1-2020

Added 3 measurements to the index list

Added change log table

7-4-2020

Specification released publicly

18-11-2020

Made introduction and ThingsML for LoRa more clear.

Specification by Paul Marcelis. Email: pjmarcelis@gmail.com

With the ThingsML playground you can compare SenML with ThingsML, as well as JSON with CBOR. .

The measurement index value starts with -24, because this value is finetuned on the CBOR specification to allow for optimal usage of this protocol for LoRaWAN applications. In CBOR a number from -24 to 23 can be encoded without additional bytes. For more information read the Major types section in the CBOR specifications .

CBOR stands for Concise Binary Object Representation and is a standard data format: . It is the more compact counterpart of JSON. That means each JSON object has its equal in CBOR and the other way around. In the SenML specification additional translation steps from JSON to CBOR are proposed to further decrease the size of SenML packs in CBOR. More information about SenML in CBOR can be found here: .

The example below shows a measurement list in both SenML and ThingsML in CBOR hexadecimal representation and CBOR diagnostic notation. The hexadecimal representation could be copy-pasted into the CBOR tool at .

This section gives the measurement index list and the way it is moderated. This list can be found on the .

[
  {"bn":"urn:dev:DEVEUI:0123456789012345:","bt":1555257479, "n":"latitude","v":51.95255,"u":"lat"},
  {"n":"longitude","v":5.32949,"u":"lon"},
  {"n":"temperature","v":29.75,"u":"Cel"},
  {"n":"batteryVoltage","v":3.59,"u":"V"},
  {"n":"extTemp","v":17.22,"u":"Cel"},
  {"n":"io","vb":false}
]
[
  {"bn":"urn:dev:DEVEUI:0123456789012345:","bt":1555257479, "v":51.95255,"i_":-22},
  {"v":5.32949,"i_":-21},
  {"v":29.75,"i_":-24},
  {"v":3.59,"i_":6},
  {"n":"extTemp","v":17.22,"u":"Cel"},
  {"n":"io","vb":false}
]
[
  {0: "latitude", 2: 51.95255, 1: "lat"},
  {0: "longitude", 2: 5.32949, 1: "lon"},
  {0: "temperature", 2: 29.75, 1: "Cel"},
  {0: "batteryVoltage", 2: 3.59, 1: "V"},
  {0: "temperature", 2: 17.22, 1: "Cel"}
]
[
  {2: 51.95255, 23: -22},
  {2: 5.32949, 23: -21},
  {2: 29.75, 23: -24},
  {2: 3.59, 23: 6},
  {2: 17.22, 23: -24}
]
[
  {-2: "urn:dev:DEVEUI:0123456789012345:", -3: 1555257479},
  {0: "latitude", 2: 51.95255, 1: "lat"},
  {0: "longitude", 2: 5.32949, 1: "lon"},
  {0: "temperature", 2: 29.75, 1: "Cel"},
  {0: "batteryVoltage", 2: 3.59, 1: "V"},
  {0: "extTemp", 2: 17.22, 1: "Cel"},
  {0: "io", 4: false}
]
[
  {-2: "urn:dev:DEVEUI:0123456789012345:", -3: 1555257479},
  {2: 51.95255, 23: -22},
  {2: 5.32949, 23: -21},
  {2: 29.75, 23: -24},
  {2: 3.59, 23: 6},
  {0: "extTemp", 2: 17.22, 1: "Cel"},
  {0: "io", 4: false}
]
Section 4.4 of the SenML specification
ThingsML for LoRa
Learn more about the Things Device SDK
Visit the ThingsML playground
https://tools.ietf.org/html/rfc7049#section-2.1
https://tools.ietf.org/html/rfc7049
https://tools.ietf.org/html/rfc8428#section-6
http://cbor.me
common measurements page
ThingsML Playground