Introduction on the SenML data representation
For Freemium projects, the way KPN Things processes device data to SenML changed (per December 13th 2022). Read all the details about the Changes in KPN SenML.
SenML is the default data format for device data in KPN Things. Currently KPN Things only supports SenML in JSON. We plan to support SenML in CBOR in the future.
A SenML message is officially called a SenML pack. One SenML pack consists of one or more SenML records. One SenML record expresses a single measurement. That way a SenML pack expresses a set of measurements
The first record in a SenML pack always contains base information. Base information expresses from which device the measurements are from (base name) and the moment the measurements were taken (base time).
There are different type of SenML records:
Number record - for communicating a number measurement
Boolean record - for communicating an on/off measurement
String record - for communicating a string measurement
Data record - for communicating raw data measurement
Below you find a non-exhaustive example of a SenML pack:
Line nr.
Explanation
2
First line with base attributes, stating measurement source and timestamp.
3
Number record
4
Number record with relative time
5
Boolean record
6
String record
7
Data record
This section explains the attributes you can use to express your measurements in SenML
When the base name and/or base time values are present in the pack, they are available in the first measurement of the pack and applicable for all the measurements in the pack.
The base name of a SenML pack states which device the measurements are from. It should contain a unique resource name. The parts of the base name, which are colon separated, are as following:
urn stands for unique resource name.
dev means we are identifying a device.
DEVEUI is the type of device identifier used to identify the device.
01234567890ABCDEF is the DevEUI, the device identifier.
In the example above we identify a LoRa device by its DEVEUI. If we would have an M2M device, we would identify it by its IMEI and the base name would be as following: urn:dev:IMEI:0123456789012345:.
A base name always ends with a colon :!
The timestamp of the measurement in UNIX timestamp format. If KPN Things receives a measurement without base time, the time of receiving the measurement is taken as base time. SenML being sent by KPN Things will always contain a base time.
Name of the individual measurement.
The (..) name MUST consist only of characters out of the set "A" to "Z", "a" to "z", and "0" to "9", as well as "-", ":", ".", "/", and "_"; furthermore, it MUST start with a character out of the set "A" to "Z", "a" to "z", or "0" to "9". - The SenML spec, RFC8428 section 4.5.1
When designing your SenML measurement format for M2M and Internet devices, you should take into account our Common measurements list. By using as much name and unit pairs from that list, you achieve higher compatibility with current and future data processing features.
Unit of the measurement. Write your u-values in camel case, so start with a lower case character.
The timestamp of an individual record relative to the base time in seconds.
Value of the number measurement.
Boolean value for a boolean measurement. Should be true or false.
String value for a string measurement.
Data value for a data measurement. Should be a base64 encoded string.
There are more attributes in the SenML specification, but these values are not used by KPN Things. You can read about them in the Sensor Measurements Lists specification:
For easier integration with KPN Things we have created client libraries in C and Python for you to use in your devices.
To make SenML more suitable for communication over LoRa, we have introduced ThingsML. ThingsML is a valid extension of SenML that introduces measurement indices. We have listed several common measurements, allowing you to only send the measurement index number instead of sending the complete name and unit of the measurement. Learn more about ThingsML.
The primary aim of this document is to provide a comprehensive guide on how to correctly interpret Sensor Measurement List (SenML) data within the KPN Things platform. This guidance is essential for ensuring accurate data processing and utilization on the receiving end of the platform.
To get the most out of SenML data, it is important to interpret it correctly. Not doing so may result in missing or incorrect data in your downstream applications. As SenML data structures can be complex and varied, understanding the nuances is vital for reliable data handling.
SenML data is structured in JSON format to ensure uniformity and readability. Following are examples of correctly structured SenML records. All the records in this paragraph represent the same SenML data, this will help you understand what the SenML that arrives at your Destination might look like.
In the above example, we can see unresolved SenML. The SenML contains a base name (bn) and a base time (bt). A SenML Record is referred to as "resolved" if it does not contain any base values, i.e., labels starting with the character b, (except for Base Version fields), and has no relative times. The message above, when "resolved", looks like this:
As mentioned earlier, all the SenML in this paragraph represents the same data. So when it is resolved, it matches the SenML records in the previous example.
The records should be in chronological order in the resolved SenML, however in this example the time values are equal so switching the order still represents the same SenML:
Numeric values should be valid JSON numbers, which means that they can have a decimal point and an exponent. Therefore the above example is equal to the following example:
Time values are relative to the base time. So if measurement records have time values, they are relative to the value of the previous base time record. If you look at the example below, the base time should be added to the time fields of the measurements to resolve the time of the measurement.
The SenML data is the same as all the previous SenML messages.
It is possible to have multiple base value records. The base value applies to all successive measurements until another of that base value is found. In the example below, a single base name is used for all the records. On the contrary, the base time is altered in the second record. Since in both record the base time and the time elements add up to the same value, this data still equals the previous SenML examples.
Working with SenML requires an understanding of the SenML data structures. Simple parsing strategies, such as string splitting, might work with one SenML representation, but could fail when dealing with a different one. When SenML is sent out to your destination, we give no guarantees on the representation.
To assist in correctly interpreting SenML, several libraries are available for different programming languages:
Java: senml-java TO BE RELEASED
These libraries are designed to handle the complexities of the SenML data and provide more reliable parsing and data extraction.
At KPN Things, we strive to adhere as closely as we can to the SenML standard. This dedication sometimes necessitates modifications on our end to align more precisely with these standards. We recommend our customers to also embrace this approach, ensuring customer applications remain robust and future-proof by closely following the SenML standard.
Understanding and correctly interpreting SenML data is crucial for accurate and reliable data processing when interacting with the KPN Things platform. We urge our users to follow best practices and consult the official SenML specification. Adhering to these guidelines not only guarantees data integrity but also maximizes the benefits derived from using the KPN Things platform.
These changes only apply to Freemium projects.
KPN Things currently rounds the SenML bt and t fields in its SenML output to whole seconds. We are going to remove or change this rounding to allow for more accurate time values. Although time values always use seconds as their unit, it means that these values may contain fractional seconds.
For example, "t" may contain the value 0.001 or -33.333333 whereas before they would be rounded as 0 or -33 respectively. Depending on the device, a "t" value may change from "t": 1.659571200E9 to "t": 1.6595711995E9. So this changes in this example from Thursday, August 4, 2022 0:00:00 to Wednesday, August 3, 2022 23:59:59.500.
Check your device specification and your implementation to see if this has an impact on you. You may be affected if your software assumes the time values are always integers.
The base values sent by KPN Things are currently presented as a separate SenML record. As each record represents a measurement, this record containing only the base values implies a measurement of "v": 0 for "n": "". This is incorrect, as such a measurement was never made. In the future, KPN Things will no longer generate a separate SenML record with only the base name and base time fields, but instead will follow the SenML specification and store the base values as needed in existing records. Note: base values can appear in any SenML record, not just the first. To properly resolve the records in a SenML pack care must be taken to follow the specification to decompress the pack correctly.
To read more on this topic see
base values can appear in any SenML record, not just the first. To properly resolve the records in a SenML pack, care must be taken to follow the specification to decompress the pack correctly.
Currently the system sends SenML like this:
In the new situation this will no longer occur. The base values will be part of the first measurement, and will look like this:
Check the software on your end to see if it can handle this.
It is advised that incoming data should be parsed and normalised to an appropriate internal representation of the , before any further processing is attempted.
JavaScript:
C#:
When in doubt you can always use the as a reference. Or take a look at the .
In the first measurement, the "v" and "u" fields of the first measures are missing (). But this indicates a sensor with the name urn:dev:DEVEUI:0123456789ABCDEF: that has no units or values, which is not a representation of reality.
