Data processing (flows)
Last updated
Last updated
Data Processing is the core of the KPN Things platform. Both in uplink and downlink communication it links the devices to the destinations and vise versa. It makes information from raw data, and translates universal instructions into device specific commands.
Data Processing can be configured on the Data Processing in Flow page.
Data Processing configuration is configured in a Flow. It expresses how data from and to your Devices should be processed by KPN Things. This processing is useful, because your Destination does not always understand what your Device is saying, and vice versa. This data should be transformed or interpreted between your Device and Destination.
We distinguish two type of actors that process data:
Decoders process data coming from your device (uplink data).
Encoders process data going to your device (downlink data).
Uplink data is processed by Decoders, Downlink data is processed by Encoders.
Uplink data configuration has to do with the data sent from your Device and possible metadata coming along when data is sent by your Device. An example of the latter is LoRa Geolocation data; location data that the LoRa network can determine when receiving the message from your Device.
We distinguish three types of data that can be enabled to forward to your Destination. Each type has its own symbol that is used in the name of the Decoder.
Downlink data configuration has to do with the data sent to your Device. Often you want to send a clear instruction to your Device, which is transformed under water into a technical data message to your Device. This is where Encoders come into play. Next to that, also raw data can be sent to your Device.
We distinguish two different ways of sending data to your Device, identified by a symbol in the name of the Encoder.
Within a Flow you configure the way your IoT data should flow through KPN Things. You link one or more Devices to one or more Destinations through the Flow, and you determine the Data Processing in between. So, if data is received from a Device that is linked to a Flow, its data will be processed as configured and then forwarded to the Destinations that are linked to the same Flow.
A Flow has the following attributes:
UUID - the technical identifier of your Flow.
Name - the name you give to your Flow.
Description (optional) - further explanation of your Flow.
This page gives you an overview of the configuration of a Flow.
It shows you:
The name and description of your Flow.
A link to the Flow detail page, where you can edit the name and description of your Flow.
The number of Devices in your Flow and a smart link to the Devices in your Flow.
A summary of the Data Processing in your Flow and a link to the Data Processing in Flow page.
Removed. The configuration of Device Twin values has been removed in favor of showing all possibles values by default.
The number of Destinations in your Flow and a smart link to the Destinations in your Flow.
A delete button to remove your Flow. All Devices and Destinations that were linked to the Flow will not be deleted, they can still be found on the All Devices page and All Destinations page respectively after you deleted your Flow.
If there is an issue with the configuration of your Flow, a warning icon is shown in the description of the concerning part of the configuration. Click on the warning label to find out more about the issue.
The smart link allows you to quickly get to the Devices and Destinations in your Flow from the overview page. Depending on the number of Devices or Destinations you have in your Flow, the card will link you to a different view:
The Flow detail page shows the details of your Flow (#1 in the screenshot) and it allows you to edit the name and description of your Flow by clicking Edit (#2 in the screenshot).
This page shows you the Devices that are linked to the current Flow.
Elements on this page are:
The number of Devices in the current Flow.
The list of devices, showing their name, device type and last message time.
When you click on a table row, you are sent to the Device detail page of the selected Device.
You can sort the Device list by clicking on the column header you want to sort by.
A button to add another Device to this Flow.
Checkboxes that allow you to start bulk actions.
After selecting one or more Devices you enter the bulk editing mode. This allows you to select several Devices (#1 in the screenshot below) or all Devices at once (#2 in the screenshot below). After you completed your selection, click on the desired action in the bulk action bar in the bottom of your screen. In this case, you can Unlink all selected Devices from the current Flow.
When you have reached the maximum number of unique devices in your Project, you will not be able to add another device to your Flow. The Add Device button will then be grey and disabled.
Also, it can occur that you have reached the total maximal amount of Devices in your account. Then you can only link an existing Device to your Flow, you cannot add a new Device. In that case the Add a new Device button will be grey and disabled.
When adding another Device to a Flow, you have two choices:
Add a new Device - register a complete new device to KPN Things and directly add it to the Flow.
Link an existing Device - link a device that is already registered to KPN Things. You can add one Device to multiple Flows.
This page is identical to the Add new Device page coming from the All Devices page, with the additional step of Decoder activation.
At this step you can select the decoders you want to enable for this device type in the current Flow by checking the corresponding decoder (like #1 in the screenshot above). If you have previously activated decoders on this Flow for this Device type, they will be already checked (like #2 in the screenshot above). Learn more about Decoders.
This page will show you an overview of all Devices already linked to your Flow and other available Devices. There are two methods to link existing Devices to a Flow:
By clicking Link on the row of a Device, you link that single Device to the Flow.
By selecting a checkbox in a row of a Device, you start with linking multiple Devices to the Flow.
After selecting one Device you can select more using the checkboxes (#1 in the screenshot below), or even select all Devices you have using the checkbox in the top row (#2 in the screenshot below). When you finalized your selection you can commit the linkage with the button Link selected (#3 in the screenshot below).
The Data Processing in Flow page shows you the configuration of the data processing for Devices in that Flow.
When you visit the page, it first shows you the Device types that are present in the Flow. For each Device type it shows the number of devices linked to the Flow (#1 in the screenshot below), and the number of Decoders and Encoders enabled for that Device type (#2 in the screenshot below.
After you clicked on one of the Device types, all available Decoders (#1 in the screenshot below) and Encoders (#2 in the screenshot below) are shown for that Device type. The toggles to the right of the list (#3 in the screenshot below) indicate whether a Decoder or Encoder is enabled for that Flow. Learn more about Data Processing configuration
If there is a possible issue with your data processing configuration, a warning icon is shown for the concerning Device type (#1 in the screenshot below) and a warning message is shown explaining the issue (#2 in the screenshot below). We offer three possible solutions for the issue:
Show you where to fix the issue (#3 in the screenshot below)
Fix it for you (#4 in the screenshot below)
Let you ignore the warning (#5 in the screenshot below)
This page shows you the Destinations linked to the current Flow.
On this page you can see:
The number of Destinations in the current Flow.
A list of the Destinations in this Flow with their name, status and type. By clicking on a Destination, you will go to the Destination detail page of the selected Destination.
A button to add another Destination to your Flow. If you have reached the maximum number of Destination links in your Project, this button will be disabled.
When you have reached the maximum number of Destination links in your Project, you will not be able to add another Destination to your Flow. The Add Destination button will then be grey and disabled.
Also, it can occur that you have reached the total maximal amount of Devices in your account. Then you can only link an existing Destination to your Flow, you cannot add a new Device. In that case the Create a new Destination option will not be available in the drop down.
When adding a Destination to a Flow, you have two options, as illustrated in the screenshot above:
Create a new Destination
Select an existing Destination to link to your Flow
This page is identical to the Add new Destination page coming from the All Destinations page.
After selecting the desired Destination in the drop down menu, click Add Destination to link the selected Destination to your Flow.
Symbol
Output data type
›
Decoded data, so incoming encoded data transformed into understandable measurements in SenML format.
+
Additional data (decoders) enabling metadata corresponding to the message send by your Device.
=
Raw data, just forwarding data like it is received by KPN Things.
