Comarch IoT Platform – user manual

Overview of the manual

The next chapter, "Getting started", provides a hands-on guide to basic configuration of the platform and the devices. It covers both the workflow setup and security management. Most of the tools and functionalities are described briefly here, and more details can be found in the following chapter "Components", which contains a description of every module of the Platform. For even more details and specifications, refer to the technical documentation.

Introduction

Comarch IoT Platform is a dedicated cloud application for all our IoT devices. With IoT Platform tailored to your needs, viewing the information collected by the devices is a breeze. We propose a solution which is:

intuitive – because navigating through the interface should never be frustrating, or consume too much of your time;
scalable – the database and its possibilities are ready to grow along with your business;
secure – all the data is encrypted, and the user access management is highly customizable;
reputable – many companies successfully use the Platform along with our IoT devices.

IoT Platform can be best described as a three-part system:

On the device side, there’s an interface which allows the devices to automatically send the data in real time, using a protocol such as MQTT or LwM2M (deprecated);
At the core lies a cloud-stored database, in which the device data is saved;
The app external interface provides easy access to the data – both through a REST or Websocket API (for vertical applications) and through a website (for humans).

The architecture of Comarch IoT Platform – Smart Metering and Asset tracking are examples of our vertical applications.

To summarize – the IoT Platform most important functionalities include management of the IoT devices, telemetry, and an interface suitable both for users and for other applications.

Getting Started

Account creation

An individual account should be created for every person working on a project. Accounts are highly customizable in terms of access rights, and provide personalized dashboards for every person involved.

A project is a container for all devices working together. Their state and indications can be viewed within a project. Projects are isolated from each other.

At this point, either you have been granted an administrator account, or you have been assigned an administrator from Comarch to begin the setup for you. Feel free to contact us if you’re not sure which is the case. If the account has been pre-created for you, you can skip the next section and proceed to Connecting your first device.

Creating new accounts for users

Creating new user accounts is essential before you can add them to the project. If you have an administrator account, you can create new user accounts for your clients or coworkers. Visit https://{host}/auth/admin/master/console/ (insert your domain name instead of host) to access the administrator console.

Provide permissions. Proceed to section "Users" in the menu on the left, and provide the tenant administrator manage-users and view-realm permissions, as shown below:
Add new users. Log in as the tenant administrator and proceed to users management console at https://sso.{host}/auth/admin/{tenant}/console/#/realms/{tenant}/users. Add new users in Manage… → Users section:
As the user, check your email. An email with a temporary password will have been sent to the user. After logging in, the user will be prompted to change their password, and then redirected to the IoT Platform.

Creating new project

Visit the project site and log in with your credentials. You’ll be redirected to the projects directory. Click "Add new project" to create a new one. You should see the following window:

Add new project dialog window; project ID is Project_1, Name is Test, and Description is Example Project Space

Project ID should be unique, but it’s not an issue at the moment as you are creating your first project. It’s best to choose something both short and descriptive, like the name of your service, or a key word that everyone will understand. Most important things to keep in mind are that it cannot be changed later and is case-sensitive.
Name does not have to be unique, and can be changed later. Think of it as something less binding and more arbitrary.
Description can (and probably should) be longer than the name, and should provide a brief summary of what the project is about.

After you click "approve", the project will be seen in the dashboard as follows:

Click on the project tile to select it. The project’s properties (name and description) can be modified by clicking the cogwheel icon:

Connecting your first device

To connect your first device, go to section "devices" and click "Add new device(s)". You should see a "Select device source" screen. You can either define device by yourself or define few devices. For your first device, it’s recommended to choose the first option. Click "next".

Option 1: Defining device by yourself (wizard)

Important

Only SenML with JSON is supported out of the box. For custom JSON messages please refer to [Adapters]

You should see an "Add device screen". Here you can provide basic information about the device.

Device ID: unique indentifier of the device. It will be needed when you decide to assign rules or a channel to this device. If this section remains empty, the system will generate a random ID for you. The device ID cannot be changed later

Name: it helps you recognize devices easily. It does not have to be unique and can be changed later.

Device type: it depends on the type of device you are connecting. Click on the input window to see an expandable list, and choose the correct device type:

Tags: tags allow grouping devices according to your needs. After assigning a tag to a device, it’s possible to search for them easily by the tag, which is especially helpful if the project contains a lot of devices. One device can have multiple tags assigned.

Attributes: values not define directly on model, like geolocation. Attributes are added in JSON format.

The second step is adding properties to the device.

Depending on the device, it could be battery level, temperature, and more – for full list of properties available for your device, refer to its documentation. One device can, of course, have many properties. Here, you can add the following information:

Property Id: unique name (address) of property of device. While creating the first property, you can for example type "online".

Name: it does not have to be unique, rather descriptive. It can be the same as ID, for example the name "online" or "temperature" are perfectly valid – they’re both short and self-descriptive.

Description: description of device property , expressed as string (text value), double (numeric value), boolean (true/false), or data (other, specific data). For property "online", "boolean" is the correct type.

Unit: unit of the property, e.g. % or dB. You can also leave this field blank, for example for a property like "online".

The third step, channels, defines ways of connecting with the IoT Platform. IoT Platform supports connection via MQTT and LwM2M (deprecated) protocols. In this step, you can configure connection addresses, topics, credentials, etc.

Warning

Setting up a channel is necessary for the device to communicate with the Platform.

The section "Available for assignment" contains created channels available to assign to a device. Most probably, there are no channels at this point and it is necessary to create one. To do so, select section "Create new channel".

Channel ID: unique identificator of the channel, like "outdoor-channel-3" or "d3th-3i54-12h4-294b".
Name: more descriptive, and not necessarily unique, name of the channel.
Description: a brief yet comprehensive description of the channel.
Type: refers to the type of the protocol – either MQTT or LwM2M (deprecated).
Device ID: list of available devices. Select the preferred device’s ID from the expandable list.
MQTT Password: channel password of your choice. After entering the password, write it down in a safe place.

Tip	To change the password in the future, select tab "channels", find the channel you’d like to change password to, and click the three dots next to channel type. Select "edit", click the "I want to type new MQTT password" checkbox, and enter the password of your choice.

Option 2: Define few devices (JSON)

It is also possible to add multiple devices using JSON-formatted text.

In the prompt text, you can see a sample formula for adding multiple devices. This view contains a built-in JSON validator, so a message will be shown if you miss any coma or a bracket. To finish, click "Add devices".

Creating a rule – introduction to rule engine

The rule engine allows you to set up a notification system in cases when you’d like to be informed about particular events, e.g. the device going offline or exceeding the limit of messages sent per hour. You don’t have to create any rule for the device setup to work, so you can skip this step now. Detailed instruction to creating rules is covered in section Rule Engine.

Introduction to dashboards

To see the values of device properties, it is necessary to create a dashboard. When you first select "Dashboards", you should see a message saying "There is no dashboard available in this project". To create the first dashboard in the project, click "Add new dashboard". One project can have multiple dashboards. When creating a dashboard, you can give other users limited access to the content on the dashboard.

Guest: read-only. Guest can’t create and modify anything on projects.
Technician: read-write. Technician can read, create and edit the content on projects.
Owner: read, write and managing – beyond the technician functions, the owner can manage users in the project.

After clicking "Add dashboard", you should see a notification confirming successful creating of the dashboard. You can edit its properties and privacy settings (access rights) by clicking the cogwheel icon and choosing "edit dashboard". You can also toggle autorefresh, and set its time interval. Now, you can add widgets to your dashboard.

Widgets

A widget is a tile on the dashboard that displays the device property of your choice. When added to the dashboard, they will look like this:

A single value widget.

Or, in case of table and graph widgets, like this:

A table widget with three properties.

A graph widget, perfect to visualise values that change in time.

Adding new widget

To add a widget, click the "Add new widget" button. You should see the following screen:

Widget title is what will be shown on the dashboard, at the top of the tile.
Widget category can be either single value, graph, or table, as pictured above;
Device – the device which property you’d like to show, as indicated by its ID
Category: Properties (this should be the only possible option in this case)
Property: select the property of your choice from the expandable list.

Adding collaborators and securing your device

To ensure restricted access to the project, remember to give permissions only to your authorized coworkers. To add more people to the project, select "Members" and click "Add new members":

Warning

Make sure the users you want to add already have an account. If you have been granted an administrator account, make sure you have followed the instructions in Creating new accounts for users.

You should see a window prompting you to add a new members to the project. To add a collaborator, type their email address and click "Add". By default, the new member will be added as a guest, but their role can be changed afterwards. To change the role, choose another from the expandable list:

Detailed information about scopes of particular roles are covered in section [Managing roles].

After completing all above steps, your project should be all set up and working. The next chapter provides more detailed information about each of the Platform’s components.

Components

This section provides more detailed description of each of the Platform’s components and their functionalities not mentioned in Getting Started section.

Device Management

List of devices and editing properties

Go to section "Devices" to see the full list of the devices in your project. To edit the device, click Action… and Edit device:

Here, among adding other attributes, you can apply geolocation tag to the device. Click on the map (or enter a JSON-formatted text, if you wish) to generate property "geolocation".

Warning

"Geolocation tag" is not a tag as used in "device tags". The resemblance is coincidental.

While editing properties, you’ll approach the name device twin. A device twin is not a physical device, but a JSON document. For the time being, you can leave the field "desired propierties" blank. We’ll talk more about device twins in section Connectivity.

Filtering devices

It’s possible to filter the devices in the project using the filter field:

You can filter the devices by ID, name, device type, or tags. To filter, you need to use query syntax, for example: name:mydevice.

Tip	If your device’s name contains spaces (for example "Temperature sensor", do not put in in quotations. Simply write `name:Temperature sensor`.)

Tip	The filter input is not case sensitive, meaning that you can filter by `name:Sensor` and `name:sensor` and achieve the same results.

Connectivity

IoT Communication Patterns

Telemetry
Command & Control
- IoT Commands
- Device-Twin control

Telemetry – what the device tells the IoT Platform

After the device changes its state, it sends its values to the Platform so you can see them in device history and visualize them on widgets. The history of all devices connected to the Platform are safely stored in Comarch Data Centers located around the world.

SenML over MQTT

Device can send its telemetry to IoT Platforms MQTT broker. MQTT is a popular publish/subscribe messaging protocol. To connect with MQTT broker, device must be assigned to a channel and use proper credentials (defined in its channel).

Format of telemetry messages is known as SenML (Sensor Measurements List) and is defined in RFC-8428. Current version of platforms telemetry supports JSON messages only. IoT Platform imposes that name (after resolution) of SenML record should follow urn:dev:ca:{tenant}:{project}:{property} pattern.

How to connect a device see Connecting your first device chapter.

Example message:

[
  { "n": "urn:dev:ca:ca:project:hub:movement_sensor", "t": 1.276020076e+09, "vb": true},
  { "n": "urn:dev:ca:ca:project:hub:battery_level", "u": "per", "t": 1.276020091e+09, "v":23.6}
]

Telemetry frame size is limited to 128kB. Frames that exceed this limit are discarded. It can also contain maximum of 128 records.

Telemetry over HTTP

Address: https://{TENANT}.{DOMAIN}/telemetry/{FORMAT}/projects/{PROJECT}/channels/{CHANNEL}
Method: POST
Payload: JSON message body (Requires header Content-Type: application/json)

Authorization: HTTP BasicAuth where:

username is MQTT channel username in format: {TENANT}.{PROJECT}.{CHANNEL}
password is MQTT channel password

This HTTP endpoint requires user to specify FORMAT, which support adapters. To use standard telemetry format, user have to specify value senml_json.

Adapters are more thoughtfully described in [Adapters]

Command & Control – what you tell the device

When you change a setting on a device – for example, you add new property "online" or change the geolocation tag – you don’t connect directly to the device; rather, you change the contents of a device twin – a JSON document associated with your device. The changes in device twin are submitted to an MQTT topic which the physical device subscribes, meaning that it watches constantly for any changes and will update its state accordingly.

The main advantage of such solution is that the physical devices don’t have to be constantly connected to the Platform. If any of them goes offline, you can still operate on it on the Platform or change its properties. When the physical device goes back online, it will update its state according to the changes you made.

Use Cases for IoT Commands

By design, Commands are to be used for requesting a specific action to perform on target device. Such action doesn’t have to be fully integrated with IoT Platform, but can use some other services.

This interaction works best for complicated tasks, which are hard to represent in a set of device properties.

Examples:

diagnostics logs upload (provide repository where to upload these),
firmware update (provide repository with images to download),
restart the device,
etc.

Use Cases for IoT Device Twin

By design, Device Twin is to be used for establishing given state on target device. Such state is defined by two JSON documents, one representing device physical state and one representing logical state, requested by the user. Device updates its physical state to match requested logical state.

This interaction works best for tasks, which can be represented as a single property change on target device.

Examples:

turning on the camera (set "camera": true in desired JSON document and wait for device to respond to it in reported JSON document),
lowering the temperature (set "temperature": 12.3 in desired JSON document and wait for device to respond to it in reported JSON document),
etc.

IoT Commands

This solution consists of several MQTT topics and bi-directional communication between devices and the cloud service. It’s similar to typical request-response model.

Glossary

Term Summary

Term	Summary
Command execution	A command process, which corresponds to a single device within execution targets
Terminal command execution status	An execution status, which is one of the final stages of command execution. Corresponds to `FAILED`, `SUCCEEDED`, `CANCELED`, `TIMED_OUT`
Non-terminal command execution status	An execution status, which is a transitive stage of command execution. Corresponds to `QUEUED`, `IN_PROGRESS`
Execution queue	A list that contains command execution in order to execute

Command execution

A command process, which corresponds to a single device within execution targets

Terminal command execution status

An execution status, which is one of the final stages of command execution. Corresponds to FAILED, SUCCEEDED, CANCELED, TIMED_OUT

Non-terminal command execution status

An execution status, which is a transitive stage of command execution. Corresponds to QUEUED, IN_PROGRESS

Execution queue

A list that contains command execution in order to execute

Commands Management REST API

Human interaction with commands management is performed via REST API available by open API.

MQTT Topics

Topic name Permissions Description

Topic name	Permissions	Description
`commands/json/{tenant}/{project}/{device}/{command}/request`	`PUBLISH`	Device queries IoT Platform for command to execute and updates command execution status
`commands/json/{tenant}/{project}/{device}/{command}/response`	`SUBSCRIBE`	Device receives answers from IoT Platform for issued requests
`commands/json/{tenant}/{project}/{device}/{command}/notify`	`SUBSCRIBE`	Device receives notification about command events (notably cancellation and timeout events)
`commands/json/{tenant}/{project}/{device}/notify-next`	`SUBSCRIBE`	Device receives notification about new command to execute

commands/json/{tenant}/{project}/{device}/{command}/request

PUBLISH

Device queries IoT Platform for command to execute and updates command execution status

commands/json/{tenant}/{project}/{device}/{command}/response

SUBSCRIBE

Device receives answers from IoT Platform for issued requests

commands/json/{tenant}/{project}/{device}/{command}/notify

SUBSCRIBE

Device receives notification about command events (notably cancellation and timeout events)

commands/json/{tenant}/{project}/{device}/notify-next

SUBSCRIBE

Device receives notification about new command to execute

Device flow summary

After connecting, device subscribes to the {…}/notify-next topic. Note, that it will not receive command executions that were added while being disconnected. It delivers only new commands for execution.
Device publishes ExecutionQuery request to {…}/request topic to retrieve the next command execution to process.
1. Because the ID might be unknown at the moment, retrieving next command execution can be done with a special $next symbol.
Device publishes ExecutionUpdateEnvelope request to {…}/request topic, moving the command execution to IN_PROGRESS state and begins processing.
1. If timeout was configured for given command execution, it would start at the moment of processing start.
2. If device had crushed while command execution, sending ExecutionQuery to {…}/$next/request topic returns current IN_PROGRESS command.
When execution is finished, ExecutionUpdateEnvelope request to {…}/request topic, moving it into SUCCEEDED status.
1. When execution had ultimately failed, the command execution should be moved do FAILURE status.
Move back to point 2, until no next command execution is available.
1. When no command executions are queued at the moment, creation of the new one will be published on {…}/notify-next topic.

Note

Additionally device needs to subscribe to {…}/notify topic to observe potential state changes to CANCELLED or TIMED_OUT states. Device may subscribe with "+" wildcard instead of concrete command id in topics. If command execution is in one of termination states the further state change is forbidden. Response for ExecutionQuery request with $next alias as command id contains error with COMMAND_NOT_FOUND code if execution queue is empty.

Commands execution order

Each device specified within the command targets list corresponds to a single execution process. Therefore each command execution is uniquely identifiable by pair commandId and deviceId. Each device can have no more than one command execution in IN_PROGRESS state.

The execution queue is a list that contains command executions with non-terminal states (either one of QUEUED or IN_PROGRESS). Command execution in IN_PROGRESS state is always on top of the execution queue. The other command executions in QUEUED state are ordered by creation time.

Device requesting Commands to Device ExecutionQuery with $next command id in topic gets the first (if any) element from execution queue. Such element may be currently being executed IN_PROGRESS command or next QUEUED command to start.

Device is allowed to interact only with the first command execution in execution queue (e.g. to change the state). Once command execution state is changed to one of terminal states, the command execution is removed from execution queue.

Limitation

There are following limitation for commands:

message send to {…}/request topic must not exceed 64 KB.
number of messages send to {…}/request topic and related with one command must not be larger than 40.
number of entries in statusDetails map in CommandExecution and CommandExecutionState documents must not be larger than 10.
size of key in statusDetails map in CommandExecution and CommandExecutionState documents must not be larger than 128 characters.
keys in statusDetails can contains only letters, digits or hyphen.
size of value in statusDetails map in CommandExecution and CommandExecutionState documents must not be larger than 2048 characters.
size of correlationId in any request must not be larger than 36 characters.

Note	Messages that exceeds limitation may be ignored by IoT Platform.

Device-Twin control

In order to control devices, device-twin API must be used. It is accessible at:

https://<tenant>.<host>/api/projects/<project>/device-twins/<deviceId>;

Data Model

Fields desired and reported are representation of device state in [SenML](https://tools.ietf.org/html/rfc8428) format.

For fields specified in SenML as doubles [JSON and XML representation](https://tools.ietf.org/html/rfc8428#section-7) values are stored as 128bit decimals to support more precise sensors.

PUT

Put request creates new device twin or replaces existing one, reported should be an empty array.

Example:

curl -XPUT  https://<tenant>.<host>/api/projects/<project>/device-twins/<deviceId> -k
-H "Private-Token: <token>" -H "Content-Type: application/json"
-d "<device twin JSON goes here>"

Device Twin JSON:

{
  "desired" : [ {
    "t" : 1585645427,
    "v" : 10.0,
    "n" : "urn:dev:ca:tenant:project:device:property"
  } ],
  "reported" : [],
  "currentChannel": "twin-channel"
}

Both desired and reported are JSON arrays

desired: State which user/or process is requesting on the device, that state is being sent to the device.
reported: Last state that the device reported; device should update this state whenever it receives new desired state and updates its state.
currentChannel: Adding this field during Put/Patch requests will trigger sending /state update over given MQTT channel.

GET

To get current device state:

curl -XGET  https://<tenant>.<host>/api/projects/<project>/device-twins/<deviceId>  -k -H "Private-Token: <token>"

Response:

{
  "desired" : [ {
    "t" : 1585645427,
    "v" : 10.0,
    "n" : "urn:dev:ca:ca:project:device:property"
  } ],
  "reported" : [ {
    "t" : 1585645427,
    "v" : 10.0,
    "n" : "urn:dev:ca:ca:project:device:property"
  } ]
}

GET HISTORY

To get history of device state:

curl -XGET  https://<tenant>.<host>/api/projects/<project>/device-twins/<deviceId>/history?size=20&since=0&until=2546260198  -k -H "Private-Token: <token>"

Where size is maximum number of returned elements and since/until are timestamps in seconds

Response:

{
  "content" : [ {
      "desired" : [ {
        "t" : 1585645427,
        "v" : 10.0,
        "n" : "urn:dev:ca:ca:project:device:property"
      } ],
      "reported" : [ {
        "t" : 1585645427,
        "v" : 10.0,
        "n" : "urn:dev:ca:ca:project:device:property"
      } ],
      "version" : 0,
      "timestampS" : 1599230957
      },
      {
      "desired" : [ {
        "t" : 1585645427,
        "v" : 12.0,
        "n" : "urn:dev:ca:ca:project:device:property"
      } ],
      "reported" : [ {
        "t" : 1585645427,
        "v" : 12.0,
        "n" : "urn:dev:ca:ca:project:device:property"
      } ],
      "version" : 1,
      "timestampS" : 1599230960
      }],
  "links" : {
    "next" : "/api/projects/project/device-twins/device/history?cursor=6LjSnuEdUWlOnw0Wt4gHpp7v9W6y8T7nL8aCEnTywfVmrsCWDsHOzAPuUsreLMTlGqflOXHDllU6tLMeUHWPGRzjdXCNC-xhhPcLP2UiCKI="
  }
}

Where links field is relative URI to query next batch of data, it can be done using following request:

curl -XGET  https://<tenant>.<host>/api/projects/<project>/device-twins/<deviceId>/history?cursor=6LjSnuEdUWlOnw0Wt4gHpp7v9W6y8T7nL8aCEnTywfVmrsCWDsHOzAPuUsreLMTlGqflOXHDllU6tLMeUHWPGRzjdXCNC-xhhPcLP2UiCKI=  -k -H "Private-Token: <token>"

PATCH

Patch request is used to update desired or reported state of device twin. It does not change omitted fields.

Example PATCH request for new state from device:

curl -XPATCH  https://<tenant>.<host>/api/projects/<project>/device-twins/<deviceId> -k
 -H "Content-Type: application/json" -H "Private-Token: <token>" -d '{patch JSON}'

Request body:

{
  "desired" : [ {
    "t" : 1585645427,
    "v" : 15.0,
    "n" : "urn:dev:ca:ca:project:device:property"
  } ]
}

Response:

{
  "desired" : [ {
    "t" : 1585645427,
    "v" : 15.0,
    "n" : "urn:dev:ca:ca:project:device:property"
  } ],
  "reported" : [ {
    "t" : 1585645427,
    "v" : 10.0,
    "n" : "urn:dev:ca:ca:project:device:property"
  } ]
}

MQTT Device API

Channel setup

Devices communicating with device twin over MQTT must use Channels. When creating new Channel it is important to set commandControl flag to true. Example DTO to create channel using POST to Channels API:

```json5
 {
   "id": "myChannel",
   "type": "MQTT",
   "password": "password",
   "devices": [ "dev1", "dev2", "dev3" ],
   "contentType": "application/senml+json",
   "telemetry": true,
   "commandControl": true,
 }
 ```

Channel permissions

Setting commandControl to true gives:

write permissions to cc/<format>/<tenant>/<project>/<channel>/get,
write permissions to cc/<format>/<tenant>/<project>/<channel>/update,
read permission for cc/<format>/<tenant>/<project>/<channel>/state.

Protocol

{
  "deviceId": <string>,
  "hubId": <string>,
  "messageId": <number - integer>,
  "version": <number>,
  "state": {
    "desired": <application/senml+json>,
    "reported": <application/senml+json>
  },
  "error": <string>
}

deviceId - used to identify device, multiple devices may access same MQTT channel, this is required to distinguish them.
hubId - optional, may be useful for devices behind hubs.
messageId - used to corelate requests to responses, it should be integer.
version - twin message protocol version, current version is 1.0.
state - desired and/or reported state of the device.
error - will be non-empty if get/update requests fail.

Topics

cc/senml_json/<tenant>/<project>/<channel>/state

Device should subscribe to this topic when it wants to receive responses to /get and /update calls. Also, if currentChannel is set then this topic receives updates whenever Patch or Put is used. Messages will not be retained so device can fetch them only if it is subscribed at the moment of emitting. Payload of state message will contain all available information.

cc/senml_json/<tenant>/<project>/<channel>/get

Device should send message to this topic when it wants to receive current state, only part of protocol message is required, ie:

{
  "deviceId": <string>,
  "hubId": <string>,
  "messageId": <number - integer>,
  "version": <number>,
}

cc/senml_json/<tenant>/<project>/<channel>/update

Device should report its state change by publishing messages to this topic. Message must contain reported field.

Rule Engine

Basic setup

Note	If you are familiar with Rule Engine basics, skip to Advanced setup.

Rule Engine is a tool which allows you to set up certain automatic actions in response to the events satisfying given conditions. The base concepts in Rule Engine are user-defined rules and actions.

Let’s say you want to be notified each time the temperature in your fridge exceeds 10°C. If this happens, you want to receive a request on your endpoint, or get an email. In this case, the condition imposed on the temperature is the rule, and the request or email is an action. Simply put, rules define the conditions that need to be satisfied by the events that arrive to the Platform (and can also describe how the input data will be processed), and when the conditions are met, they signal the actions to be executed. Rules can also signal the actions to be cancelled in some use cases – but this will be covered later.

Rules and actions are not created per device – the rules will be matched against all the events from all the devices if not specified otherwise. You can specify the devices and properties you want to monitor when creating the rule.

Actions include:

Sending an email;
Sending an HTTP request to a given endpoint;
Sending an MQTT message to a given host and topic;
Generating a system alert.

Actions can be executed immediately after the event arrives or can be scheduled for later execution. In the later case, a cancellation signal can be used to remove scheduled executions. Let’s stick with the fridge example – imagine you want to be notified when the temperature exceeds 10°C for longer than 10 minutes. So if a correct (lesser than 10°C) temperature event arrives to the Platform within 10 minutes of the first "abnormal", rule-triggering event, the rule engine will send a cancellation signal and prevent the action from happening.

This section consists of two parts – first, you’ll learn the basics of rule engine, and then further possibilities (e.g. transformations and autoresolving) will be described.

Creating your first rule & action

This section provides an example which will walk you through the process of creating a rule. For testing purposes, we’ll create an HTTP action. You can later tailor the details to your needs. As said before, there are two key elements you’ll need to create – a rule and an action. Let’s create a rule first.

Note	It is not recommended to use the tab "Rules (deprecated)". It is not supported anymore, and will be deleted soon.

In your project view, go to Rules tab:

Click the "Add new" button to begin creating a new rule. You will see the first step of rule creation wizard.

Rule name is its identifier and the name shown on the dashboard. It should be informative to your own advantage, and it must be unique.
Description allows you to further describe the rule created, with no specific restrictions.
Status allows you to initially set the rule as active or inactive. By default, the rule is active.

Click Next. In the second step, you can add transformations. Let’s say the event that allows to the Platform is your input data – this step allows you to transform this data, e.g. choose which property you want to monitor, or which devices should be affected by this rule.

Note	You will see a banner informing you to create SIGNAL as last operation. SIGNAL is one of many supported methods, all of which are covered in Advanced setup – to create your first rule, you don’t have to know them. Just use SIGNAL as your only method.

Input is the input data of the transformation. For the first transformation, it is the event that arrives to the Platform – simply type event.
functor is what allows you to filter the data. In this example, we want the event property (this.iprop) to be equal to temperature: "{{this.iprop}}" == "temperature". This will result in catching temperature information from all the devices.

Click "Next" to proceed to the summary and "Add rule" to finish adding the rule. Keep note of the rule name – you will need it to create an action for it. We will now create an action – as an example, we will use the HTTP actioner. Other actions are covered in Advanced setup.

In the Rules view, click on the three dots symbol in the Options column, and select "Edit action":

Click "Add action". You will see action creation wizard. In the first field, "Type", choose "HTTP"; you will see the following fields:

Delay allow the actions to be triggered after a set time and are covered in Advanced setup. For now, skip this step;
Description should be short and informative of the action, but it is optional;
Body is the request body;
URI is the endpoint you want to send the request to;
Method is the HTTP method you want to use;
Headers are optional, but you can specify them in this section. If you are not using any headers, remove them using the trash icon.

Click "Add action". You now can test the rule to see if you configure everything correctly – instructions are covered in Testing rules.

Note	See Alerts Integration for details about managing alerts.

Advanced setup

Other actions – alert, MQTT and email

To set up other action in place of the HTTP action described before, simply choose the desired action from the drop-down list.

Note	Currently, you can see two Alerts tabs which correspond to two versions of the rule engine. The alerts sent by deprecated rule engine will be shown in "Alerts (deprecated)" tab, and the alerts from the new rule engine – in the "Alerts" tab.

Alert action

Delay key is a unique key which identifies the delay.
Delay duration should be input as e.g. '1h5m' for 1 hour and 5 minutes.
Alert key causes the alerts with the same key to have multiple occurences instead of creating many rows in the table.
Autoresolve makes the alert resolve itself after certain conditions are met.

MQTT action

URI – your MQTT host name. Remember to specify the tcp:// protocol;
Topic – the MQTT topic you want to write to;
Username – the MQTT username of the channel;
Password – the MQTT password to the channel;
Retained – true or false, specifies whether this message should be sent to every new subscriber automatically;
QOS – Quality of Service: 0 or 1, depending on your host settings;
Message – the MQTT message you want to publish.

Email action

Email – the email address of the recipient;
Subject – email subject;
Message – contents of the email.

Data types

The data types refer to what kind of data arrives to the Platform in an event. The data types used by Rule Engine are:

boolean – True or False, e.g. whether the device is online
number - IEEE-754 64-bit floating-point number, e.g. the temperature measured by a sensor
string - UTF-8-encoded text, e.g. a custom message sent by a device
object - equivalent of JSON Object aka "string to any" map
array - equivalent of JSON Array
any - any of the following types

Note	Arrays holding specific types are defined as *array[type], e.g. array[object]*

Templating

Rule Engine uses templating to inject dynamic data to rules and actions.

Note	Data available for injection depends on the context and is discussed in detail further down.

Note	Under the hood the mustache template engine is used and all the features that come with it can be used.

Note	Objects and arrays are injected in json format.

Whenever user want to use dynamically injected data, they can do so using the following notation.

{{value}}
{{object.child}}
{{object.nested.child}}

{{object}}
{{array}}

{{#array}}-{{.}}
{{/array}}

Which for context of:

{
  "value": 1,
  "object" : { "child": true, "nested": { "child": "foobar"} },
  "array": [1, false, "bazbar"]
}

Evaluates to:

1
true
foobar

{"child":true,"nested":{"child":"foobar"}}
[1,false,"bazbar"]

- 1
- false
- bazbar

Important

Please refer to mustache documentation and examples at the bottom for more details.

Operators

The following operator are available in Rule Engine:

Arithmetic operations

+ - add, e.g. 2 + 2 ⇒ 4
- - subtract, e.g. 4 - 2 ⇒ 2
* - multiply, e.g. 2 * 4 ⇒ 8
/ - divide, e.g. 8 / 2 ⇒ 4
% - modulo, e.g. 5 % 2 ⇒ 1
** - power, e.g. 3 ** 3 ⇒ 27
> - greater, e.g. 2 > 1 ⇒ true
>=- greater or equal, e.g. 2 >= 2 ⇒ true
< - less, e.g. 2 < 4 ⇒ true
<= - less or equal, e.g. 2 <= 2 ⇒ true
== - equal, e.g. 2 == 2 ⇒ true
!= - not equal, e.g. 2 != 4 ⇒ true

Text operators

+ - add, e.g. "a" + "b" ⇒ "ab"

> - greater (by lexical order), e.g. "b" > "a" ⇒ true
>=- greater or equal (by lexical order), e.g. "b" >= "b" ⇒ true
< - less (by lexical order), e.g. "a" < "b" ⇒ true
<= - less or equal (by lexical order), e.g. "b" <= "b" ⇒ true

== - equal, e.g. "foo" == "bar" ⇒ false
!= - not equal, e.g. "foo" != "bar" ⇒ true
=~ - regex LIKE e.g. "foo_bar" =~ "foo" ⇒ true
!~ - regex NOT LIKE, e.g. "foo_bar" !~ "baz" ⇒ true

Logical operations

! - not, e.g. !false ⇒ true
&& - and, e.g. 1 == 1 && "foo" == "foo" ⇒ true
|| - or, e.g. 1 == 2 || "foo" == "foo" ⇒ true

If one wants to compare strings, they must do so using quoted notation (especially when using templates as strings are injected without quotes) e.g. "first" == "second", "{{n}}" =~ "temperature".

Warning

All operations in Rule Engine are strongly typed, e.g. 2 is not the same as "2". While some assumptions can be made about the data types (e.g. while using regex logical operator’s) user should always be wary of the data types they are using. Ignoring data types can lead to undefined behavior.

Important

Remember to correctly escape data (e.g. \") when dealing with REST API

Functions

During injection functions can be called using mustache "lambda" feature.

Function calls use the following format: {{#functionName}}<arg1>,<arg2>,…{{/functionName}}.

See "functions documentation" section for list of available functions.

Warning

Functions inside exp.* package are experimental and can change without previous notice.

Input Data

Rule Engine is prepared in a way to allow using any JSON-formatted data input, although currently it only accepts telemetry events data sent by the devices in SenML format.

Note	Comarch Iot Platform uses extensions for the standard SenML fields. Extended fields names usually start with "i".

SenML data frame is a JSON Array of telemetry data JSON Objects. In the context of IoT Platform telemetry data object is a Device Property defined together with the device (TODO: LINK).

SenML data frame example:

[
  {
    // extension: tenant name as defined by the system administrator
    "it": "tenant_name",
    // extension: project name as defined by the user
    "ip": "project_name",
    // extension: channel name as defined by the user
    "ich": "channel_name",
    // extension: device name as defined by the user
    "idev": "device_name",
    // extension: property name as defined by the user
    "iprop": "property_name",
    // extension: unique identifier for this particular data frame
    "iuuid": "91e2cb40-a105-11eb-9078-fedbf7d090e5",
    "n": "urn::dev::ca::tenant_name::project_name::device_name::property_name",
    // one of v, vb, or vs will be present based on value type (number, boolean, string)
    "v": 10,
    "vb": true,
    "vs": "some_data",
    "t": 1618901291,
    // extension: when the event was processed on IoT Platform side
    "pt": 1618901565
  }
]

Evaluation process

Every data frame (aka event; telemetry send by a device) will start the rules evaluation process:

Rule Engine will select rules based on "it" and "ip" fields matching them to rules defined under specific project.
Using user defined data transformations event data will build up the transformation context - a data map containing both the original event and all intermediate data which is the result of data transformations.
Using user defined signal transformations signals will be sent to actions when specified conditions are met.

Note	Rules and actions are defined in the context of specific Tenant and Project. That means that under specific project every "it" and "ip" fields will always be the same across all records inside SenML array.

Rules

Rule is specified with:

name - unique name that identifies the rule
data transformations - a list of operations used to filter out and calculate new data
signal transformations - a list of operations used to signal actions to be executed when specified conditions are met

Note	Data and signal operations are defined in the same list, so there can be e.g. signal transformations defined in-between data transformations.

Important

A rule is considered valid only when it contains at least one signal operation.

Important

Rule’s transformations list must always end with a signal transformation for the rule to be considered valid.

Rule can also be temporarily disabled by the user.

Transformations

When defining a rule, user defines data and signal transformations.

Data transformation is a process of preparing the data for rule evaluation that includes data filtering, calculations etc. All output data is saved under user defined keys, building up the data context.
Signal transformation is a process of checking whether user-defined conditions are met and if so, signaling the actions to be executed.
All transformations can access and inject context data into their functors.

A single transformation is defined by:

Inputs - names of array/s which elements will be used in "for-each" fashion by this transformation
Operation - name of the operation to be executed
Functor - operation-specific function; can use current input element (available as {{this}}) or previous transformations results; usually evaluates to boolean, string or number
Output - name under which the result will be stored; result is operation-specific and will be one of data types.

Important

The original event (SenML telemetry data array) that invokes rule evaluation process is always accessible under a reserved name: event, hence event is of array[object] type, usable as transformation input. Most rules will use this input for the first transformation.

Note	Transformation are executed sequentially in the same order as defined in the rule.

Available transformations

Important

Transformations are also a way of filtering the data that is relevant for specified rule. If transformation results in empty (e.g. FIND or FIND_ONE produces empty array) rule processing is terminated, and the rule expression is not evaluated.

Data transformations

FIND
- selects elements from input array[type] and stores them as array[type] under output name
- selects elements based on functor that must evaluate to boolean, e.g. "{{iprop}}" == "temperature""
- terminates on empty result
FIND_ONE
- as above, but only selects single element
- terminates when size of result is != 1 (that is also when more elements match the specified functor).
MAP
- selects all elements from input array[object] and maps them to new object fields
- new object is stored under output name
- new object has fields constructed from functor that must evaluate to string
- each new object field stores object from original array
SUM
- selects all elements from input array
- sums their values based on functor that must evaluate to number
- stores result under output name as number
MEAN
- as above, but calculates mean of functor results

Signal transformations

SIGNAL
- checks the condition specified in functor (e.g. {{thermometer.v}} < 20) and if it’s true signals the actions to be executed
- can use optional input to send a signal for each element in the input array, where current element is sent together with the data context under {{this}} key
SIGNAL_CANCEL
- checks the condition specified in functor (e.g. {{thermometer.v}} >= 20) and if it’s true, a cancellation signal is sent to the actions
- can use optional input to send a cancellation signal for each element in the input array, where current element is sent together with the data context under {{this}} key.

Transformation Inputs Functor Output Output type

Transformation	Inputs	Functor	Output	Output type
FIND	1 required, e.g. `["event"]`	Filter. Must evaluate to boolean, e.g. `"{{this.iprop}}" == "battery_lvl" && {{this.v}} < 10`	Key under which found elements will be saved, e.g. `low_battery_levels`	array[object]
FIND_ONE	1 required, e.g. `["event"]`	Filter. Must evaluate to boolean, e.g. `"{{this.idev}}" == "entrance_thermometer" && {{this.iprop}}" == "temperature"`	Key under which found element will be saved, e.g. `entrance_temperature`	object
MAP	1 required, e.g. `["event"]`	Key. Must evaluate to string. Key under which current element will be stored inside output object, e.g. `"{{this.idev}}"`	Key under which new object (storing elements under functor key) will be saved	object
SUM	1 required, e.g. `["event"]`	Value to add to total sum. Must evaluate to number, e.g. `{{this.v}}`	Key under which resulting sum will be saved	number
MEAN	1 required, e.g. `["event"]`	Value to add to total sum before. Must evaluate to number, e.g. `{{this.v}}`	Key under which resulting mean will be saved	number
SIGNAL	0 or 1, e.g. `null, [], ["event"]`	Condition. Must evaluate to boolean, e.g. `{{entrance_temperature.v}} >= 50`. If true, a signal will be sent to actions	n/a	n/a
SIGNAL_CANCEL	0 or 1, e.g. `null, [], ["event"]`	Condition. Must evaluate to boolean, e.g. `{{entrance_temperature.v}} < 50`. If true, a cancellation signal will be sent to actions	n/a	n/a

FIND