This web page is about the MQTT gateway bound to Webcom, also known as the "Webcom MQTT Gateway".
Table of contents:
- Requirements
- Overview
- Subscribe to MQTT -> Put/Push to Datasync
- Listen to Datasync -> Publish to MQTT
Requirements
In order to use the Webcom MQTT Gateway, you must fulfill the following steps:
- If you never logged on Webcom, you must create an account
- Once signed in, create at least one Webcom application
- Then, you must handle authentication whose complete explanation is beyond the scope of this document (refer to the Webcom documentation)
Only registered people will be able to configure the Webcom MQTT Gateway.
Overview
The goal of Webcom MQTT Gateway is to provide a convenient way for developers using Webcom to deal with their MQTT brokers in order to:
- subscribe to MQTT brokers and store any update to Webcom
- monitor data on Webcom and publish any update to MQTT brokers
First feature is called "Subscribe to MQTT -> Put/Push to Datasync", second one is called "Listen to Datasync -> Publish to MQTT". Following chapters explain in details what should be configured to use these two features. How to configure them is explained in another page. Please, read the following chapters before the other page...
Subscribe to MQTT -> Put/Push to Datasync
As a developer using Webcom, say that you have a temperature sensor somewhere, which value is periodically published to some specific topic on your favorite MQTT broker. And each time such a value is published, you'd like it to be stored on some path of your Webcom application. All you need to do is:
- provide the settings of your MQTT broker: URL, client id, authentication settings, topic...
- decide on which Webcom path you want this temperature value to be stored
Webcom MQTT Gateway will then subscribe to the appropriate topic on your MQTT broker and, each time a message is published, store the value contained in the message to the appropriate Webcom path.
Additionally, you may:
- decide if any update of your temperature value will:
- overwrite the previous one in Webcom through a PUT operation
- store it as an additional value to a new sub-path through a PUSH operation
- use a template in order to:
- extract a specific part of the MQTT message as the value
- build the Webcom path from MQTT message content
MQTT broker settings
Webcom MQTT Gateway needs to know how to reach your MQTT broker and which topic to subscribe to. Here are the settings we need (settings in bold are MANDATORY)
- the broker URL, for example: "
wss://iot.eclipse.org:443/ws" - the topic, for example "
my/topic" - your client ID, assigned to you by the broker,
- your username and password if any,
- the SSL certificate, needed to reach the broker, if any.
Webcom settings
Webcom MQTT Gateway needs to know how to store data in Webcom. Here are the settings we need:
- a front path, corresponding to the path if no template is in use (see below)
- a boolean, named overwrite, corresponding to:
- true: a PUT operation,
- false: a PUSH operation
- a template, which format is detailed below,
- a boolean, named use JSON, used to apply or not the template.
The template is made of reserved words, all starting with a "$" character, as follows:
$year,$month,$day,$hour,$minute,$secondand$millisecondreflecting the date when data is stored in Webcom (and not when it was published on MQTT broker)$payloadreflecting the entire JSON message retrieved from topic.
We use $payload to navigate through MQTT JSON message, separating field names with a dot, like for example: "$payload.room.temperature". Then, we can use these $payload.xxx combinations to build a Webcom path, using "/" character as a separator. The unique combination, after the last "/" separator, is considered as the value to store in the path built from previous combinations.
Warning:
- most of the time, there is a "payload" field in JSON messages retrieved from MQTT brokers. Pay attention not to be mistaken by its name and our
$payloadreserved word. Actually, the content of this "payload" field should be designed in our template by "$payload.payload". - do not try to build a Webcom path using a combination which reference is a JSON object or array in MQTT message. Only simple values (string, number, boolean) are correctly parsed, others are simply ignored.
Example #1:
Let us use an MQTT message with the following format:
{
"timestamp":"2018-07-25T09:34:34 561Z",
"payload": {
"room": {
"name":"kitchen",
"data": {
"temp":25,
"hygro":75,
"status":"normal"
}
}
}
}
Say that you just want the 'temp' value to be overwritten each time such a message is published in your /home/kitchen/temp Webcom path. In that case, you have to define the following settings:
- overwrite:
true - front path:
/home/kitchen/temp - use JSON:
true - template:
$payload.payload.room.data.temp
Example #2:
Now, say that you want the entire data regarding your kitchen in /home/kitchen Webcom path:
- front path:
/home/kitchen - template:
$payload.payload.room.data
This will store data in Webcom as follows:
|
|- home
|
|- kitchen
|
|- temp: 25
|- hygro: 75
|- status: normal
You may obtain the same result with the following settings:
- overwrite:
true - front path:
/home - use JSON:
true - template:
$payload.payload.room.name/$payload.payload.room.data
Here, "kitchen" is dynamically retrieved from the "name" field in the JSON message. The template dynamically built path from MQTT JSON field values, using the "/" character as a separator.
So, the previous example works again with the following settings:
- overwrite:
true - front path: nothing, defaults to
/ - use JSON:
true - template:
home/$payload.payload.room.name/$payload.payload.room.data
Remember that the unique $payload.xxx combination in the template, after the last "/" separator, is considered as the value to store in the path built from previous combinations. This is why we obtain "temp", 'hygro" and "status" sub-paths of the "/home/kitchen" path.
Example #3:
If you want to add a "data" sub-path to "/home/kitchen" path, you should use the following settings:
- overwrite:
true - front path: nothing, defaults to
/ - use JSON:
true - template:
home/$payload.payload.room.name/data/$payload.payload.room.data
Then, you obtain the following storage in Webcom:
|
|- home
|
|- kitchen
|
|- data
|
|- temp: 25
|- hygro: 75
|- status: normal
Example #4:
Let us see what happens with PUSH operations via the overwrite boolean:
- overwrite:
false - front path: nothing, defaults to
/ - use JSON:
true - template:
home/$payload.payload.room.name/$payload.payload.room.data
Then, you obtain the following storage in Webcom:
|
|- home
|
|- kitchen
|
|- -LAlgWvdSr-FAElPkBCo
| |
| |- temp: 25
| |- hygro: 75
| |- status: normal
|
|
|- -LAlgkVVQZgDjAhA2pJK
|
|- temp: 24
|- hygro: 80
|- status: normal
Notice that "-LAlgWvdSr-FAElPkBCo" and "-LAlgkVVQZgDjAhA2pJK" are randomly generated sub-paths to avoid overwriting data on Webcom.
Example #5:
You may have several of these sub-paths each day, however if you intend not to update data every day, or to keep only the last values of updated data each day, you could use the following settings:
- overwrite:
true - front path:
/home - use JSON:
true - template:
$year/$month/$day/$payload.payload.room.name/$payload.payload.room.data
Then, you obtain the following storage in Webcom:
|
|- home
|
|- 2018
| |
| |- 7
| |
| |- 24
| |
| |- kitchen
| |
| |- temp: 25
| |- hygro: 75
| |- status: normal
|
|
|- 2018
|
|- 7
|
|- 25
|
|- kitchen
|
|- temp: 24
|- hygro: 80
|- status: normal
Of course, it is also possible to keep last value of each month or even year.
Example #6:
And finally, you could mix the two previous examples in order to obtain several values each day, plus you could group date sub-paths to reduce tree levels:
- overwrite:
false - front path:
/home - use JSON:
true - template:
$year-$month-$day/$payload.payload.room.name/$payload.payload.room.data
Then, you obtain the following storage in Webcom:
|
|- home
|
|- 2018-10-24
| |
| |- kitchen
| |
| |- -LAlgWvdSr-FAElPkBCo
| | |
| | |- temp: 25
| | |- hygro: 75
| | |- status: normal
| |
| |
| |- -LAlgkVVQZgDjAhA2pJK
| |
| |- temp: 24
| |- hygro: 80
| |- status: normal
|
|
|- 2018-10-25
|
|- kitchen
|
|- -LHX7Xm5lsD7XqXYlP0D
| |
| |- temp: 23
| |- hygro: 74
| |- status: normal
|
|
|- -LHX9UEY4Ythei8_A0rm
|
|- temp: 22
|- hygro: 81
|- status: normal
Here are a few working and non working examples of grouped reserved words used to build sub-paths (and not value):
$year%$month.$day_$hour$minute$second$payload.room.name$payload.payload.room.name-$payload.payload.room.temp$payload.payload.room.status$payload.payload.room.name$year_$month_$day_$payload.payload.room.namefoobardoes not work as 'namefoobar' is misinterpreted ('$payload.payload.room.name-foobar' works)$year_$month_$day_$payload.payload.room.name.foobardoes not work as 'payload.room.name.foobar' references nothing in MQTT message$year_$month_$day_$payloaddoes not work as '$payload' alone, without any following '.xxx' item, is ignored$year_$month_$day_$payload.payload.room.datadoes not work as '$payload.payload.room.data' references a JSON object in MQTT message
to obtain respectively something like:
2018%10.25_170742kitchenkitchen-22normalkitchen2018_10_25_('$payload.payload.room.namefoobar', that cannot be retrieved in MQTT message, is ignored)2018_10_25_('$payload.payload.room.name.foobar', that cannot be retrieved in MQTT message, is ignored)2018_10_25_('$payload' alone, without any following '.xxx' item, is ignored)2018_10_25_('$payload.payload.room.data', as a JSON object, is ignored)
Warning: do not group combinations or use date reserved words after the last "/" separator. The only reserved word here may be $payload alone to retrieve the entire MQTT message (not very useful) or a unique $payload.xxx combination.
Example #7:
Another solution would be to use the "timestamp" field value from MQTT JSON message, as a path information. In that case, there is no need to use randomly generated sub-paths as this timestamp changes for every received MQTT message:
- overwrite:
true - front path:
/home - use JSON:
true - template:
$payload.payload.room.name/$payload.timestamp/$payload.payload.room.data
Then, you obtain the following storage in Webcom:
|
|- home
|
|- kitchen
|
|- 2018-07-24T01:53:56_560Z
| |
| |- temp: 25
| |- hygro: 75
| |- status: normal
|
|
|- 2018-07-25T09:34:34_561Z
|
|- temp: 24
|- hygro: 80
|- status: normal
Notice that forbidden characters in Webcom paths have been replaced by an underscore character (the space, in timestamp, before milliseconds in our example)
Listen to Datasync -> Publish to MQTT
As a developer using Webcom, say that any data stored on some path of your Webcom application should be, as soon as it changes, published on some topic of your favorite MQTT broker. All you need to do is:
- provide the settings of your MQTT broker: URL, client id, authentication settings, topic...
- decide which Webcom path you want to monitor (or "listen to")
Webcom MQTT Gateway will then listen to the appropriate Webcom path and, each time data is updated there, publish it through an MQTT message on appropriate topic of your MQTT broker.
MQTT broker settings
Webcom MQTT Gateway needs to know how to reach your MQTT broker and which topic to publish to. Here are the settings we need (settings in bold are MANDATORY)
- the broker URL, for example: "
wss://iot.eclipse.org:443/ws" - the topic, for example "
my/topic" - your client ID, assigned to you by the broker,
- your username and password if any,
- the SSL certificate, needed to reach the broker, if any.
Webcom settings
Webcom MQTT Gateway only needs to know which path to monitor on Webcom. Here is the setting we need:
- listen path: the path to monitor, along with all its sub paths
On Webcom, when you listen to a path, you also listen to all its sub paths, and if something changes only for a sub path, you receive information about data change for this sub path only. In that case however, Webcom MQTT Gateway will publish the entire data stored on the path, and not only data regarding the updated sub path.
Example #1:
Let us work with the Webcom data below, and say that your listen path is /home/kitchen:
|
|- home
|
|- kitchen
|
|- temp: 25
|- hygro: 75
|- status: normal
Each time something changes on at least one of "temp", "hygro" and "status" sub paths, Webcom MQTT gateway will publish the following data to your MQTT broker (example if temperature has decreased to 24):
{
"temp":24,
"hygro":75,
"status":"normal"
}
So, if you only want to publish MQTT messages about temperature, you have to use /home/kitchen/temp as the listen path and Webcom MQTT gateway will publish its only value (24 in this example) as a MQTT message.
Example #2:
Now, say that you need to format data published to your MQTT broker, for example with the famous "payload" field usually used in MQTT messages:
{
"payload": {
"temp":25,
"hygro":75,
"status":"normal"
}
}
As there is no template in "Listen to Datasync -> Publish to MQTT" feature, you need to cheat with Webcom and add a "payload" sub path level, like the following:
|
|- home
|
|- kitchen
|
|- payload
|
|- temp: 25
|- hygro: 75
|- status: normal
Of course, this only works if you keep /home/kitchen as your listen path.