BACE Webhooks

How to connect BACE to your IoT Application using Webhooks

Introduction

Webhooks are developed to send events to your application(s). The difference with classical API is that a webhook is used to actively push data to an external endpoint, instead of having the third-party poll for data.

In this chapter, we'll give a walkthrough on how to use Webhooks for measurements or events and provide some tips on how to use them most effectively for your application. A general introduction about how to use the BACE Webhooks is given in

How to use BACE Webhooks

Webhooks could be used to synchronize your asset measurements and events to your system. These measurements and events could be stored in your database to be ingested by your IoT application.

The BACE Webhook supports HTTPS URLs, and a static authentication header can be used to publish responses to subscribers. The data is sent by HTTPS Post requests.

For creating a new webhook using BACE Panel, please use following steps in Creating Webhooks

Sending Measurements as a Webhook

Measurements from your assets are forwarded as a webhook response in the order they are received by the IoT Connector. Depending on the sampling rate of IoT connector, you will receive a response for any new measurements retrieved from your assets.

If the IoT Connector cannot retrieve any new measurements from the device, for example when your asset is offline, you will not receive a webhook.

Example of one measurement

Here is an example webhook message for an IoT Connector with one measurement (aka datatype):

[{    
  "timestamp": 1634730271645,
  "timestamp_seconds": 1634730271.645,    
  "timestamp_changed": 1634730271645,    
  "datatype": 5000,    
  "value": 29.5489501953125,    
  "relations": {      
      "id_container": "caff51ce-d6e4-4329-abde-8b2a0dbc733d",      
      "id_group": "3cf26834-777e-4c46-ae0c-352de3509871",      
      "source_device": "16d7706e-b429-4778-8f8c-cfd6e2050649",      
      "id_container_data_latest": "46e6afab-f9d5-446d-bc75-c31b47e49be4"    
      },    
  "label": "Temperature",    
  "unit": "°C",    
  "icon": "fas fa-thermometer-three-quarters"  
}] 

Here is the breakdown of all attributes in the webhook response:

• Timestamp – Timestamp in milliseconds. This is the exact time of the measurement performed in the field.

• Timestamp_seconds – Timestamp in seconds, same as above.

• Timestamp_change – Timestamp in milliseconds. This is the time the value was last changed. For example, if the last value is 50, and the previous value is also 50, we don't change the timestamp from the first time we saw "50". Only when the value changes by >0.01 (in either direction) the timestamp_changed value updates to the time of that measurement.

• Datatype: Datatype of stored data. Each measurement type has a predefined or user configured datatype. An IoT Connectivity Module may be reporting the same data type (i.e. 5000 - Temperature) from many customer assets connected to it (i.e. Modbus #1 - Zwave #1)

• Value: Value of the measurement.

• Relations: Shows the relationship of the measurement:

  • id_container: Unique id of the container. Each IoT Connectivity Module will have a unique id_container.

  • id_group: Unique id of the group. Each IoT Connectivity Module will have a unique id_group. This field could be used to interpret which IoT Connectivity Module has sent the measurement.

  • source_device: Unique id of the IoT Connectivity Module or your asset sending the measurement. Each IoT Connector and Zwave device have their own unique ids. This field could be used to interpret which IoT Connectivity Module has sent the measurement. This value is same as id_physical_device explained in physical-device endpoint. The IoT Communication Module has a sticker that matches with label attribute in the physical-device endpoint's response.

  • id_container_data_latest: id_container of the latest data for this measurement. This could be useful for particular API calls if you want to request a specific measurement.

• label: Predefined or user configurable label of the measurement. You can use generic expressions like "Temperature" or use it in a more descriptive way like "SolarConverter-FaultCode"

• unit: Predefined or user configurable unit of the measurement.

• icon: User configurable field. This field could be used as a hint to your frontend what icon to use for a given datatype when displaying status indicators.

Each webhook response should contain measurements for one IoT Connector which could be differentiated by its unique group_id.

Example of Multiple Measurements

When your IoT Connector is configured to retrieve multiple measurements (aka datatypes) from your asset, you can see many data points in one webhook response. The example below shows a device with three different data types in the same response.

[
  {
    "timestamp": 1655381927834,
    "timestamp_seconds": 1655381927.834,
    "timestamp_changed": 1655381927834,
    "datatype": 2185,
    "value": 280,
    "relations": {
      "id_container": "e3ebb57a-d1b3-49ef-a5f3-f84307822b78",
      "id_group": "3f813212-63aa-49b5-aa71-c95acbf14ad6",
      "source_device": "1d551a5c-36e7-400c-a6ed-490bcc3815fa",
      "id_container_data_latest": "379c101e-0e40-4f55-8f09-d5f94d1ab2ec"
    },
    "label": "WTW-ZehnderE300-OutletTemperature",
    "unit": null,
    "icon": null
  },
  {
    "timestamp": 1655382346354,
    "timestamp_seconds": 1655382346.354,
    "timestamp_changed": 1655358046192,
    "datatype": 10000,
    "value": 165.91600036621094,
    "relations": {
      "id_container": "e3ebb57a-d1b3-49ef-a5f3-f84307822b78",
      "id_group": "3f813212-63aa-49b5-aa71-c95acbf14ad6",
      "source_device": "1d551a5c-36e7-400c-a6ed-490bcc3815fa",
      "id_container_data_latest": "543afeaa-8258-4129-8840-951e301b00eb"
    },
    "label": "Consumption 1",
    "unit": "kWh",
    "icon": null
  },
  {
    "timestamp": 1655382351594,
    "timestamp_seconds": 1655382351.594,
    "timestamp_changed": 1655382351594,
    "datatype": 10007,
    "value": 5.359999656677246,
    "relations": {
      "id_container": "e3ebb57a-d1b3-49ef-a5f3-f84307822b78",
      "id_group": "3f813212-63aa-49b5-aa71-c95acbf14ad6",
      "source_device": "6667e971-af9d-4027-8b49-2c5ca2e725b9",
      "id_container_data_latest": "10c0eb14-18fe-48aa-a1ab-c90a9348c45a"
    },
    "label": "Electrical Meter",
    "unit": "kWh",
    "icon": null
  }
]

Let's breakdown the webhook response above. From the array length, it is understood that IoT Connector with "id_group": "3f813212-63aa-49b5-aa71-c95acbf14ad6"has sent three measurements.

The first measurement has a descriptive label "label": "WTW-ZehnderE300-OutletTemperature", which includes asset model & name and name of the value. The report value is "value": 280 per the response.

The second measurement is for a measurement called "label": "Consumption 1", and measurement is "value": 165.91600036621094 in "unit": "kWh". First and second measurements have the same id_group and source_device.

Third measurement ia veryom a "label": "Electrical Meter" connected to IoT Connector. However, the third measurement is received from Z-wave plug so that's why it has a different source_device than the first two measurements.

Compact Format Overview

Certain gateways, such as BACE Go, can transmit a considerable volume of data, potentially exceeding 1MB in payload size when employing the standard format. Some backend services may find it overwhelming to process such large payloads.

Compact format is optimized to transmit essential data without verbosity. It's a way to retain crucial information in a concise manner to reduce payload size.

{
  "groups": {
    "b85dda65-1a26-49ac-a049-5e37e3dd3cf7": {
      "devices": {
        "fac7670d-a64d-4071-a4c0-a01e23d4c8e3": {
          "datatypes": {
            "2100": {
              "label": "Voltage",
              "id_container_data_latest": "d75b196e-da34-4886-a2b3-9aa7ac7b6928",
              "data": [
                [
                  1692781843650,
                  230.5
                ]
              ]
          }
        }
      }
    }
  }
}

Here is the breakdown of all attributes in the webhook response:

• groups – Unique id of the group. Each IoT Connectivity Module have a unique id_group. This field could be used to interpret which IoT Connectivity Module has sent the measurement.

• devices: Unique id of the IoT Connectivity Module or your asset sending the measurement. Each IoT Connector and Zwave device have their own unique ids. This field could be used to interpret which IoT Connectivity Module has sent the measurement. This value is same as id_physical_device explained in physical-device endpoint. The IoT Communication Module has a sticker that matches with label attribute in the physical-device endpoint's response.

• datatypes: Datatype of stored data. Each measurement has a predefined or user configured datatype. If the event is related to a specific data type, it will be reported in this attribute. A good example is for Modbus Read-error which might occur when IoT Connector is trying to poll information from a specific modbus register assigned to a data type. Each nested datatype has three attributes:

  • label: A label can either be predefined or user-defined. It can be generic like "Temperature" or descriptive like "SolarConverter-FaultCode"

  • id_container_data_latest: id_container of the latest data for this measurement. This could be useful for particular API calls if you want to request a specific measurement.

  • data: An array where the first item indicates the timestamp (in milliseconds) of the measurement, and the second item denotes its value

Sending Events as a Webhook

Like measurements, any events from your assets, IoT Communication module or IoT connector are forwarded as a webhook response to your HTTP address.

Here is an example webhook message for an IoT Connector with one measurement:

[
  {
    "id_event": 299663,
    "id_group": "4df124e4-160d-46cf-b19c-3411e6280e3c",
    "source_device": "e67c9edc-13ab-48cc-8802-765f6d185d52",
    "from_device": 1,
    "event_type": "202",
    "event_type_label": "202 - Reset Cause on boot",
    "occurred_at": 1660223629,
    "value": "1",
    "datatype": null,
    "event_value_label": "ESP_RST_POWERON",
    "event_value_description": "Reset due to power-on event",
  }
]

Here is the breakdown of all attributes in the webhook response:

• id_event – Unique id of the event.

• id_group – Unique id of the group. Each IoT Connectivity Module have a unique id_group. This field could be used to interpret which IoT Connectivity Module has sent the measurement.

• source_device – Unique id of the IoT Connectivity Module or your asset sending the event. Each IoT Connector and Zwave device have their own unique ids. This field could be used to interpret which IoT Connectivity Module has sent the measurement. This value is same as id_physical_device explained in physical-device endpoint. The IoT Communication Module has a sticker that matches with label attribute in the physical-device endpoint's response.

• from_device: If the event is generated by IoT Connector or your asset, the value is 1. If event is generated by IoT Communicator, the value is 0.

• event_type – Type of the event. Each event has a different event, (i.e. 202 is an event type reporting connectivity changes and 207 is an event type reporting Over The Air update initiation.

• event_type_label: Label of the data. A brief description of the event occurred. In this example above, the IoT Connector is reporting a boot start event.

• Occured at – Timestamp in seconds. This is the exact time of the event occurred.

• Value: Value of the event. Some event types have a value to provide more details about the event. The label and description of the event is included in the response with two different attributes. As an example, Event 202 is an event that occurs when IoT Connector is booting. The value is 1, if the device is powered on. The value is 3 if the reboot is due to a configuration change requiring reboot.

• Datatype: This will be always null for events

• event_value_label: Label of the event value. It will be published if the event has a value.

• event_value_description: Description of the event value when the event has a value.

Limitations

Webhook count:

We currently limit usage to 2 effective webhooks for a given project. We'll be happy to discuss other options if this doesn't accommodate your need.

Types of data:

Currently only measurements are pushed. However, sending IoT Connector events is in the roadmap.

Delivery Guarantee

While the Webhooks have a retry mechanism and will attempt to ensure no data is left unsent. This is a best effort approach.

The webhook mechanism will check the remote server’s HTTP status code. If the returned code is not a “success“ range (2xx) it will add the attempted data to a retry queue and make an attempt at a later time to resend it. Repeat failures are given an escalating back-off timeout, meaning that for every failed attempt to deliver the data, the next attempt is delayed even longer, to prevent the wasting of our server’s time attempting to deliver data to a party that is unreachable.

The resending of failed delivery attempts is done with no guarantee as to the order data is delivered in:

• New incoming data may be delivered before old data is retried.

• Retried data may be resent in a different order than it arrived in, due to the escalating timeout mechanism.

The retry queue is purged on a schedule:

• Messages that had to be retried, but were successful, are purged after 7 days (for record keeping to evaluate reliability).

• Messages that were not delivered are retained for 28 days. After 28 days undelivered data is deleted permanently. This means that if our server is unable to deliver a given data payload to the configured address for the webhook for longer than 28 days, that data is deleted

Testing The Webhook Functionality

The easiest way to test the webhook functionality is by using the external website like webhook.site. Upon opening this website, it will create a unique instance of a webhook endpoint just for you, which is valid for 24h.

You can share the 'Your Unique URL' in the webhook URL with us to connect any group of devices to it that is currently gathering data. As soon as data is available in IoT Connector, it will be forwarded to the webhook and should be visible on this website.