Tutorial: Webhooks

Serverless DatabaseNotifications Webhooks

In order to notify an external component, out of the scope of the client application itself, when some data are updated within the Webcom database, Webcom provides a "webhook notification" function. Such notifications are triggered as HTTP POST requests to a given webhook. Common use cases are:

  • a custom back end application can directly subscribe to some data and then get back notifications as incoming HTTP POST requests,
  • a mobile application can add a subscription on a specific path, which will notify a preconfigured “custom SMS gateway server” in order to send an SMS to some device.

Configuration

Set up a specific HTTP endpoint on your custom server

Webhooks are HTTP requests that will be sent to specific HTTP endpoints. Their message format is described below. For testing purpose you may use: the https://webhook.site/ service and the Webcom notification simulator.

Webhook configuration

The simplest way to define a webhook is the "notifications" tab of the Webcom developer console. A webhook definition mainly consists in 3 parameters:

  • The webhook label or identifier: it is used later by subscription functions to add or remove subscriptions through the corresponding webhook,
  • One or more destination URI, where to send the notifications (as HTTP POST requests) and where you should have previously set up an HTTP endpoint. Each destination URI may be parametrized to enforce a valid HTTPS certificate (default) or allow invalid ones (useful on debugging purpose).
  • A subscription duration, which specifies the validity period of a subscription on the corresponding webhook when the subscription doesn't set an explicit duration.

Optionally, a list of custom HTTP headers (pairs of header names and values) may be associated to a given webhook. This may be useful for example to handle an access token required by the targeted endpoint. Currently, only the X-webcom-webhook-token is allowed.

This configuration may also be done programmatically, typically in a devops perspective, using the following REST API:

WEBCOM_APP="<your-app-id>"
SECRET_KEY="<the-secret-key-of-the-Webcom-application" # find it in the "authentication" tab of the developer console

# Set a complete configuration with exactly one webhook labeled 'myWebhook'
curl -X PUT "https://io.datasync.orange.com/datasync/v2/$WEBCOM_APP/settings/webhooks"  \
     -H "Content-Type: application/json" \
     -H "Authorization: Bearer $SECRET_KEY" \
     -d '{"destinations":{"myWebhook":{"enabled":true,"subscribeDurationSeconds":3600000,"destinationUris":[{"uri":"https://webhook.site/<XXXX>","acceptAnyCertificate":false}],"headers":{"X-webcom-webhook-token": "bar"}}}}' 

# Check the configuration is correctly savec
curl -X GET "https://io.datasync.orange.com/datasync/v2/$WEBCOM_APP/settings"  \
     -H "Authorization: Bearer $SECKEY"

Subscription

Adding a subscription through webhook notifications relies on an API very similar to the one for mobile push notifications. It has the same limitations, except the size of the payload notifications, which is not limited, and the ciphering of notifications, which are not ciphered (because they are sent directly to the webhook without any intermediary).

You may add subscriptions using the following API on the appropriate client SDK. In this example, we subscribe to the "child addition" and "child change" events (similar to the ones available for the real-time subscriptions on the "one/child/where/to/subscribe" data node (replace “<your-app>” with your actual application identifier):

// Create a reference to a Webcom application
var ref = new Webcom("<your-app>");
var node = ref.child("one/child/where/to/subscribe");
// You MUST authenticate
ref.authAnonymously();
// Subscribe to the "child addition" and "child change" events on the node
var subscription = ref.subscribe(Webcom.Event.Child.Addition.Change, Webcom.Webhook("myWebhook", "a context"));
// The returned subscription object may be further used to cancel or update the subscription
//import com.orange.webcom.sdkv2.datasync.subscription.SubscribableEvent.*

val app = WebcomApplication.default
val manager = app.datasyncService.createManager()
val node = manager / "one/child/where/to/subscribe"
var subscription: Subscription
node.subscribeThroughWebhook(Child.AddedChanged::class, "myWebhook", context = "a context") { // it: WebcomResult<Subscription>
  when (it) {
    is WebcomResult.Success -> // the subscription has been accepted by the back end
      subscription = it.result
    is WebcomResult.Failure ->
      println("The subscription has been rejected! ${it.error.message}")
  }
}

The event type to subscribe to is specified by the Kotlin class of one of the SubscribableEvent sub-interfaces. The value of the context parameter will be attached to all subsequent sent notifications (see below).
The expirationDate or duration parameters make it possible to limit the subscription to a specific validity period. If not set, the default duration associated to the targeted webhook (see the Webhook configuration section) is used.
Finally, the includesRevocation parameter allows notifying client applications when the subscription is revoked (because either it is expired or the client app explicitly unsubscribes).

let node = Webcom.defaultApplication.dataService.node(for: "one/child/where/to/subscribe")
let descriptor = DataSubscriptionDescriptor(eventType: .child(addition: true, change: true, removal: false))
node.subscribeThroughWebhook("myWebhook", context: "a context", descriptor: descriptor) { result in
    switch result {
    case let .success(subscription):
        print("subcription succeeded")
        // subscription.cancel() // may be used to cancel the subscription
    case let .failure(error):
        print("subcription error:", error)
    }
}

The REST API is the same POST request as the one for mobile push notifications:

WEBCOM_APP="<your-app>"
AUTH_TOKEN="<Webcom-authentication-token>" # subscriptions require a valid authentication context
UID="<UID-of-the-user-authenticated-by-the-AUTH-TOKEN>"
KIND="webhook"

curl -X POST "https://io.datasync.orange.com/datasync/v2/$WEBCOM_APP/subscriptions/$UID?allowsUpdate=true" \
     -H "Content-Type: application/json" \
     -H "Authorization: Bearer $AUTH_TOKEN" \
     -d '{"notifFormatVersion": "v2", "kind":"'$KIND'", "destination":"myWebhook", "mode":"childEvent", 
          "childEventFilter":{"added":true, "removed":false, "changed":true}, "path":"/one/child/where/to/subscribe",
          "context":"a context"}'

As for mobile push notifications, the JSON document passed in the request body may contain the following additional properties:

  • The mode and childEventFilter properties specify the subscribed event type (note that the latter property may combine several child events as shown in the example above):
mode childEventFilter Subscribed event type
"value" Value Change
"noData" Value Change, except that the data of the subscribed node is never included within the sent notifications
"chidEvent" {"added":true} Child Addition
"chidEvent" {"changed":true} Child Change
"chidEvent" {"removed":true} Child Addition
  • The receivesRevocations boolean property controls whether client applications are notified when the subscription is revoked (because either it is expired or the client app explicitly unsubscribes).
    If not set, revocations will be notified.
  • The expirationTimestampSeconds numeric property assigns a specific duration to the subscription.
    If not set, the default duration set up in the webhook configuration is used.

When successful, the REST request returns a JSON document containing a referential description of the just created subscription. In particular, the id property gives the identifier of the subscription, to use further with other REST requests to update or delete the subscription.

The received JSON document may be further retrieved using a GET request:

SUBSCRIPTION_ID="<the-id-received-at-subscription>"

curl -X GET "https://io.datasync.orange.com/datasync/v2/$WEBCOM_APP/subscriptions/$UID/$SUBSCRIPTION_ID" \
     -H "Authorization: Bearer $AUTH_TOKEN"

You can now write some piece of data on the "one/child/where/to/subscribe" path of your Webcom application and check that your webhook endpoint receives a notification.

Simulation

It is possible to test webhook reception using a specific REST API. It consists in sending a fake notification to the webhook described in the request body, using the same format that the one for configuring webhooks (replace “<your-app>” with your actual application identifier):

WEBCOM_APP="<your-app>"

curl -X POST "https://io.datasync.orange.com/datasync/v2/$WEBCOM_APP/webhookSimulations" \
     -H "Content-Type: application/json" \
     -H "Authorization: Bearer $SECKEY" \
     -d '{"destination":{"enabled":true,"subscribeDurationSeconds":3600000,"destinationUris":[{"uri":"https://webhook.site/<XXX>","acceptAnyCertificate":false}],"headers":{"X-webcom-webhook-token": "foo"}}}'

Notification format

Now your webhook component has to decode received notifications. As webhooks are mainly targeted at back end components, currently none of the Webcom SDK (mainly aimed at front end applications) provides with an out-of-the-box decoding function. In the future, such a function is expected on the Webcom SDK for JavaScript for Node.js-based back end components.

Notifications are encoded in a JSON document with the following properties:

  • data: a structured JSON object representing the notified event (see below), its structure is common to mobile push notifications,
  • context: a string equal to the context parameter passed at subscription,
  • auth: a JSON object representing the authentication details of the user that added the subscription, along the same format as the Webcom authentication model.

Notified event format

The format of notified events is common to both mobile push notifications (in this case, it is ciphered as a JWE and must be deciphered before decoding) and webhook notifications. It is useful when no Webcom SDK is available for decoding it.

It is a JSON object with the following properties:

Property Type Optional Description
a
(app id)
string Mandatory The identifier of the Webcom application that sends the notification.
p
(path)
string Mandatory Path of the subscribed data node.
t
(type)
string Mandatory Type of event: either "r" for a revocation event or "v" for a data event.
m
(mode)
string For data events only (t="v") Subscription mode among "value", "noData" or "chidEvent". This value is the same as the one set at subscription.
v
(value)
JSON value For data events with m="value" only The (new) value of the subscribed data node.
c
(children)
JSON object For data events with m="childEvent" Set of child updates on the subscribed data node (see format below).
r
(reason)
string For revocation events only (t="r") Error code representing the reason why the subscription has been revoked (see the list below).
s
(status)
string For data events only (t="v") Status of the conveyed values (v and c properties) among: "OK", "OnlyKeys" or "TooBig" (see below).

The c property represents the updates on the children of the subscribed data node using a JSON object with 3 properties:

  • a (added): JSON object mapping the keys of the added children to their values (or to true if s="OnlyKeys"),
  • c (changed): JSON object mapping the keys of the changed children to their values (or to true if s="OnlyKeys"),
  • r (removed): JSON array containing the keys of the removed children.

Revocation reasons

The possible revocation reasons (r property) are:

Status code Description
permission_denied Security rules prevent read access to the subscribed data node (according to the current data and the specified authentication token).
namespace_does_not_exist The Webcom application database doesn't exist or has just been deleted.
disabled For webhook notifications, the targeted webhook is disabled, enable it in the Webcom developer console -or-
The mobile push notifications are disabled for the Webcom application, enable them in the Webcom developer console.
expired The subscription has expired.
too_big The data located at the subscribed data node is too big and can no longer be subscribed (see weight limitations) because it would result in unreasonable performances

Payload size limitations

With mobile push notifications only, the payload size of the notifications is limited to 4 KBytes.

More precisely, the whole data payload of the notified event is serialized as a JWE base64 string, and the length of this string should not exceed 4064 bytes. The corresponding size of the non-ciphered JSON payload is about 4000 bytes for notifications without ciphering and generally a bit less than 3000 bytes for ciphered ones (this depends on the chosen cipher algorithm).

If the length is 4064 bytes-long or more:

  • in "value" mode: the status property (s) is set to "TooBig" and the value property (v) is removed,
  • in "childEvent" mode:
    • if possible, the status property (s) is set to "OnlyKeys", both additions (c.a) and changes (c.c) properties contain only keys, with true as mapped values,
    • otherwise, that is the size of the data payload with only keys and true as value is too large, the status field (s) is set to "TooBig" and the additions and changes properties are empty.

When data is missing because of payload size limitation, the only way for the application to retrieve them is to invoke explicitly read functions or realtime subscription function of the Webcom SDK API.