Tutorial: Webhooks

In order to notify an external component, out of the scope of the client application itself, when some data are updated within the Webcom database, Webcom provides a "webhook notification" function. Such notifications are triggered as HTTP POST requests to a given webhook. Common use cases are:

• a custom back end application can directly subscribe to some data and then get back notifications as incoming HTTP POST requests,
• a mobile application can add a subscription on a specific path, which will notify a preconfigured “custom SMS gateway server” in order to send an SMS to some device.

Configuration

Set up a specific HTTP endpoint on your custom server

Webhooks are HTTP requests that will be sent to specific HTTP endpoints. Their message format is described below. For testing purpose you may use: the https://webhook.site/ service and the Webcom notification simulator.

Webhook configuration

The simplest way to define a webhook is the "notifications" tab of the Webcom developer console. A webhook definition mainly consists in 3 parameters:

• The webhook label or identifier: it is used later by subscription functions to add or remove subscriptions through the corresponding webhook,
• One or more destination URI, where to send the notifications (as HTTP POST requests) and where you should have previously set up an HTTP endpoint. Each destination URI may be parametrized to enforce a valid HTTPS certificate (default) or allow invalid ones (useful on debugging purpose).
• A subscription duration, which specifies the validity period of a subscription on the corresponding webhook when the subscription doesn't set an explicit duration.

Optionally, a list of custom HTTP headers (pairs of header names and values) may be associated to a given webhook. This may be useful for example to handle an access token required by the targeted endpoint. Currently, only the X-webcom-webhook-token is allowed.

This configuration may also be done programmatically, typically in a devops perspective, using the following REST API:

WEBCOM_APP="<your-app-id>"
SECRET_KEY="<the-secret-key-of-the-Webcom-application" # find it in the "authentication" tab of the developer console

# Set a complete configuration with exactly one webhook labeled 'myWebhook'
curl -X PUT "https://io.datasync.orange.com/datasync/v2/$WEBCOM_APP/settings/webhooks"  \
     -H "Content-Type: application/json" \
     -H "Authorization: Bearer $SECRET_KEY" \
     -d '{"destinations":{"myWebhook":{"enabled":true,"subscribeDurationSeconds":3600000,"destinationUris":[{"uri":"https://webhook.site/<XXXX>","acceptAnyCertificate":false}],"headers":{"X-webcom-webhook-token": "bar"}}}}' 

# Check the configuration is correctly saved
curl -X GET "https://io.datasync.orange.com/datasync/v2/$WEBCOM_APP/settings"  \
     -H "Authorization: Bearer $SECKEY"

Subscription

Adding a subscription through webhook notifications relies on an API very similar to the one for mobile push notifications. It has the same limitations, except the size of the payload notifications, which is not limited, and the ciphering of notifications, which are not ciphered (because they are sent directly to the webhook without any intermediary).

Please note that webhook subscriptions are stored on the Webcom back end and remain persistent even after your client application shuts down (as opposed to real time subscriptions, which are bound to a callback function hosted by your client application and are therefore canceled as soon as it shuts down).

You may add subscriptions using the following API on the appropriate client SDK. In this example, we subscribe to the “child addition” and “child change” events (similar to the ones available for the real-time subscriptions on the "one/child/where/to/subscribe" data node (replace “<your-app>” with your actual application identifier):

// const app = Webcom.App("<your-app>"); // UNCOMMENT if you haven't yet an instance of your app!
// In this sample, you can use either `serverlessDb` or `serverlessDbLite`
const database = app.serverlessDb;
const auth = database.authentication;
// Get a reference to the node
let node = database.rootNode.relativeNode("one/child/where/to/subscribe");

// You MUST authenticate
auth.signInAsGuest();
// Subscribe to the "child addition" and "child change" events on the node
const webhook = Webcom.Webhook("myWebhook", "a context");
let subscription;
node.subscribe(Webcom.Event.Child.Addition.Change, webhook)
    .then(sub => subscription = sub);
// The returned subscription object may be further used to cancel or update the subscription

// update: change the subscribed event type
subscription.update(Webcom.Event.Child.Addition.Change.Removal);
// cancelation (the following two lines are equivalent)
subscription.cancel();     // EITHER directly on the subscription object
node.unsubscribe(webhook); // OR indirectly on the data node object

import com.orange.webcom.sdk.datasync.subscription.SubscribableEvent.*

val app = WebcomApplication.default
val manager = app.datasyncService.createManager()
val node = manager / "one/child/where/to/subscribe"
var subscription: Subscription
node.subscribeThroughWebhook(Child.AddedChanged::class, "myWebhook", context = "a context") { // it: WebcomResult<Subscription>
  when (it) {
    is WebcomResult.Success -> // the subscription has been accepted by the back end
      subscription = it.result
    is WebcomResult.Failure ->
      println("The subscription has been rejected! ${it.error.message}")
  }
}

let node = Webcom.defaultApplication.dataService.node(for: "one/child/where/to/subscribe")
let descriptor = DataSubscriptionDescriptor(eventType: .child(addition: true, change: true, removal: false))
node.subscribeThroughWebhook("myWebhook", context: "a context", descriptor: descriptor) { result in
    switch result {
    case let .success(subscription):
        print("subcription succeeded")
        // subscription.cancel() // may be used to cancel the subscription
    case let .failure(error):
        print("subcription error:", error)
    }
}

WEBCOM_APP="<your-app>"
AUTH_TOKEN="<Webcom-authentication-token>" # subscriptions require a valid authentication context
UID="<UID-of-the-user-authenticated-by-the-AUTH-TOKEN>"
KIND="webhook"

curl -X POST "https://io.datasync.orange.com/datasync/v2/$WEBCOM_APP/subscriptions/$UID?allowsUpdate=true" \
     -H "Content-Type: application/json" \
     -H "Authorization: Bearer $AUTH_TOKEN" \
     -d '{"notifFormatVersion": "v2", "kind":"'$KIND'", "destination":"myWebhook", "mode":"childEvent", 
          "childEventFilter":{"added":true, "removed":false, "changed":true}, "path":"/one/child/where/to/subscribe",
          "context":"a context"}'

SUBSCRIPTION_ID="<the-id-received-at-subscription>"

curl -X GET "https://io.datasync.orange.com/datasync/v2/$WEBCOM_APP/subscriptions/$UID/$SUBSCRIPTION_ID" \
     -H "Authorization: Bearer $AUTH_TOKEN"

You can now write some piece of data on the "one/child/where/to/subscribe" path of your Webcom application and check that your webhook endpoint receives a notification.

Simulation

It is possible to test webhook reception using a specific REST API. It consists in sending a fake notification to the webhook described in the request body, using the same format that the one for configuring webhooks (replace “<your-app>” with your actual application identifier):

WEBCOM_APP="<your-app>"

curl -X POST "https://io.datasync.orange.com/datasync/v2/$WEBCOM_APP/webhookSimulations" \
     -H "Content-Type: application/json" \
     -H "Authorization: Bearer $SECKEY" \
     -d '{"destination":{"enabled":true,"subscribeDurationSeconds":3600000,"destinationUris":[{"uri":"https://webhook.site/<XXX>","acceptAnyCertificate":false}],"headers":{"X-webcom-webhook-token": "foo"}}}'

Notification format

Now your webhook component has to decode received notifications. As webhooks are mainly targeted at back end components, currently none of the Webcom SDK (mainly aimed at front end applications) provides with an out-of-the-box decoding function. In the future, such a function is expected on the Webcom SDK for JavaScript for Node.js-based back end components.

Notifications are encoded in a JSON document with the following properties:

• data: a structured JSON object representing the notified event (see below), its structure is common to mobile push notifications,
• context: a string equal to the context parameter passed at subscription,
• auth: a JSON object representing the authentication details of the user that added the subscription, along the same format as the Webcom authentication model.

Notified event format

The format of notified events is common to both mobile push notifications (in this case, it is ciphered as a JWE and must be deciphered before decoding) and webhook notifications. It is useful when no Webcom SDK is available for decoding it.

It is a JSON object with the following properties:

The c property represents the updates on the children of the subscribed data node using a JSON object with 3 properties:

• a (added): JSON object mapping the keys of the added children to their values (or to true if s="OnlyKeys"),
• c (changed): JSON object mapping the keys of the changed children to their values (or to true if s="OnlyKeys"),
• r (removed): JSON array containing the keys of the removed children.

Revocation reasons

The possible revocation reasons (r property) are:

Payload size limitations

With mobile push notifications only, the payload size of the notifications is limited to 4 KBytes.

More precisely, the whole data payload of the notified event is serialized as a JWE base64 string, and the length of this string should not exceed 4064 bytes. The corresponding size of the non-ciphered JSON payload is about 4000 bytes for notifications without ciphering and generally a bit less than 3000 bytes for ciphered ones (this depends on the chosen cipher algorithm).

If the length is 4064 bytes-long or more:

• in "value" mode: the status property (s) is set to "TooBig" and the value property (v) is removed,
• in "childEvent" mode:
  • if possible, the status property (s) is set to "OnlyKeys", both additions (c.a) and changes (c.c) properties contain only keys, with true as mapped values,
  • otherwise, that is the size of the data payload with only keys and true as value is too large, the status field (s) is set to "TooBig" and the additions and changes properties are empty.

When data is missing because of payload size limitation, the only way for the application to retrieve them is to invoke explicitly read functions or realtime subscription function of the Webcom SDK API.