Notification

   Whenever CloudConnect receives some event data from an asset, it will forward the event to a configured webserver via a POST HTTP request, also known as WebHook. These "WebHooks" can be used to update an external database, mirror a backup, trigger mail alert or anything you want.

WebHooks can be triggered by the following events:

  • presence - When an asset connects or disconnects (see Presence Format for details).
  • message - When an asset sends a message on any channel (see Message Format for details).
  • track - When an asset sends tracking data (see Track Format for details).
  • poke - When a third-party application sends poke data (see Poke Format for details).


You can also download the current JSON schema here - attributes in the schema will not be removed but other attributes might be added in the future so your code should handle unknown attributes

Details

  • protocol version HTTP/1.1
  • support for secure connection via HTTPS
  • persistent connection (HTTP KeepAlive) enabled to reduce HTTP connection overhead
  • data type is given with the http header "content-type=application/json"

Warning

   A word about the content-type header: some technologies always interpret the content-type and can generate an error on your server if you do not properly manage the type in question. With WCF for example (windows communication foundation), you will need to tell WCF to not try to understand the request as JSON, and to instead just pass the raw request body to the parameter. You can do that using a WebContentTypeMapper.

   When an event is created on our servers, we'll POST a notification to your URL. The body of the POST will be a JSON-encoded document.
The JSON document is a list of event objects, each object has the account, event and payload field.

Here is an example body with 5 events:

[
  {
    "meta": {
      "account": "AccountExample",
      "event": "message"
    },
    "payload": {
      "asset": "359551XXXXX1234",
      "channel": "example.channel",
      "created_at": "2012-07-25T15:07:12Z",
      "parent_id": null,
      "parent_id_str": null,
      "b64_payload": "SGVsbG8gZXhhbXBsZSBwYXlsb2Fk",
      "received_at": "2012-07-25T15:07:12Z",
      "recipient": "@@server@@",
      "recorded_at": "2012-07-25T15:07:10Z",
      "sender": "359551XXXXX6317",
      "type": "message",
      "id": 339393147572322327,
      "id_str": "339393147572322327",
      "connection_id":630740379448115201,
      "connection_id_str":"630740379448115201"
    }
  },
  {
    "meta": {
      "account": "AccountExample",
      "event": "presence"
    },
    "payload": {
      "asset": "359551XXXXX5678",
      "time": "2012-07-25T15:07:10Z",
      "type": "connect",
      "reason": "cold_boot",
      "id": 339376253389766677,
      "id_str": "339376253389766677",
      "connection_id":630740379448115201,
      "connection_id_str":"630740379448115201"
    }
  },
  {
    "meta": {
      "account": "AccountExample",
      "event": "presence"
    },
    "payload": {
      "asset": "359551XXXXX6317",
      "time": "2012-07-25T15:07:10Z",
      "type": "disconnect",
      "reason": "client_disconnect",
      "id": 339376253389766678,
      "id_str": "339376253389766678",
      "connection_id":630740379448115201,
      "connection_id_str":"630740379448115201"
    }
  },
  {
    "meta": {
      "account": "AccountExample",
      "event": "poke"
    },
    "payload": {
      "asset": "359551XXXXX6317",
      "received_at": "2012-07-25T15:07:10Z",
      "sender": "Sender",
      "namespace": "Namespace",
      "id": 339376253389766698,
      "id_str": "339376253389766698",
      "b64_payload": "SGVsbG8gZXhhbXBsZSBwYXlsb2Fk"
    }
  },
  {
    "meta": {
      "account": "AccountExample",
      "event": "track"
    },
    "payload": {
      "id": 342656641079967767,
      "id_str": "342656641079967767",
      "asset": "359551XXXXX9012",
      "recorded_at": "2012-08-03T14:25:25Z",
      "recorded_at_ms": "2012-08-03T14:25:25.000Z",
      "received_at": "2012-08-03T14:26:28Z",
      "connection_id":630740379448115201,
      "connection_id_str":"630740379448115201",
      "index":84,
      "loc": [
        2.36687,
        48.78354
      ],
      "fields": {
        "GPS_SPEED": {
          "b64_value": "AAAAKg=="
        }
      }
    }
  }
]

packet size: Maximum packet size sent to the client server.

  • 1Mb by default (the recommended size)
  • the packet size of the request can be greater than this value because the component do not split an event to match the exact size
  • this value can be lowered, but only if there is small load. Otherwise, it is possible to receive more packets than you are able to read (and send the ack)

connection timeout: Connection time limit after which we will try to reconnect to the client's server and resend the current packet.

  • Default value is 15s
  • this timeout is invoked during the initial connection setup if the server is slow to respond or overloaded.
  • this value can be increased (up to 1 minute), but only if there is small load (for the same reasons).

inactivity timeout: Acknowledgment time limit after which we will try to resend the current packet.

  • default value is 15s
  • this timeout is invoked after a successful connection has been made, but no acknowledgment has been received from the server.
  • this value can be increased (up to 1 minute), but only if there is small load (for the same reasons).

parallel requests: Maximum number of requests

  • default value is 1
  • this value can be increase when needed in order to have a bigger throughput

Multiple data packets can be aggregated, and sent in one request to the client's server.

The 2 main use-cases:

  • The client's url is up and running with devices deployed. In this case, data packets arrive on the Cloud and they are notified a few seconds later. If a packet arrives alone, it is sent alone. If there is a large number of connected devices and/or a high data throughput however (which is directly related to the client's configuration), multiple packets are likely to arrive concurrently and will be bundled and sent together.
  • The client's url is down for maintenance. In this case, packets will be queued until the url is back up and running, and the queue will then be emptied by aggregating the maximum number of data packets that will fit in the configured packet size for the url and sending them in one request.

In either case, a variable number of data packets will be sent in one request. If enough data is aggregated to reach the upper size limit, it is not abnormal to have requests with several thousand tracking data (but that depends on how many field values each tracking datum contains, as per the device's configuration).

   If the customer's HTTP web server is not reachable or doesn't acknowledge the requests in time (responding with a 200 OK HTTP code), connections are retried with an exponential backoff. The maximum amount of time between 2 retries is around 5 minutes.

We'll keep the data internally to retry for 1 week. Older data can be notified again on demand only.

The data retention period depends on your production account configuration (default: 60 days).

   Within the same data packet (or request), events order per device is ensured. Due to our cloud architecture (distributed, real-time, asynchronous), events between different devices within the same data packet can arrive out of order.

   Duplicate events can happen during several situations, always to ensure that all events are correctly handled:

  • if a service is migrated or crashes duplicated events can be created and sent to the customers
  • if a customer's backend does not respond with 200 OK the service will retry sending the same data until it does

Deduplicating

If it fits your architecture, the simplest solution is to put a unique database index on id, and to ignore unique index violations at insert time. Otherwise, keep the ids of the notifications received in a recent time window, and drop duplicate ones.

Duration of the observation window

   In the above paragraphs we mention "waiting" or "old/recent" without specifying an actual duration or age. This is because everybody's latency vs correctness compromise is different. In most cases, a window of 10-20 seconds is a good choice that will soak up all the jitter. You may want a longer window for notifications that require some cloud-side post-processing and are not latency-critical on your side. You can use different values for different notification types. Mobile devices will notify clients if significant disruptions have happened. If unsure about your usecase, contact our customer relations.

   We can set your testing endpoint on your Integration account and also on your Production account in parallel with your current production endpoint.

Keep in mind that the same data stream will be sent to the 2 endpoints, so if they are connected to the same database, you should make sure you can handle duplicate data without issues. Filters can be added to limit the test to some devices until the test phase is finished.

Firewall

   We are not able to provide you with an IP or a range of IPs to add to the configuration of your firewalls. The source IP address of the data stream can change dynamically when the platform moves instances of the component to another host or adds a new host.

Basic access authentication

Basic access authentication is available and can be enabled.

Keep in mind that this method does not encrypt the connection: your data stream is still readable.

TLS without authentication

CloudConnect handles 'https' connections without authentication, your data stream will be encrypted between CloudConnect and your endpoint but the identify of both endpoints is not ensured.

TLS with mutual authentication

Mutual authentication is available and requires a certificate exchange.

You must send MDI a Certificate Signing Request (CSR)

  1. MDI will sign it with the CA related to your account
  2. MDI will send you the signed certificate

MDI will send your team a Certificate Signing Request (CSR)

  1. You must sign it with your CA
  2. You must send MDI the signed certificate

Links

MDI example

# This is a minimal openssl config file to create self-signed/root certificate autorities (CA) and 
# node (final) certificates.
# The intent is to have a root CA directly signing node certs, but intermediate CAs can also be used.
# Node certs (one for the server, one for the client) are expected to be requested by one peer and signed 
# by the other peer.
#
# General considerations:
# This documentation assumes a unix-like platform. Always make sure to use a recent version of OpenSSL.
# Consider tracking the generated files using revision control software, like git or mercurial.
#
# Configuration:
# The values in the req_distinguished_name section should be adapted to your project.
# Everything else should probably be left as-is.
#  should be a random integer between 1 and 2^159.
# No two certs signed by the same CA should have the same serial.
#  is either 'server' or 'client' depending on which side of the connection is going to use the cert.
#  is how many days (integer) until the certificate expire and need to be renewed.
#
# Creating the root self-signed CA:
# openssl req -x509 \
#        -config openssl.conf -extensions ext_rootca -set_serial  -newkey rsa:4096 -days  \
#        -keyout .rootca.key.pem -out .rootca.cert.pem
#
# Creating a certificate signing request:
# openssl req -new \
#         -config openssl.conf -extensions ext_nodereq -newkey rsa:2048 -nodes \
#         -keyout .key.pem -out .csr
#
# Creating a certificate (signing a csr):
# openssl x509 -req \
#         -extfile openssl.conf -extensions ext_nodesign -set_serial  -days  \
#         -CAkey .rootca.key.pem -CA .rootca.cert.pem \
#         -in .csr -out ..cert.pem
# mv .key.pem ..key.pem
# rm .csr


[ req ]
distinguished_name = req_distinguished_name
prompt             = no
string_mask        = utf8only

[ req_distinguished_name ]
C            = US                      # Country (2-letter uppercase)
ST           = MI                      # State (region/county/depatement...)
L            = Deadborn                # Locality (town/city)
O            = Thecustomer             # Organisation (company/project name)
OU           = MobileDevices           # Organisation unit (subproject)
emailAddress = support@thecustomer.io  # Contact details

[ ext_rootca ]
basicConstraints = CA:TRUE, pathlen:1
keyUsage = cRLSign, keyCertSign
authorityKeyIdentifier=keyid,issuer
subjectKeyIdentifier=hash

[ ext_intca ]
basicConstraints = CA:TRUE, pathlen:0
keyUsage = cRLSign, keyCertSign
authorityKeyIdentifier=keyid,issuer
subjectKeyIdentifier=hash

[ ext_nodereq ]
basicConstraints = CA:FALSE
keyUsage = nonRepudiation, digitalSignature, keyEncipherment

[ ext_nodesign ]
basicConstraints = CA:FALSE
keyUsage = nonRepudiation, digitalSignature, keyEncipherment
authorityKeyIdentifier=keyid,issuer
subjectKeyIdentifier=hash

We recommend using RequestBin to troubleshoot webhooks.