.. _fetch_messages_webservice: Fetch Messages (Modern API) =========================== Overview -------- The Fetch Messages web service allows the app to check for new incoming messages and to synchronize message history. Additionally, it can be used to synchronize message histories across multiple devices that are linked to the same account. This ensures that messages sent from one device are synced to other devices to maintain consistency. The Fetch Messages web service is triggered whenever the mobile app is brought to the foreground and whenever an incoming message push notification is received. For desktop applications, the service is checked periodically. The Fetch Messages web service can be configured via the :ref:`account-xml`. For more information about sending push notification to the app, see :ref:`http-push` .. hint:: To get full send/receive text message functionality via web services, all the following web services need to be implemented: - :ref:`send_message_webservice` to be able to send outgoing messages - :ref:`fetch_messages_webservice` to be able to fetch incoming messages and also outgoing messages sent from other devices, implementing full server-side history synced between devices - :ref:`http-push` to be able to send push notifications to mobile devices about incoming messages - :ref:`push_token_reporter_webservice` to obtain and store push addresses of devices and create mapping between usernames and push addresses where the username is being used. If you receive an incoming message for user X on your server, you need to check the mapping created by :ref:`push_token_reporter_webservice` and see that user X is using device with push token ABC. You can then send push notification to this device using :ref:`http-push` web service. Push notifications will wake up the app on this device and make it call the :ref:`fetch_messages_webservice`, which will finally deliver the messages. Parameters ---------- Parameter templates for Fetch Messages web service can use any variables from :ref:`global-params` and :ref:`account-params` scopes. There are also additional, service-specific parameters for this service: last_known_sms_id ~~~~~~~~~~~~~~~~~ This parameter will be replaced by ``sms_id`` field of the last message **received** in the past. When the response to this web service is processed, the app updates this identifier to the ``sms_id`` of the last message in ``received_smss`` collection. Server can use this field to determine which messages need to be sent to client. In case there is no previous message, the app will send an empty string. last_known_sent_sms_id ~~~~~~~~~~~~~~~~~~~~~~ This parameter will be replaced by the ``sms_id`` of the last **sent** message. When the response to this web service is processed, the app updates this identifier to the ``sms_id`` of the last message in ``sent_smss``. Server can use this field to synchronize sent messages between devices which use the same account. .. important:: The :ref:`send_message_webservice` web service response should contain ``sms_id`` of the message which has been just sent. However, the app does not update ``last_known_sent_sms_id`` to this value; doing so would make the app skip all messages sent on other devices between the last Fetch response and the message just sent. The consequence is that every message sent from a device will be received again on the same device in ``sent_smss`` collection of Fetch web service response. The app uses the ``sms_id`` from Send web service to detect and skip the duplicate message. last_known_sending_date ``DEPRECATED`` ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Similar to above, this parameter will be replaced by the most recent `sending_date` received previously. Note that apps will send this parameter in RFC3339 format with UTC timezone. If you use a different timezone in your response, the value here will be converted. Server can use this field to determine which messages need to be sent to client. In case there is no previous message, the app will send an empty string. .. note:: This field has been ``DEPRECATED`` and is preserved only for backward compatibility. Using timestamps doesn't work well because of possible time differences between the server and client clocks. It is recommended to use the identifiers instead. Configuration ------------- The following :ref:`account-xml` keys are relevant for Fetch Messages web service configuration: ``genericSmsFetchUrl`` ~~~~~~~~~~~~~~~~~~~~~~ Contains the URL, including URL scheme, of the web service, possibly also with query string. Example: .. code-block:: none https://example.com/fetch_messages/?username=%account[username]%&password=%account[password]%&last_id=%last_known_sms_id%&last_sent_id=%last_known_sent_sms_id%&device=%installid% ``genericSmsFetchPostData`` ~~~~~~~~~~~~~~~~~~~~~~~~~~~ If filled in, the app will use POST request to send the message. Example ( for application/x-www-form-urlencoded): .. code-block:: none username=%account[username]%&password=%account[password]%&last_id=%last_known_sms_id&last_sent_id=%last_known_sent_sms_id%&device=%installid% Example ( for application/json ): .. code-block:: json { "username" : "%account[username]%", "password" : "%account[password]%", "last_id" : "%last_known_sms_id%", "last_sent_id" : "%last_known_sent_sms_id%", "device" : "%installid%" } ``genericSmsFetchContentType`` ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Specifies the value of Content-Type header to be sent in the request. If not specified, the app will default to ``application/x-www-form-urlencoded`` Response -------- The response will be considered successful if the HTTP response code is 2xx. Non-2xx responses are silently ignored. Successful response can contain array of items under ``received_smss`` key (previously `unread_smss` and `read_smss`), which represent received messages, and optionally also an array of items under ``sent_smss`` key, containing sent messages. If you want to mark some received messages as read elsewhere you need to add ``displayed`` field with non-empty value (previously those messages were contained in `read_smss` array). The records under ``received_smss`` and ``sent_smss`` arrays should be sorted by sending_date in ascending order, meaning that the oldest message should be first in the array. Messages with ``displayed`` property will not appear as new and they will not be notified to the user. Only "application/xml" and "application/json" content types are supported for the response. Each record has following fields: :sms_id: unique identifier of the message, set by provider. Optional, but if not provided, the app won't check for duplicate messages. :sending_date: date when the message was sent, in :RFC:`3339` format. :sender: address of the sender. Present in **incoming** messages only. This field should work as destination address when sending a reply. :recipient: address of recipient. Present in **outgoing** messages only. :sms_text: body of the message, UTF-8 encoded :content_type: MIME content-type of the sms_text. Typically passed from :ref:`send_message_webservice`. If omitted, ``text/plain`` is assumed. :disposition_notification: opaque string received from the :ref:`send_message_webservice`. Can be omitted if it's empty. :displayed: present in **incoming** messages only. If non-empty/true the message has already been displayed on other user's device and should not be notified. :stream_id: identifies the stream the message belongs to The response should also contain ``date`` key, which represent the current date on the server and should be based on the same clock as ``sending_date``. It must be in :RFC:`3339` format. It's used to synchronize timestamps of incoming messages which are server clock based with the timestamps of outgoing messages which are device clock based. Additional optional keys of the response related to message sync are the following: See the examples below for better understanding. Duplicate detection and removal ------------------------------- Received messages are checked for duplicities. Duplicate messages are detected based on and `sms_id` fields. In case your web service returns a message with `sms_id` which already exist on the client, it will be silently ignored. Stream matching ------------------------------- The modern Fetch Messages API supports the concept of streams. A stream is a set of one or more users who can send messages to each other. Each message belongs to exactly one stream. The stream is identified by the `stream_id` field in the message record. This eliminates the need for workarounds for the potential stream matching problem present in the legacy Fetch Messages API, where a difference between the device local contact record and the server contact record (e.g. area code) could cause the app to not match the message to the correct stream. Examples -------- GET method, XML ~~~~~~~~~~~~~~~ request: .. code-block:: http GET /fetch_messages/?username=johndow&password=12345678&last_id=&last_sent_id=&device=73C54F29105A0647 HTTP/1.1 Host: example.com Connection: close Cache-Control: max-age=0 User-Agent: CloudSoftphone/1.5.6 response: .. code-block:: http HTTP/1.1 200 OK Date: Sun, 15 Mar 2015 00:46:17 GMT Server: Apache/2.4.7 (Ubuntu) Vary: Accept-Encoding Content-Length: 123 Keep-Alive: timeout=5, max=100 Connection: Keep-Alive Content-Type: text/xml 2015-03-15T00:46:17.25Z E6E553673C54F29105A0647204C 2014-12-19T16:39:57.31-08:00 +1 (555) 123-1234 Hello World xkXRBMCL.twvNcfC9:positive-delivery, display 1 E6E553673C54F29105A0647A4F0 2014-12-19T16:39:59.77-08:00 +1 (555) 123-1234 Another message ori58sYj.5NZktzLK:positive-delivery, display 1 E6E553673C54F29105A0647A4F1 2014-12-19T16:49:19.22-08:00 +1 (555) 123-1234 application/x-acro-filetransfer+json ...encoded_content... g496ilaL.bvYAmkMm:positive-delivery, display E6E553673C54F29105A0647A4F2 2014-12-19T16:49:25.44-08:00 +1 (555) 123-1234 application/x-acro-disposition-notification ...encoded_content... 12345 2018-12-19T16:39:57.31-08:00 +1 (555) 123-4321 Sent from another device text/plain POST method, JSON, initial fetch ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ request: .. code-block:: http POST /fetch_messages/ HTTP/1.1 Host: example.com Connection: close Cache-Control: max-age=0 User-Agent: CloudSoftphone/1.5.6 Content-Type: application/json Content-Length: 183 { "username" : "johndow", "password" : "12345678", "last_id " : "", "last_sent_id " : "", "stream_id" : "", "device" : "73C54F29105A0647" } response: .. code-block:: http HTTP/1.1 200 OK Date: Sun, 15 Mar 2015 00:46:17 GMT Server: Apache/2.4.7 (Ubuntu) Vary: Accept-Encoding Content-Length: 183 Keep-Alive: timeout=5, max=100 Content-Type: application/json Connection: Keep-Alive { "date": "2015-03-15T00:46:17.25Z", "received_smss": [{ "sms_id": "E6E553673C54F29105A0647204C", "sending_date": "2014-12-19T16:39:57.31-08:00", "sender": "+1 (555) 123-1234", "sms_text": "Hello World", "disposition_notification": "xkXRBMCL.twvNcfC9:positive-delivery, display", "displayed": true, "stream_id": "stream_1" }, { "sms_id": "E6E553673C54F29105A0647A4F0", "sending_date": "2014-12-19T16:39:59.77-08:00", "sender": "+1 (555) 123-1234", "sms_text": "Another message", "disposition_notification": "ori58sYj.5NZktzLK:positive-delivery, display", "displayed": true, "stream_id": "stream_2" }, { "sms_id": "E6E553673C54F29105A0647A4F1", "sending_date": "2014-12-19T16:49:19.22-08:00", "sender": "+1 (555) 123-1234", "content_type": "application/x-acro-filetransfer+json", "sms_text": "...encoded_content...", "disposition_notification": "ori58sYj.5NZktzLK:positive-delivery, display", "stream_id": "stream_2" }, { "sms_id": "E6E553673C54F29105A0647A4F2", "sending_date": "2014-12-19T16:49:25.44-08:00", "sender": "+1 (555) 123-1234", "content_type": "application/x-acro-disposition-notification", "sms_text": "...encoded_content...", "stream_id": "stream_1" } ], "sent_smss": [{ "sms_id": "1234567889", "sending_date": "2014-12-19T14:39:57.31-08:00", "recipient": "+1 (555) 123-1234", "sms_text": "Some message", "disposition_notification": "xkXRBMCL.twvNcfC9:positive-delivery, display", "stream_id": "stream_1" }] }