Fetch Messages

Overview

If this web service is defined in account XML, the app will use it to check for new incoming messages and optionally also to sync message histories between different devices which use the same account. Messages sent on one device may be synced to other devices for consistency. The web service is called each time the app is moved to foreground and also when push notification about incoming message arrives.

For more information about sending push notification to the app, see Push Notifications over HTTP

Hint

To get full send/receive text message functionality via web services, all the following web services will have to be implemented:

  • Send Message to be able to send outgoing messages
  • Fetch Messages to be able to fetch incoming messages and also outgoing messages sent from other devices, implementing full server-side history synced between devices
  • Push Notifications over HTTP to be able to send push notifications to mobile devices about incoming messages
  • Push Token Reporter to obtain and store push addresses of devices and create mapping between usernames and push addresses where the username is being used.

If you receive an incoming message for user X on your server, you need to check the mapping created by Push Token Reporter and see that user X is using device with push token ABC. You can then send push notification to this device using Push Notifications over HTTP web service. Push notifications will wake up the app on this device and make it call the Fetch Messages, which will finally deliver the messages.

Parameters

Parameter templates for Fetch Messages web service can use any variables from Global parameters and Account parameters scopes. There are also additional, service-specific parameters for this service:

last_known_sms_id

This parameter will be replaced by sms_id field of the last message received in the past. When the response to this web service is processed, the app updates this identifier to the sms_id of the last message in received_smss collection.

Server can use this field to determine which messages need to be sent to client. In case there is no previous message, the app will send an empty string.

last_known_sent_sms_id

This parameter will be replaced by the sms_id of the last sent message. When the response to this web service is processed, the app updates this identifier to the sms_id of the last message in sent_smss.

Server can use this field to synchronize sent messages between devices which use the same account.

Important

The Send Message web service response should contain sms_id of the message which has been just sent. However, the app does not update last_known_sent_sms_id to this value; doing so would make the app skip all messages sent on other devices between the last Fetch response and the message just sent.

The consequence is that every message sent from a device will be received again on the same device in sent_smss collection of Fetch web service response. The app uses the sms_id from Send web service to detect and skip the duplicate message.

last_known_sending_date DEPRECATED

Similar to above, this parameter will be replaced by the most recent sending_date received previously. Note that apps will send this parameter in RFC3339 format with UTC timezone. If you use a different timezone in your response, the value here will be converted.

Server can use this field to determine which messages need to be sent to client. In case there is no previous message, the app will send an empty string.

Note

This field has been DEPRECATED and is preserved only for backward compatibility. Using timestamps doesn’t work well because of possible time differences between the server and client clocks. It is recommended to use the identifiers instead.

first_known_sms_id

This parameter will be replaced by sms_id field of the oldest message received in the past. When the response to this web service is processed, the app updates this identifier to the sms_id of the first message in receviced_smss collection but only if the response contains older field set to 1 or true.

Server can use this field to determine which messages need to be sent to client. In case there is no previous message, the app will send an empty string.

first_known_sent_sms_id

This parameter will be replaced by the sms_id of the oldest sent message known to the client. When the response to this web service is processed, the app updates this identifier to the sms_id of the first message in sent_smss but only if the response contains older field set to 1 or true.

Server can use this field to synchronize sent messages between devices which use the same account.

stream_id

This field is non-empty if the client wants to limit fetch to just one particular stream. It’s useful if the initial fetch does not contain the entire message history but only the few newest messages. The client can then fetch the older messages for a particular stream on demand when scrolling through the stream history.

Configuration

The following Account XML keys are relevant for Fetch Messages web service configuration:

genericSmsFetchUrl

Contains the URL, including URL scheme, of the web service, possibly also with query string.

Example:

https://example.com/fetch_messages/?username=%account[username]%&password=%account[password]%&last_id=%last_known_sms_id&last_sent_id=%last_known_sent_sms_id%&device=%installid%

genericSmsFetchPostData

If filled in, the app will use POST request to send the message.

Example ( for application/x-www-form-urlencoded):

username=%account[username]%&password=%account[password]%&last_id=%last_known_sms_id&last_sent_id=%last_known_sent_sms_id%&device=%installid%

Example ( for application/json ):

{
  "username"     : "%account[username]%",
  "password"     : "%account[password]%",
  "last_id"      : "%last_known_sms_id%",
  "last_sent_id" : "%last_known_sent_sms_id%",
  "device"       : "%installid%"
}

genericSmsFetchContentType

Specifies the value of Content-Type header to be sent in the request. If not specified, the app will default to application/x-www-form-urlencoded

Response

The response will be considered successful if the HTTP response code is 2xx.

Non-2xx responses are silently ignored.

Successful response can contain array of items under received_smss key (previously unread_smss and read_smss), which represent received messages, and optionally also an array of items under sent_smss key, containing sent messages. If you want to mark some received messages as read elsewhere you need to add displayed field with non-empty value (previously those messages were contained in read_smss array). Messages with displayed property will not appear as new and they will not be notified to the user.

Only “application/xml” and “application/json” content types are supported for the response. Each record has following fields:

sms_id:unique identifier of the message, set by provider. Optional, but if not provided, the app won’t check for duplicate messages.
sending_date:date when the message was sent, in RFC 3339 format.
sender:address of the sender. Present in incoming messages only. This field should work as destination address when sending a reply.
recipient:address of recipient. Present in outgoing messages only.
sms_text:body of the message, UTF-8 encoded
content_type:MIME content-type of the sms_text. Typically passed from Send Message. If omitted, text/plain is assumed.
disposition_notification:
 opaque string received from the Send Message. Can be omitted if it’s empty.
displayed:present in incoming messages only. If non-empty/true the message has already been displayed on other user’s device and should not be notified.
stream_id:identifies the stream the message belongs to (used in group chats)

The response should also contain date key, which represent the current date on the server and should be based on the same clock as sending_date. It must be in RFC 3339 format. It’s used to synchronize timestamps of incoming messages which are server clock based with the timestamps of outgoing messages which are device clock based.

Additional optional keys of the response related to message sync are the following:

older:non-empty/true if the listed messages are older than some specified first_known_sms_id, and first_known_sent_sms_id. Useful for syncing message histories. If the client is interested in fetching older messages in particular stream it will ask the web service again with stream_id and older (and first_known_sms_id and first_known_sent_sms_id variables set. The web service should respond with messages older than first_known_sms_id and first_known_sent_sms_id respectively. It needs to let the client know that the older mode is supported by including the older key (set to the same value as in the request) in the response.
stream_id:non-empty if the client requested messages from a particular stream only. It can be combined with other modes like older. The web service needs to include stream_id key (set to the same value as in the request) in the response to let the client know that this mode is suported.
limit:non-empty if the client requested (or server decided) to limit the total number of sent and received messages in the response. The web service needs to include limit key with the actual value in the response to indicate the number of messages was limited.
streams:array of streams the user belongs to along with the oldest sent and received sms_id in the particular stream. first_sent_sms_id, first_recv_sms_id and stream_id keys should be set for each stream item in the array.

See the examples below for better understanding.

Duplicate detection and removal

Received messages are checked for duplicities. Duplicate messages are detected based on and sms_id fields. In case your web service returns a message with sms_id which already exist on the client, it will be silently ignored.

Paging support

If you want to support syncing of the messages across multiple devices, there may be a problem with volume when the initial sync takes place. It’s recommended that only a handful (like 100) of newest messages are provided during the initial fetch. Initial fetch is identified by not having last_known_sms_id, last_known_sent_sms_id, first_known_sms_id and first_known_sent_sms_id variables set. In addition a list of all streams the user is joined to with the oldest sms_id for sent and received message in each stream should be sent in the initial fetch response, so the client can then fetch older messages in each stream. For this reason the fetch web service should be able to limit its response to a particular stream_id. To fetch older messages in a particular stream the client will set the older variable and stream_id variable that can be passed to the web service. If the web service supports these parameters it needs to let the client know about it and include older and stream_id (set to the same value as in request) in the response.

Examples

GET method, XML

request:

GET /fetch_messages/?username=johndow&password=12345678&last_id=&last_sent_id=&device=73C54F29105A0647 HTTP/1.1
Host: example.com
Connection: close
Cache-Control: max-age=0
User-Agent: CloudSoftphone/1.5.6

response:

HTTP/1.1 200 OK
Date: Sun, 15 Mar 2015 00:46:17 GMT
Server: Apache/2.4.7 (Ubuntu)
Vary: Accept-Encoding
Content-Length: 123
Keep-Alive: timeout=5, max=100
Connection: Keep-Alive
Content-Type: text/xml

<response>
    <date>2015-03-15T00:46:17.25Z</date>
    <received_smss>
        <item>
            <sms_id>E6E553673C54F29105A0647204C</sms_id>
            <sending_date>2014-12-19T16:39:57.31-08:00</sending_date>
            <sender>+1 (555) 123-1234</sender>
            <sms_text>Hello World</sms_text>
            <disposition_notification>xkXRBMCL.twvNcfC9:positive-delivery, display</disposition_notification>
            <displayed>1</displayed>
        </item>
        <item>
            <sms_id>E6E553673C54F29105A0647A4F0</sms_id>
            <sending_date>2014-12-19T16:39:59.77-08:00</sending_date>
            <sender>+1 (555) 123-1234</sender>
            <sms_text>Another message</sms_text>
            <disposition_notification>ori58sYj.5NZktzLK:positive-delivery, display</disposition_notification>
            <displayed>1</displayed>
        </item>
        <item>
            <sms_id>E6E553673C54F29105A0647A4F1</sms_id>
            <sending_date>2014-12-19T16:49:19.22-08:00</sending_date>
            <sender>+1 (555) 123-1234</sender>
            <content_type>application/x-acro-filetransfer+json</content_type>
            <sms_text>...encoded_content...</sms_text>
            <disposition_notification>g496ilaL.bvYAmkMm:positive-delivery, display</disposition_notification>
        </item>
        <item>
            <sms_id>E6E553673C54F29105A0647A4F2</sms_id>
            <sending_date>2014-12-19T16:49:25.44-08:00</sending_date>
            <sender>+1 (555) 123-1234</sender>
            <content_type>application/x-acro-disposition-notification</content_type>
            <sms_text>...encoded_content...</sms_text>
        </item>
    </received_smss>
    <sent_smss>
        <item>
            <sms_id>12345</sms_id>
            <sending_date>2018-12-19T16:39:57.31-08:00</sending_date>
            <recipient>+1 (555) 123-4321</recipient>
            <sms_text>Sent from another device</sms_text>
            <content_type>text/plain</content_type>
        </item>
    </sent_smss>
</response>

POST method, JSON, initial fetch

request:

POST /fetch_messages/ HTTP/1.1
Host: example.com
Connection: close
Cache-Control: max-age=0
User-Agent: CloudSoftphone/1.5.6
Content-Type: application/json
Content-Length: 183


{
    "username"      : "johndow",
    "password"      : "12345678",
    "last_id "      : "",
    "last_sent_id " : "",
    "first_id "     : "",
    "first_sent_id ": "",
    "older"         : "",
    "stream_id"     : "",
    "device"        : "73C54F29105A0647"
}

response:

HTTP/1.1 200 OK
Date: Sun, 15 Mar 2015 00:46:17 GMT
Server: Apache/2.4.7 (Ubuntu)
Vary: Accept-Encoding
Content-Length: 183
Keep-Alive: timeout=5, max=100
Content-Type: application/json
Connection: Keep-Alive

{
    "date"          : "2015-03-15T00:46:17.25Z",
    "received_smss"   :
    [
            {
                "sms_id"        : "E6E553673C54F29105A0647204C",
                "sending_date"  : "2014-12-19T16:39:57.31-08:00",
                "sender"        : "+1 (555) 123-1234",
                "sms_text"      : "Hello World",
                "disposition_notification" : "xkXRBMCL.twvNcfC9:positive-delivery, display"
                "displayed"     : true
                "stream_id"     : "stream_1"

            },
            {
                "sms_id"        : "E6E553673C54F29105A0647A4F0",
                "sending_date"  : "2014-12-19T16:39:59.77-08:00",
                "sender"        : "+1 (555) 123-1234",
                "sms_text"      : "Another message",
                "disposition_notification" : "ori58sYj.5NZktzLK:positive-delivery, display"
                "displayed"     : true
                "stream_id"     : "stream_2"
            },
            {
                "sms_id"        : "E6E553673C54F29105A0647A4F1",
                "sending_date"  : "2014-12-19T16:49:19.22-08:00",
                "sender"        : "+1 (555) 123-1234",
                "content_type"  : "application/x-acro-filetransfer+json",
                "sms_text"      : "...encoded_content...",
                "disposition_notification" : "ori58sYj.5NZktzLK:positive-delivery, display"
                "stream_id"     : "stream_2"
            },
            {
                "sms_id"        : "E6E553673C54F29105A0647A4F2",
                "sending_date"  : "2014-12-19T16:49:25.44-08:00",
                "sender"        : "+1 (555) 123-1234",
                "content_type"  : "application/x-acro-disposition-notification",
                "sms_text"      : "...encoded_content..."
                "stream_id"     : "stream_1"
            }
    ],
    "sent_smss"   :
    [
            {
                "sms_id"        : "1234567889",
                "sending_date"  : "2014-12-19T14:39:57.31-08:00",
                "recipient"     : "+1 (555) 123-1234",
                "sms_text"      : "Some message",
                "disposition_notification" : "xkXRBMCL.twvNcfC9:positive-delivery, display"
                "stream_id"     : "stream_1"
            }
    ],

    "streams"   :
    [
            {
                "stream_id"     : "stream_1",
                "first_sent_sms_id" : "1234567889",
                "first_recv_sms_id" : "E6E553673C54F29105A06472000"
            },
            {
                "stream_id"     : "stream_2",
                "first_sent_sms_id" : "1234567888",
                "first_recv_sms_id" : "E6E553673C54F29105A06471000"
            },
            {
                "stream_id"     : "stream_3",
                "first_recv_sms_id" : "E6E553673C54F29105A06460000"
            }
    ],
    "limit" : 4
}

POST method, JSON, paging

request:

POST /fetch_messages/ HTTP/1.1
Host: example.com
Connection: close
Cache-Control: max-age=0
User-Agent: CloudSoftphone/1.5.6
Content-Type: application/json
Content-Length: 183


{
    "username"      : "johndow",
    "password"      : "12345678",
    "last_id "      : "E6E553673C54F29105A06480000",
    "last_sent_id " : "12345678AA",
    "first_id "     : "E6E553673C54F29105A0647204C",
    "first_sent_id ": "1234567890",
    "older"         : "1",
    "stream_id"     : "ABCDEFGH",
    "device"        : "73C54F29105A0647"
}

response:

HTTP/1.1 200 OK
Date: Sun, 15 Mar 2015 00:46:17 GMT
Server: Apache/2.4.7 (Ubuntu)
Vary: Accept-Encoding
Content-Length: 183
Keep-Alive: timeout=5, max=100
Content-Type: application/json
Connection: Keep-Alive

{
    "date"          : "2015-03-15T00:46:17.25Z",
    "received_smss"   :
    [
            {
                "stream_id"     : "ABCDEFGH"
                "sms_id"        : "E6E553673C54F29105A06472000",
                "sending_date"  : "2014-12-19T15:39:57.31-08:00",
                "sender"        : "+1 (555) 123-1234",
                "sms_text"      : "Some message",
                "disposition_notification" : "xkXRBMCL.twvNcfC9:positive-delivery, display"
                "displayed"     : true

            },
            {
                "stream_id"     : "ABCDEFGH"
                "sms_id"        : "E6E553673C54F29105A06472001",
                "sending_date"  : "2014-12-19T15:39:59.77-08:00",
                "sender"        : "+1 (555) 123-1234",
                "sms_text"      : "Another message",
                "disposition_notification" : "ori58sYj.5NZktzLK:positive-delivery, display"
                "displayed"     : true
            },
            {
                "stream_id"     : "ABCDEFGH"
                "sms_id"        : "E6E553673C54F29105A06472002",
                "sending_date"  : "2014-12-19T15:49:19.22-08:00",
                "sender"        : "+1 (555) 123-1234",
                "content_type"  : "application/x-acro-filetransfer+json",
                "sms_text"      : "...encoded_content...",
                "disposition_notification" : "ori58sYj.5NZktzLK:positive-delivery, display"
            }
    ],
    "sent_smss"   :
    [
            {
                "stream_id"     : "ABCDEFGH"
                "sms_id"        : "1234567889",
                "sending_date"  : "2014-12-19T14:39:57.31-08:00",
                "recipient"     : "+1 (555) 123-1234",
                "sms_text"      : "Some message",
                "disposition_notification" : "xkXRBMCL.twvNcfC9:positive-delivery, display"
            }
    ],
    "stream_id" : "ABCDEFGH",
    "older" : "1"
}