Delta Synchronization

Introduction

A mobile app created on the Appery.io platform can work even when there is no network connection. The platform provides several ways to work in an offline mode which are introduced in this document.

📘

Offline Support

An app can work in an offline mode only with the API Express generated services.

API Express offers two types of generated services:

1. Generated services.
2. 2-way sync generated services.

Generated Services

A generated service can be created from a relational database table or from the Appery.io database collection. The only requirement is that the collection/table has only one primary key.

When a mobile app is online, a generated service caches the response of find and read operations into the local cache.

When a mobile app goes into the offline mode, a generated service gets data from local cache.

All data modification (CREATE, UPDATE, DELETE) operations in the offline mode are stored in the Client History and are executed after switching back to the online mode.

The main disadvantage of a generated service is controlling relevant data in the local cache and network overhead.

2-way Sync Generated Service

A 2-way sync generated service can be created from a relational database table or from the Appery.io database. This service has the following requirements:

1. Has only one primary key.
2. There is a column with the last modification date value filled by the database.
3. There is a column that indicates that the record was deleted from the database.
4. Genererated services are marked in API Express as 2-way sync.

A difference between a generated service and 2-way sync generated service is the presence of a FETCH operation.

AppClient keeps the last synchronization date information and based on this information fetch operation returns only a list of changed records from the last synchronization date.

The synchronization process works equally for a generated service and for a 2-way sync generated service.

When an app goes from the offline to the online mode, the synchronization process starts.

AppClient synchronizes operations from Client History one by one, in a not transactional mode. For example, if 10 data modification operations were made in the offline mode and 5 of them were performed successfully but the 6th operation failed, then AppClient stops the synchronization process and there is no option to automatically roll back the first 5 operations.

Example

To understand this process better let's see a simple example.

There is a collection called Label in the Appery.io database with only one custom field: name (type String).

A 2-way sync generated service was imported into the app via API Express Extension with the name of label.

This is the status of the client/server as you start going through this scenario:

• Collection on the server is empty.
• Client Cache is empty.
• Client History is empty.
• Last synchronization date variable (LastSyncDate) is not initialized.
• AppClient is in the online mode.

The following shows the current client/server status:

At this point, AppClient (app) creates the first label (name: item1) at 9:00. You can see the record added on the server:

AppClient invokes the FETCH operation (LastSyncDate is undefined).

API Express returns all records. In our example, it is only one record: item1.

At this point, the data on the client and server are the same. The LastSyncDate is 9:00.

Next, item1 was updated to item1_1. The update happened not from our mobile app but from an external system such as an external API or a web application. This update happened at 9:10 as shown below:

AppClient invokes the FETCH operation (LastSyncDate is 9:00).

API Express returns all records where the value of _updatedAt field is later than 9:00.
In our case, API Express returns item1_1. This step is shown below:

The data on the client and the server is the same. LastSyncDate is 9:10.

item2 on the server was created again from an external application such as an API or a web application at 9:20 as shown below:

At this point, AppClient invokes the FETCH operation (LastSyncTime is 9:10).

The FETCH operation returns item2 to the client.

Data on the client and server is the same. LastSyncDate is to 9:20.

AppClient (app) deletes item1_1 at 9:30. AppClient invokes a delete REST API call and passes item1_1 and value of the _updatedAt field.

At this point, API Express verifies that the value of _updatedAt field of item1_1 from the mobile app matches with the server value and marks the column as deleted (set the value of deleted field to true).

If the server returns a successful response, AppClient removes item1_1 from local cache.

Data on the client and the server side is now different.

Database on the server contains information about the deleted item when AppClient doesn't. LastSyncDate is 9:20.
AppClient invokes the FETCH operation. LastSyncDate is 9:20.

API Express returns item1_1 and flags that record as deleted.

AppClient can't find item1_1 in local cache and considers this situation as correct.

LastSyncDate is 9:30.

AppClient now goes into the offline mode.

AppClient invokes the FIND operation and gets the response from local cache (in our case item2).

AppClient updates item2 to item2_1.

The UPDATE operation was recorded into the Client History with new and old values (old value is required for the REVERT operation and will be explained later).

AppClient creates item3.

item3 was created with a temporary _id (t_3).

The CREATE operation was recorded into Client History.

AppClient deletes item2_1.

item4 on the server was created from an external source (for example, external API) at 9:40.

AppClient goes into the online mode from the offline mode.

Since Client History contains operations, the synchronization process is started.

AppClient performs all operations one by one and starts from updating item2 despite that item2 was eventually deleted.

Let's suppose that the synchronization process begins at 9:50 and every operation is performed for 1 minute.

AppClient invokes the first operation UPDATE and passes item2_1 to the server.

item2 was updated on the server (field _updatedAt was also updated).

After UPDATE is successfully performed, it is deleted from Client History.

AppClient invokes the second operation, CREATE and passes item3 to the server.

item3 is created on the server with _id = 4 and _updatedAt = 9:51.
AppClient updates _id and _updatedAt of item3 in local cache.

AppClient invokes last operation DELETE and passes item2_1 to the server.

item2_1 is marked as deleted and _updatedAt equals 9:52.

Since all operations were performed without errors AppClient goes into the online mode.

AppClient invokes the FETCH method and passes LastSyncDate = 9:30.

The server returns records item2_1 (deleted), item4 and item3.

LastSyncDate is 9:52.