[Proposal] Improve client-side sync diagnostics

[rkistner]

Background

When using traditional online requests, when there is some issue, it is usually obvious: The request would fail, typically producing logs on the server and an error message on the client.

When using a sync engine, the cause is often removed from the symptoms: The offline query completes, but doesn't return the data the user expects, typically because it hasn't synced yet. But the reason for the data not syncing is often completely removed from the query. During development this is often debugged by checking client logs, but in production it becomes more difficult.

Some examples of potentially silent failures:

1. connect() was never called (status.connected == false && status.connecting == false).
2. Some error is present in the sync, including network errors, authentication errors, and temporary service errors (visible in downloadError).
3. There are pending operations in the upload queue, preventing downloads from being applied (need to query the queue).
4. The upload queue is clear, but the write checkpoint has not been acknowledged yet, possibly due to replication delays (need to query the write checkpoint).
5. Data is being synced, but is behind due to replication delays (has service alerts).
6. Data is being synced, but the expected data is not synced due to sync rule issues.

The idea

We want an easy way to identify the above scenarios for any particular user in production applications. It should:

1. Not require significant app-side code changes, such as implementing custom logging solutions.
2. Protect the user's privacy - no sensitive data should be sent without the user's consent.
3. Be easy to enable in an existing application in production.
4. Not introduce too much noise and/or overhead.

We should keep in mind that:

1. Having the information available on the service could be the most convenient place to debug the issue.
2. However, a big class of issues prevent connection to the service, either temporarily or permanently, so we also need a way to show the relevant failure info directly on the client.

Current workarounds

Some information is available in the client-side status updates, for example with the useStatus() hook in React. This includes:

1. Download errors.
2. Upload errors.
3. Connection status (not connected; connecting).
4. Sync in progress (downloading).
5. Last sync timestamp.
6. Busy uploading (uploading: true) - can be inferred to mean not applying changes due to pending uploads, but does not cover all cases.

However, there is a big class of issues missing, typically only visible in logs, if at all:

1. Replication delay.
2. Not applying changes due to pending write checkpoint.

Proposals

• Improved client status reporting
• Explicit sync call

Pending proposals

• Report sync status to the service