[Proposal] Explicit sync (db.checkpoint() api)

[rkistner]

Summary

Add a db.checkpoint() API for explicit sync feedback.

Status

2025-08-14: Split out sync status changes into a separate proposal, as well as the more general Improve client-side sync diagnostics proposal
2025-08-13: Initial idea proposed. Need to evaluate whether the proposed APIs would solve the problems, and how it would interact with future diagnostics reporting to the service.

Background

For background on the problem statement, see Improve client-side sync diagnostics.
The changes here depend on Write checkpoint / checkpoint request protocol changes.

Proposal

Add a call that creates a new checkpoint request, waits for the requested checkpoint sync, throwing an error if it fails.

await db.checkpoint(options?: { timeout?: number }).

This would:

1. Check that the connection is connected or connecting (does not automatically call connect()).
2. Create a new checkpoint request (does not just re-post an existing one).
3. Wait for that checkpoint to be synced back.

The call does not automatically create a new connection, or actually change in of the sync behavior, with the exception of:

1. The new checkpoint request being created.
2. Triggering a retry immediately if the connection is in a retry back-off state.

The Promise rejects if:

1. Any new error condition is reached (current errors are not reported).
2. The timeout is reached.

This means the call would wait for pending uploads, pending downloads, and pending write checkpoints. The timeout should be used sparingly - a long wait is expected when the client is downloading millions of operations. Note that #325 already adds a timeout on waiting_for_requested_checkpoint, meaning that no additional "inactivity timeout" is needed.

The rejected error should include the specific reason for failure.

TODO

• Are there better names than checkpoint()?
• Define the error format - this is very important to make these issues easy to diagnose

Use case / implementation examples

Push notifications

This is a common use case: The user received a push notification, opens the notification, and now expects the relevant data to be available. This API would explicitly wait until the data has been synced, or fail with an explicit reason if the data cannot be synced.

Pull-to-refresh

With a sync engine, users can generally expect data to be up-to-date all the time, and in most cases no explicit sync action is required. Selective usage of useStatus() can be used to indicate to users when that is not the case, for example indicating that the user is offline.

However, adding a "pull-to-refresh" action that calls checkpoint() could help the user gain additional confidence - it can be used to give the user an explicit indication that "your data is up-to-date".

Sync on startup

The checkpoint() API could be used as an explicit method to report sync status when the application is started, or at other points in the workflow. This could be used as a "request-style" alternative to waiting for a specific sync status.

For this use case, care should be taken to not block the user from continuing if the user could work with data previously synced.

Waiting for server-side processing

At a certain part of an application workflow, the client may need to wait for a service response to be synced back. For example, let's say the user requested a PDF report to be generated, and is now waiting for the response to be synced, containing the results (including URL of the PDF).

Here, there are two scenarios:

1. The server hasn't finished generating the report yet.
2. The report is generated, but results hasn't synced back yet.

While waiting, the client can periodically call checkpoint(), to make sure that the client is up-to-date. If the checkpoint() call resolves, and the response is still not synced, the client knows the server is still generating the report, rather than the response not being synced.

Issue reporting

checkpoint() errors could be good candidates to report to an error reporting service, for example:

await db.checkpoint().catch(error => {
  Sentry.captureException(error);
  throw error;
});

[simolus3]

I like the checkpoint() idea 👍 I think we should use a name that is less overloaded within PowerSync, but I also can't come up with something better. Maybe something like syncBarrier()? But that also feels very technical.

Having better insight into what the sync client is doing right now also sounds helpful to me. We've seen a number of reports asking about why download progress seemingly stops at 1.0. That's working as intended, but because downloadProgress is the only real-time feedback we have, it's understandable why one would expect 1.0 to mean "done".
So I wonder if we could make the SyncStatusEnum more detailed, or use unions / sealed classes to attach more details on the current step of the download progress, e.g.

interface SyncProcess {
  state: 'connecting' | 'idleConnected' | { downloading: DownloadProgress } | 'applyingChanges' | 'delayAfterError';
  // If we have a checkpoint that hasn't been applied, report why:
  outstandingChanges: null | 'pendingLocalWrites' | { waitingForCheckpoint: { expected: string, current: string } };
}

// and then on sync status:
process: SyncProcess | undefined;

[rkistner]

I think we should use a name that is less overloaded within PowerSync, but I also can't come up with something better.

To an extent I went with this name initially because it is related to checkpoints in the protocol - specifically "write checkpoints". Part of the issue is that the "write checkpoints" name isn't ideal - for example in this case it is used without any actual writes.

So I wonder if we could make the SyncStatusEnum more detailed, or use unions / sealed classes to attach more details on the current step of the download progress

How this is actually structured could be specific to the SDK. For example, Rust Enums would be perfect for exactly what you've described, but there's no direct equivalent in JS. You could use a structure like you've described, but actually using that in JS could be tricky - for your example it could end up with a weird combination of == and 'downloading' in status type checks. I don't have a good answer for this yet though - how can we keep it as simple as possible to get the high-level status, while also making it easy to dig deeper into the additional details specific to that status?

[simolus3]

I agree on using what each language has to offer, my main goal is to avoid confusing states (like download progress being at 100% when we're not really downloading anymore) at a type level. Looks like {status: 'connecting'} | {status: 'downloading', progress: DownloadProgress} | ... may be a more convenient way to express that in TS.

[rkistner]

Ok, I like that structure. But next problem: There are details that are very relevant for a specific status, such as DownloadProgress for downloading. But it is not exclusive to that - you could also want to know the download progress while disconnected. I'm wondering if we should have DownloadProgress specifically as part of the downloading status, and just also keep it in the general status? (We'd need that for backwards-compatibility anyway)

I guess one way to approach this is to think about what would typically be displayed to a user, and prioritize making the mapping from status -> UI as simple as possible.

Either way, this gives me some more good ideas to use - I'll refine it into another iteration of the status updates proposal.

[simolus3]

But it is not exclusive to that - you could also want to know the download progress while disconnected

Fair point, but we don't even have that today because we don't persist the target count for buckets (and reconnecting would likely make the progress go backwards a bit if a write happened while offline). But that is a technicality, I agree on generally simplifying the status to UI mapping.

[rkistner]

Moved the status changes to a separate proposal, since this one started getting big: #325

Still need to refine the actual design there.

[stevensJourney]

I also like the concept and functionality that a checkpoint() API would introduce.

This might be more of an implementation question, so maybe not very relevant for now - but I was wondering how the checkpoint() method would work if requested while there is an upload queue present - uploading == true. It might be implied, but I'm assuming we don't want to create a new write checkpoint while uploads are in progress.

I imagine the order would be something like:

1. Checks that the connection is connected or connecting (does not automatically call connect()).
2. Waits for pending uploads to be completed if present - create a new write checkpoint.
3. Wait for that checkpoint to be synced back.

[rkistner]

If there are uploads in progress, we can skip the new write checkpoint. But it also won't break anything either if we do create another one - it would simply be superseded by the one created after the uploads.

We don't actually have to wait for that specific write checkpoint - we just have to wait for any new checkpoint to sync, which implies that either that write checkpoint or a later one has been synced back.

[stevensJourney]

I might have missed something, but I was thinking along the lines of the Waiting for server-side processing use-case.

I imagine the dev code would look something like

await powersync.execute("insert into pdf_requests ...");
await powersync.checkpoint();
// Assuming the pdf request has made it to the server at this point. 

While waiting, the client can periodically call checkpoint(), to make sure that the client is up-to-date. If the checkpoint() call resolves, and the response is still not synced, the client knows the server is still generating the report, rather than the response not being synced.

But I imagine if there are pending uploads at the time of the first powersync.checkpoint() call, and if we don't wait for those before creating the checkpoint. The checkpoint we wait for might be synced down before the pdf_request has actually been uploaded.
This is probably fine if we recommend polling checkpoint() calls. But, I imagine if we did wait for uploads when creating that checkpoint, then we would at least be guaranteed (if the checkpoint request resolves) that the server is processing the request. Or even further, if the backend processes the pdf request in the uploadData request operation and stores the result before resolving the request, we could be guaranteed that the result has already synced back to the client.

[rkistner]

Ok, I see there is a race condition that could happen with the current implementation: If the local target_op is updated while there is data in the queue, then it would break things.

It's not a fundamental design issue, but it is an implementation detail we need to take into account.