Why does RisingWave need checkpoint_frequency instead of just longer epochs?

[marigov]

Hello team! I'm trying to understand the architectural reasoning behind RisingWave's epoch and checkpoint design, specifically why we need the checkpoint_frequency parameter.

Configuration 1: Frequent epochs, infrequent checkpoints

ALTER SYSTEM SET barrier_interval_ms = 1000;     -- 1s epochs
ALTER SYSTEM SET checkpoint_frequency = 10;      -- Checkpoint every 10 epochs

Result: Checkpoints every 10 seconds

Configuration 2: Longer epochs aligned with checkpoints

ALTER SYSTEM SET barrier_interval_ms = 10000;    -- 10s epochs
ALTER SYSTEM SET checkpoint_frequency = 1;       -- Checkpoint every epoch  

Result: Checkpoints every 10 seconds (same outcome)

Why separate epochs from checkpoints? Is there a technical constraint I'm missing?

[lmatz]

Barrier/epoch determines the freshness, e.g. how fresh the materialized view is when being batch queried.

Checkpoint interval tradeoff between the time you spent on catching up when recovery and the performance during regular hours.

[marigov]

From the docs:

As is mentioned before, during checkpointing, every operator writes their changes of this epoch into storage. For the storage layer, these data are still uncommitted, i.e. not persisted to the shared storage. However, the data still need to be visible to that operator locally.

From this blog post:

From the user's point of view, a checkpoint is considered complete only when all local states of all operators in an epoch are uploaded and registered with the meta service. Both user query and recovery are based on the latest complete checkpoint.

Don't batch queries read from storage? If so, isn't the only state persisted in storage the one that's check-pointed?

[lmatz]

Don't batch queries read from storage? If so, isn't the only state persisted in storage the one that's check-pointed?

storage also includes states that are NOT persisted yet, which can also be visible to batch queries besides persisted states, depending on what users want.

[marigov]

which can also be visible to batch queries besides persisted states, depending on what users want.

How is that set?

Is there any documentation or code I could check that describes how this mechanism works?

Also, I had these default settings:

                  Name                  |             Value             |                       Description                       | Mutable 
----------------------------------------+-------------------------------+---------------------------------------------------------+---------
 barrier_interval_ms                    | 1000                          | The interval of periodic barrier.                       | t
 checkpoint_frequency                   | 1                             | There will be a checkpoint for every n barriers.        | t

I increased the checkpoint_frequency to 3600 and tried querying one of the upsert tables that consumes from a Kafka topic, but the state wasn't being updated. As soon as I dropped the checkpoint_frequency back to a low value, the updates were getting reflected when querying the table.