Skip to content

Commit 8269db6

Browse files
committed
Add recommendations on non-idempotent operations
Fixes #5242
1 parent 8117972 commit 8269db6

File tree

2 files changed

+93
-0
lines changed

2 files changed

+93
-0
lines changed

doc/platform/sharding/vshard_admin.rst

Lines changed: 87 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -551,6 +551,93 @@ In a router application, you can define the ``put`` function that specifies how
551551

552552
Learn more at :ref:`vshard-process-requests`.
553553

554+
.. _vshard-deduplication:
555+
556+
Deduplication of non-idempotent requests
557+
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
558+
559+
**Idempotent requests** produce the same result every time they are executed.
560+
For example, a data read request or a multiplication by one are both idempotent.
561+
Therefore, incrementing by one is an example of a non-idempotent operation.
562+
When such an operation is applied again, the value for the field increases by 2 instead of just 1.
563+
564+
.. note::
565+
566+
Any write requests that are intended to be executed repeatedly should be idempotent.
567+
The operations' idempotency ensures that the change is applied **only once**.
568+
569+
A request may need to be run again if an error occurs on the server or client side.
570+
In this case:
571+
572+
- Read requests can be executed repeatedly.
573+
For this purpose, :ref:`vshard.router.call() <router_api-call>` (with ``mode=read``) uses the ``request_timeout`` parameter
574+
(since ``vshard`` 0.1.28).
575+
It is necessary to pass the ``request_timeout`` and ``timeout`` parameters together, with the following requirement:
576+
577+
.. code-block:: text
578+
579+
timeout > request_timeout
580+
581+
582+
For example, if ``timeout = 10`` and ``request_timeout = 2``,
583+
within 10 seconds the router is able to make 5 attempts (2 seconds each) to send a request to different replicas
584+
until the request finally succeeds.
585+
586+
- Write requests (:ref:`vshard.router.callrw() <router_api-callrw>`) generally **cannot be re-executed** without verifying
587+
that they have not been applied before.
588+
Lack of such a check might lead to duplicate records or unplanned data changes.
589+
590+
For example, a client has sent a request to the server. The client is waiting for a response within a specified timeout.
591+
If the server sends a successful response after this time has elapsed, the client will receive an error.
592+
When re-executing this request without additional check, the operation may be applied twice.
593+
594+
A write request can be executed repeatedly without a check only if the error occurred on the server side --
595+
for example, `ER_READONLY`.
596+
597+
**Deduplication examples**
598+
599+
To ensure that the write requests (INSERT, UPDATE, UPSERT, and autoincrement) are idempotent,
600+
you should implement a check that the request is applied for the first time.
601+
602+
For example, when you add a new tuple to a space, you can use a unique insert ID to check the request.
603+
In the example below within a single transaction:
604+
605+
1. It is checked whether a tuple with the ``key`` ID exists in the ``bands`` space.
606+
2. If there is no tuple with this ID in the space, the tuple is inserted.
607+
608+
.. code-block:: lua
609+
610+
box.begin()
611+
if box.space.bands:get{key} == nil then
612+
box.space.bands:insert{key, value}
613+
end
614+
box.commit()
615+
616+
For update and upsert requests, you can create a *deduplication space* where the request IDs will be saved.
617+
*Deduplication space* is a user space that contains a list of unique identifiers.
618+
Each identifier corresponds to one applied request.
619+
This space can have any name, in the example it is called ``deduplication``.
620+
621+
In the example below, within a single transaction:
622+
623+
1. It is checked whether the ``deduplication_key`` request ID exists in the ``deduplication`` space.
624+
2. If there is no such ID, The ID is added to the deduplication space.
625+
3. If the request hasn't been applied before, it increments the specified field in the ``bands`` space by one.
626+
627+
This approach ensures that each data modification request will be executed **only once**.
628+
629+
.. code-block:: lua
630+
631+
function update_1(deduplication_key, key)
632+
box.begin()
633+
if box.space.deduplication:get{deduplication_key} == nil then
634+
box.space.deduplication:insert{deduplication_key}
635+
box.space.bands:update(key, {{'+', 'value', 1 }})
636+
end
637+
box.commit()
638+
end
639+
640+
554641
.. _vshard-maintenance:
555642

556643
Sharded cluster maintenance

doc/reference/reference_rock/vshard/vshard_router.rst

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -163,6 +163,12 @@ Router public API
163163
optional attribute containing a message with the human-readable error description,
164164
and other attributes specific for the error code.
165165

166+
.. note::
167+
168+
Any write requests that are intended to be executed repeatedly should be idempotent.
169+
The operations' idempotency ensures that the change is applied **only once**.
170+
Read more: :ref:`<vshard-deduplication>`.
171+
166172
**Examples:**
167173

168174
To call ``customer_add`` function from ``vshard/example``, say:

0 commit comments

Comments
 (0)