utubettl: fix slow take on busy utubes

If some of the utubettl for tasks at the top of the queue were busy
most of the time, `take` would slow down for every other task.
This problem is fixed by creating a new space `space_ready`. It
contains first task with `READY` status from each utube.

This solution shows great results for the stated problem, with the cost
of slowing the `put` method (it is ~3 times slower). Thus, this workaround is
disabled by default. To enable it, user should set the
`storage_mode = queue.driver.utubettl.STORAGE_MODE_UTUBE_READY_SPACE` as an option
while creating the tube. As example:
```lua
local test_queue = queue.create_tube('test_queue', 'utubettl',
        {temporary = true, storage_mode = queue.driver.utubettl.STORAGE_MODE_UTUBE_READY_SPACE})
```

Closes #228

Author: DerekBum

CHANGELOG.md

@@ -8,16 +8,16 @@ and this project adheres to [Semantic Versioning](http://semver.org/spec/v2.0.0.
 ## [Unreleased]
 
 ### Added
-- `storage_mode` option for creating a `utube` tube (#228). It enables the
-  workaround for slow takes while working with busy tubes.
+- `storage_mode` option for creating a `utube` and `utubettl` tube (#228).
+  It enables the workaround for slow takes while working with busy tubes.
 
 ### Fixed
 
 - Stuck in `INIT` state if an instance failed to enter the `running` mode
   in time (#226). This fix works only for Tarantool versions >= 2.10.0.
-- Slow takes on busy `utube` tubes (#228). The workaround could be enabled by
-  passing the `storage_mode = "utube_ready_space"` option while creating
-  the tube.
+- Slow takes on busy `utube` and `utubettl` tubes (#228). The workaround could
+  be enabled by passing the `storage_mode = "utube_ready_space"` option while
+  creating the tube.
 
 ## [1.3.3] - 2023-09-13
 

README.md

@@ -243,13 +243,48 @@ in strict FIFO order.
 
 This queue type is effectively a combination of `fifottl` and `utube`.
 
+It is advised not to use `utubettl` methods inside transactions with
+`read-confirmed` isolation level. It can lead to errors when trying to make
+parallel tube methods calls with mvcc enabled.
+
 The following options can be specified when creating a `utubettl` queue:
   * `temporary` - boolean - if true, the contents of the queue do not persist
 on disk
   * `if_not_exists` - boolean - if true, no error will be returned if the tube
 already exists
   * `on_task_change` - function name - a callback to be executed on every
 operation
+  * `storage_mode` - string - one of
+      * `queue.driver.utubettl.STORAGE_MODE_DEFAULT` ("default") - default
+        implementation of `utubettl`
+      * `queue.driver.utubettl.STORAGE_MODE_UTUBE_READY_SPACE`
+        ("utube_ready_space") - allows processing `take` requests faster, but
+        by the cost of `put` operations speed. Right now this option is enabled
+        only for `memtx` engine.
+
+      Here is a benchmark comparison of these two modes:
+    * Benchmark for simple `put` and `take` methods. 30k utubes are created
+      with a single task each. Task creation time is calculated. After that
+      30k consumers are calling `take` + `ack`, each in the separate fiber.
+      Time to ack all tasks is calculated. The results are as follows:
+
+      |         | put (30k) | take+ack |
+      |---------|-----------|----------|
+      | default | 200ms     | 1.7s     |
+      | ready   | 320ms     | 1.8s     |
+    * Benchmark for the busy utubes. 10 tubes are created.
+      Each contains 1000 tasks. After that, 10 consumers are created (each works
+      on his tube only, one tube — one consumer). Each consumer will
+      `take`, then `yield` and then `ack` every task from their utube
+      (1000 tasks each).
+      After that, we can also run this benchmark with 10k tasks on each utube,
+      100k tasks and 140k tasks. But all that with 10 utubes and 10 consumers.
+      The results are as follows:
+
+      |         | 1k    | 10k  | 50k  | 140k  |
+      |---------|-------|------|------|-------|
+      | default | 80s   | 1.6h | 100h | 1000h |
+      | ready   | 520ms | 5.4s | 28s  | 83s   |
 
 The following options can be specified when putting a task in a
 `utubettl` queue: