api: native crud module support

Adds native api support for crud module [1]
to use it from a connection object.

1. github.com/tarantool/crud

Closes #205

Author: GRISHNOV

Diff for: .github/workflows/packing.yml

@@ -99,6 +99,12 @@ jobs:
       - name: Install test requirements
         run: pip3 install -r requirements-test.txt
 
+      - name: Install the crud module for testing purposes
+        run: |
+          curl -L https://tarantool.io/release/2/installer.sh | bash
+          sudo apt install -y tt
+          tt rocks install crud
+
       - name: Run tests
         run: make test-pure-install
 
@@ -331,6 +337,12 @@ jobs:
       - name: Install test requirements
         run: pip3 install -r requirements-test.txt
 
+      - name: Install the crud module for testing purposes
+        run: |
+          curl -L https://tarantool.io/release/2/installer.sh | bash
+          sudo apt install -y tt
+          tt rocks install crud
+
       - name: Run tests
         run: make test-pure-install
 
@@ -492,6 +504,12 @@ jobs:
       - name: Install test requirements
         run: pip3 install -r requirements-test.txt
 
+      - name: Install the crud module for testing purposes
+        run: |
+          curl -L https://tarantool.io/release/2/installer.sh | bash
+          sudo apt install -y tt
+          tt rocks install crud
+
       - name: Run tests
         run: make test-pure-install
 

Diff for: .github/workflows/reusable_testing.yml

@@ -39,4 +39,10 @@ jobs:
       - name: Install test requirements
         run: pip3 install -r requirements-test.txt
 
+      - name: Install the crud module for testing purposes
+        run: |
+          curl -L https://tarantool.io/release/2/installer.sh | bash
+          sudo apt install -y tt
+          tt rocks install crud
+
       - run: make test

Diff for: .github/workflows/testing.yml

@@ -85,6 +85,12 @@ jobs:
       - name: Install test requirements
         run: pip3 install -r requirements-test.txt
 
+      - name: Install the crud module for testing purposes
+        run: |
+          curl -L https://tarantool.io/release/2/installer.sh | bash
+          sudo apt install -y tt
+          tt rocks install crud
+
       - name: Run tests
         run: make test
 
@@ -143,6 +149,15 @@ jobs:
       - name: Install test requirements
         run: pip3 install -r requirements-test.txt
 
+      - name: Install the crud module for testing purposes
+        # This is a workaround with TARANTOOL_DIR and should be reworked later.
+        # See more here: https://github.com/tarantool/tt/issues/282
+        run: |
+          source tarantool-enterprise/env.sh
+          curl -L https://tarantool.io/release/2/installer.sh | bash
+          sudo apt install -y tt
+          tt rocks install crud TARANTOOL_DIR=$PWD/tarantool-enterprise
+
       - name: Run tests
         run: |
           source tarantool-enterprise/env.sh
@@ -196,6 +211,12 @@ jobs:
       - name: Install test requirements
         run: pip3 install -r requirements-test.txt
 
+      - name: Install the crud module for testing purposes
+        run: |
+          curl -L https://tarantool.io/release/2/installer.sh | bash
+          sudo apt install -y tt
+          tt rocks install crud
+
       - name: Run tests
         run: make test-pure-install
 

Diff for: CHANGELOG.md

@@ -9,6 +9,8 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 ### Added
 - Support custom packer and unpacker factories (#191).
 
+- Support [crud module](https://github.com/tarantool/crud) native API (#205).
+
 ### Changed
 
 ### Fixed

Diff for: docs/source/api/submodule-crud.rst

@@ -0,0 +1,4 @@
+module :py:mod:`tarantool.crud`
+================================
+
+.. automodule:: tarantool.crud

Diff for: docs/source/index.rst

@@ -63,6 +63,7 @@ API Reference
    api/module-tarantool.rst
    api/submodule-connection.rst
    api/submodule-connection-pool.rst
+   api/submodule-crud.rst
    api/submodule-dbapi.rst
    api/submodule-error.rst
    api/submodule-mesh-connection.rst

Diff for: docs/source/quick-start.rst

@@ -347,3 +347,123 @@ with out-of-band message processing:
 
     print(callback_res)
     >>> [[[100, 1]], [[200, 1]], [[300, 1]]]
+
+
+Interaction with the crud module
+----------------------------------
+
+Through the :class:`~tarantool.Connection` object, you can access 
+`crud module <https://github.com/tarantool/crud>`_ methods:
+
+.. code-block:: python
+
+    >>> import tarantool
+    >>> from tarantool.error import CrudModuleError, CrudModuleManyError, DatabaseError
+    >>> conn = tarantool.Connection(host='localhost',port=3301)
+
+    >>> conn.crud_
+    conn.crud_count(                conn.crud_insert(               conn.crud_insert_object_many(   
+    conn.crud_min(                  conn.crud_replace_object(       conn.crud_stats(                
+    conn.crud_unflatten_rows(       conn.crud_upsert_many(          conn.crud_delete(               
+    conn.crud_insert_many(          conn.crud_len(                  conn.crud_replace(              
+    conn.crud_replace_object_many(  conn.crud_storage_info(         conn.crud_update(               
+    conn.crud_upsert_object(        conn.crud_get(                  conn.crud_insert_object(        
+    conn.crud_max(                  conn.crud_replace_many(         conn.crud_select(               
+    conn.crud_truncate(             conn.crud_upsert(               conn.crud_upsert_object_many(
+
+As an example, consider :meth:`~tarantool.Connection.crud_insert` and :meth:`~tarantool.Connection.crud_insert_object_many`. 
+It is recommended to enclose calls in the try-except construction as follows:
+
+.. code-block:: python
+
+    # Insert without exception:
+    >>> res = conn.crud_insert('tester', (3500,300,'Rob'))
+    >>> res
+    <tarantool.crud.CrudResult object at 0x11a56e320>
+    >>> res.
+    res.metadata  res.rows
+    >>> res.rows
+    [[3500, 300, 'Rob']]
+    >>> res.metadata
+    [{'name': 'id', 'type': 'unsigned'}, {'name': 'bucket_id', 'type': 'unsigned'}, {'name': 'name', 'type': 'string'}]
+
+    # Insert with exception (duplicate key exists):
+    >>> try:
+    ...     res = conn.crud_insert('tester', (3500,300,'Rob'))
+    ... except CrudModuleError as e:
+    ...     exc_crud = e
+    ... 
+    >>> exc_crud
+    CrudModuleError(0, 'Failed to insert: Duplicate key exists in unique index "primary_index" in space "tester" with old tuple - [3500, 300, "Rob"] and new tuple - [3500, 300, "Rob"]')
+    >>> exc_crud.extra_info_error
+    <tarantool.crud.CrudError object at 0x10a276950>
+    >>> exc_crud.extra_info_error.
+    exc_crud.extra_info_error.class_name  exc_crud.extra_info_error.err         exc_crud.extra_info_error.file        exc_crud.extra_info_error.line        exc_crud.extra_info_error.str       
+    >>> exc_crud.extra_info_error.class_name
+    'InsertError'
+    >>> exc_crud.extra_info_error.str
+    'InsertError: Failed to insert: Duplicate key exists in unique index "primary_index" in space "tester" with old tuple - [3500, 300, "Rob"] and new tuple - [3500, 300, "Rob"]'
+
+    # In case of batch operation (*_many), CrudModuleManyError exception contains both result and errors (if there is a problem with at least one row).
+    >>> try:
+    ...     res = conn.crud_insert_object_many('tester', ({'id':3,'bucket_id':100,'name':'Ann'}, {'id':4,'bucket_id':100,'name':'Sam'}), {'timeout':100, 'rollback_on_error':False})
+    ... except CrudModuleManyError as e:
+    ...     exc_crud = e
+    ... 
+    >>> exc_crud
+    CrudModuleManyError(0, 'Got multiple errors, see errors_list')
+    >>> exc_crud.success_list # some of the rows were inserted.
+    <tarantool.crud.CrudResult object at 0x11a56f310>
+    >>> exc_crud.success_list.rows
+    [[1, 100, 'Bob'], [2, 100, 'Rob']]
+    >>> exc_crud.errors_list # some of the rows were not inserted.
+    [<tarantool.crud.CrudError object at 0x11a56e9e0>, <tarantool.crud.CrudError object at 0x11a56f490>]
+    >>> exc_crud.errors_list[0].str
+    'CallError: Failed for 037adb3a-b9e3-4f78-a6d1-9f0cdb6cbefc: Function returned an error: Duplicate key exists in unique index "primary_index" in space "tester" with old tuple - [3500, 300, "Rob"] and new tuple - [3500, 100, "Mike"]'
+    >>> exc_crud.errors_list[1].str
+    'InsertManyError: Failed to flatten object: FlattenError: Object is specified in bad format: FlattenError: Unknown field "second_name" is specified'
+    
+    # If there are no problems with any rows, the entire response will be contained in the res variable.
+    >>> res = conn.crud_insert_object_many('tester', ({'id':3,'bucket_id':100,'name':'Ann'}, {'id':4,'bucket_id':100,'name':'Sam'}), {'timeout':100, 'rollback_on_error':False})
+    >>> res.rows
+    [[3, 100, 'Ann'], [4, 100, 'Sam']]
+
+If module crud not found on the router or user has not sufficient grants:
+
+.. code-block:: python
+
+    >>> try:
+    ...     res = conn.crud_insert('tester', (22221,300,'Rob'))
+    ... except DatabaseError as e:
+    ...     exc_db = e
+    ... 
+    >>> exc_db
+    DatabaseError(33, "Procedure 'crud.insert' is not defined. Ensure that you're calling crud.router and user has sufficient grants")
+    >>> exc_db.extra_info
+    BoxError(type='ClientError', file='/tmp/tarantool-20221003-6335-edruh3/tarantool-2.10.3/src/box/lua/call.c', line=112, message="Procedure 'crud.insert' is not defined", errno=0, errcode=33, fields=None, prev=None)
+
+Using :meth:`~tarantool.Connection.crud_select` and :meth:`~tarantool.Connection.crud_unflatten_rows`:
+
+.. code-block:: python
+
+    >>> res = conn.crud_select('tester', [], {'first':2})
+    >>> res
+    <tarantool.crud.CrudResult object at 0x10a276d10>
+    >>> res.rows
+    [[1, 100, 'Mike'], [2, 100, 'Bill']]
+    >>> res.metadata
+    [{'name': 'id', 'type': 'unsigned'}, {'name': 'bucket_id', 'type': 'unsigned'}, {'name': 'name', 'type': 'string'}]
+    >>> r = conn.crud_unflatten_rows(res.rows, res.metadata)
+    >>> r
+    [{'id': 1, 'bucket_id': 100, 'name': 'Mike'}, {'id': 2, 'bucket_id': 100, 'name': 'Bill'}]
+
+Using :meth:`~tarantool.Connection.crud_truncate` and :meth:`~tarantool.Connection.crud_len`:
+
+.. code-block:: python
+
+    >>> res = conn.crud_len('tester')
+    >>> res
+    26
+    >>> res = conn.crud_truncate('tester')
+    >>> res
+    True