diff --git a/doc/book/admin/instance_config.rst b/doc/book/admin/instance_config.rst index 054f6e7d64..64cb813802 100644 --- a/doc/book/admin/instance_config.rst +++ b/doc/book/admin/instance_config.rst @@ -20,7 +20,7 @@ The main steps of creating and preparing the application for deployment are: In this section, a `sharded_cluster `_ application is used as an example. This cluster includes 5 instances: one router and 4 storages, which constitute two replica sets. -.. image:: admin_instances_dev.png +.. image:: /book/admin/admin_instances_dev.png :align: left :width: 700 :alt: Cluster topology @@ -171,7 +171,7 @@ define instances to run on each machine by changing the content of the ``instanc - On the developer's machine, this file might include all the instances defined in the cluster configuration. - .. image:: admin_instances_dev.png + .. image:: /book/admin/admin_instances_dev.png :align: left :width: 700 :alt: Cluster topology @@ -184,7 +184,7 @@ define instances to run on each machine by changing the content of the ``instanc - In the production environment, this file includes instances to run on the specific machine. - .. image:: admin_instances_prod.png + .. image:: /book/admin/admin_instances_prod.png :align: left :width: 700 :alt: Cluster topology diff --git a/doc/book/index.rst b/doc/book/index.rst index aed53f9158..37138cb9ee 100644 --- a/doc/book/index.rst +++ b/doc/book/index.rst @@ -17,6 +17,7 @@ User's Guide box/index admin/index monitoring/index + cluster_app connectors ../reference/reference_sql/index faq diff --git a/doc/code_snippets/snippets/config/instances.enabled/application_role/config.yaml b/doc/code_snippets/snippets/config/instances.enabled/application_role/config.yaml new file mode 100644 index 0000000000..2258022f07 --- /dev/null +++ b/doc/code_snippets/snippets/config/instances.enabled/application_role/config.yaml @@ -0,0 +1,7 @@ +groups: + group001: + replicasets: + replicaset001: + instances: + instance001: + roles: [ greeter ] diff --git a/doc/code_snippets/snippets/config/instances.enabled/application_role/greeter.lua b/doc/code_snippets/snippets/config/instances.enabled/application_role/greeter.lua new file mode 100644 index 0000000000..e49da0d052 --- /dev/null +++ b/doc/code_snippets/snippets/config/instances.enabled/application_role/greeter.lua @@ -0,0 +1,6 @@ +-- greeter.lua -- +return { + validate = function() end, + apply = function() require('log').info("Hi from the 'greeter' role!") end, + stop = function() end, +} diff --git a/doc/code_snippets/snippets/config/instances.enabled/application_role/instances.yml b/doc/code_snippets/snippets/config/instances.enabled/application_role/instances.yml new file mode 100644 index 0000000000..aa60c2fc42 --- /dev/null +++ b/doc/code_snippets/snippets/config/instances.enabled/application_role/instances.yml @@ -0,0 +1 @@ +instance001: diff --git a/doc/code_snippets/snippets/config/instances.enabled/application_role_cfg/byeer.lua b/doc/code_snippets/snippets/config/instances.enabled/application_role_cfg/byeer.lua new file mode 100644 index 0000000000..f7cf4d2bad --- /dev/null +++ b/doc/code_snippets/snippets/config/instances.enabled/application_role_cfg/byeer.lua @@ -0,0 +1,9 @@ +-- byeer.lua -- +local log = require('log').new("byeer") + +return { + dependencies = { 'greeter' }, + validate = function() end, + apply = function() log.info("Bye from the 'byeer' role!") end, + stop = function() end, +} diff --git a/doc/code_snippets/snippets/config/instances.enabled/application_role_cfg/config.yaml b/doc/code_snippets/snippets/config/instances.enabled/application_role_cfg/config.yaml new file mode 100644 index 0000000000..190cf800a5 --- /dev/null +++ b/doc/code_snippets/snippets/config/instances.enabled/application_role_cfg/config.yaml @@ -0,0 +1,10 @@ +groups: + group001: + replicasets: + replicaset001: + instances: + instance001: + roles: [ greeter ] + roles_cfg: + greeter: + greeting: 'Hi' diff --git a/doc/code_snippets/snippets/config/instances.enabled/application_role_cfg/greeter.lua b/doc/code_snippets/snippets/config/instances.enabled/application_role_cfg/greeter.lua new file mode 100644 index 0000000000..2be88cde8a --- /dev/null +++ b/doc/code_snippets/snippets/config/instances.enabled/application_role_cfg/greeter.lua @@ -0,0 +1,23 @@ +-- greeter.lua -- +local log = require('log').new("greeter") + +local function validate(cfg) + if cfg.greeting then + assert(type(cfg.greeting) == "string", "'greeting' should be a string") + assert(cfg.greeting == "Hi" or cfg.greeting == "Hello", "'greeting' should be 'Hi' or 'Hello'") + end +end + +local function apply(cfg) + log.info("%s from the 'greeter' role!", cfg.greeting) +end + +local function stop() + log.info("The 'greeter' role is stopped") +end + +return { + validate = validate, + apply = apply, + stop = stop, +} diff --git a/doc/code_snippets/snippets/config/instances.enabled/application_role_cfg/instances.yml b/doc/code_snippets/snippets/config/instances.enabled/application_role_cfg/instances.yml new file mode 100644 index 0000000000..aa60c2fc42 --- /dev/null +++ b/doc/code_snippets/snippets/config/instances.enabled/application_role_cfg/instances.yml @@ -0,0 +1 @@ +instance001: diff --git a/doc/code_snippets/snippets/config/instances.enabled/application_role_http_api/README.md b/doc/code_snippets/snippets/config/instances.enabled/application_role_http_api/README.md new file mode 100644 index 0000000000..e5014fae80 --- /dev/null +++ b/doc/code_snippets/snippets/config/instances.enabled/application_role_http_api/README.md @@ -0,0 +1,22 @@ +# HTTP API + +A sample application showing how to implement a custom `http-api` role. + +## Running + +To start an application, execute the following command in the [config](../../../config) directory: + +```console +$ tt start application_role_http_api +``` + +## Making test requests + +To test the API, make the following requests: + +```console +$ curl -X GET --location "http://0.0.0.0:8080/band?limit=7" \ + -H "Accept: application/json" +$ curl -X GET --location "http://0.0.0.0:8080/band/5" \ + -H "Accept: application/json" +``` diff --git a/doc/code_snippets/snippets/config/instances.enabled/application_role_http_api/config.yaml b/doc/code_snippets/snippets/config/instances.enabled/application_role_http_api/config.yaml new file mode 100644 index 0000000000..76e7b9fe74 --- /dev/null +++ b/doc/code_snippets/snippets/config/instances.enabled/application_role_http_api/config.yaml @@ -0,0 +1,11 @@ +groups: + group001: + replicasets: + replicaset001: + instances: + instance001: + roles: [ http-api ] + roles_cfg: + http-api: + host: '127.0.0.1' + port: 8080 diff --git a/doc/code_snippets/snippets/config/instances.enabled/application_role_http_api/data.lua b/doc/code_snippets/snippets/config/instances.enabled/application_role_http_api/data.lua new file mode 100644 index 0000000000..775d00d668 --- /dev/null +++ b/doc/code_snippets/snippets/config/instances.enabled/application_role_http_api/data.lua @@ -0,0 +1,32 @@ +local data = {} + +data.add_sample_data = function() + box.watch('box.status', function() + if box.info.ro then + return + end + + box.schema.space.create('bands', { if_not_exists = true }) + box.space.bands:format({ + { name = 'id', type = 'unsigned' }, + { name = 'band_name', type = 'string' }, + { name = 'year', type = 'unsigned' } + }) + box.space.bands:create_index('primary', { parts = { 'id' } }) + box.space.bands:create_index('band', { parts = { 'band_name' } }) + box.space.bands:create_index('year_band', { parts = { { 'year' }, { 'band_name' } } }) + + box.space.bands:insert { 1, 'Roxette', 1986 } + box.space.bands:insert { 2, 'Scorpions', 1965 } + box.space.bands:insert { 3, 'Ace of Base', 1987 } + box.space.bands:insert { 4, 'The Beatles', 1960 } + box.space.bands:insert { 5, 'Pink Floyd', 1965 } + box.space.bands:insert { 6, 'The Rolling Stones', 1962 } + box.space.bands:insert { 7, 'The Doors', 1965 } + box.space.bands:insert { 8, 'Nirvana', 1987 } + box.space.bands:insert { 9, 'Led Zeppelin', 1968 } + box.space.bands:insert { 10, 'Queen', 1970 } + end) +end + +return data diff --git a/doc/code_snippets/snippets/config/instances.enabled/application_role_http_api/http-api.lua b/doc/code_snippets/snippets/config/instances.enabled/application_role_http_api/http-api.lua new file mode 100644 index 0000000000..e54d20b45d --- /dev/null +++ b/doc/code_snippets/snippets/config/instances.enabled/application_role_http_api/http-api.lua @@ -0,0 +1,65 @@ +-- http-api.lua -- +local httpd +local json = require('json') + +local function validate(cfg) + if cfg.host then + assert(type(cfg.host) == "string", "'host' should be a string containing a valid IP address") + end + if cfg.port then + assert(type(cfg.port) == "number", "'port' should be a number") + assert(cfg.port >= 1 and cfg.port <= 65535, "'port' should be between 1 and 65535") + end +end + +local function apply(cfg) + if httpd then + httpd:stop() + end + httpd = require('http.server').new(cfg.host, cfg.port) + local response_headers = { ['content-type'] = 'application/json' } + httpd:route({ path = '/band/:id', method = 'GET' }, function(req) + local id = req:stash('id') + local band_tuple = box.space.bands:get(tonumber(id)) + if not band_tuple then + return { status = 404, body = 'Band not found' } + else + local band = { id = band_tuple['id'], + band_name = band_tuple['band_name'], + year = band_tuple['year'] } + return { status = 200, headers = response_headers, body = json.encode(band) } + end + end) + httpd:route({ path = '/band', method = 'GET' }, function(req) + local limit = req:query_param('limit') + if not limit then + limit = 5 + end + local band_tuples = box.space.bands:select({}, { limit = tonumber(limit) }) + local bands = {} + for _, tuple in pairs(band_tuples) do + local band = { id = tuple['id'], + band_name = tuple['band_name'], + year = tuple['year'] } + table.insert(bands, band) + end + return { status = 200, headers = response_headers, body = json.encode(bands) } + end) + httpd:start() +end + +local function stop() + httpd:stop() +end + +local function init() + require('data'):add_sample_data() +end + +init() + +return { + validate = validate, + apply = apply, + stop = stop, +} diff --git a/doc/code_snippets/snippets/config/instances.enabled/application_role_http_api/instances.yml b/doc/code_snippets/snippets/config/instances.enabled/application_role_http_api/instances.yml new file mode 100644 index 0000000000..aa60c2fc42 --- /dev/null +++ b/doc/code_snippets/snippets/config/instances.enabled/application_role_http_api/instances.yml @@ -0,0 +1 @@ +instance001: diff --git a/doc/code_snippets/snippets/sharding/instances.enabled/sharded_cluster/sharded_cluster-scm-1.rockspec b/doc/code_snippets/snippets/sharding/instances.enabled/sharded_cluster/sharded_cluster-scm-1.rockspec index d88dc912ee..5be1567c6b 100644 --- a/doc/code_snippets/snippets/sharding/instances.enabled/sharded_cluster/sharded_cluster-scm-1.rockspec +++ b/doc/code_snippets/snippets/sharding/instances.enabled/sharded_cluster/sharded_cluster-scm-1.rockspec @@ -5,7 +5,7 @@ source = { } dependencies = { - 'vshard == 0.1.26' + 'vshard == 0.1.27' } build = { type = 'none'; diff --git a/doc/code_snippets/snippets/sharding/instances.enabled/sharded_cluster_crud/config.yaml b/doc/code_snippets/snippets/sharding/instances.enabled/sharded_cluster_crud/config.yaml index 77a5a5d8b0..9135ca916e 100644 --- a/doc/code_snippets/snippets/sharding/instances.enabled/sharded_cluster_crud/config.yaml +++ b/doc/code_snippets/snippets/sharding/instances.enabled/sharded_cluster_crud/config.yaml @@ -2,10 +2,10 @@ credentials: users: replicator: password: 'topsecret' - roles: [replication] + roles: [ replication ] storage: password: 'secret' - roles: [sharding] + roles: [ sharding ] iproto: advertise: @@ -19,9 +19,11 @@ sharding: groups: storages: - roles: [storage] + roles: [ roles.crud-storage ] + app: + module: storage sharding: - roles: [storage] + roles: [ storage ] replication: failover: manual replicasets: @@ -48,9 +50,19 @@ groups: listen: - uri: '127.0.0.1:3305' routers: - roles: [router] + roles: [ roles.crud-router ] + roles_cfg: + roles.crud-router: + stats: true + stats_driver: metrics + stats_quantiles: false + stats_quantile_tolerated_error: 0.001 + stats_quantile_age_buckets_count: 5 + stats_quantile_max_age_time: 180 + app: + module: router sharding: - roles: [router] + roles: [ router ] replicasets: router-a: instances: diff --git a/doc/code_snippets/snippets/sharding/instances.enabled/sharded_cluster_crud/router.lua b/doc/code_snippets/snippets/sharding/instances.enabled/sharded_cluster_crud/router.lua index c09e13003c..61ccb2c40b 100644 --- a/doc/code_snippets/snippets/sharding/instances.enabled/sharded_cluster_crud/router.lua +++ b/doc/code_snippets/snippets/sharding/instances.enabled/sharded_cluster_crud/router.lua @@ -1,17 +1 @@ local vshard = require('vshard') -local crud = require('crud') - -local M = {} - -function M.validate() -end - -function M.apply() - crud.init_router() -end - -function M.stop() - crud.stop_router() -end - -return M diff --git a/doc/code_snippets/snippets/sharding/instances.enabled/sharded_cluster_crud/sharded_cluster_crud-scm-1.rockspec b/doc/code_snippets/snippets/sharding/instances.enabled/sharded_cluster_crud/sharded_cluster_crud-scm-1.rockspec index ca26ede2b6..fb61077796 100644 --- a/doc/code_snippets/snippets/sharding/instances.enabled/sharded_cluster_crud/sharded_cluster_crud-scm-1.rockspec +++ b/doc/code_snippets/snippets/sharding/instances.enabled/sharded_cluster_crud/sharded_cluster_crud-scm-1.rockspec @@ -5,8 +5,8 @@ source = { } dependencies = { - 'vshard == 0.1.26', - 'crud == 1.4.3' + 'vshard == 0.1.27', + 'crud == 1.5.2' } build = { type = 'none'; diff --git a/doc/code_snippets/snippets/sharding/instances.enabled/sharded_cluster_crud/storage.lua b/doc/code_snippets/snippets/sharding/instances.enabled/sharded_cluster_crud/storage.lua index 4ddfdf2e65..f692015c03 100644 --- a/doc/code_snippets/snippets/sharding/instances.enabled/sharded_cluster_crud/storage.lua +++ b/doc/code_snippets/snippets/sharding/instances.enabled/sharded_cluster_crud/storage.lua @@ -1,29 +1,17 @@ -local crud = require('crud') - -local M = {} - -function M.validate() -end - -function M.apply() - crud.init_storage() - if box.info.ro ~= true then - box.schema.create_space('bands', { - format = { - { name = 'id', type = 'unsigned' }, - { name = 'bucket_id', type = 'unsigned' }, - { name = 'band_name', type = 'string' }, - { name = 'year', type = 'unsigned' } - }, - if_not_exists = true - }) - box.space.bands:create_index('id', { parts = { 'id' }, if_not_exists = true }) - box.space.bands:create_index('bucket_id', { parts = { 'bucket_id' }, unique = false, if_not_exists = true }) +box.watch('box.status', function() + if box.info.ro then + return end -end - -function M.stop() - crud.stop_storage() -end -return M + box.schema.create_space('bands', { + format = { + { name = 'id', type = 'unsigned' }, + { name = 'bucket_id', type = 'unsigned' }, + { name = 'band_name', type = 'string' }, + { name = 'year', type = 'unsigned' } + }, + if_not_exists = true + }) + box.space.bands:create_index('id', { parts = { 'id' }, if_not_exists = true }) + box.space.bands:create_index('bucket_id', { parts = { 'bucket_id' }, unique = false, if_not_exists = true }) +end) diff --git a/doc/code_snippets/snippets/sharding/templates/basic/{{.name}}-scm-1.rockspec.tt.template b/doc/code_snippets/snippets/sharding/templates/basic/{{.name}}-scm-1.rockspec.tt.template index 2806c482ce..b34bf8e015 100644 --- a/doc/code_snippets/snippets/sharding/templates/basic/{{.name}}-scm-1.rockspec.tt.template +++ b/doc/code_snippets/snippets/sharding/templates/basic/{{.name}}-scm-1.rockspec.tt.template @@ -5,7 +5,7 @@ source = { } dependencies = { - 'vshard == 0.1.25' + 'vshard == 0.1.27' } build = { type = 'none'; diff --git a/doc/concepts/configuration.rst b/doc/concepts/configuration.rst index 900a480562..a895e31e32 100644 --- a/doc/concepts/configuration.rst +++ b/doc/concepts/configuration.rst @@ -58,18 +58,22 @@ A cluster's topology includes the following elements, starting from the lower le You can flexibly configure a cluster's settings on different levels: from global settings applied to all groups to parameters specific for concrete instances. +.. NOTE:: + + All the available options are documented in the :ref:`Configuration reference `. + .. _configuration_file: Configuration in a file -~~~~~~~~~~~~~~~~~~~~~~~ +----------------------- This section provides an overview on how to configure Tarantool in a YAML file. .. _configuration_instance_basic: Basic instance configuration -**************************** +~~~~~~~~~~~~~~~~~~~~~~~~~~~~ The example below shows a sample configuration of a single Tarantool instance: @@ -86,7 +90,7 @@ The example below shows a sample configuration of a single Tarantool instance: .. _configuration_scopes: Configuration scopes -******************** +~~~~~~~~~~~~~~~~~~~~ This section shows how to control a scope the specified configuration option is applied to. Most of the configuration options can be applied to a specific instance, replica set, group, or to all instances globally. @@ -129,6 +133,9 @@ Most of the configuration options can be applied to a specific instance, replica :emphasize-lines: 1-3 :dedent: +Configuration scopes above are listed in the order of their precedence -- from highest to lowest. +For example, if the same option is defined at the instance and global level, the instance's value takes precedence over the global one. + .. NOTE:: @@ -138,7 +145,7 @@ Most of the configuration options can be applied to a specific instance, replica .. _configuration_replica_set_scopes: Configuration scopes: Replica set example -***************************************** +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ The example below shows how specific configuration options work in different configuration scopes for a replica set with a manual failover. You can learn more about configuring replication from :ref:`Replication tutorials `. @@ -171,55 +178,131 @@ You can learn more about configuring replication from :ref:`Replication tutorial .. _configuration_application: +.. _configuration_application_roles: + +Enabling and configuring roles +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +An application role is a Lua module that implements specific functions or logic. +You can turn on or off a particular role for certain instances in a configuration without restarting these instances. -Loading an application -********************** +There can be built-in Tarantool roles, roles provided by third-party Lua modules, or custom roles that are developed as a part of a cluster application. +This section describes how to enable and configure roles. +To learn how to develop custom roles, see :ref:`application_roles`. -Using Tarantool as an application server, you can run your own Lua applications. -In the ``app`` section, you can load the application and provide a custom application configuration in the ``cfg`` section. -In the example below, the application is loaded from the ``myapp.lua`` file placed next to the YAML configuration file: -.. literalinclude:: /code_snippets/snippets/config/instances.enabled/application/config.yaml +.. _configuration_application_roles_enable: + +Enabling a role +*************** + +To turn on or off a role for a specific instance or a set of instances, use the :ref:`roles ` configuration option. +The example below shows how to enable the ``roles.crud-router`` role provided by the `CRUD `__ module using the :ref:`roles ` option: + +.. literalinclude:: /code_snippets/snippets/sharding/instances.enabled/sharded_cluster_crud/config.yaml :language: yaml + :start-at: roles.crud-router + :end-at: roles.crud-router :dedent: -To get a value of the custom ``greeting`` property in the application code, -use the ``config:get()`` function provided by the :ref:`config ` module. +Similarly, you can enable the ``roles.crud-storage`` role to make instances act as CRUD storages: -.. literalinclude:: /code_snippets/snippets/config/instances.enabled/application/myapp.lua - :language: lua +.. literalinclude:: /code_snippets/snippets/sharding/instances.enabled/sharded_cluster_crud/config.yaml + :language: yaml + :start-at: roles.crud-storage + :end-at: roles.crud-storage :dedent: -As a result of :ref:`starting ` the *instance001*, a log should contain the following line: +Example on GitHub: `sharded_cluster_crud `_ -.. code-block:: console - main/103/interactive/myapp I> Hello from app, instance001! +.. _configuration_application_roles_configure: -The ``app`` section can be placed in any :ref:`configuration scope `. -As an example use case, you can provide different applications for storages and routers in a sharded cluster: +Configuring a role +****************** -.. code-block:: yaml +The :ref:`roles_cfg ` option allows you to specify the configuration for each role. +In this option, the role name is the key and the role configuration is the value. + +The example below shows how to enable statistics on called operations by providing the ``roles.crud-router`` role's configuration: + +.. literalinclude:: /code_snippets/snippets/sharding/instances.enabled/sharded_cluster_crud/config.yaml + :language: yaml + :start-at: roles.crud-router + :end-at: stats_quantile_max_age_time + :dedent: + +Example on GitHub: `sharded_cluster_crud `_ + + + +.. _configuration_application_roles_scopes: + +Roles and configuration scopes +****************************** + +As the most of configuration options, roles and their configurations can be defined at :ref:`different levels `. +Given that the ``roles`` option has the ``array`` type and ``roles_cfg`` has the ``map`` type, there are some specifics of applying the configuration: + +- For ``roles``, an instance's role takes precedence over roles defined at another level. + In the example below, ``instance001`` has only ``role3``: + + .. code-block:: yaml + + # ... + replicaset001: + roles: [ role1, role2 ] + instances: + instance001: + roles: [ role3 ] + + Learn more about the order of precedence for different configuration scopes in :ref:`configuration_scopes`. + +- For ``roles_cfg``, the following rules are applied: + + - If a configuration *for the same role* is provided at different levels, an instance configuration takes precedence over the configuration defined at another level. + In the example below, ``role1.greeting`` is ``'Hi'``: + + .. code-block:: yaml + + # ... + replicaset001: + roles_cfg: + role1: + greeting: 'Hello' + instances: + instance001: + roles: [ role1 ] + roles_cfg: + role1: + greeting: 'Hi' + + - If the configurations *for different roles* are provided at different levels, both configurations are applied at the instance level. + In the example below, ``instance001`` has ``role1.greeting`` set to ``'Hi'`` and ``role2.farewell`` set to ``'Bye'``: + + .. code-block:: yaml + + # ... + replicaset001: + roles_cfg: + role1: + greeting: 'Hi' + instances: + instance001: + roles: [ role1, role2 ] + roles_cfg: + role2: + farewell: 'Bye' - groups: - storages: - app: - module: storage - # ... - routers: - app: - module: router - # ... -Learn more about using Tarantool as the application server from :ref:`Developing applications with Tarantool `. .. _configuration_predefined_variables: Predefined variables -******************** +~~~~~~~~~~~~~~~~~~~~ In a configuration file, you can use the following predefined variables that are replaced with actual values at runtime: @@ -234,14 +317,14 @@ In the example below, ``{{ instance_name }}`` is replaced with *instance001*. :language: yaml :dedent: -As a result, the :ref:`paths to snapshots and write-ahead logs ` differ for different instances. +As a result, the paths to :ref:`snapshots and write-ahead logs ` differ for different instances. .. _configuration_environment_variable: Environment variables -~~~~~~~~~~~~~~~~~~~~~ +--------------------- For each configuration parameter, Tarantool provides two sets of predefined environment variables: @@ -259,66 +342,93 @@ To see all the supported environment variables, execute the ``tarantool`` comman $ tarantool --help-env-list -Below are a few examples that show how to set environment variables of different types, like *string*, *number*, *array*, or *map*: +.. NOTE:: -* String. In this example, ``TT_LOG_LEVEL`` is used to set a logging level to ``CRITICAL``: + There are also special ``TT_INSTANCE_NAME`` and ``TT_CONFIG`` environment variables that can be used to :ref:`start ` the specified Tarantool instance with configuration from the given file. - .. code-block:: console +Below are a few examples that show how to set environment variables of different types, like *string*, *number*, *array*, or *map*. - $ export TT_LOG_LEVEL='crit' +.. _configuration_environment_variable_string: -* Number. In this example, a logging level is set to ``CRITICAL`` using a corresponding numeric value: +String +~~~~~~ - .. code-block:: console +In this example, ``TT_LOG_LEVEL`` is used to set a logging level to ``CRITICAL``: - $ export TT_LOG_LEVEL=3 +.. code-block:: console -* Array. The examples below show how to set the ``TT_SHARDING_ROLES`` variable that accepts an array value. - Arrays can be passed in two ways: using a *simple* ... + $ export TT_LOG_LEVEL='crit' - .. code-block:: console - $ export TT_SHARDING_ROLES=router,storage +.. _configuration_environment_variable_number: - ... or *JSON* format: +Number +~~~~~~ - .. code-block:: console +In this example, a logging level is set to ``CRITICAL`` using a corresponding numeric value: - $ export TT_SHARDING_ROLES='["router", "storage"]' +.. code-block:: console - The *simple* format is applicable only to arrays containing scalar values. + $ export TT_LOG_LEVEL=3 -* Map. To assign map values to environment variables, you can also use *simple* or *JSON* formats. - In the example below, ``TT_LOG_MODULES`` sets different logging levels for different modules using a *simple* format: +.. _configuration_environment_variable_array: - .. code-block:: console +Array +~~~~~ - $ export TT_LOG_MODULES=module1=info,module2=error +The examples below show how to set the ``TT_SHARDING_ROLES`` variable that accepts an array value. +Arrays can be passed in two ways: using a *simple* ... - In the next example, ``TT_APP_CFG`` is used to specify the value of a custom configuration property for a :ref:`loaded application ` using a *JSON* format: +.. code-block:: console - .. code-block:: console + $ export TT_SHARDING_ROLES=router,storage - $ export TT_APP_CFG='{"greeting":"Hi"}' +... or *JSON* format: - The *simple* format is applicable only to maps containing scalar values. +.. code-block:: console -* Array of maps. In the example below, ``TT_IPROTO_LISTEN`` is used to specify a :ref:`listening host and port ` values: + $ export TT_SHARDING_ROLES='["router", "storage"]' - .. code-block:: console +The *simple* format is applicable only to arrays containing scalar values. - $ export TT_IPROTO_LISTEN=['{"uri":"127.0.0.1:3311"}'] - You can also pass several listening addresses: +.. _configuration_environment_variable_map: - .. code-block:: console +Map +~~~ - $ export TT_IPROTO_LISTEN=['{"uri":"127.0.0.1:3311"}','{"uri":"127.0.0.1:3312"}'] +To assign map values to environment variables, you can also use *simple* or *JSON* formats. +In the example below, ``TT_LOG_MODULES`` sets different logging levels for different modules using a *simple* format: +.. code-block:: console -.. NOTE:: + $ export TT_LOG_MODULES=module1=info,module2=error + +In the next example, ``TT_ROLES_CFG`` is used to specify the value of a custom configuration for a :ref:`role ` using a *JSON* format: + +.. code-block:: console + + $ export TT_ROLES_CFG='{"greeter":{"greeting":"Hello"}}' + +The *simple* format is applicable only to maps containing scalar values. - There are also special ``TT_INSTANCE_NAME`` and ``TT_CONFIG`` environment variables that can be used to :ref:`start ` the specified Tarantool instance with configuration from the given file. + +.. _configuration_environment_variable_array_of_maps: + +Array of maps +~~~~~~~~~~~~~ + +In the example below, ``TT_IPROTO_LISTEN`` is used to specify a :ref:`listening host and port ` values: + +.. code-block:: console + + $ export TT_IPROTO_LISTEN=['{"uri":"127.0.0.1:3311"}'] + +You can also pass several listening addresses: + +.. code-block:: console + + $ export TT_IPROTO_LISTEN=['{"uri":"127.0.0.1:3311"}','{"uri":"127.0.0.1:3312"}'] @@ -328,7 +438,7 @@ Below are a few examples that show how to set environment variables of different .. _configuration_etcd_overview: Centralized configuration -~~~~~~~~~~~~~~~~~~~~~~~~~ +------------------------- .. include:: /concepts/configuration/configuration_etcd.rst :start-after: ee_note_centralized_config_start @@ -355,7 +465,7 @@ Learn more from the following guide: :ref:`configuration_etcd`. .. _configuration_precedence: Configuration precedence -~~~~~~~~~~~~~~~~~~~~~~~~ +------------------------ Tarantool configuration options are applied from multiple sources with the following precedence, from highest to lowest: @@ -368,85 +478,6 @@ If the same option is defined in two or more locations, the option with the high -.. _configuration_options_overview: - -Configuration options overview ------------------------------- - -This section gives an overview of some useful configuration options. -All the available options are documented in the :ref:`Configuration reference `. - -.. _configuration_options_connection: - -Connection settings -~~~~~~~~~~~~~~~~~~~ - -To configure an address used to listen for incoming requests, use the ``iproto.listen`` option. -The example below shows how to set a listening IP address for ``instance001`` to ``127.0.0.1:3301``: - -.. literalinclude:: /code_snippets/snippets/config/instances.enabled/iproto_listen_address/config.yaml - :start-at: instance001 - :end-at: '127.0.0.1:3301' - :language: yaml - :dedent: - -You can learn more from the :ref:`configuration_connections` topic. - - -.. _configuration_options_access_control: - -Access control -~~~~~~~~~~~~~~ - -The ``credentials`` section allows you to create users and grant them the specified privileges. -In the example below, a ``dbadmin`` user with the specified password is created: - -.. literalinclude:: /code_snippets/snippets/config/instances.enabled/credentials/config.yaml - :language: yaml - :start-at: credentials: - :end-at: T0p_Secret - :dedent: - -To learn more, see the :ref:`configuration_credentials` section. - - -.. _configuration_options_memory: - -Memory -~~~~~~ - -The :ref:`memtx.memory ` option specifies how much :ref:`memory ` -Tarantool allocates to actually store data. - -.. literalinclude:: /code_snippets/snippets/config/instances.enabled/memtx/config.yaml - :language: yaml - :start-at: memtx: - :end-at: 1073741824 - :dedent: - -When the limit is reached, ``INSERT`` or ``UPDATE`` requests fail with :ref:`ER_MEMORY_ISSUE `. - -Learn more: :ref:`In-memory storage configuration `. - -.. _configuration_options_directories: - -Snapshots and write-ahead logs -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -The :ref:`snapshot.dir ` and :ref:`wal.dir ` -options can be used to configure directories for storing snapshots and write-ahead logs. -For example, you can place snapshots and write-ahead logs on different hard drives for better reliability. - -.. code-block:: yaml - - instance001: - snapshot: - dir: '/media/drive1/snapshots' - wal: - dir: '/media/drive2/wals' - -To learn more about the persistence mechanism in Tarantool, see the :ref:`Persistence ` section. -Read more about snapshot and WAL configuration: :ref:`Persistence `. .. toctree:: diff --git a/doc/how-to/app/app_roles.rst b/doc/how-to/app/app_roles.rst new file mode 100644 index 0000000000..96b6bdcbb4 --- /dev/null +++ b/doc/how-to/app/app_roles.rst @@ -0,0 +1,481 @@ +.. _application_roles: + +Application roles +================= + +An application role is a Lua module that implements specific functions or logic. +You can turn on or off a particular role for certain instances in a :ref:`configuration ` without restarting these instances. +A role is run when a configuration is loaded or reloaded. + +Roles can be divided into the following groups: + +- Tarantool's built-in roles. + For example, the ``config.storage`` role can be used to make a Tarantool replica set act as a :ref:`configuration storage `. +- Roles provided by third-party Lua modules. + For example, the `CRUD `__ module provides the ``roles.crud-storage`` and ``roles.crud-router`` roles that enable CRUD operations in a sharded cluster. +- Custom roles that are developed as a part of a cluster application. + For example, you can create a custom role to define a stored procedure or implement a supplementary service, such as an email notifier or a replicator. + +This section describes how to develop custom roles. +To learn how to enable and configure roles, see :ref:`configuration_application_roles`. + +.. NOTE:: + + Don't confuse application roles with other role types: + + - A role is a container for privileges that can be granted to users. Learn more in :ref:`access_control_concepts_roles`. + - A role of a replica set in regard to sharding. Learn more in :ref:`vshard_config_sharding_roles`. + + + + +.. _roles_create_custom_role_config: + +Providing a role configuration +------------------------------ + +A custom role can be configured in the same way as roles provided by Tarantool or third-party Lua modules. +You can learn more from :ref:`configuration_application_roles`. + +This example shows how to enable and configure the ``greeter`` role, which is implemented in the next section: + +.. literalinclude:: /code_snippets/snippets/config/instances.enabled/application_role_cfg/config.yaml + :language: yaml + :start-at: instance001 + :dedent: + +The role's configuration provided in ``roles_cfg`` can be accessed when :ref:`validating ` and :ref:`applying ` this configuration. + +Given that a role is a :ref:`Lua module `, a role's name is passed to ``require()`` to obtain the module. +When :ref:`developing an application `, you can place a file with a role's code next to a cluster's configuration file. + + + +.. _roles_create_custom_role: + +Creating a custom role +---------------------- + +.. _roles_create_custom_role_overview: + +Overview +~~~~~~~~ + +Creating a custom role includes the following steps: + +1. Define a function that validates a role's configuration. +2. Define a function that applies a validated configuration. +3. Define a function that stops a role. +4. (Optional) Define roles from which this custom role depends on. + +As a result, a role's module should return an object that has corresponding functions and fields specified: + +.. code-block:: lua + + return { + validate = function() -- ... -- end, + apply = function() -- ... -- end, + stop = function() -- ... -- end, + dependencies = { -- ... -- }, + } + +The examples below show how to do this. + +.. NOTE:: + + Code snippets shown in this section are included from the following application: `application_role_cfg `_. + + + + +.. _roles_create_custom_role_validate: + +Validating a role configuration +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +To validate a role's configuration, you need to define the :ref:`validate([cfg]) ` function. +The ``cfg`` argument provides access to the :ref:`role's configuration ` and check its validity. + +In the example below, the ``validate()`` function is used to validate the ``greeting`` configuration value: + +.. literalinclude:: /code_snippets/snippets/config/instances.enabled/application_role_cfg/greeter.lua + :language: lua + :start-at: local function validate + :end-before: local function apply + :dedent: + +If the configuration is not valid, ``validate()`` reports an unrecoverable error by throwing an error object. + + + + +.. _roles_create_custom_role_apply: + +Applying a role configuration +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +To apply the validated configuration, define the :ref:`apply([cfg]) ` function. +As the ``validate()`` function, ``apply()`` provides access to a role's configuration using the ``cfg`` argument. + +In the example below, the ``apply()`` function uses the :ref:`log ` module to write a role's configuration value to the log: + +.. literalinclude:: /code_snippets/snippets/config/instances.enabled/application_role_cfg/greeter.lua + :language: lua + :start-at: local function apply + :end-before: local function stop + :dedent: + + + +.. _roles_create_custom_role_stop: + +Stopping a role +~~~~~~~~~~~~~~~ + +To stop a role, use the :ref:`stop() ` function. + +In the example below, the ``stop()`` function uses the :ref:`log ` module to indicate that a role is stopped: + +.. literalinclude:: /code_snippets/snippets/config/instances.enabled/application_role_cfg/greeter.lua + :language: lua + :start-at: local function stop + :end-before: return + :dedent: + +When you've defined all the role's functions, you need to return an object that has corresponding functions specified: + +.. literalinclude:: /code_snippets/snippets/config/instances.enabled/application_role_cfg/greeter.lua + :language: lua + :start-at: return + :dedent: + + + +.. _roles_create_custom_role_dependencies: + +Role dependencies +~~~~~~~~~~~~~~~~~ + +To define a role's dependencies, use the :ref:`dependencies ` field. +In this example, the ``byeer`` role has the ``greeter`` role as the dependency: + +.. literalinclude:: /code_snippets/snippets/config/instances.enabled/application_role_cfg/byeer.lua + :language: lua + :dedent: + +A role cannot be started without its dependencies. +This means that all the dependencies of a role should be defined in the ``roles`` configuration parameter: + +.. code-block:: yaml + + instance001: + roles: [ greeter, byeer ] + +You can find the full example here: `application_role_cfg `_. + + + +.. _roles_create_custom_role_init: + +Adding initialization code +~~~~~~~~~~~~~~~~~~~~~~~~~~ + +You can add initialization code to a role by defining and calling a function with an arbitrary name at the top level of a module, for example: + +.. code-block:: lua + + local function init() + -- ... -- + end + + init() + +For example, you can :ref:`create spaces `, define :ref:`indexes `, or :ref:`grant privileges ` to specific users or roles. + +See also: :ref:`roles_create_space`. + + + +.. _roles_create_space: + +Specifics of creating spaces +~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +To create a space in a role, you need to make sure that the target instance is in read-write mode (its :ref:`box.info.ro ` is ``false``). +You can check an instance state by subscribing to the ``box.status`` event using :ref:`box.watch() `: + +.. code-block:: lua + + box.watch('box.status', function() + -- creating a space + -- ... + end) + + +.. NOTE:: + + Given that a role may be enabled when an instance is already in read-write mode, + you also need to execute schema initialization code from :ref:`apply() `. + To make sure a space is created only once, use the :ref:`if_not_exists ` option. + + + +.. _roles_life_cycle: + +Roles life cycle +---------------- + +A role’s life cycle includes the stages described below. + +.. _roles_life_cycle_loading_roles: + +1) *Loading roles* + + On each run, all roles are loaded in the order they are specified in the :ref:`configuration `. + This stage takes effect when a role is enabled or an instance with this role is restarted. + At this stage, a role executes the :ref:`initialization code `. + + A role cannot be started if it has :ref:`dependencies ` that are not specified in a configuration. + + .. NOTE:: + + Dependencies do not affect the order in which roles are loaded. + However, the ``validate()``, ``apply()``, and ``stop()`` functions are executed taking dependencies into account. + Learn more in :ref:`roles_life_cycle_dependencies_specifics`. + + +.. _roles_life_cycle_stopping_roles: + +2) *Stopping roles* + + This stage takes effect during a configuration reload when a role is removed from the configuration for a given instance. + Note that all ``stop()`` calls are performed before any ``validate()`` or ``apply()`` calls. + This means that old roles are stopped first, and only then new roles are started. + +.. _roles_life_cycle_validating_role_config: + +3) *Validating a role's configurations* + + At this stage, a configuration for each role is validated using the corresponding :ref:`validate() ` function in the same order in which they are specified in the configuration. + +.. _roles_life_cycle_applying_role_config: + +4) *Applying a role's configurations* + + At this stage, a configuration for each role is applied using the corresponding :ref:`apply() ` function in the same order in which they are specified in the configuration. + + +All role's functions report an unrecoverable error by throwing an error object. +If an error is thrown in any phase, applying a configuration is stopped. +If starting or stopping a role throws an error, no roles are stopped or started afterward. +An error is caught and shown in :ref:`config:info() ` in the ``alerts`` section. + + +.. _roles_life_cycle_dependencies_specifics: + +Executing functions for dependent roles +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +For roles that :ref:`depend ` on each other, their ``validate()``, ``apply()``, and ``stop()`` functions are executed taking into account the dependencies. +Suppose, there are three independent and two dependent roles: + +.. code-block:: none + + role1 + role2 + role3 + └─── role4 + └─── role5 + +- ``role1``, ``role2``, and ``role5`` are independent roles. +- ``role3`` depends on ``role4``, ``role4`` depends on ``role5``. + +The roles are enabled in a configuration as follows: + +.. code-block:: yaml + + roles: [ role1, role2, role3, role4, role5 ] + +In this case, ``validate()`` and ``apply()`` for these roles are executed in the following order: + +.. code-block:: none + + role1 -> role2 -> role5 -> role4 -> role3 + +Roles removed from a configuration are stopped in the order reversed to the order they are specified in a configuration, taking into account the dependencies. +Suppose, all roles except ``role1`` are removed from the configuration above: + +.. code-block:: yaml + + roles: [ role1 ] + +After reloading a configuration, ``stop()`` functions for the removed roles are executed in the following order: + +.. code-block:: none + + role3 -> role4 -> role5 -> role2 + + + + +.. _roles_example_custom_role: + +Example: Role without a configuration +------------------------------------- + +The example below shows how to enable the custom ``greeter`` role for ``instance001``: + +.. literalinclude:: /code_snippets/snippets/config/instances.enabled/application_role/config.yaml + :language: yaml + :start-at: instance001 + :end-at: greeter + :dedent: + +The implementation of this role looks as follows: + +.. literalinclude:: /code_snippets/snippets/config/instances.enabled/application_role/greeter.lua + :language: lua + :dedent: + +Example on GitHub: `application_role `_ + + + +.. _roles_example_custom_role_with_config: + +Example: Role with a configuration +---------------------------------- + +The example below shows how to enable the custom ``greeter`` role for ``instance001`` and specify the configuration for this role: + +.. literalinclude:: /code_snippets/snippets/config/instances.enabled/application_role_cfg/config.yaml + :language: yaml + :start-at: instance001 + :end-at: greeting + :dedent: + +The implementation of this role looks as follows: + +.. literalinclude:: /code_snippets/snippets/config/instances.enabled/application_role_cfg/greeter.lua + :language: lua + :dedent: + +Example on GitHub: `application_role_cfg `_ + + + + +.. _roles_example_custom_role_http_api: + +Example: HTTP API +----------------- + +The example below shows how to enable and configure the ``http-api`` custom role: + +.. literalinclude:: /code_snippets/snippets/config/instances.enabled/application_role_http_api/config.yaml + :language: yaml + :start-at: instance001 + :end-at: 8080 + :dedent: + +The implementation of this role looks as follows: + +.. literalinclude:: /code_snippets/snippets/config/instances.enabled/application_role_http_api/http-api.lua + :language: lua + :dedent: + +Example on GitHub: `application_role_http_api `_ + + + + + +.. _roles_api_reference: + +API Reference +------------- + +.. container:: table + + .. rst-class:: left-align-column-1 + .. rst-class:: left-align-column-2 + + .. list-table:: + :widths: 35 65 + + * - **Members** + - + + * - :ref:`validate([cfg]) ` + - Validate a role's configuration. + + * - :ref:`apply([cfg]) ` + - Apply a role's configuration. + + * - :ref:`stop() ` + - Stop a role. + + * - :ref:`dependencies ` + - Define a role's dependencies. + + + + +.. _roles_api_reference_validate: + +.. function:: validate([cfg]) + + Validate a role's configuration. + This function is called on instance startup or when the :ref:`configuration is reloaded ` for the instance with this role. + Note that the ``validate()`` function is called regardless of whether the role's configuration or any field in a cluster's configuration is changed. + + ``validate()`` should throw an error if the validation fails. + + :param cfg: a role's role configuration to be validated. + This parameter provides access to configuration options defined in :ref:`roles_cfg.\ `. + To get values of configuration options placed outside ``roles_cfg.``, use :ref:`config:get() `. + + See also: :ref:`roles_create_custom_role_validate` + + +.. _roles_api_reference_apply: + +.. function:: apply([cfg]) + + Apply a role's configuration. + ``apply()`` is called after ``validate()`` is executed for all the enabled roles. + As the ``validate()`` function, ``apply()`` is called on instance startup or when the configuration is reloaded for the instance with this role. + + ``apply()`` should throw an error if the specified configuration can't be applied. + + .. NOTE:: + + Note that ``apply()`` is not invoked if an instance switches to read-write mode when :ref:`replication.failover ` is set to ``election`` or ``supervised``. + You can check an instance state by subscribing to the ``box.status`` event using :ref:`box.watch() `. + + :param cfg: a role's role configuration to be applied. + This parameter provides access to configuration options defined in :ref:`roles_cfg.\ `. + To get values of configuration options placed outside ``roles_cfg.``, use :ref:`config:get() `. + + See also: :ref:`roles_create_custom_role_apply` + + +.. _roles_api_reference_stop: + +.. function:: stop() + + Stop a role. + This function is called on configuration reload if the role is removed from ``roles`` for the given instance. + + See also: :ref:`roles_create_custom_role_stop` + +.. _roles_api_reference_dependencies: + +.. data:: dependencies + + (Optional) Define a role's dependencies. + + :rtype: table + + See also: :ref:`roles_create_custom_role_dependencies` + diff --git a/doc/how-to/app/index.rst b/doc/how-to/app/index.rst index 9600843978..452b4caf7e 100644 --- a/doc/how-to/app/index.rst +++ b/doc/how-to/app/index.rst @@ -12,6 +12,7 @@ in C or C++. .. toctree:: :maxdepth: 1 + app_roles launching_app creating_app using_ide diff --git a/doc/reference/configuration/configuration_reference.rst b/doc/reference/configuration/configuration_reference.rst index bb76dd970a..23769f69c7 100644 --- a/doc/reference/configuration/configuration_reference.rst +++ b/doc/reference/configuration/configuration_reference.rst @@ -11,6 +11,100 @@ This topic describes all :ref:`configuration parameters ` provide Most of the configuration options described in this reference can be applied to a specific instance, replica set, group, or to all instances globally. To do so, you need to define the required option at the :ref:`specified level `. + + +.. _configuration_reference_app: + +app +--- + +Using Tarantool as an application server, you can run your own Lua applications. +In the ``app`` section, you can load the application and provide an application configuration in the ``app.cfg`` section. + +.. NOTE:: + + ``app`` can be defined in any :ref:`scope `. + +- :ref:`app.cfg ` +- :ref:`app.file ` +- :ref:`app.module ` + + +.. _configuration_reference_app_cfg: + +.. confval:: app.cfg + + **Since:** :doc:`3.0.0 `. + + A configuration of the application loaded using ``app.file`` or ``app.module``. + + **Example** + + In the example below, the application is loaded from the ``myapp.lua`` file placed next to the YAML configuration file: + + .. literalinclude:: /code_snippets/snippets/config/instances.enabled/application/config.yaml + :language: yaml + :end-at: greeting + :dedent: + + Example on GitHub: `application `_ + + | + | Type: map + | Default: nil + | Environment variable: TT_APP_CFG + + +.. _configuration_reference_app_file: + +.. confval:: app.file + + **Since:** :doc:`3.0.0 `. + + A path to a Lua file to load an application from. + + | + | Type: string + | Default: nil + | Environment variable: TT_APP_FILE + + +.. _configuration_reference_app_module: + +.. confval:: app.module + + **Since:** :doc:`3.0.0 `. + + A Lua module to load an application from. + + **Example** + + The ``app`` section can be placed in any :ref:`configuration scope `. + As an example use case, you can provide different applications for storages and routers in a sharded cluster: + + .. code-block:: yaml + + groups: + storages: + app: + module: storage + # ... + routers: + app: + module: router + # ... + + | + | Type: string + | Default: nil + | Environment variable: TT_APP_MODULE + + + + + + + .. _configuration_reference_audit: audit_log @@ -2586,8 +2680,8 @@ The ``process`` section defines configuration parameters of the Tarantool proces If not specified, defaults to the current working directory. Other directory and file parameters, if set as relative paths, - are interpreted as relative to ``process.work_dir``. For example, - :ref:`directories for storing snapshots and write-ahead logs ` + are interpreted as relative to ``process.work_dir``, for example, directories for storing + :ref:`snapshots and write-ahead logs `. | | Type: string @@ -2974,7 +3068,7 @@ The ``replication`` section defines configuration parameters related to :ref:`re roles ----- -This section describes configuration parameters related to roles. +This section describes configuration parameters related to :ref:`application roles `. .. NOTE:: @@ -2990,6 +3084,11 @@ This section describes configuration parameters related to roles. **Since:** :doc:`3.0.0 `. + Specify the roles of an instance. + To specify a role's configuration, use the :ref:`roles_cfg ` option. + + See also: :ref:`configuration_application_roles` + | | Type: array | Default: nil @@ -3002,6 +3101,12 @@ This section describes configuration parameters related to roles. **Since:** :doc:`3.0.0 `. + Specify a role's configuration. + This option accepts a role name as the key and a role's configuration as the value. + To specify the roles of an instance, use the :ref:`roles ` option. + + See also: :ref:`configuration_application_roles` + | | Type: map | Default: nil