Skip to content

Latest commit

 

History

History
151 lines (101 loc) · 6.42 KB

stats-service.adoc

File metadata and controls

151 lines (101 loc) · 6.42 KB

Stats Service

Stats Service is a service that helps Monarc applications aggregate statistics from its different instances and serves Monarc applications with the data, plain or transformed to different formats for the charts' views. The stats, displayed in the Monarc Front Office local user interface on Global Dashboard tab, is called Local Instance Statistics.

Different installations of Stats Service can also share the statistics to the Global Statistics (Monarc dashboard) that can be used for the community and future weather forecast. The shared data do not have an identification of Monarc Front Office instance.

An architecture of Monarc Front Offices and Back Offices instance(s) communicated with Stats Service(s) and Global Statistics can be found here.

Installation of the Stats Service

Depends on the Monarc project infrastructure the Stats Service can be installed in different places.

In case if the Monarc installation has a Back Office instance, which means that you most probably have multiple Front Office instances, it is recommended to install Stats Service on the same server where the Back Office app is installed. The reason is that all the Front Office instances as well as Back Office need to be able to communicate with Stats Service. But having a separate (virtual) server for StatsService is also a good solution.

The installation instructions as well as system requirements can be found here. Stats Service project is available on GitHub.

Accounts on the Stats Service

In order to send and get the statistics to/from the Stats Service you need to create accounts for each of your Front Office instances.

Note. This operation is done by ansible, but in case of installations of only Front Office instance(s) or if ansible is not used, the process can be done manually.

There is a console command to create an account, needs to be executed from your Stats Service home folder:

  contrib/create_client.sh {YourClientName}

where {YourClientName} is a name of your Front Office instance.

The output should contain the following lines (for example):

  ~/stats-service$ contrib/create_client.sh NC3
  UUID: c08e380f-7ec0-43ff-bca2-4505856739e0
  Name: NC3
  Role: 1
  Token: ECsBPF5Lh3NreoSUTsxwDESri0UuIkpmHcHk6Zr8lXXQX_TVNAEnGv_6hGvmyBVw9XONpUCw2KIGoufRJ4jCFB
  Created at: 2021-01-01 12:51:18.776037

Configuration of Front Office instance

Note. Ansible does the job for us, the configuration is needed when it is not used.

When the client is created on Stats Service side, the generated token need to be set in the Front Office(es) configuration file(s).

The other option to be setup is the Stats Service baseUrl. The value depends on its installation and can be, for instance, an ip address of the Back Office server. The default port is 5005.

Configuration of the cronjob

The user, under which the crontab is executed, should have read and execute permissions to the following files:

  • /var/lib/monarc/fo/scripts/collect-stats-for-all-clients.sh

  • /var/lib/monarc/fo/vendor/monarc/frontoffice/bin/console

Note. The permissions are set by ansible during the deployment process, but in case if ansible is not used, then permissions can be set like:

  chmod 755 /var/lib/monarc/fo/scripts/collect-stats-for-all-clients.sh`
  chmod 755 /var/lib/monarc/fo/vendor/monarc/frontoffice/bin/console`

The cronjob needs to be scheduled on daily base to have a precise stats data evolution and changes.

It is possible to create a cronjob file (name can be e.g. stats-collector) in the directory /var/etc/cron.d/ with the following content:

  05 00 * * * www-data /var/lib/monarc/fo/bin/console monarc:collect-stats >> /var/log/crontab/collect-stats.log

In case of multiple Front Office instances:

  05 00 * * * www-data /var/lib/monarc/fo/scripts/collect-stats-for-all-clients.sh >> /var/log/crontab/collect-stats.log

Or modify the crontab for a particular user (e.g. www-data): sudo -u www-data crontab -e

  05 00 * * * /var/lib/monarc/fo/bin/console monarc:collect-stats >> /var/log/crontab/collect-stats.log

In case of multiple Front Office instances:

  05 00 * * * /var/lib/monarc/fo/scripts/collect-stats-for-all-clients.sh >> /var/log/crontab/collect-stats.log

To be able to log the stats collection output, needs to be created the file in log directory with the write permissions for the user, which executes the cronjob:

  sudo mkdir -p /var/log/crontab
  sudo touch /var/log/crontab/collect-stats.log
  sudo chmod 0766 /var/log/crontab/collect-stats.log

Recommendations

It is recommended to test the integration between Front Office and Stat Service by executing the same command scheduled in the crontab:

  /var/lib/monarc/fo/bin/console monarc:collect-stats

In case of multiple Front Office instances:

  /var/lib/monarc/fo/scripts/collect-stats-for-all-clients.sh >> /var/log/crontab/collect-stats.log

As an output in the log file collect-stats.log should be list of UUIDs of analyses, the statistics collected for and sent to Stats Service.

Challenges

In case if your Front Office and Stats Service installed on different physical or virtual servers and there is a proxy configuration, you need to be sure that the internal connection goes directly. The user, which executes the cronjob, can have no-proxy environment setting, as far as it’s not needed for internal communication.

The servers connectivity can be traced with the network monitoring.

Global Dashboard on Stats Service

In order to make the web interface of Global Dashboard on Stats Service accessible, should be configured a virtual host and proxy settings to forward the particular requests to the service. It can be configured the same way as your Back Office instance access.