Release/23.04 (openimis#167)

* ajustement 5_base/data

* OP-975 Added additional filters to uspAPIGetClaims (openimis#157)

* OP-1066: modular IMIS: time correlations don't work (openimis#158)

* SubmitSingleClaim SP is now includes historical timestamp for ProductItems/ProductServices

* SubmitSingleClaim SP is now taking historical data for items/services

* Historical Price lists are taken care while processing claims

* OTC-780: update business logic (openimis#160)

* OTC-780: update business logic: items/serives left is now being calculate as a sum of quantity in all valid claims assigned to insuree

* OTC-780: refactor the code

* Update openmis-module-test.yml

* Devision by 0 while calculating Allocated amount (openimis#159)

* change encoding to UTF-8

* UTF-8 + LF

* Merge docker and fix SQL

* remove run that never stops

* to unblock CI on other repo

* fix end date for batch process

the old code does a process date between start and stop date BUT processdate is a DateTime so any datetime after 0.00 on the last day was NOT taken into account

* Capitation : Wrong formula to calculate allocation is fixed

* Change type of env INIT_MODE from 'empty' to empty

* Update Dockerfile

* Update Dockerfile

* Fixed the INIT_MODE variable

* utf-8

* Update docker.yml

---------

Co-authored-by: mngoe <[email protected]>
Co-authored-by: Kamil Malinowski <[email protected]>
Co-authored-by: Hiren Soni <[email protected]>
Co-authored-by: Jan <[email protected]>
Co-authored-by: Patrick Delcroix <[email protected]>
Co-authored-by: Patrick Delcroix <[email protected]>
Co-authored-by: Hiren Soni <[email protected]>
Co-authored-by: Damian Borowiecki <[email protected]>

Author: 7 people

Diff for: .github/workflows/docker.yml

@@ -0,0 +1,25 @@
+name: publish
+on:
+  pull_request:
+  workflow_dispatch:
+    inputs:
+      tag:
+        default: latest
+        description: build tag
+        required: false
+jobs:
+  publish-docker-image:
+    runs-on: ubuntu-latest
+    steps:
+    - uses: actions/checkout@v2
+    - name: Login to GitHub Container Registry
+      uses: docker/login-action@v1
+      with:
+        registry: ghcr.io
+        username: ${{ github.actor }}
+        password: ${{ secrets.GITHUB_TOKEN }}
+    - name: Build the hello-docker Docker image
+      run: |
+         TAG_INPUT=${{ github.event.inputs.tag }}
+         docker build . --tag ghcr.io/openimis/openimis-mssql:${TAG_INPUT:-"latest"}
+         docker push ghcr.io/openimis/openimis-mssql:${TAG_INPUT:-"latest"}

Diff for: .github/workflows/file-merge.yml

@@ -15,7 +15,7 @@ jobs:
     steps:
       - uses: actions/checkout@v2
       - name: Merge me!
-        run : bash concatenate_files.sh
+        run : bash script/concatenate_files.sh
       - name: Publish artifact
         uses: actions/upload-artifact@master
         with:

Diff for: .github/workflows/openmis-module-test.yml

@@ -11,7 +11,7 @@ on:
 
 jobs:
   run_test:
-    runs-on: ubuntu-latest
+    runs-on: ubuntu-20.04
     services:
       mssql:
         image: mcr.microsoft.com/mssql/server:2017-latest
@@ -34,7 +34,7 @@ jobs:
           path: './new'
       - name: generate new files
         working-directory: ./new
-        run: bash concatenate_files.sh
+        run: bash script/concatenate_files.sh
       - name: install linux packages
         run: |
             mkdir ./oldMain 

Diff for: Dockerfile

@@ -0,0 +1,15 @@
+FROM  mcr.microsoft.com/mssql/server:2017-latest
+ARG ACCEPT_EULA=Y
+ENV ACCEPT_EULA=N
+ARG SA_PASSWORD=IMISuserP@s
+ENV SA_PASSWORD=IMISuserP@s
+ENV DB_USER_PASSWORD=IMISuserP@s
+ENV DB_NAME=IMIS
+ENV DB_USER=IMISUser
+ENV INIT_MODE=empty
+RUN mkdir -p /app
+COPY script/* /app/
+COPY sql /app/sql
+WORKDIR /app
+RUN chmod a+x /app/*.sh
+CMD /bin/bash ./entrypoint.sh

Diff for: README.md

@@ -41,6 +41,68 @@ For deployment please read the [installation manual](http://openimis.readthedocs
 Please read [CONTRIBUTING.md](https://gist.github.com/PurpleBooth/b24679402957c63ec426) for details on our code of conduct, and the process for submitting pull requests to us.
 -->
 
+## openIMIS dockerized database
+
+
+
+| :bomb: Disclaimer : NOT FOR PRODUCTION USE :bomb: |
+| --- |
+| This repository provides a dockerized openIMIS database. It provides a quick setup for development, testing or demoing. ***It is NOT INTENDED FOR PRODUCTION USE.*** |
+
+
+### ENV
+
+- INIT_MODE if set to demo will init the database to demo (works only if not yet init)
+- SQL_SCRIPT_URL url to init scripts
+- **ACCEPT_EULA** must be set to Y to accept MS SQL EULA
+- SA_PASSWORD default: IMISuserP@s
+- DB_USER_PASSWORD defautl: IMISuserP@s
+- DB_NAMEdefautl: IMIS
+- DB_USER defautl: IMISUser
+
+
+### gettingstarted
+
+Please look for the directions on the openIMIS Wiki: https://openimis.atlassian.net/wiki/spaces/OP/pages/963182705/MO1.1+Install+the+modular+openIMIS+using+Docker
+
+Using the provided docker file, you can build a docker image running a SQL Server 2017, with a restored openIMIS backup database.
+This is done by giving the following ARGs to the docker build command:
+```
+docker build \
+  --build-arg ACCEPT_EULA=Y \
+  --build-arg SA_PASSWORD=<your secret password> \
+  . \
+  -t openimis-db
+```
+
+optinnaly 
+```
+--build-arg SQL_SCRIPT_URL=<url to the sql script to create the database> \
+  --build-arg DB_USER_PASSWORD=StrongPassword
+  --build-arg DB_USER=IMISUser
+  --build-arg DB_NAME=IMIS
+```
+***Notes***:
+* by setting the ACCEPT_EULA=Y, you explicitely accept [Microsoft EULA](https://go.microsoft.com/fwlink/?linkid=857698) for the dockerized SQL Server 2017. Please ensure you read it and use the provided software according to the terms of that license.
+* choose a strong password (at least 8 chars,...)... or SQL Server will complain
+
+
+To start the image in a docker container: `docker run -p 1433:1433 openimis-db`
+To restore the backup inside the container:
+* To spot the ID of the container: `docker container ls` (spot the row with openimis-db IMAGE name)
+
+
+***Note:***
+the container will check if the database exist, if it doesnot it will take the latest demo release version and deploy it , SQL_SCRIPT_URL is per defautl set to "https://github.com/openimis/database_ms_sqlserver/releases/latest/download/sql-files.zip"
+to have data retention when container are recreated volums need to be configured as microsoft docs suggest
+* <host directory>/data:/var/opt/mssql/data : database files
+* <host directory>/log:/var/opt/mssql/log : logs files
+* <host directory>/secrets:/var/opt/mssql/secrets : secrets
+
+The database is writen within the container. If you want to keep your data between container execution, stop/start the container via `docker stop <CONTAINER ID>` / `docker start <CONTAINER ID>` (using `docker run ... ` recreates a new container from the image... thus without any data)
+
+
+
 ## Versioning
 
 We use [SemVer](http://semver.org/) for versioning. For the versions available, see the [tags on this repository](https://github.com/openimis/web_app_vb/tags). 

Diff for: concatenate_files.ps1

@@ -0,0 +1,8 @@
+#!/bin/bash
+
+mkdir output
+cat sql/migrations/1_migration_latest.sql sql/stored_procedures/*.sql > output/fullMigrationScipt.sql
+cat sql/base/*.sql sql/stored_procedures/*.sql > output/fullEmptyDatabase.sql
+cat sql/base/*.sql sql/stored_procedures/*.sql sql/demo/*.sql> output/fullDemoDatabase.sql
+cat sql/base/*.sql sql/stored_procedures/*.sql sql/offline/central.sql > output/fullOfflineCentralDatabase.sql
+cat sql/base/*.sql sql/stored_procedures/*.sql sql/offline/hf.sql > output/fullOfflineHFDatabase.sql

Diff for: script/concatenate_files.sh

@@ -0,0 +1,4 @@
+# https://www.softwaredeveloper.blog/initialize-mssql-in-docker-container
+
+# Run Microsoft SQl Server and initialization script (at the same time)
+/app/run-initialization.sh & /opt/mssql/bin/sqlservr

Diff for: script/entrypoint.sh

@@ -0,0 +1,7 @@
+#!/usr/bin/env bash
+data=$(/opt/mssql-tools/bin/sqlcmd -S localhost -U SA -P $SA_PASSWORD -Q "SELECT COUNT(*)  FROM master.dbo.sysdatabases WHERE name = N'$DB_NAME'" | tr -dc '0-9'| cut -c1 )
+if [ ${data} -eq "1" ]; then
+    exit 0
+else
+    exit 1
+fi

Diff for: script/healthcheck.sh

@@ -0,0 +1,38 @@
+#!/bin/bash
+
+
+# Wait to be sure that SQL Server came up
+sleep 60s
+
+
+# DATABSE initialisation
+
+echo "Database initialisaton"
+# if the table does not exsit it will create the table
+
+# get "1" if the database exist : tr get only the integer, cut only the first integer (the second is the number of row affected)
+data=$(/opt/mssql-tools/bin/sqlcmd -S localhost -U SA -P $SA_PASSWORD -Q "SELECT COUNT(*)  FROM master.dbo.sysdatabases WHERE name = N'$DB_NAME'" | tr -dc '0-9'| cut -c1 )
+if [ ${data} -eq "0" ]; then
+        echo 'download full demo database'
+        echo 'create database user'
+        /opt/mssql-tools/bin/sqlcmd -S localhost -U SA -P $SA_PASSWORD -Q "CREATE LOGIN $DB_USER WITH PASSWORD='${SA_PASSWORD}', CHECK_POLICY = OFF"
+        echo "merging files"
+        ./concatenate_files.sh
+        echo 'create database'
+        #/opt/mssql-tools/bin/sqlcmd -S localhost -U SA -P $SA_PASSWORD -Q "DROP DATABASE IF EXISTS $DB_NAME"
+        /opt/mssql-tools/bin/sqlcmd -S localhost -U SA -P $SA_PASSWORD -Q "CREATE DATABASE $DB_NAME"
+
+        if [ "$INIT_MODE" = "demo" ]; then
+                /opt/mssql-tools/bin/sqlcmd -S localhost -U SA -P $SA_PASSWORD -i output/fullDemoDatabase.sql -d $DB_NAME | grep . | uniq -c
+        else
+                /opt/mssql-tools/bin/sqlcmd -S localhost -U SA -P $SA_PASSWORD -i output/fullEmptyDatabase.sql -d $DB_NAME | grep . | uniq -c
+        fi
+        echo ' give to the user the access to the database'
+        /opt/mssql-tools/bin/sqlcmd -S localhost -U SA -P $SA_PASSWORD -Q "EXEC sp_changedbowner '$DB_USER'" -d $DB_NAME
+else
+        echo "database already existing, nothing to do"
+fi
+
+# manual cleaning command
+# /opt/mssql-tools/bin/sqlcmd -S localhost -U SA -P $SA_PASSWORD -Q "DROP DATABASE $DB_NAME"
+# /opt/mssql-tools/bin/sqlcmd -S localhost -U SA -P $SA_PASSWORD -Q "DROP  LOGIN  $DB_USER"

Diff for: script/run-initialization.sh

@@ -91,7 +91,7 @@ INSERT [dbo].[tblIdentificationTypes] ([IdentificationCode], [IdentificationType
 INSERT [dbo].[tblIdentificationTypes] ([IdentificationCode], [IdentificationTypes], [AltLanguage], [SortOrder]) VALUES (N'V', N'Voter Card', N'Carte d''électeur', NULL)
 SET IDENTITY_INSERT [dbo].[tblIMISDefaults] ON 
 
-INSERT [dbo].[tblIMISDefaults] ([DefaultID], [PolicyRenewalInterval], [FTPHost], [FTPUser], [FTPPassword], [FTPPort], [FTPEnrollmentFolder], [AssociatedPhotoFolder], [FTPClaimFolder], [FTPFeedbackFolder], [FTPPolicyRenewalFolder], [FTPPhoneExtractFolder], [FTPOffLineExtractFolder], [AppVersionBackEnd], [AppVersionEnquire], [AppVersionEnroll], [AppVersionRenewal], [AppVersionFeedback], [AppVersionClaim], [OffLineHF], [WinRarFolder], [DatabaseBackupFolder], [OfflineCHF], [SMSLink], [SMSIP], [SMSUserName], [SMSPassword], [SMSSource], [SMSDlr], [SMSType], [AppVersionFeedbackRenewal], [AppVersionImis]) VALUES (1, 14, N'', N'', N'', 0, N'/Images/Submitted', N'/Images/Updated', N'', N'', N'', N'', N'', CAST(1.2 AS Decimal(3, 1)), CAST(0.0 AS Decimal(3, 1)), CAST(0.0 AS Decimal(3, 1)), CAST(0.0 AS Decimal(3, 1)), CAST(0.0 AS Decimal(3, 1)), CAST(0.0 AS Decimal(3, 1)), 0, N'C:\Program Files (x86)\WinRAR\', N'', 0, N'', N'', N'', N'', N'', 1, 1, CAST(1.2 AS Decimal(3, 1)), CAST(0.0 AS Decimal(3, 1)))
+INSERT [dbo].[tblIMISDefaults] ([DefaultID], [PolicyRenewalInterval], [FTPHost], [FTPUser], [FTPPassword], [FTPPort], [FTPEnrollmentFolder], [AssociatedPhotoFolder], [FTPClaimFolder], [FTPFeedbackFolder], [FTPPolicyRenewalFolder], [FTPPhoneExtractFolder], [FTPOffLineExtractFolder], [AppVersionBackEnd], [AppVersionEnquire], [AppVersionEnroll], [AppVersionRenewal], [AppVersionFeedback], [AppVersionClaim], [OffLineHF], [WinRarFolder], [DatabaseBackupFolder], [OfflineCHF], [SMSLink], [SMSIP], [SMSUserName], [SMSPassword], [SMSSource], [SMSDlr], [SMSType], [AppVersionFeedbackRenewal], [AppVersionImis]) VALUES (1, 14, N'', N'', N'', 0, N'/Images/Submitted', N'Images/Updated', N'', N'', N'', N'', N'', CAST(1.2 AS Decimal(3, 1)), CAST(0.0 AS Decimal(3, 1)), CAST(0.0 AS Decimal(3, 1)), CAST(0.0 AS Decimal(3, 1)), CAST(0.0 AS Decimal(3, 1)), CAST(0.0 AS Decimal(3, 1)), 0, N'C:\Program Files (x86)\WinRAR\', N'', 0, N'', N'', N'', N'', N'', 1, 1, CAST(1.2 AS Decimal(3, 1)), CAST(0.0 AS Decimal(3, 1)))
 SET IDENTITY_INSERT [dbo].[tblIMISDefaults] OFF
 INSERT [dbo].[tblIMISDefaultsPhone] ([RuleName], [RuleValue], [Usage]) VALUES (N'AllowInsureeWithoutPhoto', 0, 'Allow synchronization of Insurees without a Photo.')
 INSERT [dbo].[tblIMISDefaultsPhone] ([RuleName], [RuleValue], [Usage]) VALUES (N'AllowFamilyWithoutPolicy', 0, 'Allow synchronization of Families without a Policy.')