|
1 | | -# Sci-ModoM |
| 1 | +# Sci-ModoM: The quantitative database of transcriptome-wide high-throughput RNA modification sites |
2 | 2 |
|
3 | | -The application consists of the following components: |
4 | | - |
5 | | -- Database: MariaDB |
6 | | -- Server: A REST-API backend using the SQLAlchemy Database Toolkit for Python and Flask |
7 | | -- Client: A Web-GUI using Vue.js |
8 | | - |
9 | | -## General setup |
10 | | - |
11 | | -### Data import |
12 | | - |
13 | | -At launch time, the app uses files defined in `SetupService` to populate the database. These files **must** exist and be located under `IMPORT_PATH` (development) or `HOST_IMPORT_DIR` (production). By default, this path is located under _server/import_, and can be specified in the environment file. See [Flask CLI - Setup](https://dieterich-lab.github.io/scimodom/flask.html#setup) for details. |
14 | | - |
15 | | -### Assembly and annotation |
16 | | - |
17 | | -Assembly and annotation files must be added before any modification data can be uploaded to the database. See [Flask CLI - Setup](https://dieterich-lab.github.io/scimodom/flask.html#setup) for details. |
18 | | - |
19 | | -### Project and data management |
20 | | - |
21 | | -Modification data is organised into projects and datasets, and must be formatted according to the [bedRMod specifications](https://dieterich-lab.github.io/scimodom/bedrmod.html). Consult the **Documentation** tab on the Web-GUI, or [Flask CLI - Project and data management](https://dieterich-lab.github.io/scimodom/flask.html#project-and-data-management) for details. |
22 | | - |
23 | | -### Dependencies |
24 | | - |
25 | | -Sci-ModoM depends on [Bedtools](https://bedtools.readthedocs.io/en/latest/) v2.31.0. There is nothing to do for production. The version is specified in the [Dockerfile](docker/app_container/Dockerfile). For development, it is recommended to use [pre-compiled binaries](https://bedtools.readthedocs.io/en/latest/content/installation.html#downloading-a-pre-compiled-binary) with the correct version number. |
26 | | - |
27 | | -## Production setup |
28 | | - |
29 | | -The recommended way to run Sci-ModoM in production is to use Podman to create, manage, and deploy the application and the database containers, see [Container setup](docker/CONTAINER_SETUP.md) for details. |
30 | | - |
31 | | -## Development setup |
32 | | - |
33 | | -### Database setup |
34 | | - |
35 | | -Set up a MariaDB database. One way to do this is to run a MariaDB container image, see the _General_ and _Development setup_ sections in [Container setup](docker/CONTAINER_SETUP.md). |
36 | | - |
37 | | -### Server setup |
38 | | - |
39 | | -Create a Python3 virtual environment and activate it: |
40 | | - |
41 | | -```bash |
42 | | -python3 -m venv ~/.virtualenvs/scimodom |
43 | | -source ~/.virtualenvs/scimodom/bin/activate |
44 | | -``` |
45 | | - |
46 | | -Get the source code and install: |
47 | | - |
48 | | -```bash |
49 | | -git clone https://github.com/dieterich-lab/scimodom.git |
50 | | -cd scimodom/server |
51 | | -pip install --upgrade pip setuptools |
52 | | -pip --verbose install -e '.[dev,tests,docs]' 2>&1 | tee install.log |
53 | | -``` |
54 | | - |
55 | | -**Note:** The package depends on [mysqlclient](https://pypi.org/project/mysqlclient/). You may have to install MySQL development headers and libraries! |
56 | | - |
57 | | -Set up your environment configuration in _server/.env_: |
58 | | - |
59 | | -```bash |
60 | | -DATABASE_URI=mysql+mysqldb://scimodom: [email protected]:3307/scimodom |
61 | | -SECRET_KEY=SECRET_KEY |
62 | | -SESSION_COOKIE_SAMESITE=None |
63 | | -SESSION_COOKIE_SECURE=True |
64 | | -SMTP_SERVER=mail-server.my-site.org |
65 | | - |
66 | | - |
67 | | -HTTP_PUBLIC_URL=http://localhost:5173/ |
68 | | -UPLOAD_PATH=/path/to/upload |
69 | | -DATA_PATH=/path/to/data |
70 | | -BEDTOOLS_TMP_PATH=/path/to/tmp |
71 | | -``` |
72 | | - |
73 | | -where `PASSWORD` is the password for the scimodom user for the MariaDB database in _docker/secrets/mariadb-scimodom_, `3307` is the `MARIADB_DEV_PORT` (we recommend to use a non-standard port _e.g._ 3307 to avoid clashes with a local MariaDB/MySQL installation), and `SECRET_KEY` is the key found in _docker/secrets/flask-secret_, see [Container setup](docker/CONTAINER_SETUP.md) for details. You need to adjust the paths, and make sure they are valid and exist. |
74 | | - |
75 | | -**Important:** If the host name _localhost_ is used in the `DATABASE_URI` the database driver will assume that the database is contacted using a named |
76 | | -socket. That will not work if a container is used! |
77 | | - |
78 | | -You can also create a _server/.flaskenv_ file with Flask-only variables: |
| 3 | +Sci-ModoM is a quantitative database of RNA modifications dedicated to novel assays that provide transcriptome-wide information at |
| 4 | +single-base resolution. |
79 | 5 |
|
80 | | -```bash |
81 | | -FLASK_APP=src/scimodom/app |
82 | | -FLASK_DEBUG=True |
83 | | -``` |
| 6 | +<p align="center"> |
| 7 | + <a href="https://dieterich-lab.github.io/scimodom/index.html"><img alt="Sci-ModoM" src="https://github.com/dieterich-lab/scimodom/raw/master/docs/source/_static/logo.png"></a> |
| 8 | +</p> |
84 | 9 |
|
85 | | -#### Running the application |
| 10 | +<p align="center"> |
| 11 | + |
| 12 | +[](https://doi.org/10.5281/zenodo.13911907) |
| 13 | +[](https://www.gnu.org/licenses/agpl-3.0) |
| 14 | +</p> |
86 | 15 |
|
87 | | -Start the database container under the _docker_ directory, see [Container setup](docker/CONTAINER_SETUP.md) (Development setup). |
88 | | -Under the _server_ directory, initialize the database schema: |
| 16 | +--- |
89 | 17 |
|
90 | | -```bash |
91 | | -alembic upgrade head |
92 | | -``` |
| 18 | +## Documentation |
93 | 19 |
|
94 | | -and start the API backend: |
95 | | - |
96 | | -```bash |
97 | | -flask run |
98 | | -``` |
99 | | - |
100 | | -Most Python IDEs can run this process in the integrated debugger. You are now ready to add assemblies, annotations, projects, and data (General setup). |
101 | | - |
102 | | -##### Email functionality, local login and registration |
103 | | - |
104 | | -To register in development mode, use the _Sign up_ button. This requires a functional email server. You first need build the frontend (see Client setup below). Once you receive a link via email, click on this link, but change the frontend server address to that of the Flask development server URL, _e.g._ change http://localhost:5173/ to http://localhost:5000. This is only necessary if you run the database using a container and connect it with the local Flask application. |
105 | | - |
106 | | -Note that email functionality may be limited, as your mail server must be willing to relay emails for your `SMTP_FROM_ADDRESS`, _e.g._ Google or Gmail addresses will most likely not work. This may be problematic if you wish to register, as registration is done via a link sent by email. One way to avoid this problem is to patch the database. Open a python console under your environment and do the following |
107 | | - |
108 | | -```python |
109 | | -from werkzeug.security import generate_password_hash |
110 | | -generate_password_hash("mypassword", method="pbkdf2") |
111 | | -# this will return e.g. 'pbkdf2:sha256:600000$vpYjirPAT8xBuPHo$1001474730f96085cdafbf0f159d12e20ec36342b4faddbf226d637c695ee642' |
112 | | -``` |
113 | | - |
114 | | -Then go to the database, see _e.g._ [Container setup - Manual database connection](docker/CONTAINER_SETUP.md) and do the following: |
115 | | - |
116 | | -```mysql |
117 | | -INSERT INTO user (email, state, password_hash) VALUES ('test@uni-heidelberg', 'active', 'pbkdf2:sha256:600000$vpYjirPAT8xBuPHo$1001474730f96085cdafbf0f159d12e20ec36342b4faddbf226d637c695ee642'); |
118 | | -``` |
119 | | - |
120 | | -A new user is now registered, and you can login using whatever email address you used _e.g._ "test@uni-heidelberg" with the chosen password _e.g._ "mypassword". |
121 | | - |
122 | | -### Client setup |
123 | | - |
124 | | -The first time, you need to install the local packages that the project needs, go to the _client_ directory and execute: |
125 | | - |
126 | | -```bash |
127 | | -npm install |
128 | | -``` |
129 | | - |
130 | | -This creates a _node_modules_ folder. You are now ready to bring up the frontend |
131 | | - |
132 | | -```bash |
133 | | -npm run dev |
134 | | -``` |
135 | | - |
136 | | -The application is now available at http://localhost:5173/, and any change you make _e.g._ to the HTML code will be reflected in the page you see in your browser. |
137 | | - |
138 | | -To test the bundled frontend, run: |
139 | | - |
140 | | -```bash |
141 | | -npm run build |
142 | | -``` |
143 | | - |
144 | | -This will populate the folder _dist_ with the bundled static HTML/JavaScript code as it should be deployed in production. |
145 | | -The server can now also serve this code. The complete application is now also available under the Flask development server URL _e.g._ at http://127.0.0.1:5000. |
146 | | - |
147 | | -### Development hints |
148 | | - |
149 | | -#### Pre-commit and static type checker |
150 | | - |
151 | | -Under the _server_ directory: |
152 | | - |
153 | | -```bash |
154 | | -# the first time, you might have to |
155 | | -pre-commit install |
156 | | -# the first time pre-commit runs on a file it will automatically download, install, and run the hook |
157 | | -# runs on all file at commit or run manually |
158 | | -pre-commit run --all-files |
159 | | -# to run individual hooks use pre-commit run <hook_id> |
160 | | - |
161 | | -# run static type checker |
162 | | -mypy -p scimodom |
163 | | -``` |
164 | | - |
165 | | -#### Tests |
166 | | - |
167 | | -To execute the tests, run under the _server_ directory: |
168 | | - |
169 | | -```bash |
170 | | -pytest tests |
171 | | -``` |
| 20 | +The application consists of the following components: |
172 | 21 |
|
173 | | -#### Test automation |
| 22 | +- Database: MariaDB (migrations handled using Alembic) |
| 23 | +- Server: A REST-API backend using the SQLAlchemy Database Toolkit for Python and Flask |
| 24 | +- Client: A Web-GUI using Vue.js and the PrimeVue Tailwind CSS based UI component library |
174 | 25 |
|
175 | | -The Bedtools version is specified in the [Jenkinsfile](Jenkinsfile). |
| 26 | +Consult the [Documentation](https://dieterich-lab.github.io/scimodom/index.html) for installation instructions, containers orchestration, general information about the data model or how to use the CLI, and for the latest bedRMod specifications. |
176 | 27 |
|
177 | | -#### Database schema updates |
| 28 | +General documentation is also available online at [Sci-ModoM - Documentation](https://scimodom.dieterichlab.org/documentation/about). |
178 | 29 |
|
179 | | -The database schema is tracked using Alembic. Changes to the database must be coded at two locations: |
| 30 | +## How to report issues |
180 | 31 |
|
181 | | -- An Alembic migration script under server/migrations/versions |
182 | | -- The model must be updated in server/src/scimodom/database/models.py |
| 32 | +For bugs, issues, or feature requests, use the [bug tracker](https://github.com/dieterich-lab/scimodom/issues). Follow the instructions and guidelines given in the templates. |
183 | 33 |
|
184 | | -Any change to the schema is generally tracked with: |
| 34 | +## How to cite |
185 | 35 |
|
186 | | -```bash |
187 | | -alembic revision [--autogenerate] -m "message" |
188 | | -alembic upgrade head |
189 | | -``` |
| 36 | +Etienne Boileau, Haradl Wilhelmi, Anne Busch, Andrea Cappannini, Andreas Hildebrand, Janusz M. Bujnicki, Christoph Dieterich. [Sci-ModoM: a quantitative database of transcriptome-wide high-throughput RNA modification sites](none), _Nucleic Acids Research_, ... |
190 | 37 |
|
191 | 38 | ## License |
192 | 39 |
|
|
0 commit comments