Skip to content

Commit 1718168

Browse files
wanghan-iapcmHan Wang
wanghan-iapcm
and
Han Wang
authored
Add documentation (#79)
* fix bugs of import functions * fix bug of context * add example of new input * new format of submit input. add dargs check. * command line option for old-style compatibility * fix s3 config of commands * add showkey, fix minor issues * add missing file * fix bug * add guidance for dpgen2 command and input scripts. Co-authored-by: Han Wang <wang_han@iapcm.ac.cn>
1 parent 3a4d08f commit 1718168

13 files changed

Lines changed: 354 additions & 5 deletions

File tree

‎docs/alloy_configs.rst‎

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,7 @@
1+
Alloy configs
2+
=============
3+
4+
.. _alloy configs:
5+
.. dargs::
6+
:module: dpgen2.utils.alloy_conf
7+
:func: generate_alloy_conf_args

‎docs/cli.rst‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,7 @@
11
Command line interface
22
======================
33

4+
.. _fullcli:
45
.. argparse::
56
:module: dpgen2.entrypoint.main
67
:func: main_parser

‎docs/index.rst‎

Lines changed: 6 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -12,8 +12,13 @@ DPGEN2 is the 2nd generation of the Deep Potential GENerator.
1212
:maxdepth: 3
1313
:caption: User Guide
1414

15+
quickcli
1516
cli
17+
input.md
18+
submit_args
1619
op_configs
20+
alloy_configs
21+
step_configs
1722

1823
.. _developer-guide:
1924

@@ -31,4 +36,4 @@ DPGEN2 is the 2nd generation of the Deep Potential GENerator.
3136
* :ref:`search`
3237

3338
.. _feedback:
34-
.. _affiliated packages:
39+
.. _affiliated packages:

‎docs/input.md‎

Lines changed: 178 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,178 @@
1+
(inputscript)=
2+
# Guide on writing input scripts for dpgen2 commands
3+
4+
## Preliminaries
5+
6+
The reader of this doc is assumed to be familiar with the concurrent learning algorithm that the dpgen2 implements. If not, one may check [this paper](https://doi.org/10.1016/j.cpc.2020.107206).
7+
8+
## The input script for all dpgen2 commands
9+
10+
For all the dpgen2 commands, one need to provide `dflow2` global configurations. For example,
11+
```json
12+
"dflow_config" : {
13+
"host" : "http://address.of.the.host:port"
14+
},
15+
"dflow_s3_config" : {
16+
"s3_endpoint" : "address.of.the.s3.sever:port"
17+
},
18+
```
19+
The `dpgen` simply pass all keys of `"dflow_config"` to `dflow.config` and all keys of `"dflow_s3_config"` to `dflow.s3_config`.
20+
21+
22+
## The input script for `submit` and `resubmit`
23+
24+
The full documentation of the `submit` and `resubmit` script can be [found here](submitargs). This documentation provides a fast guide on how to write the input script.
25+
26+
In the input script of `dpgen2 submit` and `dpgen2 resubmit`, one needs to provide the definition of the workflow and how they are executed in the input script. One may find an example input script in the [dpgen2 Al-Mg alloy example](../examples/almg/input.json).
27+
28+
The definition of the workflow can be provided by the following sections:
29+
30+
### Inputs
31+
32+
This section provides the inputs to start a dpgen2 workflow. An example for the Al-Mg alloy
33+
```json
34+
"inputs": {
35+
"type_map": ["Al", "Mg"],
36+
"mass_map": [27, 24],
37+
"init_data_sys": [
38+
"path/to/init/data/system/0",
39+
"path/to/init/data/system/1"
40+
],
41+
}
42+
```
43+
The key `"init_data_sys"` provides the initial training data to kick-off the training of deep potential (DP) models.
44+
45+
46+
### Training
47+
48+
This section defines how a model is trained.
49+
```json
50+
"train" : {
51+
"type" : "dp",
52+
"numb_models" : 4,
53+
"config" : {},
54+
"template_script" : {
55+
"_comment" : "omitted content of tempalte script"
56+
},
57+
"_comment" : "all"
58+
}
59+
```
60+
The `"type" : "dp"` tell the traning method is `"dp"`, i.e. calling [DeePMD-kit](https://github.com/deepmodeling/deepmd-kit) to train DP models.
61+
The `"config"` key defines the training configs, see [the full documentation](rundptrainargs).
62+
The `"template_script"` provides the template training script in `json` format.
63+
64+
65+
### Exploration
66+
67+
This section defines how the configuration space is explored.
68+
```json
69+
"explore" : {
70+
"type" : "lmp",
71+
"config" : {
72+
"command": "lmp -var restart 0"
73+
},
74+
"max_numb_iter" : 5,
75+
"conv_accuracy" : 0.9,
76+
"fatal_at_max" : false,
77+
"f_trust_lo": 0.05,
78+
"f_trust_hi": 0.50,
79+
"configurations": [
80+
{
81+
"lattice" : ["fcc", 4.57],
82+
"replicate" : [2, 2, 2],
83+
"numb_confs" : 30,
84+
"concentration" : [[1.0, 0.0], [0.5, 0.5], [0.0, 1.0]]
85+
}
86+
{
87+
"lattice" : ["fcc", 4.57],
88+
"replicate" : [3, 3, 3],
89+
"numb_confs" : 30,
90+
"concentration" : [[1.0, 0.0], [0.5, 0.5], [0.0, 1.0]]
91+
}
92+
],
93+
"stages": [
94+
{ "_idx": 0, "ensemble": "nvt", "nsteps": 20, "press": null, "conf_idx": [0], "temps": [50,100], "trj_freq": 10, "n_sample" : 3 },
95+
{ "_idx": 1, "ensemble": "nvt", "nsteps": 20, "press": null, "conf_idx": [1], "temps": [50,100], "trj_freq": 10, "n_sample" : 3 }
96+
],
97+
}
98+
```
99+
The `"type" : "lmp"` means that configurations are explored by LAMMPS DPMD runs.
100+
The `"config"` key defines the lmp configs, see [the full documentation](runlmpargs).
101+
The `"configurations"` provides the initial configurations (coordinates of atoms and the simulation cell) of the DPMD simulations. It is a list. The elements of the list can be
102+
103+
- `list[str]`: The strings provides the path to the configuration files.
104+
- `dict`: Automatic alloy configuration generator. See [the detailed doc](alloy_configs) of the allowed keys.
105+
106+
The `"stages"` defines the exploration stages. It is a list of `dict`s, with each `dict` defining a stage. The `"ensemble"`, `"nsteps"`, `"press"`, `"temps"`, `"traj_freq"` keys are self-explanatory. `"conf_idx"` pickes initial configurations of DPMD simulations from the `"configurations"` list, it provides the index of the element in the `"configurations"` list. `"n_sample"` tells the number of confgiruations randomly sampled from the set picked by `"conf_idx"` for each thermodynamic state. All configurations picked by `"conf_idx"` has the same possibility to be sampled. The default value of `"n_sample"` is `null`, in this case all picked configurations are sampled. In the example, each stage have 3 samples and 2 thermodynamic states (NVT, T=50 and 100K), then each iteration run 3x2=6 NVT DPMD simulatins.
107+
108+
109+
### FP
110+
111+
This section defines the first-principle (FP) calculation .
112+
113+
```json
114+
"fp" : {
115+
"type" : "vasp",
116+
"config" : {
117+
"command": "source /opt/intel/oneapi/setvars.sh && mpirun -n 16 vasp_std"
118+
},
119+
"task_max": 2,
120+
"pp_files": {"Al" : "vasp/POTCAR.Al", "Mg" : "vasp/POTCAR.Mg"},
121+
"incar": "vasp/INCAR",
122+
"_comment" : "all"
123+
}
124+
```
125+
The `"type" : "vasp"` means that first-principles are VASP calculations.
126+
The `"config"` key defines the vasp configs, see [the full documentation](runvaspargs).
127+
The `"task_max"` key defines the maximal number of vasp calculations in each dpgen2 iteration.
128+
The `"pp_files"` and `"incar"` keys provides the pseudopotential files and the template incar file.
129+
130+
131+
### Configuration of dflow step
132+
133+
The execution units of the dpgen2 are the dflow `Step`s. How each step is executed is defined by the `"step_configs"`.
134+
```json
135+
"step_configs":{
136+
"prep_train_config" : {
137+
"_comment" : "content omitted"
138+
},
139+
"run_train_config" : {
140+
"_comment" : "content omitted"
141+
},
142+
"prep_explore_config" : {
143+
"_comment" : "content omitted"
144+
},
145+
"run_explore_config" : {
146+
"_comment" : "content omitted"
147+
},
148+
"prep_fp_config" : {
149+
"_comment" : "content omitted"
150+
},
151+
"run_fp_config" : {
152+
"_comment" : "content omitted"
153+
},
154+
"select_confs_config" : {
155+
"_comment" : "content omitted"
156+
},
157+
"collect_data_config" : {
158+
"_comment" : "content omitted"
159+
},
160+
"cl_step_config" : {
161+
"_comment" : "content omitted"
162+
},
163+
"_comment" : "all"
164+
},
165+
```
166+
The configs for prepare training, run training, prepare exploration, run exploration, prepare fp, run fp, select configurations, collect data and concurrent learning steps are given correspondingly.
167+
168+
The readers are refered to [this page](stepconfigargs) for a full documentation of the step configs.
169+
170+
Any of the config in the `step_configs` can be ommitted. If so, the configs of the step is set to the default step configs, which is provided by the following section, for example,
171+
```json
172+
"default_step_config" : {
173+
"template_config" : {
174+
"image" : "dpgen2:x.x.x"
175+
}
176+
},
177+
```
178+
The way of writing the `default_step_config` is the same as any step config in the `step_configs`. One may refer to [this page](stepconfigargs) for full documentation.

‎docs/op_configs.rst‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -3,18 +3,21 @@ OP Configs
33

44
RunDPTrain
55
----------
6+
.. _rundptrainargs:
67
.. dargs::
78
:module: dpgen2.op.run_dp_train
89
:func: config_args
910

1011
RunLmp
1112
------
13+
.. _runlmpargs:
1214
.. dargs::
1315
:module: dpgen2.op.run_lmp
1416
:func: config_args
1517

1618
RunVasp
1719
-------
20+
.. _runvaspargs:
1821
.. dargs::
1922
:module: dpgen2.op.run_vasp
2023
:func: config_args

‎docs/quickcli.md‎

Lines changed: 86 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,86 @@
1+
# Guide on dpgen2 commands
2+
3+
One may use dpgen2 through command line interface. A full documentation of the cli is found [here](fullcli)
4+
5+
## Submit a workflow
6+
The dpgen2 workflow can be submitted via the `submit` command
7+
```bash
8+
dpgen2 submit input.json
9+
```
10+
where `input.json` is the input script. A guide of writing the script is found [here](inputscript).
11+
When a workflow is submitted, a ID (WFID) of the workflow will be printed for later reference.
12+
13+
## Check the convergence of a workflow
14+
The convergence of stages of the workflow can be checked by the `status` command. It prints the indexes of the finished stages, iterations, and the accurate, candidate and failed ratio of explored configurations of each iteration.
15+
```bash
16+
$ dpgen2 status input.json WFID
17+
# stage id_stg. iter. accu. cand. fail.
18+
# Stage 0 --------------------
19+
0 0 0 0.8333 0.1667 0.0000
20+
0 1 1 0.7593 0.2407 0.0000
21+
0 2 2 0.7778 0.2222 0.0000
22+
0 3 3 1.0000 0.0000 0.0000
23+
# Stage 0 converged YES reached max numb iterations NO
24+
# All stages converged
25+
```
26+
27+
## Watch the progress of a workflow
28+
The progress of a workflow can be watched on-the-fly
29+
```bash
30+
$ dpgen2 watch input.json WFID
31+
INFO:root:steps iter-000000--prep-run-train----------------------- finished
32+
INFO:root:steps iter-000000--prep-run-lmp------------------------- finished
33+
INFO:root:steps iter-000000--prep-run-fp-------------------------- finished
34+
INFO:root:steps iter-000000--collect-data------------------------- finished
35+
INFO:root:steps iter-000001--prep-run-train----------------------- finished
36+
INFO:root:steps iter-000001--prep-run-lmp------------------------- finished
37+
...
38+
```
39+
The artifacts can be downloaded on-the-fly with `-d` flag.
40+
41+
42+
## Show the keys of steps
43+
44+
Each dpgen2 step is assigned a unique key. The keys of the finished steps can be checked with `showkey` command
45+
```bash $ dpgen2 watch input.json WFID
46+
0 : init--scheduler
47+
1 : init--id
48+
2 : iter-000000--prep-train
49+
3 -> 6 : iter-000000--run-train-0000 -> iter-000000--run-train-0003
50+
7 : iter-000000--prep-run-train
51+
8 : iter-000000--prep-lmp
52+
9 -> 17 : iter-000000--run-lmp-000000 -> iter-000000--run-lmp-000008
53+
18 : iter-000000--prep-run-lmp
54+
19 : iter-000000--select-confs
55+
20 : iter-000000--prep-fp
56+
21 -> 24 : iter-000000--run-fp-000000 -> iter-000000--run-fp-000003
57+
25 : iter-000000--prep-run-fp
58+
26 : iter-000000--collect-data
59+
27 : iter-000000--block
60+
28 : iter-000000--scheduler
61+
29 : iter-000000--id
62+
30 : iter-000001--prep-train
63+
31 -> 34 : iter-000001--run-train-0000 -> iter-000001--run-train-0003
64+
35 : iter-000001--prep-run-train
65+
36 : iter-000001--prep-lmp
66+
37 -> 45 : iter-000001--run-lmp-000000 -> iter-000001--run-lmp-000008
67+
46 : iter-000001--prep-run-lmp
68+
47 : iter-000001--select-confs
69+
48 : iter-000001--prep-fp
70+
49 -> 52 : iter-000001--run-fp-000000 -> iter-000001--run-fp-000003
71+
53 : iter-000001--prep-run-fp
72+
54 : iter-000001--collect-data
73+
55 : iter-000001--block
74+
56 : iter-000001--scheduler
75+
57 : iter-000001--id
76+
```
77+
78+
79+
## Resubmit a workflow
80+
81+
If a workflow stopped abnormally, one may submit a new workflow with some steps of the old workflow reused.
82+
```bash
83+
dpgen2 resubmit input.json WFID --reuse 0-49
84+
```
85+
The steps of workflow WDID 0-49 will be reused in the new workflow. The indexes of the steps are printed by `dpgen2 showkey`. In the example, all the steps before the `iter-000001--run-fp-000000` will be used in the new workflow.
86+

‎docs/step_configs.rst‎

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,9 @@
1+
Step Configs
2+
============
3+
4+
.. _stepconfigargs:
5+
Configurations for dflow steps
6+
------------------------------
7+
.. dargs::
8+
:module: dpgen2.utils.step_config
9+
:func: step_conf_args

‎docs/submit_args.rst‎

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,7 @@
1+
Arguments of the submit script
2+
==============================
3+
4+
.. _submitargs:
5+
.. dargs::
6+
:module: dpgen2.entrypoint.submit_args
7+
:func: submit_args

‎dpgen2/entrypoint/main.py‎

Lines changed: 25 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -21,6 +21,9 @@
2121
from .status import (
2222
status,
2323
)
24+
from .showkey import (
25+
showkey,
26+
)
2427
from .download import (
2528
download,
2629
)
@@ -93,6 +96,21 @@ def main_parser() -> argparse.ArgumentParser:
9396
"-o", "--old-compatible", action='store_true', help="compatible with old-style input script used in dpgen2 < 0.0.6."
9497
)
9598

99+
##########################################
100+
# show key
101+
parser_showkey = subparsers.add_parser(
102+
"showkey",
103+
help="Print the keys of the successful DPGEN2 steps",
104+
formatter_class=argparse.ArgumentDefaultsHelpFormatter,
105+
)
106+
parser_showkey.add_argument(
107+
"CONFIG", help="the config file in json format."
108+
)
109+
parser_showkey.add_argument(
110+
"ID", help="the ID of the existing workflow."
111+
)
112+
113+
96114
##########################################
97115
# status
98116
parser_status = subparsers.add_parser(
@@ -213,6 +231,13 @@ def main():
213231
status(
214232
wfid, config,
215233
)
234+
elif args.command == "showkey":
235+
with open(args.CONFIG) as fp:
236+
config = json.load(fp)
237+
wfid = args.ID
238+
showkey(
239+
wfid, config,
240+
)
216241
elif args.command == "download":
217242
with open(args.CONFIG) as fp:
218243
config = json.load(fp)

0 commit comments

Comments
 (0)