Improve documentation LOBSTER and PHONON workflow (#1117)

* add info on jobflow remote and lobster workflow

* remove file and fix linting

* add another phonon example

* add tutorial for phonons including mock_vasp

* add tutorial test

* test tutorial test

* test tutorial test

* test tutorial test

* test tutorial test

* split up workflow again

* split up workflow again

* split up workflow again

* split up workflow again

* add automerge

* add automerge

* add mock lobster and a lobster tutorial

* fix bug in tutorial and fix calc summary

* add plot

* fix linting

* fix linting

* more type hints

* fix linting

* fix linting

* fix linting

* fix linting

* fix more linting

* fix final linting

* Update pyproject.toml

* adapt tutorial

* fix linting and add a tmp dir to tutorial execution

* remove file

* print ref and actual values in testing.lobster.verify_inputs ValueError

---------

Co-authored-by: Janosh Riebesell <[email protected]>

Author: JaGeo

.github/workflows/testing.yml

@@ -89,6 +89,7 @@ jobs:
           micromamba activate a2
           pytest --splits 3 --group ${{ matrix.split }} --durations-path tests/.pytest-split-durations --splitting-algorithm least_duration --ignore=tests/ase --cov=atomate2 --cov-report=xml
 
+
       - uses: codecov/codecov-action@v1
         if: matrix.python-version == '3.10' && github.repository == 'materialsproject/atomate2'
         with:
@@ -150,7 +151,7 @@ jobs:
       - name: Test Notebooks
         run: |
           micromamba activate a2
-          pytest --nbmake ./tutorials --ignore=./tutorials/openmm_tutorial.ipynb
+          pytest --nbmake ./tutorials --ignore=./tutorials/openmm_tutorial.ipynb --ignore=./tutorials/force_fields
 
       - name: Test ASE
         env:
@@ -165,6 +166,81 @@ jobs:
           token: ${{ secrets.CODECOV_TOKEN }}
           file: ./coverage.xml
 
+  test-force-field-notebook:
+    # prevent this action from running on forks
+    if: github.repository == 'materialsproject/atomate2'
+
+    services:
+      local_mongodb:
+        image: mongo:4.0
+        ports:
+          - 27017:27017
+
+    runs-on: ubuntu-latest
+    defaults:
+      run:
+        shell: bash -l {0} # enables conda/mamba env activation by reading bash profile
+    strategy:
+      matrix:
+        python-version: ["3.10", "3.11", "3.12"]
+
+    steps:
+      - name: Check out repo
+        uses: actions/checkout@v4
+
+      - name: Set up micromamba
+        uses: mamba-org/setup-micromamba@main
+
+      - name: Create mamba environment
+        run: |
+          micromamba create -n a2 python=${{ matrix.python-version }} --yes
+
+      - name: Install uv
+        run: micromamba run -n a2 pip install uv
+
+      - name: Install conda dependencies
+        run: |
+          micromamba install -n a2 -c conda-forge enumlib packmol bader openbabel openff-toolkit==0.16.2 openff-interchange==0.3.22 --yes
+
+      - name: Install dependencies
+        run: |
+          micromamba activate a2
+          python -m pip install --upgrade pip
+          mkdir -p ~/.abinit/pseudos
+          cp -r tests/test_data/abinit/pseudos/ONCVPSP-PBE-SR-PDv0.4 ~/.abinit/pseudos
+          uv pip install .[strict,strict-forcefields,tests,abinit]
+          uv pip install torch-runstats
+          uv pip install --no-deps nequip==0.5.6
+
+      - name: Install pymatgen from master if triggered by pymatgen repo dispatch
+        if: github.event_name == 'repository_dispatch' && github.event.action == 'pymatgen-ci-trigger'
+        run: |
+          micromamba activate a2
+          uv pip install --upgrade 'git+https://github.com/materialsproject/pymatgen@${{ github.event.client_payload.pymatgen_ref }}'
+
+      - name: Forcefield tutorial
+        env:
+          MP_API_KEY: ${{ secrets.MP_API_KEY }}
+
+        # regenerate durations file with `pytest --store-durations --durations-path tests/.pytest-split-durations`
+        # Note the use of `--splitting-algorithm least_duration`.
+        # This helps prevent a test split having no tests to run, and then the GH action failing, see:
+        # https://github.com/jerry-git/pytest-split/issues/95
+        # However this `splitting-algorithm` means that tests cannot depend sensitively on the order they're executed in.
+        run: |
+          micromamba activate a2
+          pytest --nbmake ./tutorials/force_fields
+
+
+      - uses: codecov/codecov-action@v1
+        if: matrix.python-version == '3.10' && github.repository == 'materialsproject/atomate2'
+        with:
+          token: ${{ secrets.CODECOV_TOKEN }}
+          name: coverage
+          file: ./coverage.xml
+
+
+
   docs:
     runs-on: ubuntu-latest
 
@@ -186,7 +262,7 @@ jobs:
         run: sphinx-build docs docs_build
 
   automerge:
-    needs: [lint, test-non-ase, test-notebooks-and-ase, docs]
+    needs: [lint, test-non-ase, test-notebooks-and-ase, test-force-field-notebook, docs]
     runs-on: ubuntu-latest
 
     permissions:

docs/user/codes/vasp.md

@@ -248,6 +248,25 @@ adjust them if necessary. The default might not be strict enough
 for your specific case.
 ```
 
+You can use the following code to start the standard version of the workflow:
+```py
+from atomate2.vasp.flows.phonons import PhononMaker
+from pymatgen.core.structure import Structure
+
+structure = Structure(
+    lattice=[[0, 2.13, 2.13], [2.13, 0, 2.13], [2.13, 2.13, 0]],
+    species=["Mg", "O"],
+    coords=[[0, 0, 0], [0.5, 0.5, 0.5]],
+)
+
+phonon_flow = PhononMaker(min_length=15.0, store_force_constants=False).make(
+    structure=struct
+)
+```
+
+
+
+
 ### Gruneisen parameter workflow
 
 Calculates mode-dependent Grüneisen parameters with the help of Phonopy.
@@ -352,8 +371,85 @@ lobster = update_user_incar_settings(lobster, {"NPAR": 4})
 run_locally(lobster, create_folders=True, store=SETTINGS.JOB_STORE)
 ```
 
-It is, however,  computationally very beneficial to define two different types of job scripts for the VASP and Lobster runs, as VASP and Lobster runs are parallelized differently (MPI vs. OpenMP).
-[FireWorks](https://github.com/materialsproject/fireworks) allows one to run the VASP and Lobster jobs with different job scripts. Please check out the [jobflow documentation on FireWorks](https://materialsproject.github.io/jobflow/tutorials/8-fireworks.html#setting-the-manager-configs) for more information.
+There are currently three different ways available to run the workflow efficiently, as VASP and LOBSTER rely on a different parallelization (MPI vs. OpenMP).
+One can use a job script (with some restrictions), or [Jobflow-remote](https://matgenix.github.io/jobflow-remote/) / [Fireworks](https://github.com/materialsproject/fireworks) for high-throughput runs.
+
+
+#### Running the LOBSTER workflow without database and with one job script only
+
+It is possible to run the VASP-LOBSTER workflow efficiently with a minimal setup.
+In this case, you will run the VASP calculations on the same node as the LOBSTER calculations.
+In between, the different computations you will switch from MPI to OpenMP parallelization.
+
+For example, for a node with 48 cores, you could use an adapted version of the following SLURM script:
+
+```bash
+#!/bin/bash
+#SBATCH -J vasplobsterjob
+#SBATCH -o ./%x.%j.out
+#SBATCH -e ./%x.%j.err
+#SBATCH -D ./
+#SBATCH --mail-type=END
+#SBATCH [email protected]
+#SBATCH --time=24:00:00
+#SBATCH --nodes=1
+#This needs to be adapted if you run with different cores
+#SBATCH --ntasks=48
+
+# ensure you load the modules to run VASP, e.g., module load vasp
+module load my_vasp_module
+# please activate the required conda environment
+conda activate my_environment
+cd my_folder
+# the following script needs to contain the workflow
+python xyz.py
+```
+
+The `LOBSTER_CMD` now needs an additional export of the number of threads.
+
+```yaml
+VASP_CMD: <<VASP_CMD>>
+LOBSTER_CMD: OMP_NUM_THREADS=48 <<LOBSTER_CMD>>
+```
+
+
+#### Jobflow-remote
+Please refer first to the general documentation of jobflow-remote: [https://matgenix.github.io/jobflow-remote/](https://matgenix.github.io/jobflow-remote/).
+
+```py
+from atomate2.vasp.flows.lobster import VaspLobsterMaker
+from pymatgen.core.structure import Structure
+from jobflow_remote import submit_flow, set_run_config
+from atomate2.vasp.powerups import update_user_incar_settings
+
+structure = Structure(
+    lattice=[[0, 2.13, 2.13], [2.13, 0, 2.13], [2.13, 2.13, 0]],
+    species=["Mg", "O"],
+    coords=[[0, 0, 0], [0.5, 0.5, 0.5]],
+)
+
+lobster = VaspLobsterMaker().make(structure)
+
+resources = {"nodes": 3, "partition": "micro", "time": "00:55:00", "ntasks": 144}
+
+resources_lobster = {"nodes": 1, "partition": "micro", "time": "02:55:00", "ntasks": 48}
+lobster = set_run_config(lobster, name_filter="lobster", resources=resources_lobster)
+
+lobster = update_user_incar_settings(lobster, {"NPAR": 4})
+submit_flow(lobster, worker="my_worker", resources=resources, project="my_project")
+```
+
+The `LOBSTER_CMD` also needs an export of the threads.
+
+```yaml
+VASP_CMD: <<VASP_CMD>>
+LOBSTER_CMD: OMP_NUM_THREADS=48 <<LOBSTER_CMD>>
+```
+
+
+
+#### Fireworks
+Please first refer to the general documentation on running atomate2 workflows with fireworks: [https://materialsproject.github.io/atomate2/user/fireworks.html](https://materialsproject.github.io/atomate2/user/fireworks.html)
 
 Specifically, you might want to change the `_fworker` for the LOBSTER runs and define a separate `lobster` worker within FireWorks:
 
@@ -389,6 +485,16 @@ lpad = LaunchPad.auto_load()
 lpad.add_wf(wf)
 ```
 
+
+The `LOBSTER_CMD` can now be adapted to not include the number of threads:
+
+```yaml
+VASP_CMD: <<VASP_CMD>>
+LOBSTER_CMD: <<LOBSTER_CMD>>
+```
+
+#### Analyzing outputs
+
 Outputs from the automatic analysis with LobsterPy can easily be extracted from the database and also plotted:
 
 ```py
@@ -425,42 +531,6 @@ for number, (key, cohp) in enumerate(
     plotter.save_plot(f"plots_cation_anion_bonds{number}.pdf")
 ```
 
-#### Running the LOBSTER workflow without database and with one job script only
-
-It is also possible to run the VASP-LOBSTER workflow with a minimal setup.
-In this case, you will run the VASP calculations on the same node as the LOBSTER calculations.
-In between, the different computations you will switch from MPI to OpenMP parallelization.
-
-For example, for a node with 48 cores, you could use an adapted version of the following SLURM script:
-
-```bash
-#!/bin/bash
-#SBATCH -J vasplobsterjob
-#SBATCH -o ./%x.%j.out
-#SBATCH -e ./%x.%j.err
-#SBATCH -D ./
-#SBATCH --mail-type=END
-#SBATCH [email protected]
-#SBATCH --time=24:00:00
-#SBATCH --nodes=1
-#This needs to be adapted if you run with different cores
-#SBATCH --ntasks=48
-
-# ensure you load the modules to run VASP, e.g., module load vasp
-module load my_vasp_module
-# please activate the required conda environment
-conda activate my_environment
-cd my_folder
-# the following script needs to contain the workflow
-python xyz.py
-```
-
-The `LOBSTER_CMD` now needs an additional export of the number of threads.
-
-```yaml
-VASP_CMD: <<VASP_CMD>>
-LOBSTER_CMD: OMP_NUM_THREADS=48 <<LOBSTER_CMD>>
-```
 
 (modifying_input_sets)=
 Modifying input sets

pyproject.toml

@@ -123,7 +123,7 @@ strict = [
 strict-forcefields = [
     "calorine==3.0",
     "chgnet==0.4.0",
-    "mace-torch>=0.3.6",
+    "mace-torch==0.3.10",
     "matgl==1.1.3",
     "quippy-ase==0.9.14; python_version < '3.12'",
     "sevenn==0.10.3",

src/atomate2/lobster/schemas.py

@@ -597,6 +597,7 @@ def from_directory(
             "bva_comp": True,
             **calc_quality_kwargs,
         }
+
         cal_quality_dict = Analysis.get_lobster_calc_quality_summary(
             path_to_poscar=structure_path,
             path_to_vasprun=vasprun_path,
@@ -844,34 +845,32 @@ def from_directory(
         calc_quality_text = None
         describe = None
         describe_ionic = None
-        if analyze_outputs:
-            if (
-                icohplist_path.exists()
-                and cohpcar_path.exists()
-                and charge_path.exists()
-            ):
-                (
-                    condensed_bonding_analysis,
-                    describe,
-                    sb_all,
-                ) = CondensedBondingAnalysis.from_directory(
-                    dir_name,
-                    save_cohp_plots=save_cohp_plots,
-                    plot_kwargs=plot_kwargs,
-                    lobsterpy_kwargs=lobsterpy_kwargs,
-                    which_bonds="all",
-                )
-                (
-                    condensed_bonding_analysis_ionic,
-                    describe_ionic,
-                    sb_ionic,
-                ) = CondensedBondingAnalysis.from_directory(
-                    dir_name,
-                    save_cohp_plots=save_cohp_plots,
-                    plot_kwargs=plot_kwargs,
-                    lobsterpy_kwargs=lobsterpy_kwargs,
-                    which_bonds="cation-anion",
-                )
+
+        if analyze_outputs and (
+            icohplist_path.exists() and cohpcar_path.exists() and charge_path.exists()
+        ):
+            (
+                condensed_bonding_analysis,
+                describe,
+                sb_all,
+            ) = CondensedBondingAnalysis.from_directory(
+                dir_name,
+                save_cohp_plots=save_cohp_plots,
+                plot_kwargs=plot_kwargs,
+                lobsterpy_kwargs=lobsterpy_kwargs,
+                which_bonds="all",
+            )
+            (
+                condensed_bonding_analysis_ionic,
+                describe_ionic,
+                sb_ionic,
+            ) = CondensedBondingAnalysis.from_directory(
+                dir_name,
+                save_cohp_plots=save_cohp_plots,
+                plot_kwargs=plot_kwargs,
+                lobsterpy_kwargs=lobsterpy_kwargs,
+                which_bonds="cation-anion",
+            )
             # Get lobster calculation quality summary data
 
             calc_quality_summary = CalcQualitySummary.from_directory(
@@ -971,7 +970,9 @@ def from_directory(
             if describe_ionic is not None
             else None,
             strongest_bonds_cation_anion=sb_ionic,
-            calc_quality_summary=calc_quality_summary,
+            calc_quality_summary=calc_quality_summary
+            if calc_quality_summary is not None
+            else None,
             calc_quality_text=" ".join(calc_quality_text)
             if calc_quality_text is not None
             else None,