Skip to content

Add MatNova model#318

Open
zzz-sl wants to merge 93 commits into
janosh:mainfrom
zzz-sl:main
Open

Add MatNova model#318
zzz-sl wants to merge 93 commits into
janosh:mainfrom
zzz-sl:main

Conversation

@zzz-sl

@zzz-sl zzz-sl commented Mar 12, 2026

Copy link
Copy Markdown
Contributor

Description

Hi Matbench-Discovery team,
We are submitting MatNova-mptrj model to the leaderboard.

Please let us know if any additional adjustments are required.

Best

Checklist

Please open PRs as draft and only mark as ready to review after checking off the following items:

  • [√] I created a new folder and YAML metadata file models/<arch_name>/<model_variant>.yml for my submission. arch_name is the name of the architecture and model_variant.yml includes things like author details, training set names and important hyperparameters.
  • [√] I added the new model as a new attribute Model.<model_variant> enum on the Model enum in enums.py.
  • [√] I uploaded the energy/force/stress model prediction file for the WBM test set to Figshare or another cloud storage service (<yyyy-mm-dd>-<model_variant>-preds.csv.gz).
  • [√] I uploaded the model-relaxed structures file to Figshare or another cloud storage service in JSON lines format (<yyyy-mm-dd>-wbm-IS2RE-FIRE.jsonl.gz). JSON Lines allows fast loading of small numbers of structures with pandas.read_json(lines=True, nrows=100) for inspection.
  • [√] I uploaded the phonon predictions to Figshare or another cloud storage service (<yyyy-mm-dd>-kappa-103-FIRE-<values-of-dist|fmax|symprec>.gz).
  • [√] I included the urls to the Figshare files in the YAML metadata file (models/<arch_name>/<model_variant>.yml). If not using Figshare I have included the urls to the cloud storage service in the description of the PR.
  • [√] I included the test script (test_<arch_name>_<task>.py for task in discovery, kappa, diatomics) that generated the prediction files.

Marking your PR as ready to review will trigger an automated code review, please address any issues raised by the automated review. For the maintainers minimizing the final human review process enables us to merge your submissions much faster!

Additional Information (Optional)

  • I included a training script (train_<arch_name>.py) if I trained a model specifically for this benchmark.
  • I included a readme.md with additional details about my model.

Summary by CodeRabbit

  • New Features

    • Added the matNova-mptrj model with comprehensive benchmarking for phonons, geometry optimization, and discovery.
    • Introduced thermal conductivity calculation workflows.
    • Added structure relaxation and discovery evaluation pipelines that save and merge prediction results for analysis.
  • Documentation

    • Updated OC20 dataset resource link.

zzz-sl and others added 30 commits May 13, 2025 10:29
add esnet model
Correct the check error
Correct the check error
Add the comment # noqa: S301
Make modifications based on the check results
Make modifications based on the check results
add prediction results file
update checkpoint_url, pred_file_url and test set metrics
@CompRhys

Copy link
Copy Markdown
Collaborator

running just prepare-model-submission matnova_mptrj gives very different results for the kappa 103 metrics, see contributing.md

@zzz-sl

zzz-sl commented Mar 17, 2026

Copy link
Copy Markdown
Contributor Author

Thanks for the reminder! There was an error in the kappa 103 JSON submission. Both the JSON file and the corresponding metrics results have now been updated.

@CompRhys

Copy link
Copy Markdown
Collaborator

can you please run just prepare-model-submission matnova_mptrj again and upload the figures and per element errors json that it creates as those are needed for the site.

@zzz-sl

zzz-sl commented Mar 19, 2026

Copy link
Copy Markdown
Contributor Author

The required figures and per-element errors JSON have been uploaded.

@janosh janosh force-pushed the main branch 5 times, most recently from 11b8e76 to 1e4ecf1 Compare March 25, 2026 08:44
affiliation: Ningbo Institute of Artificial Intelligence Industry, Ningbo, China
orcid: https://orcid.org/0000-0002-7185-5044
github: https://github.com/zzz-sl/
repo: https://github.com/zzz-sl/matNova

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

populating these URLs feels like the last blocker to merge.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@zzz-sl Bump on populating this repo

@janosh janosh force-pushed the main branch 2 times, most recently from 610666a to 0edb8a4 Compare April 14, 2026 15:03
@janosh janosh force-pushed the main branch 3 times, most recently from 4c72f3f to dc6b114 Compare April 30, 2026 00:23
@CompRhys CompRhys added the MLIP (compliant) A model trained on allowed MP data label May 19, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

MLIP (compliant) A model trained on allowed MP data

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants