Citation suggestion: ELoRA (ICML 2025) — PEFT for equivariant GNNs

[hyjwpk]

Hi MACE team,

We are the authors of ELoRA: Low-Rank Adaptation for Equivariant GNNs (OpenReview), published at ICML 2025.

We noticed that MACE v0.3.15 introduced LoRA-based fine-tuning for equivariant interatomic potentials, a direction we are very glad to see gaining traction in the community.

To our knowledge, ELoRA was the first published work to formally study and address the challenge of parameter-efficient fine-tuning for SO(3) equivariant GNNs, and was made publicly available several months before v0.3.15. While the implementations differ, with ELoRA applying low-rank decomposition directly to the path-dependent weight matrices and MACE's approach introducing an equivariant bottleneck layer in activation space, the two are mathematically equivalent for o3.Linear layers: both reduce to a rank-r update along each shared irrep path. We appreciate that your implementation was independently developed, and we raise this purely in the spirit of keeping the academic record connected.

If you feel it is appropriate, we would appreciate a citation in the relevant documentation or any accompanying publication.

Finally, we would like to express our sincere appreciation for MACE. Its quality and openness made it possible for us to build on top of it, and we are grateful for the work your team has put into it.

Best regards,
The ELoRA Authors

[vue1999]

Thanks for bringing this to our attention. We were not aware of ELoRA when developing our LoRA fine-tuning implementation, which was done independently.
Our approach was motivated by the original LoRA literature and differs in some implementation details, including our focus on linear layers within MACE as well as extensions to other components such as the readout MLP. That said, the broad direction is clearly closely related to the direction explored in ELoRA.
We appreciate you highlighting the connection. If we prepare a write-up of this functionality, we will make sure to cite your work appropriately.

[ilyes319]

Hey! Thanks for sharing this. We should add it as background in the doc page with context and pointing out the differences as @vue1999 said.

Note that we welcome PRs to the repo of your implementation + docs with proper references to your paper.

[hyjwpk]

Thank you both for the kind and thoughtful responses!

After reviewing your implementation more carefully, we found that our approaches are mathematically equivalent for o3.Linear layers, and we appreciate that your injection-based design is more elegant than our original implementation. The main additional contribution from ELoRA is support for Contraction layers.

We have integrated our Contraction support following your existing injection pattern and submitted two PRs for code and docs.

Please let us know if any adjustments are needed.