feat: add MACE interface#224
Conversation
modified: sample_inputs/input.in.mace modified: src/mace_mpi_api.F90
new file: interfaces/MACE/README.md
|
@FelyCZ this is super exciting, thanks! Is this PR ready for review? |
|
Hi @danielhollas, I have tested the code on a local machine (WSL Ubuntu 22.04.5 LTS) and it yielded no problems. |
|
Perfect, thanks! I just gave you Triage permissions to the repo which should give you enough rights to ask for review and run tests on GitHub (I think). I'll leave it up to @JanosJiri regarding the level of permissions that you should have (perhaps you should be a member of the PHOTOX Dev Team which has write permissions to this repo?). Thanks for adding unit tests! One of them is failing though, can you take a look? The build failure of GFortran 9 is unrelated to these changes so don't worry about it, I'll fix that separately before we merge this. Can you also please run the autoformatter? https://github.com/PHOTOX/ABIN/blob/master/CONTRIBUTING.md#code-formatting Once the format and test pass on GitHub please ping me for review, thanks! |
|
Thanks for the Triage role! I've fixed the unit test and run the autoformat script. The actions are passing now. |
I could have a look, but I won't be able to get to it for quite some time. Sorry, I'm just overwhelmed with my thesis and finishing all papers before I leave the group. Also, I have very limited knowledge of MPI, so I won't be able to review it reliably (at least now).
@FelyCZ, I can add you to the PHOTOX Dev Team](https://github.com/orgs/PHOTOX/teams/devs) if you want and if you plan to work on ABIN or anything else in the repo. There is no official policy for that. |
|
@FelyCZ if you merge in master that should fix the gcc9 build. |
There was a problem hiding this comment.
Just a quick comment, I don't think we should be including the MACE-OFF model in the repo, it's a huge binary blob, and it will likely become obsolete soon as new models come out.
We can definitely provide instructions and link about where it can be downloaded though!
Is this file needed for tests? If so, we should download it as part of the MACE installation script perhaps.
There was a problem hiding this comment.
I agree that it's not entirely necessary. The main idea was to provide users with a quick way to test the interface without any further configuration.
The file is not necessary for the tests; however, the default configuration (&mace namelist) uses the file as fallback.
There was a problem hiding this comment.
Hi, @danielhollas,
Are you still of the opinion that we should remove the model from the repo?
There was a problem hiding this comment.
Hi, yes I think we should remove it. Please add instructions somewhere (perhaps a README file) where we can tell the users how to download it.
Sorry for the slow review, this is a big PR and I want to have a careful look and we also plan to test it soon.
danielhollas
left a comment
There was a problem hiding this comment.
Did a very quick initial review, I hope these comments are useful.
|
|
||
| git clone "${REPO_URL}" "${REPO_DIR}" && cd "$REPO_DIR" | ||
|
|
||
| pip install --upgrade pip |
There was a problem hiding this comment.
This will install the package into the global python environment (unless the user is already inside a venv or conda. I think we should create a fresh venv for the user.
| git clone "${REPO_URL}" "${REPO_DIR}" && cd "$REPO_DIR" | ||
|
|
||
| pip install --upgrade pip | ||
| pip install . |
There was a problem hiding this comment.
Why are we installing MACE from the repo? We should I think install the package from PyPI.
|
|
||
| pip install --upgrade pip | ||
| pip install . | ||
| pip install mpi4py |
There was a problem hiding this comment.
To be clear, I think we should do something like this:
python -m venv .mace_venv
.mace_venv/bin/python3 -m pip install mace-torch mpi4py
| self.device = str(torch_tools.init_device(device)) | ||
|
|
||
| import os | ||
| if os.path.isfile(model_path): |
There was a problem hiding this comment.
This interface is a bit confusing. Should we instead have two diferrent (mutually exclusive) parameters, model_path and model_name?
| if os.path.isfile(model_path): | ||
| # Load from local file | ||
| try: | ||
| self.model = torch.jit.load(f=model_path, map_location=device).to(device) |
There was a problem hiding this comment.
I am a little confused as to why we're importing the model directly to torch, and not using MACE specific functions / interface for it?
There was a problem hiding this comment.
We didn't use the default pre-trained model; we used a custom, fine-tuned model based on MACE-OFF-23. However, the MACE functions do not support the use of custom models (afaik). Therefore the interface is based on eval_configs.py file.
There was a problem hiding this comment.
Can you say a bit more about your fine-tuned model? Is it still in a *.model format?
The mace.calculators.MACECalculator definitely supports arbitrary models, not just the pre-trained ones. See for example the code here:
| port_name = MPI.Open_port() | ||
| log(f"MPI port opened: {port_name}") | ||
|
|
||
| port_file = "mace_port.txt.1" |
There was a problem hiding this comment.
Let's define this as a "constant" at the top of the script, something like:
MACE_PORT_FILE = "mace_port.txt.1"| When using OpenMPI, you need an `ompi-server` running for the connection handshake: | ||
|
|
||
| ```console | ||
| # 1. Start ompi-server |
There was a problem hiding this comment.
Is this really the case? We don't seem to need something like this in the TeraChem interface?
|
|
||
| ## Configuration | ||
|
|
||
| MACE settings are controlled via the `&mace` namelist in the ABIN input file (`input.in`). Below is the full list of available parameters: |
There was a problem hiding this comment.
So I am not totally opposed to this, but note that generally we don't put interface-specific parameters in the abin input file since ABIN doesn't really need to know about these.
I feel that in this case it would be better (and more general!) to hava a YAML configuration file in the interface folder which we can than load directly into MACE (I believe MACE supports this natively).
|
|
||
| namelist /qmmm/ natqm, natmm, q, LJ_rmin, LJ_eps, mm_types | ||
|
|
||
| namelist /mace/ mace_model, mace_device, mace_default_dtype, & |
There was a problem hiding this comment.
Yeah, I don't really think we should be parsing MACE specific keywords here, they are irrelevant from ABIN's point of view (and note that since MACE excosystem is still evolving a lot, we need to have the flexibility to add new options easily, without having to add them here and recompile ABIN
| end if | ||
| end subroutine finalize_mace | ||
|
|
||
| subroutine wait_for_mace(comm) |
There was a problem hiding this comment.
This code was added to TeraChem interface because a typing TC calculation takes (much) more than a 1 second so we could save a whole CPU by instructing ABIN to sleep a bit while waiting for TC.
In MACE case, the hope is that it is actually very quick, so I don't think this case makes sense. I'd just remove it for now to keep things simple? (happy to hear otherwise though!)
There was a problem hiding this comment.
Thinking about this more, I don't think we should provide the install_mace.sh script, there's simply too many variables about how to handle the python environment (and how to handle Torch installation in particular).
I'd rather have a comprehensive installation section in the README. Given the (unfortunate) fact that the mace package is not published on conda, I think we should encourage users to use the uv package installer, in particular because it has an advanced handling of pytorch.
https://docs.astral.sh/uv/guides/integration/pytorch/
i.e. something like this
uv venv
uv pip install mace-torch mpi4py --torch-backend=auto
To install CPU-only version
uv pip install mace-torch mpi4py --torch-backend=cpu
I am happy to help write this down before we merge this PR.
|
Hi @FelyCZ, just wanted to ask if you'll have time to work on this further in the near future?(No worries if not) I am hoping to do some testing on this branch next week, as we will need this interface soon in our own MACE project. |
|
Hi @danielhollas , I am very sorry but I have so many things with deadlines piling up now. I probably won't be able to work on this at least till the end of May. |
|
@FelyCZ no problem, thanks for letting me know! I obviously don't want to take over your great work, but also since we need to press on with our project, I am wondering what would be best for now, I see couple of options:
Let me know what would you prefer, I am fine either way. Also let me know if there's anybody else in Petr's group that would be interested in working on this (I know Jirka is busy, perhaps somebody else?) |
Add MACE Interface via MPI
Description
This PR adds a new interface for the MACE (Machine Learning Atomic Cluster Expansion) potential to ABIN. MACE is integrated via an MPI-based client-server architecture: a Python server loads and evaluates the MACE ML model, while the Fortran side communicates coordinates, energies, and forces over MPI. This follows the same pattern used for the existing TeraChem (
_tera_) interface.The interface supports configurable model paths, device selection (CPU/CUDA), precision settings, and various MACE-specific options -- all controlled through a new
&macenamelist in the ABIN input file.File Changes
Fortran Core
src/mace_mpi_api.F90mod_mace_mpi). Handles MPI port connection to the Python server, sends atom types/config/coordinates, and defines all MACE namelist variables (model path, device, dtype, batch size, etc.).src/force_mace.F90mod_force_mace). Orchestrates per-bead force evaluation -- sends coordinates viasend_mace, receives energy and forces viareceive_mace. Supports OpenMP.src/forces.F90_mace_case to theforce_wrapperpotential selector, routing toforce_mace().src/init.F90&macenamelist reading and MACE interface/server initialization whenpot='_mace_'.src/mpi_wrapper.F90_mace_to the MPI-required potential check so non-MPI builds raise a clear error.src/Makefilemace_mpi_api.oandforce_mace.oto the build.Python Server
interfaces/MACE/mace_server.pyinterfaces/MACE/README.md&macenamelist parameters. Includes MACE-OFF licensing notice.interfaces/MACE/MACE-OFF23_medium.modelTests
tests/MACE/test.shtests/MACE/mock_mace_server.pymace-torch/PyTorch.tests/MACE/input.inpot='_mace_'and&macenamelist.tests/MACE/mini.xyztests/MACE/*.refenergies.dat.ref,forces.xyz.ref,movie.xyz.ref,velocities.xyz.ref,temper.dat.ref).tests/test.shMACEin the MPI test folder list.unit_tests/test_mace.pfunit_tests/Makefilemacetarget to the unit test build, test runner, and clean targets.Utilities & Docs
utils/run.mace_mpi_abin.shdev_scripts/install_mace.shmace-torch,mpi4py).sample_inputs/input.in.mace&macenamelist configuration.README.md