# Engine Development Tutorial ## Introduction This tutorial will guide you through the process of implementing support for MDI in an existing engine. If you just want a quick tl;dr summary of the minimum steps involved in implementing an MDI interface, see the following: - If your engine is written in a compiled language, build and link your engine against MDI just like any other library. If it is written in Python, just import the MDI Library. - Add a way for end-users of your engine to send runtime options to the MDI Library, preferably through a `-mdi` command-line option. - Call `MDI_Init()` and `MDI_Accept_communicator()` as early as possible in your engine. - Add some server-like code that allows your engine to listen for commands via MDI and respond to them appropriately. - If your engine uses the Message Passing Interface (MPI), be aware that the MDI Library provides a replacement for MPI_COMM_WORLD, and that some MDI functions should only be called by `rank 0`. ## Step 1: Prepare Basic Requirements In order to simplify the process of implementing, testing, and analyzing the capabilities of an MDI engine in a portable environment, this tutorial makes use of several tools. These tools include Git, GitHub, MDI Mechanic, and Docker. Please install each of these tools now: - If you don't already have a GitHub account, create one now. - If you have never used Git, you may wish to work through [a quick tutorial on Git](https://education.molssi.org/python_scripting_cms/09-version-control/index.html), first. - Install MDI Mechanic. This can be done using pip, (*i.e.* `pip install mdimechanic`). - Install Docker and launch Docker Desktop, if applicable. You don't need to create a DockerHub account. You also don't need to know much about using Docker, as MDI Mechanic will handle those details for you. Note that although the above are requirements of this tutorial, none of them are required of end-users running your code. ## Step 2: Initialize an MDI Report Repository In this step of the tutorial, we will create a new GitHub repository to assist in the process of implementing, testing, and maintaining MDI support in your code. This new repository will be separate and independent from any repositories already associated with your code, and will henceforth be referred to as your **report repository**. Create your report repository by making a new repository on GitHub. This repository does **not** need to include the source code of your engine, so you can make this repository publically accessible even if your source code is maintained privately. Don't initialize the repository with a `README` file or a `.gitignore` file. You can select whatever license you prefer; since this repository is separate from the repository that holds your engine's source code, it does not need to be the same license governing your engine's code. Clone the newly created repository onto your local machine: ```Bash git clone git@github.com:/.git cd ``` Now use MDI Mechanic to create the initial structure for this report repository: ```Bash cd mdimechanic startproject --enginereport ``` This will add several new files to your report repository, including one called `mdimechanic.yml`. ## Step 3: Configure the MDI Mechanic YAML File The `mdimechanic.yml` file created in the previous step is used by MDI Mechanic to build your engine and to test and analyze its functionality as an MDI engine. If you have used continuous integration (CI) testing services in the past, you will likely recognize many similarities between `mdimechanic.yml` and the YAML files that are often used by those services. Open `mdimechanic.yml` in your favorite text editor, and you will see that MDI Mechanic pre-populated this file with a basic template. This tutorial will go over each field in `mdimechanic.yml` in detail, but the following is a quick summary: - **code_name:** The name of your code, which is used when printing out information. - **image_name:** MDI Mechanic will create an Docker image, which will contain a highly portable environment that can be used to reproducibly build and run your engine. This field sets the name of the engine MDI Mechanic will create. - **build_image**: This provides a script that is used to build the Docker image that MDI Mechanic builds. It corresponds to the steps required to prepare an environment with all of your engine's dependencies, and is comparable to a `before_install` step in some CI services. - **build_engine**: This provides a script to build your engine. It is executed within the context of the Docker image built by MDI Mechanic, and is comparable to an `install` step in some CI services. - **validate_engine**: This provides a script to verify that your engine has been built successfully. It is comparable to a `script` step in some CI services. - **engine_tests**: This provides scripts used to test MDI functionality in your engine. For now, just replace the value of `code_name` with the name of your engine, and set the value of `image_name` to something appropriate. The naming convention for Docker images is `/`, and we recommend that you follow this convention when setting `image_name`. If in doubt, you can set `image_name` to `/mdi_report`. ## Step 4: Define Your Engine's Build Environment Using MDI Mechanic This tutorial uses MDI Mechanic, which in turn runs your code within the context of a Docker image. In crude terms, you can think of an image as being a simulated duplicate of another computer, which has a different environment from yours (*i.e.* different installed libraries and system settings), and might be running an entirely different operating system. The image created by MDI Mechanic is based on the Ubunto Linux distribution. Starting from the basic Linux environment, MDI Mechanic installs some basic compilers (gcc, g++, and gfortran), an MPI library (MPICH), Python 3, and a handful of other dependencies (make and openssh). To finish building the image, MDI Mechanic executes whatever script you've provided in the `build_image` section of `mdimechanic.yml`. You should now fill out `build_image` with an appropriate script that installs any dependencies necessary to compile your engine (if your engine is written in a compiled language) or to run your engine (if your engine is written in an interpreted language). To do this, imagine that someone handed you a Linux computer that is completely new and unused, except that the compilers and libraries mentioned in the preceeding paragraph have been installed on it. What would you need to do in order to install all the dependencies for your code? The answer to this question corresponds to the script you need to provide to `build_image`. ## Step 5: Build Your Engine Using MDI Mechanic After you've finished with the `build_image` script, it is time to write the `build_engine` script. This script will be executed within the context of the image you described in the `build_image` script, so it will have access to any dependencies you installed in that script (and *only* those dependencies). To write the `build_engine` script, ask yourself "What would someone need to type into their terminal to acquire a copy of my code's source and compile it?"; the answer to this question corresponds to the script you need to provide to `build_engine`. Here are a few important details to keep in mind as you write the `build_engine` script: - The initial working directory for the `build_engine` script is the top-level directory of your report repository. - The `build_engine` script can access and manipulate any files within your report repository, including creating new files and subdirectories. It does not have access to any other files or directories on you filesystem (for Docker afficianados: the report repository's top-level directory is mounted within the image to `/repo`). - It is recommended that your `build_engine` script should download your engine repository's source code to a `source` subdirectory within your report repository. - It is recommended that your `build_engine` script should build/install your engine repository's source code to a `build` subdirectory within your report repository. - If your engine is **not** open-source, it may not be possible to simply download the source code via a command like `git clone`. In this case, you should write the `build_engine` script with the assumption that your engine's source code has been manually copied by the end-user into a `source` subdirectory within the report repository's top-level directory. Uponing cloning the report repository, it will be the responsibility of the user to copy your engine's source code into the correct location, assuming they have access to it. Note that you **absolutely should not** include any private information (*i.e.* software keys, private ssh keys, private source code, *etc.*) in `mdimechanic.yml` or any other file that is commited to your report repository. The `build` and `source` directories are included in the `.gitignore` file of the report; this prevents source code that is temporarily stored in those locations from being accidentally committed, unless `.gitignore` is overridden. Override `.gitignore` at your peril, and always be aware of anything you are committing to the repository. At this point, you can execute `mdimechanic build` in the top-level directory of your report repository. If this command executes successfully, great! If not, work to correct any problems with the build process before continuing to the next step. ## Step 6: Validate the Engine Build At this point, modify the `validate_engine` field in `mdimechanic.yml` so that it performs a simple test to confirm that the engine was actually built. The script should return a non-zero exit code upon failure. If your code is written in a compiled language, this can be as simple as a check to confirm the existence of the executable file: \code{.yml} validate_engine: - ENGINE_EXECUTABLE_PATH="build/" - | if test -f "$ENGINE_EXECUTABLE_PATH"; then echo "$ENGINE_EXECUTABLE_PATH exists" else echo "Could not find engine executable: $ENGINE_EXECUTABLE_PATH" exit 1 fi \endcode If your code is written in Python, you might instead confim that your code can be imported (*i.e.* `python -c "import "`). After providing a `validate_engine` script, run `mdimechanic report` in the top-level directory of your MPI-report repository. This will perform a series of tests to confirm whether your engine supports MDI correctly. The first of these tests simply runs the `validate_engine` script. Since we haven't even started implementing MDI functionality in your engine yet, it is expected that MDI Mechanic will report errors shortly after starting. After `mdimechanic report` stops (most likely throwing an error), you should find that there is a new `README.md` file in your MDI-report repository. This file contains the full report from MDI Mechanic. To properly view the file, you can either commit the file and push it to GitHub, where it can be viewed at your MDI-report repository's GitHub page, or you can install an offline markdown viewer (such as `grip`) to view it. There isn't much to see now, but hopefully you can see that there is a green `working` badge next to the step labeled "The engine builds successfully". If not, review the error messages from `mdimechanic report` to try to work out what when wrong, before moving on to the next step. ## Step 7: Provide an Example Input When you run `mdimechanic report`, MDI Mechanic tries to run a series of tests to determine whether and to what extent your code supports MDI. To do this, MDI Mechanic attempts to launch your code, establish a connection between it and numerous test drivers, and then report the results. At this point in the tutorial, MDI Mechanic has no information about how to launch your code, so it can't run these tests. We will now supply MDI Mechanic with everything it needs to run a calculation using your code. In `mdimechanic.yml` you will find an `engine_tests` field. This field can contain a list of scripts, each of which is intended to launch a single calculation with your code. For now, we only want to supply a single test script. The relevant part of `mdimechanic.yml` reads: ```yaml engine_tests: # Provide at least one example input that can be used to test your code's MDI functionality - script: - echo "Insert commands to run an example calculation here" - exit 1 ``` Replace the script in the `script` field here so that, when executed, it will launch a calculation using your code. This likely means that you will need to add one or more input files to your MDI-report repository, which we recommend placing in a `tests` subdirectory. Your `mdimechanic.yml` might end up looking something like this: ```yaml engine_tests: - script: - cd tests/test1 - ../../${ENGINE_EXECUTABLE_PATH} -in test.inp ``` The exact nature of the test calculation doesn't matter very much. It should be a short calculation, since it will be run many times. The calculation might involve a simulation of a Lennard-Jones fluid, evaluation of the single-point energy of a water molecule, or some other small computatation. The most important thing about the test script is that **it must return a non-zero exit value if your engine exits due to an error**. ## Step 8: Make the MDI Library Available to the Engine In this step, we will ensure that MDI functions can be called from your engine. ::::{tab-set} :sync-group: category :::{tab-item} C++ :sync: key1 Your engine must be compiled and linked against the MDI Library. The MDI Library is released under a highly permissive BSD-3 License, and developers of MDI-enabled codes are encouraged to copy the MDI Library directly into distributions of their software. If your code uses Git for version control, you can include the MDI Library in your engine's source code repository as either a subtree (recommended) or a submodule. To incorporate a distribution of the MDI Library into your engine as a subtree, you can execute the following command in the top directory of the *engine's* Git repository (**not** in the top directory of the *report* repository): ```bash git subtree add --prefix=lib/mdi https://github.com/MolSSI-MDI/MDI_Library master --squash ``` The argument to the `--prefix` option indicates the location where the MDI Library source code will reside, and can be changed to better fit your engine's directory structure. You must then modify your engine's build process to build the MDI Library and link against it. The MDI Library builds using CMake. If your engine also builds using CMake, you can simply include the MDI Library as a CMake subpackage. Otherwise, you can add a few lines to your engine's existing build scripts to execute CMake and build the MDI Library. The following lines illustrate how the MDI Library could be built, assuming that the source code for the MDI Library is located in `lib/mdi`: ```bash mkdir -p lib/mdi/build cd lib/mdi/build cmake -Dlibtype=STATIC -Dlanguage=C -DCMAKE_INSTALL_PREFIX=../install .. make make install ``` The following CMake configuration options are likely to be useful: - **-Dlibtype**: Set this to `STATIC`. - **-Dlanguage**: Set this to the language of the code you intend to link to the MDI library. Valid options are `C`, `CXX` (for C++), `Fortran`, and `Python`. - **-DCMAKE_C_COMPILER**: Set this to the C compiler used to build your engine (if applicable). - **-DCMAKE_Fortran_COMPILER**: Set this to the Fortran compiler used to build your engine (if applicable). - **-DCMAKE_INSTALL_PREFIX**: Set this to the destination directory for the installation. Finally, during the link stage of your build process, you will need to ensure that your code is linked against the MDI Library. In the case of the above example build process, the compiled static library file will be located at `${CMAKE_INSTALL_PREFIX}/lib/mdi`, and will typically be called `libmdi.a` on POSIX systems. ::: :::{tab-item} Fortran :sync: key2 Your engine must be compiled and linked against the MDI Library. The MDI Library is released under a highly permissive BSD-3 License, and developers of MDI-enabled codes are encouraged to copy the MDI Library directly into distributions of their software. If your code uses Git for version control, you can include the MDI Library in your engine's source code repository as either a subtree (recommended) or a submodule. To incorporate a distribution of the MDI Library into your engine as a subtree, you can execute the following command in the top directory of the *engine's* Git repository (**not** in the top directory of the *report* repository): ```bash git subtree add --prefix=lib/mdi https://github.com/MolSSI-MDI/MDI_Library master --squash ``` The argument to the `--prefix` option indicates the location where the MDI Library source code will reside, and can be changed to better fit your engine's directory structure. You must then modify your engine's build process to build the MDI Library and link against it. The MDI Library builds using CMake. If your engine also builds using CMake, you can simply include the MDI Library as a CMake subpackage. Otherwise, you can add a few lines to your engine's existing build scripts to execute CMake and build the MDI Library. The following lines illustrate how the MDI Library could be built, assuming that the source code for the MDI Library is located in `lib/mdi`: ```bash mkdir -p lib/mdi/build cd lib/mdi/build cmake -Dlibtype=STATIC -Dlanguage=Fortran -DCMAKE_INSTALL_PREFIX=../install .. make make install ``` The following CMake configuration options are likely to be useful: - **-Dlibtype**: Set this to `STATIC`. - **-Dlanguage**: Set this to the language of the code you intend to link to the MDI library. Valid options are `C`, `CXX` (for C++), `Fortran`, and `Python`. - **-DCMAKE_C_COMPILER**: Set this to the C compiler used to build your engine (if applicable). - **-DCMAKE_Fortran_COMPILER**: Set this to the Fortran compiler used to build your engine (if applicable). - **-DCMAKE_INSTALL_PREFIX**: Set this to the destination directory for the installation. Finally, during the link stage of your build process, you will need to ensure that your code is linked against the MDI Library. In the case of the above example build process, the compiled static library file will be located at `${CMAKE_INSTALL_PREFIX}/lib/mdi`, and will typically be called `libmdi.a` on POSIX systems. ::: :::{tab-item} Python :sync: key3 First, install the MDI Library during the `build_image` step in `mdimechanic.yml`. This can be done trivially using `pip` (*e.g.* `pip install pymdi`). Your engine can then import the MDI Library where needed (*e.g.* `import mdi`). ::: :::: ## Step 9: Support User Input of the MDI Options Your code should allow users to set certain MDI parameters at runtime. Typically, end-users should be able to set these parameters through the use of a `-mdi` command-line option when your engine is launched. In this case, the user should be able to launch your code by doing something along the lines of: ```bash engine_exectable --mdi "-name engine -role ENGINE -method TCP -hostname localhost -port 8021" ``` The details of how you read this command-line option are beyond the scope of this tutorial, but should conform to whatever existing method you use to read command-line options. The argument to the `--mdi` command-line option should be represented as a `char*` in C++, a `CHARACTER` array in Fortran, and a `String` in Python. Subsequent steps in this tutorial will assume that you have named the corresponding variable `mdi_options`. We understand that some codes prefer to eschew command-line options where possible. If it is preferable not to introduce support for a `--mdi` command-line option, ensure that there is some other mechanism for the user to provide the MDI parameters at runtime. Regardless of how you decide to handle user input of the MDI options, you'll need to modify your `engine_tests` script to ensure that MDI Mechanic can pass this information to your code at runtime. When running tests, MDI Mechanic handles selection of MDI options by setting an `MDI_OPTIONS` environment variable. If you decide to use a command-line `--mdi` option to allow user input of the MDI options, you'll need to pass this environment variable as the argument to your `--mdi` option. For example: ```yaml engine_tests: - script: - cd tests/test1 - ../../${ENGINE_EXECUTABLE_PATH} --mdi ${MDI_OPTIONS} -in test.inp ``` ## Step 10: Initialize the MDI Library Your code must initialize the MDI Library by calling the `MDI_Init()` function. This is a straightforward process, but there are a couple of important details to keep in mind if you are using both MDI and the Message Passing Interface (MPI): - If your code uses MPI, you should call `MDI_Init()` after the call to `MPI_Init()` (or `MPI_Init_thread()`, if applicable). Aside from the restriction, `MDI_Init()` should be called as early in your code as possible. It is a best practice to call `MDI_Init()` immediately after calling `MPI_Init()`. Calling MPI functions (other than `MPI_Init()` or `MPI_Init_thread()`) before calling `MDI_Init()` can lead to bugs. - Immediately following the call to `MDI_Init()`, the `MDI_MPI_get_world_comm()` function should be called. This function accepts a pointer to an MPI communicator as its only argument. Upon return, this pointer will point to an MPI intra-communicator that spans all ranks associated with your engine. This intra-communicator should be used whenever you would otherwise use `MPI_COMM_WORLD`. You should **never** perform MPI operations involving `MPI_COMM_WORLD` in an MDI-enabled code, as `MPI_COMM_WORLD` will in certain contexts span ranks that are not associated with your engine, but which are instead associated with the driver or other engines. The following code snippets provide a guide to correctly initializing MDI and MPI together in C++, Fortran, and Python. ::::{tab-set} :sync-group: category :::{tab-item} C++ :sync: key1 ```c++ #include #include "mdi.h" /* User-selected options for the MDI Library This should be obtained at runtime from a "-mdi" command-line option */ char *mdi_options; /* MPI intra-communicator for all processes running this code It should be set to MPI_COMM_WORLD prior to the call to MDI_Init(), as shown below Afterwards, you should ALWAYS use this variable instead of MPI_COMM_WORLD */ MPI_Comm world_comm; /* Pointer to world_comm */ MPI_Comm *world_comm_ptr; /* MDI communicator used to communicate with the driver */ MDI_Comm mdi_comm = MDI_COMM_NULL; /* Rank of this process in the MDI-created intra-communicator */ int myrank = 0; /* Function to initialize both MPI and MDI */ initialize(int argc, char** argv) { /* If using MPI, it should be initialized before MDI */ MPI_Init(&argc, &argv); /* MDI should be initialized immediately after MPI */ MDI_Init(&argc, &argv); MDI_MPI_get_world_comm(world_comm_ptr); /* Following this point, *world_comm_ptr should be used whenever you would otherwise have used MPI_COMM_WORLD */ /* Get the rank of this process, within the MDI-created intra-communicator */ MPI_Comm_rank(*world_comm_ptr, my_rank); /* Accept a connection from an external driver */ if ( my_rank == 0 ) { MDI_Accept_communicator(&mdi_comm); } } ``` After implementing the call to `MDI_Init()`, you should recompile the code to confirm that your executable is linked to the MPI Library. ::: :::{tab-item} Fortran :sync: key2 ```f90 SUBROUTINE initialize ( mdi_options, world_comm, my_rank, mdi_comm ) USE mpi, ONLY : MPI_COMM_WORLD USE mdi, ONLY : MDI_Init, MDI_COMM_NULL ! ! User-selected options for the MDI Library ! This should be obtained at runtime from a "-mdi" command-line option ! CHARACTER(len=1024), INTENT(IN), OPTIONAL :: mdi_options ! ! MPI intra-communicator for all processes running this code ! It should be set to MPI_COMM_WORLD prior to the call to MDI_Init(), as shown below ! Afterwards, you should ALWAYS use this variable instead of MPI_COMM_WORLD ! INTEGER, INTENT(INOUT) :: world_comm ! ! Rank of this process within the MDI-created intra-communicator ! INTEGER, INTENT(OUT) :: my_rank ! ! MDI communicator, obtained from MDI_Accept_communicator ! INTEGER, INTENT(OUT) :: mdi_comm ! ! Error flag used in MDI calls ! INTEGER :: ierr ! ! If using MPI, it should be initialized before MDI ! CALL MPI_Init(ierr) ! ! MDI should be initialized immediately after MPI ! IF ( PRESENT(mdi_options) ) THEN CALL MDI_Init(mdi_options, ierr) CALL MDI_MPI_get_world_comm(world_comm) END IF ! ! Following this point, world_comm should be used whenever you would otherwise have used MPI_COMM_WORLD ! ! ! Get the rank of this process, within the MDI-created intra-communicator ! CALL MPI_Comm_rank(world_comm, my_rank, ierr) ! ! Accept a connection from an external driver ! IF ( my_rank .eq. 0 ) THEN CALL MDI_Accept_communicator( mdi_comm, ierr ) END IF END SUBROUTINE initialize ``` After implementing the call to `MDI_Init()`, you should recompile the code to confirm that your executable is linked to the MPI Library. ::: :::{tab-item} Python :sync: key3 ```python # Import the MDI Library import mdi # Attempt to import mpi4py try: from mpi4py import MPI use_mpi4py = True except ImportError: use_mpi4py = False # Get the command-line options for MDI ... # Initialize MDI mdi.MDI_Init(mdi_options) world_comm = mdi.MDI_MPI_get_world_comm() # Get the MPI rank of this process if world_comm is not None: my_rank = world_comm.Get_rank() else: my_rank = 0 # Accept a connection from an external driver if my_rank == 0: mdi_comm = MDI_Accept_communicator() ``` ::: :::: ## Step 11: Support Basic MDI Communication In this step, we are going to introduce some basic code that will finally allow external drivers to connect to your code and ask it to do useful things for them. First, identify a point in your code when it would be appropriate for the code to accept instructions (in the form of MDI commands) from an external driver. The chosen point should occur after your code has completed basic initialization operations (reading input files, doing basic system setup, calling `MDI_Init()`, *etc.*). It should also be practical to implement support for a reasonable number of MDI commands at whatever point you select. The [MDI Standard](https://molssi-mdi.github.io/MDI_Library/html/mdi_standard.html) defines numerous commands that driver developers might want to send to your code. You won't need to support all of the available commands, but it is advisable to support some of the more common commands, such as commands that request or change the nuclear coordinates (`COORDS`, respectively), as well as commands that request the energy (` #include "mdi.h" run_mdi(char *node_name, int my_rank, MPI_Comm world_comm, MDI_Comm mdi_comm) { /* Exit flag for the main MDI loop */ bool exit_flag = false; /* MDI command from the driver */ command = new char[MDI_COMMAND_LENGTH]; /* Main MDI loop */ while (not exit_flag) { /* Receive a command from the driver */ if ( my_rank == 0 ) { MDI_Recv_command(command, mdi_comm); } MPI_Bcast(command, MDI_COMMAND_LENGTH, MPI_CHAR, 0, world_comm); /* Confirm that this command is actually supported at this node */ int command_supported = 0; MDI_Check_command_exists(node_name, command, MDI_COMM_NULL, &command_supported); if ( command_supported != 1 ) { /* Note: Replace this with whatever error handling method your code uses */ MPI_Abort(world_comm, 1); } /* Respond to the received command */ if ( strcmp(command, "EXIT") == 0 ) { exit_flag = true; } else { /* The received command is not recognized by this engine, so exit Note: Replace this with whatever error handling method your code uses */ MPI_Abort(world_comm, 1); } // Free any memory allocations delete [] command; } ``` ::: :::{tab-item} Fortran :sync: key2 Call this subroutine as `CALL run_mdi("@DEFAULT", my_rank, world_comm, mdi_comm)`. ```f90 SUBROUTINE run_mdi( node_name, my_rank, world_comm, mdi_comm ) USE mdi, ONLY : MDI_Send, MDI_Recv, MDI_Recv_Command, & MDI_Accept_Communicator, & MDI_CHAR, MDI_DOUBLE, MDI_INT, & MDI_COMMAND_LENGTH, MDI_NAME_LENGTH, & MDI_COMM_NULL ! ! MDI command from the driver ! CHARACTER :: node_name(MDI_NAME_LENGTH) ! ! Rank of this process in world_comm ! If you are not using MPI, you can set my_rank = 0 ! INTEGER, INTENT(IN) :: my_rank ! ! MDI-created intra-communicator ! INTEGER, INTENT(IN) :: world_comm ! ! MDI communicator, obtained from MDI_Accept_communicator ! INTEGER, INTENT(IN) :: mdi_comm ! ! MDI command from the driver ! CHARACTER, ALLOCATABLE :: command(:) ! ! Error flag for MDI functions ! INTEGER :: ierr ! ! Flag to indicate whether a received command is supported ! INTEGER :: command_supported ! ! Allocate the command array ! ALLOCATE( command(MDI_COMMAND_LENGTH) ) ! ! Main MDI loop ! mdi_loop: DO ! ! Receive a command from the driver ! IF ( my_rank .eq. 0 ) THEN CALL MDI_Recv_command( command, mdi_comm, ierr ) WRITE(*,*) "MDI Engine received a command: ",trim(command) END IF ! ! Broadcast the command to all ranks ! Note: Remove this line if not using MPI ! CALL MPI_Bcast( header, MDI_COMMAND_LENGTH, MPI_CHAR, 0, world_comm ) ! ! Confirm that this command is actually supported at this node ! command_supported = 0; CALL MDI_Check_command_exists(node_name, command, MDI_COMM_NULL, command_supported, ierr); IF ( command_supported .ne. 1 ) THEN ! Note: Replace this with whatever error handling method your code uses CALL MPI_Abort(world_comm, 1); END IF ! ! Respond to the received command ! SELECT CASE ( trim( command ) ) CASE( "EXIT" ) RETURN CASE DEFAULT ! ! The received command is not recognized by this engine, so exit ! Note: Replace this with whatever error handling method your code uses ! WRITE(*,*) "MDI Engine received unrecognized command: ",trim(command) CALL MPI_Abort(world_comm, 1); END SELECT END DO mdi_loop END SUBROUTINE run_mdi ``` ::: :::{tab-item} Python :sync: key3 ```python import mdi def run_mdi(node_name, my_rank, world_comm, mdi_comm): exit_flag = False # Main MDI loop while not exit_flag: # Receive a command from the driver if self.my_rank == 0: command = mdi.MDI_Recv_command(self.comm) else: command = None # Broadcast the command to all ranks, if using MPI if world_comm is not None: command = world_comm.bcast(command, root=0) # Confirm that this command is actually supported at this node if not mdi.MDI_Check_node_exists(node_name, command): raise Exception('MDI Engine received unsupported command: ' + str(command)) # Respond to the received command if command == "EXIT": exit_flag = True else: # The received command is not recognized by this engine, so exit # Note: Replace this with whatever error handling method your code uses raise Exception('MDI Engine received unrecognized command: ' + str(command)) ``` ::: :::: ## Step 12: Register the Node and Commands MDI requires you to "register" a list of all nodes and commands your engine supports. This allows you, as an engine developer, to inform any drivers of what your code can do. MDI provides two functions that allow you to do this: `MDI_Register_node()` and `MDI_Register_command()`. The engine we have developed thus far in the tutorial only supports a single node, and that node only supports a single command. As a result, we need to call `MDI_Register_node()` once to register the `@DEFAULT` node and call `MDI_Register_command()` once to register the `EXIT` command. Fortunately, this is a very simple process. The only argument to the `MDI_Register_node()` function is the name of the node, `@DEFAULT`. The `MDI_Register_command()` function accepts two arguments: the name of the node, and the name of the command that we are registering at that node, `EXIT`. In time, you may add support for additional commands at the default node, making additional calls to `MDI_Register_command()` for each newly supported command. If you add additional nodes, each node will have its own list of registered commands, which may be different from the list of commands supported at the `@DEFAULT` node. For now, place the following code immediately after your engine's call to `MDI_Init()`. It is important that it be called prior to the call to `MDI_Accept_communicator()`. ::::{tab-set} :sync-group: category :::{tab-item} C++ :sync: key1 ```c++ /* Register all supported commands and nodes */ MDI_Register_node("@DEFAULT"); MDI_Register_command("@DEFAULT", "EXIT"); ``` ::: :::{tab-item} Fortran :sync: key2 ```fortran ! Register all supported commands and nodes CALL MDI_Register_node("@DEFAULT", ierr); CALL MDI_Register_command("@DEFAULT", "EXIT", ierr); ``` ::: :::{tab-item} Python :sync: key3 ```python # Register all supported commands and nodes mdi.MDI_Register_node("@DEFAULT") mdi.MDI_Register_command("@DEFAULT", "EXIT") ``` ::: :::: ## Step 13: Add Support for Additional Commands It may not look like much yet, but you have now established a basic MDI interface! If you generate a new report by typing `mdimechanic report` at the command line, MDI Mechanic should confirm that your engine passes all of the "Basic Functionality Tests". If your engine is still failing that test, you should return to the previous steps of this tutorial to determine what went wrong. Assuming that your MDI interface is functioning as expected, you can now begin the process of implementing support for more commands. Whenever you want to add support for a new command, you will need to do the following: -# Add code to the "Main MDI loop" in `run_mdi()` that will respond appropriately to the new command. -# Add a call to `MDI_Register_command()` to register support for that command. Examine the commands specified by the [MDI Standard](https://molssi-mdi.github.io/MDI_Library/html/mdi_standard.html), and implement support for the ones that seem relevant for your engine. Each command in the MDI Standard describes exactly how the engine is expected to respond. Often, the engine is expected to either send or receive information to/from the driver. This is accomplished using the `MDI_Send()` and `MDI_Recv()` functions, respectively. If you are using MPI, you should be aware that all MDI-based communication must take place through `rank 0`. Only `rank 0` should call `MPI_Send()`, `MPI_Recv()`, and `MPI_Recv_Command()`. Depending on how you have distributed data structures across ranks, you may need to do `MPI_Gather()` or similar operations to collect the data onto `rank 0` before calling `MDI_Send()`. Likewise, you may need to do `MPI_Scatter()` or similar operations to correctly distribute data after calling `MDI_Recv()`. ## Step 14: Add Support for Additional Nodes Whenever you add a new node, you must also add a call to `MDI_Register_node()` to register support for that node. Commands are registered separately for each node, so any commands that are supported at the new node must be registered for it. For example, if you implement five nodes, and each of them supports the `EXIT` command, you will need to call the `MDI_Register_command()` function five times, each time with a different node as the first argument and with `EXIT` as the second argument.