 |
Molssi Driver Interface Library
|
|
Molssi Driver Interface Library
|
This tutorial will guide you through the process of implementing a support for MDI in an existing engine.
If you just want a quick tl;dr summary of the minimum steps involved in implementing an MDI interface, see the following:
-mdi command-line option.MDI_Init() and MDI_Accept_communicator() as early as possible in your engine.rank 0.
In order to simplify the process of implementing, testing, and analyzing the capabilities of an MDI engine in a portable environment, this tutorial makes use of several tools. These tools include Git, GitHub, MDI Mechanic, and Docker. Please install each of these tools now:
pip install mdimechanic).Note that although the above are requirements of this tutorial, none of them are required of end-users running your code.
In this step of the tutorial, we will create a new GitHub repository to assist in the process of implementing, testing, and maintaining MDI support in your code. This new repository will be separate and independent from any repositories already associated with your code, and will henceforth be referred to as your report repository.
Create your report repository by making a new repository on GitHub. This repository does not need to include the source code of your engine, so you can make this repository publically accessible even if your source code is maintained privately. Don't initialize the repository with a README file or a .gitignore file. You can select whatever license you prefer; since this repository is separate from the repository that holds your engine's source code, it does not need to be the same license governing your engine's code.
Clone the newly created repository onto your local machine:
Now use MDI Mechanic to create the initial structure for this report repository:
This will add several new files to your report repository, including one called mdimechanic.yml.
The mdimechanic.yml file created in the previous step is used by MDI Mechanic to build your engine and to test and analyze its functionality as an MDI engine. If you have used continuous integration (CI) testing services in the past, you will likely recognize many similarities between mdimechanic.yml and the YAML files that are often used by those services. Open mdimechanic.yml in your favorite text editor, and you will see that MDI Mechanic pre-populated this file with a basic template.
This tutorial will go over each field in mdimechanic.yml in detail, but the following is a quick summary:
before_install step in some CI services.install step in some CI services.script step in some CI services.For now, just replace the value of code_name with the name of your engine, and set the value of image_name to something appropriate. The naming convention for Docker images is <organization_name>/<image_name>, and we recommend that you follow this convention when setting image_name. If in doubt, you can set image_name to <engine_name>/mdi_report.
This tutorial uses MDI Mechanic, which in turn runs your code within the context of a Docker image. In crude terms, you can think of an image as being a simulated duplicate of another computer, which has a different environment from yours (i.e. different installed libraries and system settings), and might be running an entirely different operating system. The image created by MDI Mechanic is based on the Ubunto Linux distribution. Starting from the basic Linux environment, MDI Mechanic installs some basic compilers (gcc, g++, and gfortran), an MPI library (MPICH), Python 3, and a handful of other dependencies (make and openssh). To finish building the image, MDI Mechanic executes whatever script you've provided in the build_image section of mdimechanic.yml.
You should now fill out build_image with an appropriate script that installs any dependencies necessary to compile your engine (if your engine is written in a compiled language) or to run your engine (if your engine is written in an interpreted language). To do this, imagine that someone handed you a Linux computer that is completely new and unused, except that the compilers and libraries mentioned in the preceeding paragraph have been installed on it. What would you need to do in order to install all the dependencies for your code? The answer to this question corresponds to the script you need to provide to build_image.
After you've finished with the build_image script, it is time to write the build_engine script. This script will be executed within the context of the image you described in the build_image script, so it will have access to any dependencies you installed in that script (and only those dependencies). To write the build_engine script, ask yourself "What would someone need to type into their terminal to acquire a copy of my code's source and compile it?"; the answer to this question corresponds to the script you need to provide to build_engine. Here are a few important details to keep in mind as you write the build_engine script:
build_engine script is the top-level directory of your report repository.build_engine script can access and manipulate any files within your report repository, including creating new files and subdirectories. It does not have access to any other files or directories on you filesystem (for Docker afficianados: the report repository's top-level directory is mounted within the image to /repo).build_engine script should download your engine repository's source code to a source subdirectory within your report repository.build_engine script should build/install your engine repository's source code to a build subdirectory within your report repository.git clone. In this case, you should write the build_engine script with the assumption that your engine's source code has been manually copied by the end-user into a source subdirectory within the report repository's top-level directory. Uponing cloning the report repository, it will be the responsibility of the user to copy your engine's source code into the correct location, assuming they have access to it. Note that you absolutely should not include any private information (i.e. software keys, private ssh keys, private source code, etc.) in mdimechanic.yml or any other file that is commited to your report repository. The build and source directories are included in the .gitignore file of the report; this prevents source code that is temporarily stored in those locations from being accidentally committed, unless .gitignore is overridden. Override .gitignore at your peril, and always be aware of anything you are committing to the repository.At this point, you can execute mdimechanic build in the top-level directory of your report repository. If this command executes successfully, great! If not, work to correct any problems with the build process before continuing to the next step.
At this point, modify the validate_engine field in mdimechanic.yml so that it performs a simple test to confirm that the engine was actually built. The script should return a non-zero exit code upon failure. If your code is written in a compiled language, this can be as simple as a check to confirm the existence of the executable file:
If your code is written in Python, you might instead confim that your code can be imported (i.e. python -c "import <engine_name>").
After providing a validate_engine script, run mdimechanic report in the top-level directory of your MPI-report repository. This will perform a series of tests to confirm whether your engine supports MDI correctly. The first of these tests simply runs the validate_engine script. Since we haven't even started implementing MDI functionality in your engine yet, it is expected that MDI Mechanic will report errors shortly after starting. After mdimechanic report stops (most likely throwing an error), you should find that there is a new README.md file in your MDI-report repository. This file contains the full report from MDI Mechanic. To properly view the file, you can either commit the file and push it to GitHub, where it can be viewed at your MDI-report repository's GitHub page, or you can install an offline markdown viewer (such as grip) to view it. There isn't much to see now, but hopefully you can see that there is a green working badge next to the step labeled "The engine builds successfully". If not, review the error messages from mdimechanic report to try to work out what when wrong, before moving on to the next step.
When you run mdimechanic report, MDI Mechanic tries to run a series of tests to determine whether and to what extent your code supports MDI. To do this, MDI Mechanic attempts to launch your code, establish a connection between it and numerous test drivers, and then report the results. At this point in the tutorial, MDI Mechanic has no information about how to launch your code, so it can't run these tests.
We will now supply MDI Mechanic with everything it needs to run a calculation using your code. In mdimechanic.yml you will find an engine_tests field. This field can contain a list of scripts, each of which is intended to launch a single calculation with your code. For now, we only want to supply a single test script. The relevant part of mdimechanic.yml reads:
Replace the script in the script field here so that, when executed, it will launch a calculation using your code. This likely means that you will need to add one or more input files to your MDI-report repository, which we recommend placing in a tests subdirectory. Your mdimechanic.yml might end up looking something like this:
The exact nature of the test calculation doesn't matter very much. It should be a short calculation, since it will be run many times. The calculation might involve a simulation of a Lennard-Jones fluid, evaluation of the single-point energy of a water molecule, or some other small computatation. The most important thing about the test script is that it must return a non-zero exit value if your engine exits due to an error.
In this step, we will ensure that MDI functions can be called from your engine.
The argument to the --prefix option indicates the location where the MDI Library source code will reside, and can be changed to better fit your engine's directory structure.
You must then modify your engine's build process to build the MDI Library and link against it. The MDI Library builds using CMake. If your engine also builds using CMake, you can simply include the MDI Library as a CMake subpackage. Otherwise, you can add a few lines to your engine's existing build scripts to execute CMake and build the MDI Library. The following lines illustrate how the MDI Library could be built, assuming that the source code for the MDI Library is located in lib/mdi:
The following CMake configuration options are likely to be useful:
STATIC.C, CXX (for C++), Fortran, and Python.Finally, during the link stage of your build process, you will need to ensure that your code is linked against the MDI Library. In the case of the above example build process, the compiled static library file will be located at ${CMAKE_INSTALL_PREFIX}/lib/mdi, and will typically be called libmdi.a on POSIX systems.
The argument to the --prefix option indicates the location where the MDI Library source code will reside, and can be changed to better fit your engine's directory structure.
You must then modify your engine's build process to build the MDI Library and link against it. The MDI Library builds using CMake. If your engine also builds using CMake, you can simply include the MDI Library as a CMake subpackage. Otherwise, you can add a few lines to your engine's existing build scripts to execute CMake and build the MDI Library. The following lines illustrate how the MDI Library could be built, assuming that the source code for the MDI Library is located in lib/mdi:
The following CMake configuration options are likely to be useful:
STATIC.C, CXX (for C++), Fortran, and Python.Finally, during the link stage of your build process, you will need to ensure that your code is linked against the MDI Library. In the case of the above example build process, the compiled static library file will be located at ${CMAKE_INSTALL_PREFIX}/lib/mdi, and will typically be called libmdi.a on POSIX systems.
build_image step in mdimechanic.yml. This can be done trivially using pip (e.g. pip install pymdi). Your engine can then import the MDI Library where needed (e.g. import mdi).
Your code should allow users to set certain MDI parameters at runtime. Typically, end-users should be able to set these parameters through the use of a -mdi command-line option when your engine is launched. In this case, the user should be able to launch your code by doing something along the lines of:
The details of how you read this command-line option are beyond the scope of this tutorial, but should conform to whatever existing method you use to read command-line options. The argument to the -mdi command-line option should be represented as a char* in C++, a CHARACTER array in Fortran, and a String in Python. Subsequent steps in this tutorial will assume that you have named the corresponding variable mdi_options.
We understand that some codes prefer to eschew command-line options where possible. If it is preferable not to introduce support for a -mdi command-line option, ensure that there is some other mechanism for the user to provide the MDI parameters at runtime.
Your code must initialize the MDI Library by calling the MDI_Init() function. This is a straightforward process, but there are a couple of important details to keep in mind if you are using both MDI and the Message Passing Interface (MPI):
MDI_Init() after the call to MPI_Init() (or MPI_Init_thread(), if applicable). Aside from the restriction, MDI_Init() should be called as early in your code as possible. It is a best practice to call MDI_Init() immediately after calling MPI_Init(). Calling MPI functions (other than MPI_Init() or MPI_Init_thread()) before calling MDI_Init() can lead to bugs.MDI_Init(), the MDI_MPI_get_world_comm() function should be called. This function accepts a pointer to an MPI communicator as its only argument. Upon return, this pointer will point to an MPI intra-communicator that spans all ranks associated with your engine. This intra-communicator should be used whenever you would otherwise use MPI_COMM_WORLD. You should never perform MPI operations involving MPI_COMM_WORLD in an MDI-enabled code, as MPI_COMM_WORLD will in certain contexts span ranks that are not associated with your engine, but which are instead associated with the driver or other engines.The following code snippets provide a guide to correctly initializing MDI and MPI together in C++, Fortran, and Python.
After implementing the call to MDI_Init(), you should recompile the code to confirm that your executable is linked to the MPI Library.
After implementing the call to MDI_Init(), you should recompile the code to confirm that your executable is linked to the MPI Library.
In this step, we are going to introduce some basic code that will finally allow external drivers to connect to your code and ask it to do useful things for them. First, identify a point in your code when it would be appropriate for the code to accept instructions (in the form of MDI commands) from an external driver. The chosen point should occur after your code has completed basic initialization operations (reading input files, doing basic system setup, calling MDI_Init(), etc.). It should also be practical to implement support for a reasonable number of MDI commands at whatever point you select. The MDI Standard defines numerous commands that driver developers might want to send to your code. You won't need to support all of the available commands, but it is advisable to support some of the more common commands, such as commands that request or change the nuclear coordinates (<COORDS and >COORDS, respectively), as well as commands that request the energy (<ENERGY) number of atoms (<NATOMS), or (<FORCES). Try to select a point where it will be possible to fulfill some of these requests. When in doubt, select a point that is reached early in your code's execution. This tutorial will subsequently refer to the point you have selected as the MDI node.
At the MDI node, you will need to insert some code (probably in the form of a called function) that handles the process of establishing communication with the external driver, accepting MDI commands from the driver, and responding to the commands appropriately. For the purpose of this tutorial, we will implement all of this functionality in a function called run_mdi(). Examples of a minimalistic run_mdi() function are provided below, in C++, Fortran, and Python. You can simply copy the function into your codebase and call run_mdi() at your MDI node.
run_mdi("@DEFAULT", my_rank, world_comm, mdi_comm).
CALL run_mdi("@DEFAULT", my_rank, world_comm, mdi_comm).
MDI requires you to "register" a list of all nodes and commands your engine supports. This allows you, as an engine developer, to inform any drivers of what your code can do. MDI provides two functions that allow you to do this: MDI_Register_node() and MDI_Register_command(). The engine we have developed thus far in the tutorial only supports a single node, and that node only supports a single command. As a result, we need to call MDI_Register_node() once to register the @DEFAULT node and call MDI_Register_command() once to register the EXIT command.
Fortunately, this is a very simple process. The only argument to the MDI_Register_node() function is the name of the node, @DEFAULT. The MDI_Register_command() function accepts two arguments: the name of the node, and the name of the command that we are registering at that node, EXIT. In time, you may add support for additional commands at the default node, making additional calls to MDI_Register_command() for each newly supported command. If you add additional nodes, each node will have its own list of registered commands, which may be different from the list of commands supported at the @DEFAULT node.
For now, place the following code immediately after your engine's call to MDI_Init(). It is important that it be called prior to the call to MDI_Accept_communicator().
It may not look like much yet, but you have now established a basic MDI interface! If you generate a new report by typing mdimechanic report at the command line, MDI Mechanic should confirm that your engine passes all of the "Basic Functionality Tests". If your engine is still failing that test, you should return to the previous steps of this tutorial to determine what went wrong.
Assuming that your MDI interface is functioning as expected, you can now begin the process of implementing support for more commands. Whenever you want to add support for a new command, you will need to do the following:
run_mdi() that will respond appropriately to the new command.MDI_Register_command() to register support for that command.Examine the commands specified by the MDI Standard, and implement support for the ones that seem relevant for your engine. Each command in the MDI Standard describes exactly how the engine is expected to respond. Often, the engine is expected to either send or receive information to/from the driver. This is accomplished using the MDI_Send() and MDI_Recv() functions, respectively.
If you are using MPI, you should be aware that all MDI-based communication must take place through rank 0. Only rank 0 should call MPI_Send(), MPI_Recv(), and MPI_Recv_Command(). Depending on how you have distributed data structures across ranks, you may need to do MPI_Gather() or similar operations to collect the data onto rank 0 before calling MDI_Send(). Likewise, you may need to do MPI_Scatter() or similar operations to correctly distribute data after calling MDI_Recv().
Whenever you add a new node, you must also add a call to MDI_Register_node() to register support for that node. Commands are registered separately for each node, so any commands that are supported at the new node must be registered for it. For example, if you implement five nodes, and each of them supports the EXIT command, you will need to call the MDI_Register_command() function five times, each time with a different node as the first argument and with EXIT as the second argument.
1.8.17