Molssi Driver Interface Library
Tutorials

User Tutorial

In this tutorial we will use an MDI driver to perform a simple Ab Initio Molecular Dynamics (AIMD) simulation, using Quantum ESPRESSO (QE) to calculate forces and LAMMPS to update the atomic coordinates each time step.

Compiling the Engines

We will need access to a QM code and an MM code, both of which must be capable of being run as an MDI engine. This tutorial will describe the use of Quantum ESPRESSO (QE) as the QM code and LAMMPS as the MM code; however, you can use any MDI-enabled QM and MM codes you prefer. A list of codes that support MDI can be found here, along with instructions on how to compile them with MDI support.

Before moving to the next step, compile both QE and LAMMPS (or your preferred QM and MM packages) with MDI support by following the instructions provided.

Compiling the AIMD Driver

The MDI Standard supports the creation of many different types of drivers for a broad variety of applications. For this tutorial, we will download a simple AIMD driver:

git clone git@github.com:MolSSI/MDI_AIMD_Driver.git

First, navigate to the directory where you want the driver to be compiled. For this tutorial, we will do:

mkdir MDI_AIMD_Driver/build
cd MDI_AIMD_Driver/build

Then compile using CMake:

cmake ..
make

Running the Simulation

We will now run a simple AIMD simulation.

In MDI_AIMD_Driver/tests/locations there are three single-line text files, each of which is intended to indicate the location of one of the codes you installed above. Edit each of them so that they provide the absolute path to the corresponding code's executable.

There is a simple test calculation in MDI_AIMD_Driver/tests/water. Within this directory, the data subdirectory contains standard input files for LAMMPS and QE. The input files correspond to a small water box of 8 water molecule. They are exactly like any non-MDI input files for their respective codes, except for data/lammps.in. Near the end of data/lammps.in is an mdi fix, followed by an mdi command:

fix 1 all mdi
...
mdi

These two lines are required in any LAMMPS input file that is intended to be used in conjuction with MDI. The mdi fix allows LAMMPS to communicate with the driver at the appropriate points throughout the simulation, while the mdi LAMMPS command indicates the point when LAMMPS begins listening for MDI commands from the driver.

There are two scripts associated with the water test: tcp.sh and mpi.sh. The tcp.sh script will run the simulation using the TCP/IP method for communicating between the driver and the engines. It consists of the following code:

#location of required codes
DRIVER_LOC=$(cat ../locations/MDI_AIMD_Driver)
LAMMPS_LOC=$(cat ../locations/LAMMPS)
QE_LOC=$(cat ../locations/QE)
#remove old files
if [ -d work ]; then
rm -r work
fi
#create work directory
cp -r data work
cd work
cd work
${QE_LOC} -mdi "-role ENGINE -name QM -method TCP -port 8021 -hostname localhost" -in qe.in > qe.out &
#launch LAMMPS
${LAMMPS_LOC} -mdi "-role ENGINE -name MM -method TCP -port 8021 -hostname localhost" -in lammps.in > lammps.out &
#launch driver
${DRIVER_LOC} -mdi "-role DRIVER -name driver -method TCP -port 8021" &
wait

The first few lines simply read the location of the codes from the files in MDI_AIMD_Driver/tests/locations and copy the LAMMPS and QE input files into a subdirectory called work. Then the script launches QE. The QE launch command is exactly the same as a normal run of QE, except for the addition of a -mdi option. This option provides information to the MDI Library regarding how the codes will communicate with one another. Its argument is a string, which itself consists of a series of options that are read by the MDI Library. In this case, it provides the following options:

  • role: Indicates whether this code will run as an ENGINE or a DRIVER.
  • name: Identifies the purpose of this engine. Each MDI driver will expect its engines to be named according to a particular standard. The MDI_AIMD_Driver engine expects one of the engines to be named "QM" and the other to be named "MM".
  • method: The communication method used to transfer information between the driver and the engines.
  • port: The port number that the driver will be listening over. Only used if the method is TCP.
  • hostname: The host name of the driver. Only used by engines and if the method is TCP.

Note the presence of an ampersand ("&") at the end of each run command, which causes the codes to run in the background. Without doing this, the script would launch QE and would wait until QE terminated before ever launching LAMMPS and the driver, leading to an indefinite hang. Finally, the wait command at the end ensures that the driver and the engines are permitted to terminate before the script terminates.

You can now execute the tcp.sh script, which should print out the energies of LAMMPS and QE for several timesteps of a short AIMD simulation.

If you compiled everything (QE, LAMMPS, MDI_AIMD_Driver, and the internal copies of the MDI Library) with MPI, you can also execute the mpi.sh script, which will run the same simulation using MPI for the inter-code communication. The output produced by QE and LAMMPS can be found in work/qe.out and work/lammps.out, respectively.

Driver Development Tutorial

Please complete the User Tutorial before starting this tutorial.

In this tutorial we will set up a simple driver for running Ab Initio Molecular Dynamics (AIMD) simulations. We will use a quantum chemistry (QM) code to compute the forces, while using a molecular mechanics (MM) code to propagate the atomic coordinates each time step.

Creating a New Project

The easiest way to start work on a new driver is to use the MDI Driver Cookiecutter, which will automatically do most of the preparatory work for you. Using the cookiecutter will require that you have Python and Cookiecutter on your machine. If you do not already have Python installed, follow the directions on the Python Website and install both Python and the pip installer tool. To install Cookiecutter, type:

pip install cookiecutter

If you are running on an external machine and do not have write permission to the system directories, you may need to add the –user option:

pip install --user cookiecutter

Now use Cookiecutter to create a new driver project:

cookiecutter git@github.com:MolSSI/MDI_Driver_Cookiecutter.git

When prompted for the repo_name, type aimd, and then when prompted to select a language, type 1 for C++. This will create a new directory called aimd and populate it with some of the things you will need for the driver, including a copy of the MDI Library. The overall directory structure is:

.
└── aimd
├── CMakeLists.txt
├── aimd
├── CMakeLists.txt
├── STUBS_MPI
│   └── mpi.h
└── aimd.cpp
└── mdi
...

Writing the Driver

Open the file called aimd.cpp. It contains the following code:

#include <iostream>
#include <mpi.h>
#include <stdexcept>
#include <string.h>
#include "mdi.h"
using namespace std;
int main(int argc, char **argv) {
// Initialize the MPI environment
MPI_Comm world_comm;
MPI_Init(&argc, &argv);
// Initialize MDI
if ( MDI_Init(&argc, &argv) ) {
throw std::runtime_error("The MDI library was not initialized correctly.");
}
// Confirm that MDI was initialized successfully
int initialized_mdi;
if ( MDI_Initialized(&initialized_mdi) ) {
throw std::runtime_error("MDI_Initialized failed.");
}
if ( ! initialized_mdi ) {
throw std::runtime_error("MDI not initialized: did you provide the -mdi option?.");
}
// Get the correct MPI intra-communicator for this code
if ( MDI_MPI_get_world_comm(&world_comm) ) {
throw std::runtime_error("MDI_MPI_get_world_comm failed.");
}
// Connect to the engines
// <YOUR CODE GOES HERE>
// Perform the simulation
// <YOUR CODE GOES HERE>
// Send the "EXIT" command to each of the engines
// <YOUR CODE GOES HERE>
// Synchronize all MPI ranks
MPI_Barrier(world_comm);
MPI_Finalize();
return 0;
}

The first few lines of code simply initialize both MPI and and MDI. Don't worry if you don't have access to an MPI library - the code will just fall back to a set of dummy MPI functions provided in STUBS_MPI. The call to the MDI_MPI_get_world_comm() function obtains from MDI an MPI intra-communicator that spans all MPI ranks associated with the driver. This call is important when using the MPI protocol for MDI communication, because in that context, MPI_COMM_WORLD spans all ranks associated with both the driver and the engine(s). In general, an MDI-enabled code should call MDI_MPI_get_world_comm() immediately after calling MDI_Init(), and the resulting MPI intra-communicator should be used instead of MPI_COMM_WORLD throughout the remainder of the code.

After initializing MDI, we need to connect the driver to its engines. For this particular tutorial, we will use a QM engine to calculate forces and an MM engine to update the atomic coordinates each timestep. In the MDI standard, engines request a connection to a driver. The driver will need to call the MDI_Accept_communicator() function to accept each connection. The general format of this functional call looks like this:

MDI_Comm comm;

This will assign a new MDI communicator to comm, which functions very similarly to an MPI communicator and is used in certain MDI function calls to identify which engine is expected to send/receive data to/from the driver. Our AIMD driver will connect to two different engines, so we will be calling MDI_Accept_communicator() twice. We don't know the order in which the engines will request a connection to the driver, so we will need some way to determine which engine is the QM code and which engine is the MM code.

This can be accomplished through the use of the "<NAME" command. The entire MDI standard is built around the idea that drivers can send "commands" to engines, each of which is defined by the MDI standard and has a specific outcome. The "<NAME" command tells the engine to send the driver a string of length MDI_NAME_LENGTH that identifies the engine. The end-user is responsible for indicating the name of each engine at runtime, using the -name command line argument that was described in the User Tutorial. As authors of a driver, we will need to decide what we expect each of the engines to be named and clearly document that decision for the benefit of the end-users. For the purpose of this tutorial, we will expect the quantum mechanics code to be called "QM" and the molecular mechanics code to be named "MM".

Sending the "<NAME" command to an engine and then receiving the engine's name can be done as follows:

char* code_name = new char[MDI_NAME_LENGTH];
MDI_Send_command("<NAME", comm);
MDI_Recv(code_name, MDI_NAME_LENGTH, MDI_CHAR, comm);

The following code accepts connections from two engines and assigns the communicator from the engine named "MM" to mm_comm and assigns the communicator from the engine named "QM" to qm_comm. Replace the comment that reads "// Connect to the engines" with:

// Connect to the engines
MDI_Comm mm_comm = MDI_COMM_NULL;
MDI_Comm qm_comm = MDI_COMM_NULL;
int nengines = 2;
for (int iengine=0; iengine < nengines; iengine++) {
MDI_Comm comm;
// Determine the name of this engine
char* engine_name = new char[MDI_NAME_LENGTH];
MDI_Send_command("<NAME", comm);
MDI_Recv(engine_name, MDI_NAME_LENGTH, MDI_CHAR, comm);
cout << "Engine name: " << engine_name << endl;
if ( strcmp(engine_name, "MM") == 0 ) {
if ( mm_comm != MDI_COMM_NULL ) {
throw runtime_error("Accepted a communicator from a second MM engine.");
}
mm_comm = comm;
}
else if ( strcmp(engine_name, "QM") == 0 ) {
if ( qm_comm != MDI_COMM_NULL ) {
throw runtime_error("Accepted a communicator from a second QM engine.");
}
qm_comm = comm;
}
else {
throw runtime_error("Unrecognized engine name.");
}
delete[] engine_name;
}

We are now ready to use MDI to orchestrate an AIMD simulation. In broad terms, during each dynamics iteration we will send a set of atomic coordinates from the MM engine to the QM engine, order the QM engine to send the driver the corresponding atomic forces, send those forces to the MM engine, and order the MM engine to perform a time step.

Keep in mind that the driver doesn't know anything about the simulated system beyond what it can query by sending MDI commands. MDI engines initialize basic information about the system using whatever input file format the engine's developer chose. When the MM and QM engines are launched, they will each read their standard input files, perform basic initialization tasks, and then request a connection to the driver. The driver can request information about the engine's system by sending certain MDI commands. For example, to determine the number of atoms in the MM engine's system, you could do:

// Receive the number of atoms from the MM engine
int natoms;
MDI_Send_command("<NATOMS", mm_comm);
MDI_Recv(&natoms, 1, MDI_INT, mm_comm);

Similarly, to learn the coordinates of the atoms in the MM engine's system, you could do:

// Receive the coordinates from the MM engine
double coords[3*natoms];
MDI_Send_command("<COORDS", mm_comm);
MDI_Recv(&coords, 3*natoms, MDI_DOUBLE, mm_comm);

You could then update the coordinates of the QM engine's system:

// Send the coordinates to the QM engine
MDI_Send_command(">COORDS", qm_comm);
MDI_Send(&coords, 3*natoms, MDI_DOUBLE, qm_comm);
Note
As we will discuss later in Using the Driver, this driver assumes that the engines have both been initialized with the same number of atoms, the same atom types and ordering of atom types, and the same cell dimensions.

The following code will handle all of the work associated with driving an AIMD simulation Replace the comment that reads "// Perform the simulation" with:

// Perform the simulation
int niterations = 10; // Number of MD iterations
int natoms;
double qm_energy;
double mm_energy;
// Receive the number of atoms from the MM engine
MDI_Send_command("<NATOMS", mm_comm);
MDI_Recv(&natoms, 1, MDI_INT, mm_comm);
// Allocate the arrays for the coordinates and forces
double coords[3*natoms];
double forces[3*natoms];
// Have the MM engine initialize a new MD simulation
MDI_Send_command("@INIT_MD", mm_comm);
// Perform each iteration of the simulation
for (int iiteration = 0; iiteration < niterations; iiteration++) {
// Receive the coordinates from the MM engine
MDI_Send_command("<COORDS", mm_comm);
MDI_Recv(&coords, 3*natoms, MDI_DOUBLE, mm_comm);
// Send the coordinates to the QM engine
MDI_Send_command(">COORDS", qm_comm);
MDI_Send(&coords, 3*natoms, MDI_DOUBLE, qm_comm);
// Have the MM engine proceed to the @FORCES node
MDI_Send_command("@FORCES", mm_comm);
// Get the QM energy
MDI_Send_command("<ENERGY", qm_comm);
MDI_Recv(&qm_energy, 1, MDI_DOUBLE, qm_comm);
// Get the MM energy
MDI_Send_command("<ENERGY", mm_comm);
MDI_Recv(&mm_energy, 1, MDI_DOUBLE, mm_comm);
// Receive the forces from the QM engine
MDI_Send_command("<FORCES", qm_comm);
MDI_Recv(&forces, 3*natoms, MDI_DOUBLE, qm_comm);
// Send the forces to the MM engine
MDI_Send_command(">FORCES", mm_comm);
MDI_Send(&forces, 3*natoms, MDI_DOUBLE, mm_comm);
// Have the MM engine proceed to the @COORDS node, which completes the timestep
MDI_Send_command("@COORDS", mm_comm);
cout << "timestep: " << iiteration << " " << mm_energy << " " << qm_energy << endl;
}

The above code does the following:

  1. Queries the number of atoms from the MM engine and allocates appropriately sized arrays for the coordinates and forces
  2. Orders the MM engine to initialize a new MD simulation
  3. Begins an iterative loop over MD iterations
    1. Receives the atomic coordinates from the MM engine and updates the QM engine with them
    2. Receives the energy of the QM and MM systems
    3. Receives the atomic forces from the QM engine and sends them to the MM engine
    4. Orders the MM engine update the atomic coordinates

Finally, we should send an MDI command to cause the engines to exit. Replace the comment that reads "// Send the "EXIT" command to each of the engines" with the following:

// Send the "EXIT" command to each of the engines
MDI_Send_command("EXIT", mm_comm);
MDI_Send_command("EXIT", qm_comm);

Driver Compilation

The cookiecutter came with everything you need to build with CMake. To compile, simply navigate to the directory where you want the driver to be built, then do the following, replacing "<driver_top_directory>" with the path to top directory of your driver repository.

cmake <driver_top_directory>
make

Using the Driver

To test the driver, you will need access to a QM engine and an MM engine. The User Tutorial describes how to compile QE and LAMMPS for this purpose.

You will also need appropriate input files for a test simulation. The test files in the MDI_AIMD_Driver repository (see the User Tutorial) will work fine, as will the scripts. You can simply edit MDI_AIMD_Driver/tests/locations/MDI_AIMD_Driver to point to your new repository and run the scripts in the manner described in the User Tutorial.

Final Notes

Note that although we used QE as the QM code and LAMMPS as the MM code, we swap out QE for any QM engine with MDI support or LAMMPS for any MM engine with MDI support. Furthermore, nothing about our AIMD driver strictly requires that the code named "QM" actually corresponds to a quantum mechanics code. You could, for example, use LAMMPS as the QM code, while another instance of LAMMPS serves as the MM code. Doing so would allow you to run a calculation that will produce to same results as simply running an MD simulation entirely within a single instance of LAMMPS. Although not generally useful for production runs, this can be a good way to benchmark the cost of the computational overhead introduced by using our driver as a middleman between two codes.

In other cases, it may be desirable to use two different MM codes as the engines if, for example, you wish to use a force field from one MM code and a thermostat from another MM code. The only requirement on the engines is that the "QM" code supports all of the MDI commands sent by the AIMD driver to the "QM" engine, and that the "MM" code supports all of the MDI commands sent by the AIMD driver to the "MM" engine.

MDI_MPI_get_world_comm
int MDI_MPI_get_world_comm(void *world_comm)
Obtain the MPI communicator that spans the single code corresponding to the calling rank.
Definition: mdi.c:1565
MDI_Initialized
int MDI_Initialized(int *flag)
Indicates whether MDI_Init has been called.
Definition: mdi.c:184
MDI_COMM_NULL
const MDI_Comm MDI_COMM_NULL
value of a null communicator
Definition: mdi.c:55
MDI_Send
int MDI_Send(const void *buf, int count, MDI_Datatype datatype, MDI_Comm comm)
Send data through the MDI connection.
Definition: mdi.c:233
MDI_DOUBLE
const int MDI_DOUBLE
double precision float data type
Definition: mdi.c:77
MDI_NAME_LENGTH
const int MDI_NAME_LENGTH
length of an MDI name in characters
Definition: mdi.c:49
MDI_Recv
int MDI_Recv(void *buf, int count, MDI_Datatype datatype, MDI_Comm comm)
Receive data through the MDI connection.
Definition: mdi.c:257
MDI_Send_command
int MDI_Send_command(const char *buf, MDI_Comm comm)
Send a command of length MDI_COMMAND_LENGTH through the MDI connection.
Definition: mdi.c:293
MDI_Init
int MDI_Init(int *argc, char ***argv)
Initialize communication through the MDI library.
Definition: mdi.c:112
MDI_Accept_communicator
MDI_Comm MDI_Accept_communicator(MDI_Comm *comm)
Accept a new MDI communicator.
Definition: mdi.c:208
MDI_CHAR
const int MDI_CHAR
character data type
Definition: mdi.c:79
MDI_INT
const int MDI_INT
integer data type
Definition: mdi.c:59