Software Compatibility

e4s-cl wraps itself around a MPI command and inserts a container launch before the command is run. This implies an understanding of both the process launcher and the container technology to use. This section details the software e4s-cl was developped for and tested with.

MPI implementations

e4s-cl is able to launch executables compiled with a different version than the one available on the system at runtime.

MPI families

With the introduction of the MPICH ABI initiative and the MPI implementations that followed it, we see the emergence of multiple ‘families’ of MPI implementations. Members within a family can usually swap their libraries at runtime, while this is impossible accross families.

If you are unsure about the implementation used to compile a binary, a simple check is to run ldd on the executable and looking at the MPI library listed as a dependency. The below table lists the usual libray name in the soname column.

MPI implementation	MPICH ABI compatible	Family	soname
OpenMPI	No	openmpi	libmpi.so.40
MPICH	Yes	mpich	libmpi.so.12
CRAY MPICH	Yes	mpich	libmpi.so.12
MVAPICH	Yes	mpich	libmpi.so.12
Intel MPI	Yes*	mpich	libmpi.so.12

Enabling translation

e4s-cl is able to detect the MPI implementation used on the host from the input parameters; however the implementation used in the container cannot be ascertained easily. MPI translation is thus not needed when running executables with an implementation from the same family as the host’s; when this is not the case, one can enable it from the command line using the --from flag:

# Enable translation from an OpenMPI binary to a MPICH-based profile
e4s-cl --from openmpi srun -n 4 [...] path/to/executable

This will install and leverage WI4MPI (git repos here), which is a wrapper interface that performs translations from one MPI implementation to another. This allows e4s-cl to function even when substituting two MPI libraries that are ABI-incompatible.

Container backends

During a launch command, a list of required files will be computed on each target node, and bound to a container spawned on those nodes. Container backends are the container technologies e4s-cl is able to use.

apptainer, podman, shifter and singularity are supported. The docker backend can be enabled by installing e4s-cl with the docker extra feature.

Most of the container backends have some degree of configuration available to them. Configuration is done by setting the corresponding values in the backends section of the configuration file, or by passing a given environment variable.

An option passed in the environment will override the same option set in the configuration file.

apptainer

apptainer support is achieved using its command line interface. apptainer has to be installed on the system and available for it to be used by e4s-cl.

The configuration file section for apptainer is backends/apptainer. The available options are:

Configuration variable	Environment variable	Type	Default	Description
executable	E4S_CL_APPTAINER_EXECUTABLE	String	""	Path to the apptainer executable to use.
options	E4S_CL_APPTAINER_OPTIONS	List	[]	Options to pass to the spawned apptainer process.
run_options	E4S_CL_APPTAINER_EXEC_OPTIONS	List	[]	Options to pass to the exec command of the spawned apptainer process.

docker

docker support

docker support in HPC is limited at best. Due to its architecture, it will most likely not work with your MPI implementation. To enable docker support from e4s-cl, install with the docker extra enabled.

docker support is achieved using the docker-py module. The docker daemon has to be installed and running on the system to be accessed.

There are no configuration options for the docker backend.

Warning

Using docker with MPI

Several MPI implementations expect their processes to inherit opened file descriptors; because of docker’s client-daemon architecture, this is not possible. To use docker images with MPI, it is encouraged to used podman.

podman

podman support is achieved using its command line interface. podman has to be installed on the system and available for it to be used by e4s-cl.

The configuration file section for podman is backends/podman. The available options are:

Configuration variable	Environment variable	Type	Default	Description
executable	E4S_CL_PODMAN_EXECUTABLE	String	""	Path to the podman executable to use.
options	E4S_CL_PODMAN_OPTIONS	List	[]	Options to pass to the spawned podman process.
run_options	E4S_CL_PODMAN_RUN_OPTIONS	List	[]	Options to pass to the run command of the spawned podman process.

shifter

The shifter container technology is an interesting case, as it offers much of what e4s-cl is trying to propose, but in a much more restrictive way.

Warning

shifter MPI modules

Some MPI libraries will be configured as modules for shifter and imported automatically. You might not need e4s-cl ! Check the configuration file in /etc/shifter/udiRoot.conf to see if the MPI library you are trying to use is imported.

Warning

Binding files with shifter

shifter is the only container backend that does not support binding files to the container. shifter also forbids binding directories to certain locations, like /etc. This is circumvented by e4s-cl by copying all required libraries in a temporary directory, then binding it to the container. Files are ignored.

The configuration file section for shifter is backends/shifter.

Configuration variable	Environment variable	Type	Default	Description
executable	E4S_CL_SHIFTER_EXECUTABLE	String	""	Path to the shifter executable to use.
options	E4S_CL_SHIFTER_OPTIONS	List	[]	Options to pass to the spawned shifter process.

singularity

singularity support is achieved using its command line interface. singularity has to be installed on the system and available for it to be used by e4s-cl.

The configuration file section for singularity is backends/singularity. The available options are:

Configuration variable	Environment variable	Type	Default	Description
executable	E4S_CL_SINGULARITY_EXECUTABLE	String	""	Path to the singularity executable to use.
options	E4S_CL_SINGULARITY_OPTIONS	List	[]	Options to pass to the spawned singularity process.
run_options	E4S_CL_SINGULARITY_EXEC_OPTIONS	List	[]	Options to pass to the exec command of the spawned singularity process.

barebones

barebones support is achieved by bypassing the use of a container technology.

The configuration file section for barebones is backends/barebones. The available options are:

Configuration variable	Environment variable	Type	Default	Description
options	E4S_CL_BAREBONES_OPTIONS	List	[]	Options to pass to the spawned process.
run_options	E4S_CL_BAREBONES_EXEC_OPTIONS	List	[]	Options to pass to the script of the spawned process.

Other container backends

More container technologies can be supported. Create an issue on github or write a dedicated module in e4s_cl/cf/containers. Refer to e4s_cl/cf/containers/__init__.py for details.

Process launchers

The following process managers were successfully tested with e4s-cl:

• The stock mpirun of multiple MPI distributions;

• LLNL’s SLURM using srun;

• CRAY’s ALPS using aprun;

• IBM’s JSM using jsrun.

Support implies the automatic detection of parameters. If a launcher is not supported, use the ‘--’ syntax to separate launcher and process arguments.

Additional options can be configured through the configuration file or the environment:

Configuration variable	Environment variable	Type	Default	Description
launcher_options	E4S_CL_LAUNCHER_OPTIONS	List	[]	List of options to pass to the launcher