How it works#

e4s-cl uses file binding and environment manipulation to allow accessing host libraries from inside of the container. The following outlines how this is done.

MPI translation#

The MPI standard abstracts a great deal of work from the developer to make development as simple as possible. MPI libraries have to do a lot behind the scenes, and different approaches are the reason so many library implementations exist.

This means switching between libraries is not a plug-and-play experience. The standard is loose enough for the binary interfaces to differ slightly, and even if they match, the dependencies might differ and prevent operation.

e4s-cl simplifies this by tracking MPI library vendor, version and dependencies to facilitate this switch. A binary compiled with one library can then be used with another, to allow making full use of individual libraries.

When libraries are compatible at the binary level, they are swapped seamlessly using the dynamic linker. If not, Wi4MPI is used to translate MPI calls between implementations.

Launch procedure#

container launcher elements diagram

Diagram of execution using an MPI launcher#

e4s-cl will first parse the command line and identify the launcher from the final command. This is done by using a list of known launcher and options, and can fail; use dashed (--) to explicitly split the two.

The identified launcher is then used to spawn worker processes. Those processes take over the original command given on the command line, and prepare the environment before running this same command in a container.

If no launcher is given, the process is just created on the same host.

This preparation consists of multiple steps:

  • The requested library list is resolved, completed if needed, then checked for compatibility. Files in the processed list are then bound to the container in a special directory (/.e4s-cl/hostlibs) to be made accessible by the linker through the environment.

  • The requested files are bound in-place, meaning they will appear to the contained process as they do on the host. This depends on the backend as some container technologies do not allow files to be bound.

  • A script is prepared to run the given command after the potential source script, then bound to the container in the /.e4s-cl directory.

Library profiling#

When tasked to create a profile from a library, e4s-cl will use ptrace to intercept the system calls used by a binary using that library. Intercepting the open and openat calls allows e4s-cl to inspect their parameters and determine the filenames of all the files the program opens.

This file list is then sorted in two categories: files and libraries.

  • Files are files that the program will expect to find at a specific location. This includes configuration files, metadata files, or shared objects that are loaded from a hardcoded path.

  • Libraries are files that are loaded from the standard dynamic linker. Their actual location does not matter as long as they are available in the linker search path.

The command to run this analysis is e4s-cl profile detect, that will run the given command. e4s-cl init will also do so and compile a sample MPI binary if not given any.