Bench-in-situ with Guix

This is a DRAFT DOCUMENT
Table of content

This approach creates a guix environment with the build dependencies of the benchmarks, like gcc or openmpi. The environment is defined in the following manifest:

;; manifest.scm
(use-modules (guix build-system gnu)
             (ice-9 match))

(define stdenv
  (map (lambda* (pkg)
         (match pkg
           ((_ value _ ...)
            value)))
    (standard-packages)))

(concatenate-manifests
  (list
    (specifications->manifest
      (list
        "bash"
        "gcc-toolchain"

        "cmake"
        "openmpi@4"
        "pdi"
        "pdiplugin-mpi"
        "pdiplugin-user-code"
        "pdiplugin-decl-hdf5-parallel"
        "kokkos"
        "paraconf"
        "pkg-config"
        "libyaml"))
    (packages->manifest stdenv)))

Building the pack

On a machine with guix, and the guix-hpc channel, the following commands produces the pack.sqsh singularity image.

$ guix pack \
    --format=squashfs \
    -r pack.sqsh \
    -S /bin=bin \
    -S /etc=etc \
    -m ./manifest.scm

The singularity image can be copied to the target server with scp.

Loading the image

Jean-Zay

This machine requires you to load the image into a safe directory first, and to load a compute node:

$ idrcontmgr cp pack.sqsh
$ srun \
    --pty \
    --ntasks=2 \
    --mpi=pmix_v3 \
    --hint=nomultithread \
    bash

Getting a shell within the singularity only requires:

$ singularity \
    shell \
    $SINGULARITY_ALLOWED_DIR/pack.sqsh

Building and running

$ git clone https://github.com/Maison-de-la-Simulation/bench-in-situ
$ mkdir build && cd build
$ cmake -DSESSION=MPI_SESSION -DKokkos_ENABLE_OPENMP=ON -DEuler_ENABLE_PDI=ON ..
$ make
$ ./main ../setup.ini ../io_chkpt.yml
--------------------------------------------------------------------------
WARNING: There was an error initializing an OpenFabrics device.

  Local host:   r1i3n1
  Local device: hfi1_1
--------------------------------------------------------------------------
main: error: Invalid user for SlurmUser slurm, ignored
main: fatal: Unable to process configuration file