[Bug] Mybinder: SingleNodeExecutor notebook crashes

[jan-janssen]

While the notebooks run without any issue in the continuous integration environment they somehow cause a memory overload on mybinder when it comes to running the MPI parallel python functions. The example that breaks is:

from executorlib import SingleNodeExecutor

def calc_mpi(i):
    from mpi4py import MPI

    size = MPI.COMM_WORLD.Get_size()
    rank = MPI.COMM_WORLD.Get_rank()
    return i, size, rank


with SingleNodeExecutor(resource_dict={"cores": 2}, block_allocation=True) as exe:
    fs = exe.submit(calc_mpi, 3)
    print(fs.result())

[jan-janssen]

With 1.5.3 it still works but also requires a large amount of memory getting close to the memory limit of mybinder, with 1.6.0 it crashes with an out of memory issue without any noticeable change in the configuration.