Conda Environments

Adding a conda environment to a container may be easily done using cotainr.

Make sure you have the rights to use the packages in your conda environment

When adding a conda environment, it is the responsibility of the user of cotainr to make sure they have the necessary rights to use the conda channels/repositories and packages specified in the conda environment, e.g. if using the default Anaconda repositories.

As an example, consider the following conda environment file, my_conda_env.yml:

Base image requirement

In order to include a conda environment in your container, you must use a base image in which bash is installed.

my_conda_env.yml

channels:
  - conda-forge
dependencies:
  - python=3.11.0
  - numpy=1.23.5

A container based on the official Ubuntu 24.04 DockerHub image, containing this conda environment, may be built using:

$ cotainr build my_conda_env_container.sif --base-image=docker://ubuntu:24.04 --conda-env=my_conda_env.yml

The conda environment is automatically activated when the container is run, allowing for directly using the Python and NumPy versions installed in the conda environment, e.g.

$ singularity exec my_conda_env_container.sif python3 --version
Python 3.11.0
$ singularity exec my_conda_env_container.sif python3 -c "import numpy; print(numpy.__version__)"
1.23.5

You must accept the Miniforge license to add a conda environment

Bootstrapping of the conda environment in the container is done using Miniforge. As part of the bootstrap process, you must accept the Miniforge license terms. This can be done either by accepting them during the container build process, or by specifying the --accept-licenses option when invoking cotainr build.

Iterate on the conda environment before building the container

If you are unsure about what needs to go into your my_conda_env.yml file for conda to correctly resolve your environment, you may want to iterate on the content of the my_conda_env.yml file by updating it and using conda env create --file my_conda_env.yml to test that it resolves and installs correctly outside of the container. Only once it installs correctly, should you proceed to building the container with cotainr build. That way you can potentially save a lot of time by not having to rebuild the container multiple times while iterating on your conda environment.

Pip packages

cotainr does not support creating a container directly from a pip requirements.txt file. However, pip packages may be included in a conda environment, e.g. updating my_conda_env.yml to

my_conda_env.yml

channels:
  - conda-forge
dependencies:
  - python=3.11.0
  - numpy=1.23.5
  - pip
  - pip:
    - scipy==1.9.3

allows for installing SciPy via pip.

Pip packages from private repositories

Pip packages from a private GitHub repository are accessible using an ssh key. You can use this option by enabling ssh-agent forwarding to GitHub on the host machine on which cotainr is used.

For example, with my_conda_env.yml containing the following:

my_conda_env.yml

channels:
  - conda-forge
dependencies:
  - python=3.11.0
  - git
  - openssh
  - pip
  - pip:
    - "--editable=git+ssh://git@github.com/foo/bar.git@SOMEHASHCODE#egg=baz"

where github.com:foo/bar.git is a private repository.

The need for using ssh-agent is fundamentally an apptainer limitation/feature and not related to cotainr per se. For this to work, the directory pointed to on the host by the SSH_AUTH_SOCK environment variable must be bound to the container. If echo $SSH_AUTH_SOCK already points to one of the directories bound by default, e.g. /tmp, everything should work. Otherwise, another solution must be found, as cotainr does not expose directory binding from apptainer.