Skip to content

Managing Conda environments on the NCShare Cluster

You can setup conda environments to help tailor the Open OnDemand JupyterLab instances to your specific research. See below for how to setup conda environments and then how to launch a JupyterLab application to utilize those conda environments.

One-Time Setup Steps

1.) Login to the NCShare OnDemand and launch an interactive shell

2.) Install Miniconda

Download the installer: 

wget -O Miniforge3.sh "https://github.com/conda-forge/miniforge/releases/latest/download/Miniforge3-$(uname)-$(uname -m).sh"
Run the installation: 
bash Miniforge3.sh
and then follow the instructions. It will offer to update your ~/.bashrc, (init), say no if you use multiple environments. Say yes if you will only be using this environment.

Restart your shell: 

exec bash
Delete your installation file to save space: 
rm Miniforge3.sh

Now you should be an run conda install, pip install, create environments, etc.

Managing Conda Environments

  1. Creating a conda environment Users who will be using Python through SSH and OnDemand should setup virtual environments using miniconda with the desired Python version and packages. After installing Minconda, create a virtual environment. In the example below, myenv is the name of the environment and python=3.9 is the python version number.

conda create -n myenv python=3.9
conda activate myenv
conda install ipykernel
python -m ipykernel install --user --name myenv
Note: if you did not update your .bashrc file, you will need to load conda first.

  1. Activating your environment First, to use your environment, you need to activate your environment: 
    conda activate myenv
    You will know your environment is activated because it will show up on your command line: 
    (myenv) kk338@dcc-login-03  ~ $
  2. Installing any packages needed in that environment Then, you can install any packages you need using conda or pip. If your environment is activated, conda keeps track of which packages are installed where. Some packages will require pip to install - your new conda environment has it’s own pip install.

Accessing Jupyter Lab through NCShare OnDemand portal

  1. Login at ood.ncshare.org
  2. Click on Interactive Apps in the top navigation menu

  3. Click on Jupyter Lab Apptainer

    a. Walltime: This is the number of hours to allocate to your job's duration. Input the number of hours you would like the server to remain active (please try to remain small, as it will continue running even if you are not using it)

    b. CPUs (Threads) and Memory: Input the desired amount of memory, and CPUs (try to start small with only a few gigabytes of memory and cores)

    e. Press the blue "Launch" button on the bottom of the page

  4. After pressing the blue "launch" button, your job will be queued to start a Jupyter Lab server. You should see this automatically

  5. Wait a few minutes for the Jupyter Lab server to finish launching. The status will automatically change from "Starting" to "Running" when the server is ready
  6. Press the blue "Connect to Jupyter" button when the server is running to access your Jupyter Lab server
  7. You should be able to change kernel as normal

Transferring data for use with JupyterLab

You can transfer data via the Open OnDemand website. We recommend transferring individual files this way. If you need to transfer larger data sets, please see our section on data transfers utilizig SCP or rsync.

To transfer an individual file using OnDemand:

  1. Login at ood.ncshare.org
  2. Click on Files > Home Directory
  3. Here, you should see your home user directory (/hpc/home/userID). Look for the blue "Upload" and "Download" buttons.
  4. Selecting either the "Upload" or "Download" button brings up a window where you can select files to either upload or download. Selecting upload will copy files from your local machine to the directory specified by the OnDemand path (e.g., '/hpc/home/userID)'.
  5. If you need to copy files into another directory, simply click on "Change directory" and enter the directory path.