JASMIN Help Site logo JASMIN Help Site logo
  • Docs 
  • Guides 
  • Training 
  • Discussions   

  •   Search this site  

Can't find what you're looking for?

Try our Google custom search, across all JASMIN sites

Docs
  • getting started
    • get started with jasmin
    • generate ssh key pair
    • get jasmin portal account
    • get login account
    • beginners training workshop
    • how to contact us about jasmin issues
    • jasmin status
    • jasmin training accounts
    • tips for new users
    • how to login
    • multiple account types
    • present ssh key
    • reconfirm email address
    • reset jasmin account password
    • ssh auth
    • storage
    • understanding new jasmin storage
    • update a jasmin account
  • interactive computing
    • interactive computing overview
    • check network details
    • login servers
    • login problems
    • graphical linux desktop access using nx
    • sci servers
    • tenancy sci analysis vms
    • transfer servers
    • jasmin notebooks service
    • jasmin notebooks service with gpus
    • creating a virtual environment in the notebooks service
    • project specific servers
    • dask gateway
    • access from vscode
  • batch computing
    • lotus overview
    • slurm scheduler overview
    • slurm queues
    • lotus cluster specification
    • how to monitor slurm jobs
    • how to submit a job
    • how to submit an mpi parallel job
    • example job 2 calc md5s
    • orchid gpu cluster
    • slurm status
    • slurm quick reference
  • software on jasmin
    • software overview
    • quickstart software envs
    • python virtual environments
    • additional software
    • community software esmvaltool
    • community software checksit
    • compiling and linking
    • conda environments and python virtual environments
    • conda removal
    • creating and using miniforge environments
    • idl
    • jasmin sci software environment
    • jasmin software faqs
    • jaspy envs
    • matplotlib
    • nag library
    • name dispersion model
    • geocat replaces ncl
    • postgres databases on request
    • running python on jasmin
    • running r on jasmin
    • rocky9 migration 2024
    • share software envs
  • data transfer
    • data transfer overview
    • data transfer tools
    • globus transfers with jasmin
    • bbcp
    • ftp and lftp
    • globus command line interface
    • globus connect personal
    • gridftp ssh auth
    • rclone
    • rsync scp sftp
    • scheduling automating transfers
    • transfers from archer2
  • short term project storage
    • apply for access to a gws
    • elastic tape command line interface hints
    • faqs storage
    • gws etiquette
    • gws scanner ui
    • gws scanner
    • gws alert system
    • install xfc client
    • xfc
    • introduction to group workspaces
    • jdma
    • managing a gws
    • secondary copy using elastic tape
    • share gws data on jasmin
    • share gws data via http
    • using the jasmin object store
    • configuring cors for object storage
  • long term archive storage
    • ceda archive
  • mass
    • external access to mass faq
    • how to apply for mass access
    • moose the mass client user guide
    • setting up your jasmin account for access to mass
  • for cloud tenants
    • introduction to the jasmin cloud
    • jasmin cloud portal
    • cluster as a service
    • cluster as a service kubernetes
    • cluster as a service identity manager
    • cluster as a service slurm
    • cluster as a service pangeo
    • cluster as a service shared storage
    • adding and removing ssh keys from an external cloud vm
    • provisioning tenancy sci vm managed cloud
    • sysadmin guidance external cloud
    • best practice
  • workflow management
    • rose cylc on jasmin
    • using cron
  • uncategorized
    • mobaxterm
    • requesting resources
    • processing requests for resources
    • acknowledging jasmin
    • approving requests for access
    • working with many linux groups
    • jasmin conditions of use
  • getting started
    • get started with jasmin
    • generate ssh key pair
    • get jasmin portal account
    • get login account
    • beginners training workshop
    • how to contact us about jasmin issues
    • jasmin status
    • jasmin training accounts
    • tips for new users
    • how to login
    • multiple account types
    • present ssh key
    • reconfirm email address
    • reset jasmin account password
    • ssh auth
    • storage
    • understanding new jasmin storage
    • update a jasmin account
  • interactive computing
    • interactive computing overview
    • check network details
    • login servers
    • login problems
    • graphical linux desktop access using nx
    • sci servers
    • tenancy sci analysis vms
    • transfer servers
    • jasmin notebooks service
    • jasmin notebooks service with gpus
    • creating a virtual environment in the notebooks service
    • project specific servers
    • dask gateway
    • access from vscode
  • batch computing
    • lotus overview
    • slurm scheduler overview
    • slurm queues
    • lotus cluster specification
    • how to monitor slurm jobs
    • how to submit a job
    • how to submit an mpi parallel job
    • example job 2 calc md5s
    • orchid gpu cluster
    • slurm status
    • slurm quick reference
  • software on jasmin
    • software overview
    • quickstart software envs
    • python virtual environments
    • additional software
    • community software esmvaltool
    • community software checksit
    • compiling and linking
    • conda environments and python virtual environments
    • conda removal
    • creating and using miniforge environments
    • idl
    • jasmin sci software environment
    • jasmin software faqs
    • jaspy envs
    • matplotlib
    • nag library
    • name dispersion model
    • geocat replaces ncl
    • postgres databases on request
    • running python on jasmin
    • running r on jasmin
    • rocky9 migration 2024
    • share software envs
  • data transfer
    • data transfer overview
    • data transfer tools
    • globus transfers with jasmin
    • bbcp
    • ftp and lftp
    • globus command line interface
    • globus connect personal
    • gridftp ssh auth
    • rclone
    • rsync scp sftp
    • scheduling automating transfers
    • transfers from archer2
  • short term project storage
    • apply for access to a gws
    • elastic tape command line interface hints
    • faqs storage
    • gws etiquette
    • gws scanner ui
    • gws scanner
    • gws alert system
    • install xfc client
    • xfc
    • introduction to group workspaces
    • jdma
    • managing a gws
    • secondary copy using elastic tape
    • share gws data on jasmin
    • share gws data via http
    • using the jasmin object store
    • configuring cors for object storage
  • long term archive storage
    • ceda archive
  • mass
    • external access to mass faq
    • how to apply for mass access
    • moose the mass client user guide
    • setting up your jasmin account for access to mass
  • for cloud tenants
    • introduction to the jasmin cloud
    • jasmin cloud portal
    • cluster as a service
    • cluster as a service kubernetes
    • cluster as a service identity manager
    • cluster as a service slurm
    • cluster as a service pangeo
    • cluster as a service shared storage
    • adding and removing ssh keys from an external cloud vm
    • provisioning tenancy sci vm managed cloud
    • sysadmin guidance external cloud
    • best practice
  • workflow management
    • rose cylc on jasmin
    • using cron
  • uncategorized
    • mobaxterm
    • requesting resources
    • processing requests for resources
    • acknowledging jasmin
    • approving requests for access
    • working with many linux groups
    • jasmin conditions of use
  1.   Interactive computing
  1. Home
  2. Docs
  3. Interactive computing
  4. Creating a virtual environment in the JASMIN Notebooks Service

Creating a virtual environment in the JASMIN Notebooks Service

 

Notebook  
Notebook  
Share via
JASMIN Help Site
Link copied to clipboard

Adding custom Python packages to Jupyter Notebooks

On this page
Step 1: Creating a virtual environment   Step 2: Making the notebooks service recognise your new kernel   Step 3: Using your new kernel   Other tips & useful knowledge   Activating an environment without it being a kernel   Can I install packages from inside my Python code?   Can I use conda instead of a virtual environment?   Can I get rid of my old kernels from the Notebooks Service?  

Creating a virtual environment is useful to allow a discrete set of extra packages to be installed to meet specific requirements. This allows a user to run multiple environments with different dependencies without conflicts.

There are a number of ways to create a virtual environment to use with the Notebooks Service. This document outlines the most common and recommended methods, and then some other ways which you might find useful.

Please note that environments created for the Notebooks Service will not work on the JASMIN scientific analysis servers or the LOTUS batch processing.

Step 1: Creating a virtual environment  

This step creates a Python virtual environment, and allows you to install packages into it.

 
These commands are intended for use at the Jupyter Notebooks shell, not on the JASMIN sci machines

To get started, open the JASMIN Notebooks Service  and in the launcher click the terminal button.

Opening the terminal
Opening the terminal
 

Don’t worry if you see this: this is a known issue but should not cause you a problem.

id: cannot find name for user ID NNNNN
[I have no name!@jupyter-user notebooks-misc]$ 

Then, type these commands at the bash shell which appears.

First, make a directory in which to store your virtual environments. You can put this wherever you like, as long as you reference the same place later. You could store several virtual environments within this directory, for different purposes. Then, change into that directory.

mkdir ~/nb_envs
cd ~/nb_envs

Next, create a new empty virtual environment. We recommended including the --system-site-packages argument which will allow you to add packages on top of jaspy, rather than starting completely from scratch.

python -m venv name-of-environment --system-site-packages

Then, activate the specific virtual environment created above, which will allow you to install packages.

source name-of-environment/bin/activate

If you want to be able to use your virtual environment as a Jupyter Notebook kernel (recommended), you should install ipykernel using pip.

pip install ipykernel

You can then install whatever packages you need in the environment.

pip install pyjokes

If you change your mind and need to add more packages in the future, it is simple to activate the virtual environment in the same way as above and use pip to install more packages.

Step 2: Making the notebooks service recognise your new kernel  

These steps are also run from the notebooks’ service shell, as above.

If you aren’t still there from the last step, cd to the location of your venv.

cd ~/nb_envs

If it isn’t already active, activate the virtual environment.

source name-of-environment/bin/activate

Running the following command will make the Notebooks Service notice your new virtual environment, and include it in the list of kernels which you can run code with. You only have to do this once.

python -m ipykernel install --user --name=name-of-environment

Step 3: Using your new kernel  

Select kernel, in this case: 'name-of-environment'
Select kernel, in this case: 'name-of-environment'

You can then choose this kernel from the Jupyter Notebook homepage, or from the top right of any open notebook. No changes to the Python code within are required.

Kernel name shown in notebook title tab
Kernel name shown in notebook title tab

Other tips & useful knowledge  

Activating an environment without it being a kernel  

If you follow Step 1 above to create a virtual environment, it is possible to use the packages from this environment in a Python file without making it a kernel. While this can be useful, it has the very distinct disadvantage of hardcoding the path to your virtual environment in your Python code. For this reason we discourage using this method with a medium level of severity. To do this, simply add the following code to your Python file before any imports. Adjust the venv_path variable to be correct for the venv you created.

import sys
import pathlib
import platform

venv_path = "~/nb_envs/name-of-environment"

py_version = platform.python_version_tuple()
sys.path.append(
    str(
        pathlib.Path(
            f"{venv_path}/lib/python{py_version[0]}.{py_version[1]}/site-packages/"
        ).expanduser()
    )
)

Explanation: this adds the site-packages folder from your venv directly to the path Python uses to search for packages ($PYTHONPATH). This lets Python find them to import.

Can I install packages from inside my Python code?  

We very strongly recommend NOT trying to install Python packages from inside notebook code. pip isn’t designed for it, and it is almost always easier to activate the venv as above and install things that way.

If you wish to record the set of packages inside your venv so you can install them en-masse later, pip has the facility to do this. To export a list of packages that exist inside a venv, from the notebook’s bash shell with the virtual environment in question activated:

pip freeze > requirements.txt

To install a list of packages which have been exported:

pip install -r requirements.txt

Exporting packages in this way is also useful for sharing your environment with others, reinstalling when it breaks etc. It’s a good idea to keep the requirements file alongside the code in version control. If your code becomes more complex it is probably more sensible to make it a Python package, and install it as one, but doing that is outside the scope of this document.

If you really must, you can call pip from inside your notebook like this: (after first updating the packages variable to be the ones you want to install.)

import sys
import subprocess as sp

packages = ['pyjokes']

sp.check_call([sys.executable, '-m', 'pip', 'install'] + packages)

Can I use conda instead of a virtual environment?  

Yes, no problem.

To create a conda environment, simply run the following at the JASMIN Notebook shell:

conda create --name name-insert-here ipykernel

Install any packages you which to use in the environment:

conda install --name name-insert-here pyjokes

Make the Notebooks Service recognise your environment as a kernel:

conda run --name name-insert-here python -m ipykernel install --user --name name-insert-here

Can I get rid of my old kernels from the Notebooks Service?  

Yes.

To list the names of kernels you have installed, run the following at the JASMIN Notebook shell:

jupyter kernelspec list

To remove one of them, run:

jupyter kernelspec uninstall insert-name-here
Last updated on 2024-10-08 as part of:  updates for r9 (992863c87)
On this page:
Step 1: Creating a virtual environment   Step 2: Making the notebooks service recognise your new kernel   Step 3: Using your new kernel   Other tips & useful knowledge   Activating an environment without it being a kernel   Can I install packages from inside my Python code?   Can I use conda instead of a virtual environment?   Can I get rid of my old kernels from the Notebooks Service?  
Follow us

Social media & development

   

Useful links

  • CEDA Archive 
  • CEDA Catalogue 
  • JASMIN 
  • JASMIN Accounts Portal 
  • JASMIN Projects Portal 
  • JASMIN Cloud Portal 
  • JASMIN Notebooks Service 
  • JASMIN Community Discussions 

Contact us

  • Helpdesk
UKRI/STFC logo
UKRI/NERC logo
NCAS logo
NCEO logo
Accessibility | Terms and Conditions | Privacy and Cookies
Copyright © 2025 Science and Technology Facilities Council.
Hinode theme for Hugo licensed under Creative Commons (CC BY-NC-SA 4.0).
JASMIN Help Site
Code copied to clipboard