Skip to content

Overview

The EPCC SHS GPU Cluster provides access to full Nvidia A100 40GB GPUs. The SHS GPU Cluster is built upon Kubernetes and only accepts containerised workloads.

Current Configuration

The full specification of the SHS GPU Cluster as of October 2025 is as follows:

  • 672 CPU Cores (AMD EPYC)
  • 2.54 TiB Memory
  • BeeGFS Storage upto 1.3 PiB
  • 24 Nvidia A100 40GB (Full GPU, no MIG)

Quotas

This is the full configuration of the Cluster.

Each project will have access to a quota across this shared configuration.

Changes to the default quota must be discussed and agreed between the Safe Haven Information Governance function and the EPCC Service Manager for the Safe Haven in question and may incur an additional cost.

SHS GPU Cluster Access

Early Adopters

The EPCC SHS GPU Cluster is currently open to early adopters. Please contact the Research Coordinator team of your Safe Haven to discuss.

Users should have a SHS Account as the SHS GPU Cluster is only accessible through Safe Haven Virtual Machines.

Research co-ordinators can request that specific user-accounts have access to the SHS GPU Cluster through a service request to the their dedicated helpdesk.

When submitting a request, the Research Co-ordinator must provide:

  • Project/Study name
  • VM name from which access to the cluster will be granted
  • Username
  • Number of GPUs required (by default, 4 GPUs are assigned)

Each project will be given a namespace within the SHS GPU Cluster to operate in.

This project namespace will normally be formatted as the SHS Safe Heaven and Project code appended with "ns", e.g. nsh-2024-0000-ns for the nsh Safe Haven project with code 2024-0000.

Once access to the SHS GPU Cluster has been confirmed, SHS project VMs will be configured to use the SHS GPU cluster. How to access VMs is described in Safe Haven Services Access.

Project users can access the GPU Cluster using their FreeIPA credentials. When executing a kubectl command, if no valid token is cached locally, you will be prompted to enter your FreeIPA username and password. Upon successful authentication, a token is generated and cached locally, remaining valid for 90 days or until it is deleted. Once the token expires, kubectl will prompt you again for your FreeIPA credentials in the same format.

Authentication Prompt

You will first be asked to select the authentication provider — choose **freeIpaProvider**
Auth providers:
0 - localProvider
1 - freeIpaProvider
Select auth provider:

All user accounts with the project kubeconfig file downloaded can access the SHS GPU Cluster using the kubectl command line tool.

A quick check to see if a user has access to the SHS GPU Cluster can be completed by typing kubectl -n <project-namespace> get jobs into the command line.

If this is the first time you have connected to the GPU Cluster the response should be No resources found in <project-namespace> namespace.

SHS GPU Cluster vs SHS GPU-Enabled VMs

The SHS GPU Cluster is a container-based service which is accessed from SHS VMs.

This allows a project to access multiple GPUs on the GPU Cluster, unlike a GPU-enabled VM, which is limited to a small number (1–2) of GPUs of a single type.

VMs enabled for the SHS GPU Cluster need not have attached GPUs.

Project Quotas

A standard project namespace has the following initial quota (subject to ongoing review):

  • CPU: 100 Cores
  • Memory: 500 GiB
  • GPU: 4

Quota is a maximum on a Shared Resource

A project quota is the maximum proportion of the GPU Cluster available for use by that project.

Any submitted job requests that would exceed the total project quota will be queued.

Project Queues

The SHS GPU Cluster has been using the Kueue system since May 2025. The use of this is detailed in our Kueue documentation.

Job Queuing

During periods of high demand, jobs will be queued awaiting resource availability on the GPU Cluster.

As a general rule, the higher the GPU/CPU/Memory resource request of a single job, the longer the job will queue before enough resources are free on a single node for it to be allocated.

Furthermore, a project may have a quota of up to 12 GPUs but due to demand may only be able to access a smaller number at any given time.

Additional Service Information

Additional information on the service can be found in the following pages:

SHS GPU Cluster Tutorial

This tutorial teaches users how to submit tasks to the SHS GPU Cluster. It assumes knowledge of Kubernetes (see below for further reading).

Lesson Objective
Getting started with Kubernetes a. What is Kubernetes?
b. How to send a task to a GPU node.
c. How to define the GPU resources needed.
Building and Testing Containers a. How to build container images.
b. How to test containers locally.
c. Best practices for container setup.
Using Containers in the SHS a. How to run containers inside the SHS.
b. Security considerations for SHS containers.
c. How to manage container dependencies.
Creating Your First Pod Template a. What is a Kubernetes pod template.
b. How to define pod resources.
c. How to submit a pod to the GPU cluster.
Accessing the BeeGFS Inside the SHS GPU Cluster a. How to access BeeGFS from the SHS GPU Cluster.
b. How to read/write data efficiently.
c. Best practices for using shared storage.
Requesting Persistent Volumes a. How pods use persistent storage (Persistent Volumes and Persistent Volume Claims).
b. How SHS GPU Cluster users access pre-provisioned storage via Persistent Volume Claims.
c. How to attach Persistent Volume Claims to pods using volumeMounts and proper securityContext.

Further Reading and Help