Managing Files and Data in the SHS GPU Cluster
What is the BeeGFS?
BeeGFS (the BeeGFS Parallel File System) is a high-performance, distributed file system designed for environments requiring fast and scalable I/O operations; it is ideal for GPU-accelerated workloads in HPC settings. Within the EPCC SHS, BeeGFS is mounted using the BeeGFS CSI Driver, which enables dynamic provisioning and access to BeeGFS volumes inside containerised GPU jobs running on Kubernetes.
This approach ensures secure, performant, and flexible access to shared datasets across all GPU compute nodes.
Key Point
BeeGFS is deployed within the SHS. Therefore, the same restrictions apply: BeeGFS storage is
fully isolated from the internet. You cannot download data directly from public sources
(e.g., GitHub, external APIs), and copying, recording, or extracting any files from BeeGFS
outside of the SHS is strictly prohibited unless explicitly approved through the appropriate
Information Governance processes. It is also prohibited to use the BeeGFS to share data with
other users or with projects to which you have access.
Storage Overview
The BeeGFS client is installed on the shs-sdf01 VM, where the file system is mounted at /mnt/clstr-beegfs. This VM (shs-sdf01) is used to synchronize data between the desktop VM environment and the BeeGFS file system.
This setup allows users to prepare and transfer code and datasets between the project space and BeeGFS, making them accessible to GPU jobs through Kubernetes Persistent Volumes (PVs), which are directly provisioned via the BeeGFS CSI driver.
There are three main file storage environments, as follows:
-
Desktop VM
/homeFile System Accessible only when logged in via the remote desktop. Local to the VM and not backed up. -
SHS GPU Cluster BeeGFS Mounted directly on GPU compute nodes using the BeeGFS CSI driver. Provides high-performance, parallel file access for compute jobs. BeeGFS data can be synchronized from the desktop VM via the
shs-sdf01node, where the BeeGFS client is installed. This storage environment is declared in Kubernetes job definitions using Persistent Volume Claims (PVCs). -
Project Data in
/safe_dataAccessible only from the desktop VM. Backed up and used for long-term data storage. Not accessible from GPU compute nodes. It has slower I/O performance compared to BeeGFS.
Important Note
The /safe_data file system cannot be used directly within GPU jobs. Instead, synchronize necessary data to BeeGFS volumes for compute workloads and transfer results back to /safe_data if long-term storage or backup is required.
Accessing the BeeGFS in the SHS GPU Cluster
Logging into the GPU File System
Users can access the BeeGFS shared file system by SSH-ing into the shs-sdf01 node from their VM desktop terminal. This is used to prepare or synchronize data for GPU workloads.
Hello World Example
On the VM desktop terminal:
ssh shs-sdf01
<Enter your VM password>
echo "Hello BeeGFS World"
exit
BeeGFS vs VM file systems
The BeeGFS storage is mounted on GPU compute nodes and is separate from all following data locations:
- The VM file system (e.g. your desktop environment),
- The project data space (
/safe_data), - Any local
/homedirectories.
To make files available for GPU analysis jobs, they must be explicitly transferred between these environments.
Example showing separate BeeGFS and VM file systems
cd ~
touch test.txt
ls
# test.txt is visible on the desktop VM
ssh shs-sdf01
<Enter your VM password>
ls
# test.txt is NOT here (VM and BeeGFS file systems are separate)
exit
# Use scp to copy from desktop VM to BeeGFS-accessible path
scp test.txt shs-sdf01:/mnt/clstr-beegfs/<safe_heaven>/<project_id>/users/<username>/
ssh shs-sdf01
<Enter your VM password>
ls /mnt/clstr-beegfs/<safe_heaven>/<project_id>/users/<username>/
# test.txt is now here in BeeGFS
Example copying data between project data space and BeeGFS
Transferring and synchronising data sets between the project data space and the BeeGFS is easier with the rsync command (rather than manually checking and copying files/folders with scp). rsync only transfers files that are different between the two targets; more detail is available in its manual.
# BeeGFS Sync Workflow
## Syncing Project Data with BeeGFS
**On the VM desktop terminal:**
man rsync # check instructions for using rsync
# Sync project folder to BeeGFS mount point on shs-gpucl-fs01
rsync -avP /safe_data/my_project/ shs-sdf01:/mnt/clstr-beegfs/<safe_heaven>/<project_id>/shared
# Conduct analysis on GPU cluster using Kubernetes jobs accessing /mnt/clstr-beegfs/<safe_heaven>/<project_id>/
# After analysis, sync results back to /safe_data (if needed)
rsync -avP shs-sdf01:/mnt/clstr-beegfs/<safe_heaven>/<project_id>/users/<username>/ /safe_data/<my_project>/results/
Optionally remove the project folder from BeeGFS if no longer needed.