DUG is a Perth based HPC as service company located in West Perth.
It provide some resources for us to do the research, which includes access to A100 GPU.
To access that, you will first need to contact them to setup your accounts and projects. In this step, you will need to provide your ssh pub key to them, you can generate a new one if you want.
Login
To login, you will need to load your provided ssh key first via the command
ssh-add ~/.ssh/your_rsa_key # not the .pub one
Then you can update your ~/.ssh/config
Add this chunk
Host dug
HostName mcc_uwa
User your_username
ProxyJump your_username@perth.mccloud.dug.com
IdentityFile /path/to/your/dug_rsa_key
After this you should be able to login to the login node via ssh dug
It will prompt you a password requirements, type it in, then you are in.
Run on the HPC
Keep in mind with the architecture diagram above, you do not have much storage places inside the login node (Which is 10GB for DUG here), so you will need to go to your data directory in DUG. Similar to the concept for Kaya, you will need to go to sbatch or group directory.
Create conda env
This example is provided by Kai
module load conda # load the conda first
mkdir -p /data/uwa_multimodality/your_directory/env/conda
conda create -p /data/uwa_multimodality/your_directory/env/conda/text2video-finetune python=3.10 # specify env install path
conda create --name ENV_NAME --clone /data/uwa_multimodality/your_directory/env/conda/text2video-finetune
# you should not run exactly the same code as this, need to change based on your situation
This will create the conda environment under your data directory
Also you will want to get the conda init every time your login automatically, add the below section into the .bashrc