How to set up the Unified Model for your first climate simulation


The UK Met Office's Unified Model is the atmospheric part of the ACCESS model, it was used by CAWCR for the AMIP and CMIP model inter-comparison projects and is the main atmospheric model used by BoM for weather forecasting.

Researchers at the CoE access the UM by logging in to the supercomputers at the National Compute Infrastructure (NCI). NCI hosts a range of computing facilities, from supercomputers to data archiving & analysis, and is used by a wide range of researchers in different fields from across Australia.

Step 1: NCI Account


Obtain a NCI account and get connected to a compute project so that you can log on to the supercomputer

Step 2: ACCESS Registration


Due to licensing restrictions the UM is not available to just anyone with an NCI account. Join the ACCESS project to get access to the model.

Step 3: Connect to NCI


Once your NCI account is set up and you've been registered as an ACCESS member you can start using all these accounts by connecting to the supercomputer. To do this run:
ssh -Y abc123@raijin.nci.org.au
from the command line, substituting abc123 with your own username.You can update your NCI password by going to https://my.nci.org.auTry running
qstat
This gives you the full list of what everyone's running on the supercomputer, usually there's a bit under a thousand different jobs running at any one time.Other helpful commands to know are
nqstat
nci_account
lquota
nqstat shows the running jobs for only your own project, which is in general a more manageable list. nci_account show you what resources are available in your current project for compute time and storage, lquota shows you how much space is available in your home directory.

Step 4: Setting up Accessdev


The user interface for the UM is hosted on a server called accessdev. You can connect to accessdev by running:
ssh -Y abc123@accessdev.nci.org.au
If this does not work, email climate_help so we can check your account has been added properly.

The UM's source code is stored in a subversion repository. Building the model happens automatically, however the system needs to be able to see the source code. Check this works by running
svn info https://access-svn.nci.org.au/svn/um
If you have any trouble accessing this repository, email climate_help - your account may need to be given read permissions. The repository will ask to store your NCI password in plain text. Agree to this, otherwise the automatic build system won't be able to get access.

To download the model source code the UMUI will need to access your NCI password. Your password is stored securely using a program called `gpg-agent`. To set this up, run
accesssvn-setup
This will set up gpg-agent for your account, but you'll need to log out of accessdev and then back in again in order to start it up. Once it's started, you can save your NCI password with the command
access-auth
Passwords are saved for 12 hours.

The UM's interface is on accessdev, however the model actually runs on the supercomputer. To make this work you need to set things up so that accessdev can talk to the supercomputer, which is done using SSH keys. These allow you to connect to another computer without having to enter your password every time - for the details on how to do this see the wiki entry on SSH Agents.

You'll also need to set the same DATAOUTPUT variable you created on Raijin. If you use csh add to ~/.login:
setenv DATAOUTPUT /short/$PROJECT/$USER/UM_ROUTDIR
or if you use bash add to ~/.bashrc:
export DATAOUTPUT=/short/$PROJECT/$USER/UM_ROUTDIR

Step 5: The UM User Interface


With all of this setup out of the way you can finally start the UM's user interface and run a model, by running
umuix

devumui.png

You'll be greeted with a table showing a list of UM experiments. A couple are available by default, these are what we call 'standard runs', commonly used & tested jobs set up to be simple to use and base new experiments off of. If you press the 'All' button at the bottom of the window you can see everybody's UM experiments, press 'User' to go back to just your own.

The majority of existing runs in the database have been set up for NCI's old Vayu supercomputer. Converting runs is pretty easy, but for the moment we'll look at runs that have already been set up to run on Raijin.

The usual workflow with the UM is to copy an existing job, modify it to your liking then submit it to the supercomputer. To start off with we'll forgo most of that and try submitting one of the standard runs. Click the folder next to 'vabg' to show the model runs available.

dev-vabg.png

At the moment this experiment has the AMIP configuration of ACCESS version 1.3 as well as a low-resolution NWP test. You can get more details on a model run by clicking on the notepad in the leftmost column. Open up the N48 experiment a, either by double-clicking or selecting it and pressing the 'View...' button.

dev-vabga.pngi

The left panel of the interface shows configuration sections, the right panel shows the available settings within each section. Feel free to take a look at what's available.

To submit a run you first need to create a set of job scripts, do this by pressing the Process button on the right. You need to do this every time you make a change in the settings. Once the processing is done press Submit to send the job to the supercomputer.

A UM model run happens in three stages. First the code is downloaded from subversion in the extract stage, then the model is compiled in the build stage, then the actual model run happens. The model is usually rebuilt for each experiment because the model has many different science sections, enabling or disabling these changes what code needs to be added to the model. Usually building the model takes about 20 min, the model's run time depends on how many processors you're using and how long a model it is.

To find the model's output you'll need to go back to Raijin. Check the run's status with nqstat, when it's done the output will be in the folder you created earlier:
ls /short/$PROJECT/$USER/UM_ROUTDIR/$USER/vabga

You can also find run logs in the folder ~/um_output.