Artemis User Guide Documentation

27 Pages·2017·0.19 MB·English

Artemis User Guide Documentation
Release 1.0
Stephen Kolmann
Mar 13, 2017

Contents:
1 GettingStarted 1
2 StorageonArtemis 5
3 AccessfilesinafileexplorerusingSSHFS 7
4 Interactiveaccesstoyourallocation 9
5 Openingfreeview,matlab,etc. 11
6 AddingMatlabextensions 13
7 TransferringdatafromRDS(RCOS)toArtemis 15
8 SubmittingJobstoArtemis 17
9 Jobmonitoring/management 19
10 Parallelprocessing(DRAFT) 21
11 Glossary 23
i
ii
1
CHAPTER
Getting Started
Access to Artemis
For instructions to obtain an Artemis account, see the following page: http://sydney.edu.au/research_support/hpc/
access/index.shtml
Specifically,touseyourallocation(alloc-op),jointheprojectcalledFFbigdata(RDS-SMS-FFbigdata-RW).Youwill
needtoarrangeforsomeonetoupdateFFbigdata’sRDMPandaddyouasacontributor.
Projects and UniKeys
UniKey
OnceyouhavebeengrantedaccesstoArtemis,youloginusingyourUniKeycredentials. Throughouttheguide,we
usetheexampleUniKeyabcd1234wheneverweneedtospecifyaUniKey. Remembertoreplaceabcd1234with
yourUniKeywhereverneeded.
Project
YourprojectnameisFFbigdata. RememberthisprojectnameasyouwilluseitoftenonArtemis. Inrarecircum-
stances,youmayneedtoknowyourfullprojectname,whichisRDS-SMS-FFbigdata-RW.
Linux Command Line
Artemis is a Linux-based HPC with a command-line interface. If you have no background in Linux or the Linux
commandline,thereareanumberofexcellenttutorialsonlinethatwillhelpyougetstarted. Arecommendedstarting
tutorialcanbefoundhere. Onceyoufinishtutorial4,youwillhaveenoughcommandlineknowledgetostartusing
Artemis.
1
ArtemisUserGuideDocumentation,Release1.0
Logging into Artemis Access to Artemis is via SSH (Secure Shell). SSH requires you specify a username and a
hostname to log in. The hostname of Artemis is hpc.sydney.edu.au. Therefore, the command to log into
Artemisis:
ssh [email protected]
Forthisproject,youneedtousegraphicaluserinterface(GUI)programsonArtemis,soitisbesttologintoArtemis
usingtheinstructionsinthenextsection.
Opening GUI windows
Itispossibletoopenprogramswithagraphicaluserinterface(GUI)onArtemisifyouhaveanXserverrunningon
your computer and connect to Artemis via SSH with either the -X or -Y options. Programs on Artemis with GUIs
includegedit(atexteditor),Matlab,freeview,andqdec.
MacOS
OnMacOS,youneedtoinstallXQuartz. AfterinstallingXQuartz, openaterminalwindowusingMacOS’sbuilt-in
terminalprogram,thenSSHtoArtemisusingthe-Xor-Yoptions.
ssh -X [email protected]
Windows
On Windows, I recommend installing MobaXTerm: http://mobaxterm.mobatek.net/. After installation, you can log
intoArtmeisbyopeningMobaXTerm,clicking“StartLocalTerminal”,thenloginviaSSH:
ssh -X [email protected]
To get the best performance from GUI windows, use a computer with a graphics card. You can tell MobaXTerm to
use your graphics card by clicking Settings in the top menu bar, then Configuration, then click the X11 tab and set
OpenGLaccelerationto“Hardware”.
Linux
IfyouareonaLinuxcomputer,suchasUbuntuorLinuxMint,thenyouarealreadyrunninganXserver. Therefore,
youcanopenprogramswithGUIsifyoulogintoArtemisusingthe-Xor-Yoptions:
ssh -X [email protected]
Text Editors
ItispossibletoedittextfilesonArtemisusingtexteditorssuchasNano, GEdit, VimorEmacs. Forpeoplenewto
Linux,IrecommendGEditIfyoufindyourselfeditingtextfilesfrequentlyonArtemisandwanttoinvesttimetolearn
amorepowerfultexteditor,thenitisworthusingeitherEmacsorVim.
To use GEdit on Artemis, you can simply type “gedit &” while logged into Artemis (assuming you have logged in
usingtheinstructionsintheprevioussection):
2 Chapter1. GettingStarted
ArtemisUserGuideDocumentation,Release1.0
[abcd1234@login4]$ gedit &
1.5. TextEditors 3
ArtemisUserGuideDocumentation,Release1.0
4 Chapter1. GettingStarted
2
CHAPTER
Storage on Artemis
ArtemisusesaLustrefilesystemthatisaccessiblefromanycomputenodeandhasexcellentI/Operformance(espe-
ciallyforlargefiles). TheLustrefilesystemcontainsthefollowingstoragespaces:
• Home(/home/abcd1234)10GBperuserquota.
• Project(/project/FFbigdata)5TBperprojectquota.
• Scratch(/scratch/FFbigdata)Noquota. SharedbyeveryoneonArtemis.
Datawillnotbebackedupsowecannotofferanyabilitytoretrievelostdata. Forlongtermstoragepleasestoreall
dataintheResearchDataStore(RDS).
/home/abcd1234
Each researcher has their own, personal home directory on Artemis. This can be used to store program code, batch
scriptsandotherfiles. Pleasenotethatthehomedirectoryhaslimitedspace(10GB)andisintendedforstoringcode
andconfigurationinformation.
Tocheckyour/homequotausage,runthefollowingcommand:
pquota
/project/FFbigdata
FFbigdatahas5TBof/projectstorage,sharedbetweenallmembersofthisproject. Ifyourequiremoredisk
space,use/scratch/FFbigdatainstead.
Tocheckyourquotain/project/FFbigdata,usepquota.
5
ArtemisUserGuideDocumentation,Release1.0
/scratch/FFbigdata
The /scratch directory has approximately 204 Terabytes of space. To see how much is available, use the following
command:
df -h /scratch
Toseehowmuchspaceyourprojectisusing:
lfs quota -hg RDS-SMS-FFbigdata-RW /scratch
Fordetailsaboutdiskusagebyspecificsubdirectories,usethiscommand:
du -sh /scratch/FFbigdata/directory
wherethelastargumentisthedirectoryyouwanttoknowthesizeof.
File Permissions
ThedefaultfilepermissionsforfilesonArtemisare:
• usershaveread/writeaccesstofilestheyown
• yourgrouphasreadaccesstogroupmember’sfiles
Ifyouwouldlikegroupmemberstohaveread/writeaccesstoyourfilesonArtemis,runthiscommand:
chmod -R g+rwx directory
wheredirectoryisthedirectoryyouwanttochangethepermissionson. The-Rflagwillrecursivelychangethe
permissionsofallfilesandfoldersinsidethedirectory.
6 Chapter2. StorageonArtemis