NASA Technical Reports Server (NTRS) 20060019116: Information Power Grid (IPG) Tutorial 2003

39 Pages·36.9 MB·English

by  NASA Technical Reports Server (NTRS)

Source of Acquisition
NASA Ames Research Center
!= B
IPG Tutorial Introduction
~ ~~~~
This tutonal is designed to pronde intonation for both Lhse new to Grids, and
Information Power Grid those who wish to incorporateG nd Sewes into hira pplrntlons
Tutorial
wha-t w e mtl cover'
In.t roduction
WhatareGridJ?
February 4,2003 *. Current State 01 IPG
C%ewlewolGndMiddlemvs
- FutureDireChm c_
U-s ing the Gnd
Crowne Plaza Cabana - Prerequisites
Hotel - Using Basic Componentso f the Gnd
Accessing Gnd Servms from an applmhon
Palo Alto, CA - P.ro gramming Models In Use
. Parameter Studies
- Multi-DIswphMry
Data-Mining
- Data Management With hS RB
B BZ
Our Theme Grids
W
- -
I'm a scientist who has Grids provide the infrastructure
-
- a design to study to.. d yCnMalmpuictea rlelysc counrmglos m(gesrnaeterk i nandde pueninqdueen) ny managed:
- a computer code to run a simulation . Data sources (static and live)
- Scientific lnsrmmenk (wmd Tunnels. Micmsmpes. Simulators. eh.)
data about the design object -
- to. b uild large scale mllaborative problem solving environments that are:
- heard about the Grid and I'm curious . msleffeclive
- socure
Tell me more!
Grid software is "middleware"
Grids are made up of a collection of independent
resources (computers, storage, instruments, networks,
etc) that can be tightly or loosely coupled, to solve a
particular problem, as needed, via grid software
1
B Characteristics Usually Found in
Why Grids? EE
Grids
-
For NASA and the general community today Grid An underlying security infrastructure such as the Grid Security
Infrastructure (GSI). which is based on public key technology
middleware: -
- Protectionf or at least authentication informationa s it flows from resourcB
provides tools to accesshse data sources (databases, - to resource
- instruments, ... ) Readily accessible information a h tt he resources on Me Grid via a
provides tools to access computlng (unique and generic) - single mechanism, the Grid Information Service (GIS)
-
Is an enabler of large scale collaboration Single signa
Dynamicaliy responding to needs is a key selling point of a grid. - A seamless processing environment
Independentr esources can be joined as appropriate to solve a - An infrastructure mat is scalable to a large number of resources
problem.
- The ability for the grid components to cross administrative boundaries
Pmvides tools for development of Frameworks
Provide tods to enable the building of a frameworks for appllcatiow.
Provide value added service to the NASA user base for utilizing
msourm on the gnd in new and more efticient ways
B
W Before the Grid kP- W First Step to the Grid .
-
User I - independent sites User M- etacenter
ihnadredpweanrdee anntd nI Application I Trewsoouo rrcmeos rceo nnected
- software , in a controlled user
envimment
- independent user ids A layer of abstraction is added mat hides some of
secunty policy the complexities associated with running jobs in a
resolving the complexities of requiring local distributed computing environment. however. Constraints
mcoancnheinceti.o n to the limitatio;, exist - scionmglme onna marec hsipteaccteu re
common scheduler
2
c
1
!=
The Grid Today
DRcquest info Common Middleware
from heg nd -
abstracts
%trrsponsc independent, APls
BMMake selccuon huaserdr widasr, ei,n stoof atw are, Resswerrvciceess a trhea at caccecsessesd i nvfiraa svtarurioctuusr ein AtePlllisg ent -. automatic selection o,
and submit lob
service layer wth resources
daeddfiende d APls DTehvee lorepseurl ct:a Tnh feo cSucsie onnti sstc aienndc Ae papnldic naotito onn 1. infomatiolon users
comprehensive > systems management accwnltess
secuntv,
allows for site
autonomy
-
provides a common
infrastructure based
on middleware
-.._ -.-
I ,
A .
k
How the User Sees a Grid IPG Overview
w
A set of grid functions that are available as Persistent stuff
-
Application programmer interfaces (APls)
- Grid computing software
- Command-line functions
After authentication, functions can be used to - Globus
-
Spawn jobs on different pmcesscfs with a single command
- Access data on remote systems
-
Move data from one processor to another
-
Support the communication between pmgrams executing on
different processors
- Discover the properties of computational reswrces available on
the grid using the grid information service
- Use a broker to select the best place for a job to fun and then
negotiate the reservation and execution (coming soon).
3
B
IPG Components
* Globus 2.0
-
G-r id Information Sewices (GIs)
Monitoring and Directory SeMca (MOS 2.0)
-
D-a ta Transfer:
- Servers: GridFTP. GAS. gsissh
- Clients: gsi-ncllp. gsi-scp
Portal
- - Launchpad
Certificate Authority
- - local to IPG
Data Management
- - storage resource broker
IPG Services
-
Resource Brcker vl
Langley Research Center - Job Manager vl -.-
Y
Bf
!= B
State of the IPG W Globus Toolkit
- .
Server Nodes Grid computing middleware
- 1024 node, single system - 24 node SGI 02K, Glenn - Software between the hardware and high-levels ervices
- image SGI at Ames Research Center - - Basic libraries and services
- 512 node SGI 02K. AmeS - 24 node SGI 02K. Ames Most common middleware used in grids
256 node SG0I2 K. Ames - 16 node SGI 02K. Langley Primary implementation in C
- - All libraries. command-line programs. dient and SBNer
Secondary implementation in Java
-
Java COG
-
Client Nodes - Many of the dlent libraries
- 16 node SGI 0300, JPL Current version is 2.2.3
- - 4 node SGI 0300, Goddard -- 2.0 Installed on many IPG systems
- Wide area networkinterconnects of at least 100 MbitLs 2.2.3 installed on a few
http2hww.globus.org
Storage resources: 50-10 0 Terabytes of archival
infomation/data storaqe uniformly and securely
,...-
accessible from all IPG sys!ems
4
Globus Deployment
k
Arc-re
w
h B
Current IPG Services Current IPG Services
F==-
* JobManager
- R..el iSabetly of e fxileesc utot ep ae -jsotba ge
Executabletoru"
. - I-&* dkcaly. nvimnmsnl vll,iai.b*l
set Of mes to pmt-stage
* R-e source Broker
Provide suggestions on where to run a job
- Input
.. Which W and 0% am aaeptable
How to create a Job Manager Job for a selected host
- Se.l ection made using host and OS constraints and host load
. lnteracbve system:I free CPUs
Bat& system: Amunt of work in queue I X CPUs
- ou.t put
Ordered list of Job Manager Jobs (suggest& systsms)
5
Open Grid Services
B
Ongoing Work
Archihx2,ur-e
.
Open Grid Services Architecture . New framework for creating grid SEWiCES
B-a sed on web services
Globus Toolkit version 3 St.a ndards m build and use distribuled seMces
. sa- eesmwm Ianguag.: WSDL
IPG Services . sW,im.Iwma~wm:SOAP
. locatbn: UDDl (rm u dk OGSA)
Data Grid OGSA extends web sewices with
-
Requirements for ~w\na~,r m)rfaxssu ch as providing Sewice data and
MhfiCatKlnS
-- Se~cmea nagement (ueaW~d.n smmon. lifetimes)
Secunty
*. Implementeda top Apache Axis
Standardizingi n me Grid Forum
- Architwre: mtp1hmw.ggf.orgIossa-w
- Implementahon:h ~/hmw.gridtorum.org/SARCHffiSl.hbn
-.-
E
IPG Services
Globus Toolkit Version 3 (GT3)
W
.
L--a rgJEaenv tcairhe-balyan nsgeeedw fi nrimosmtpele aGmdT oef2n C ttao-tb ioGansT e3d .- GIn--ot eaUlRll:ig seLeesoron-cust arrpceteeiso aconfhiuieanrdrcdpa aerce tspefeeerirlsneetidcnctcesioe.n asntaca eonsdms rpe, uqautllinoirecgam tieonntss. cost.l oad. peffoormancew edidions
--- AG-lTp h3Ga wT v3ile lb rpasrsiooevndi d ooenf eOGqGTu3Siv Aca ulerrnet nstelyr vaicveasila tob leG T2 . C--r e aGPlibtiiecvrake sr ndiye iarsset stceoytom smntedeomesv,e p (s,o eesrn tp spdeyasaettfnheystm jw o ashbn)icd lofh re oUdmmJaOit ma s yn(Ismlstese mlmo .-m SinpodeweCp ilye wndhechn te joxbe atablesa nd
IPG won’t transition to GT3 soon (guess a year) . R-e liable execution of systemdependentj ob
- Need version 1 and time to evaluate Applicabon exdonh file management
T-ra nGsloitbiouns wshilol muladi nhtaaivne m maniny iomf athl eim epxaiscttin ogn p roougrr aumsse rs D--y nCRamhunai crag apincpgci riedesmosn uasrcn Wsd u imasceoc utoto au re npmteinromgtea ancsmnt luonctainlg m sYu;tnemls
. - IPG Services will switch to GT3 in a transparent manner . - Resourceplang
hQxJlwww.globus.wg/ogsa . W-o rSkpfleocwify m aannd aegxeemarenn Wt WMm
Implemented as OGSA sewices
-.-
A.
6
B Biz
Data Grid Outline
.
- Goal: lntelllgenlfy manage data sets in a grid
Ales identifiedb y logical file names - In.t roduction
* F-il es have physical locations . What are Grids?
- CalledrWcas . Current State of IPG
- LUoscsaf otorn W of rceapnik masa isw mQ a idnetaleintee drs plicas . OFuvteurrveie Dwir oefc tGiornids Middleware
- Replicas can ta rslected intelligedy - Using the Gnd
- Replims CM ta managed intellgsnaV * Prerequisites
* Maintain colleclions of files *- Using Basic Components of the Gnd
* Maintain metadata about files Accessing Gnd SeMces horn an applicatton
* A-m ss using libraries and mmmand-llnet ocis - P-r ogramming Models In Use
May or may not reremble Unu file systsm wage - Parameter Studies
,. * Hear more during Storage Resource Broker (SRB) session of tutorial - Multi-Disciplinary
. * Important new area for the IPG Data-Mining
- Data Management Wh heS RB
ks B
Our Theme USING the GRID
-
I’m a scientist who Prerequisites
- Has a design to study - Getting an Account
- An application to run a simulation - Getting a Certificate
- Data about the design object How to Use the Grid
- - I have an idea of what a Grid is and want to use it - Understanding how it works
Now what do I have to do? - Basic Submission command-line
- File Management
- - Finding Resources on the Grid
Adding IPG services to Applications
- The IPG Job Manager
- The IPG Job Broker
m.._
Y
7
Bl
Prerequisites Getting an Account
W- hat I need to make use of IPG Services Easiest way is to use the on-line account request
Get an account form at:
Create an IPG Certificate
Learn how to use and access Grid Services -
- stay tuned for the rest of this tutorial Ames, Glenn, and Langley accept approved
requests from this form.
JPL and Goddard are just joining as client sites
and have separate forms at this time.
b
Group ID I’ve Been Approved! !!E
F““ W
- -
The account request form requires a Group ID or - I received information about my accounts
GID - I can log in
- Obtained from a Principle Investigator (PI) - Select a Grid system to be your “Home” system
-
Approves your account request
- Provides a group ID (GID) Terminology: “Grid System”, or Grid-enabled
system is one that is running Grid middleware.
To become a PI - At a minimum my “home” system should have:
-
Write a proposal for a Grid Related Project
*- The Globus client comm‘ands
-
Contact the NASA Center you are most likely to be - The Grid Security Infrastructure (GSI) libraries
affiliated with - Allows interactive login
Will hold my credentials
8
rn
What’s Next Getting a Certificate
-
Ok, The next several slides describe:
- I know what a grid is - What a certificate is
- I have my amunt - How to request one
- - What a proxy is
Now what do I have to do? - How to generate a proxy
- Next I need to get a certificate (my credentials)
What is a Certificate?
* A certificate is like a passport; A Request the certificate with ipg-cert-request command
establishes your identity - Example:
* A certificate contains: Yourofficial credential %. .i.p g-cert-rwest
- dMisyt iNnagmuies,h ceadln leadm thee subject name or issued by a ccmfied agent - Pr.o mpts lo verify information obtained from the system:
a- Name
- My PuMic Key . Organizationala ffiliation
Center Affiliationo f the system on which I execute the command
- MyPrivateKey - Requests a pass phrase to generate and enaypt the private key
- T(Ch.Ae ) i dtheantt sityig onfe tdh eth Ce ecreiirftmifictiaotne Authority - Ernail address to mtify me when the certificate is ready fw retrieval
A CA is similar to a mrtlfication
* oTcerhgretaif iCnciaAzta’est iodcnieg rittlafiwl ssig mnea tvuarelid inny m oef me Your official credential I -me public key and certificate request are sent to the CA for signing
cBrtl6cal.3 issued by a certified agent
,..-
9
I k
Getting My Certificate Part 2 Display Certificate Information
- -
I
I've got mail! Instructing me to execute the command Use the grid-cert-info command to display
ipg-cert-retrieve your certificate information
- Included in the mail is the exact format of the command I should - For example:
execute to get my signed certificate
- Example: % grid-cart-info -all
Issuer: -rid, OINational Aeronautics and
% ipg-cart-retrievs -id Ox14b Space Acbninistration, OU=Ames Research Center,
CNICertificate Manager
* A- d ire-c tory named .globus is created Validity:
In- i t you will see two files: Not Before: N m 20 20:30:18 2002 BIT
us*rcert.psm- is yourpublic key Not After : N W 20 20:30:18 2003 GMT
- userkey.p am - if ywr private key Subject: O-Grid, (tNati0~1A eronautics and
Space Acbninistration, OW-Arms Research Center,
Grid Administrators are noMied lo add you lo a file that maps your C.N=.G.ao rge 8. Myers
subject name to your user ID on all systems you have access to
-
More information at:
http://.m.ipg.n~a..gov/ipyaera/g.ttingst.rt.dlc.rt_top.hhnl
" -<-.
Summary he What's Next?
-
OK,
- Got my accounts,
- got my certificate,
- - I'm ready to go!
Not so fast ...
- we want to introduce the concept of delegation.
- The idea is to give processes on the remote system to
act on your behalf.
- In Grid Security Infrastructure this is done with what is
called a proxy.
10