An Introduction to the

By: Roman Olschanowsky
       roman2u@sdsc.edu
 Outline
SDSC and History of SRB
–Example Project

Introduction to SRB
–Discussion on SRB basics
–SRB Clients

Overview of a Data Grid

–Infrastructure
–Topology

Teragrid Demo
–How to use your TG SRB account
–How to access Digital Data Collections
www.sdsc.edu 
Human infrastructure:  
Experienced multi­
disciplinary staff support a 
broad spectrum of national 
science, engineering
and technology projects
Archival
Systems

DataStar Blue Gene/L

18 PB (Due 12/04)
IBM Power4
Networking
2.8/5.7 TF
15.6 TF Visualization
Storage
 and Compute
Resources

TeraGrid Linux
Storage Area Cluster (IA64)
Network Disk Sun F15K
Disk
1.4 PB Server 4.4 TF
Sites Using the SRB
Academia Sinica, Taiwan
ASCC, Computing Centre, Taiwan
Australian National University
Bedford Oceanography,Canada
Bioinformatics Institute, Singapore
CSIRO, Australia
Data Storage Institute, Singapore
EGEE, French National Center
GeoForschungsZentrum, Germany
James Cook University, Australia
KEK High Energy Physics, Japan
Max Planck Institute, Netherlands
Parallab, Norway
South Australian Advanced Computing
UIB (Parallab) , Norway
University of Amsterdam
University of Cambridge, Astronomy
University of Cambridge, e-Science
University of Edinburgh
University of Genoa, Italy
University of Hong Kong
Univrsity of Manchester
University of Oslo
University of Southampton
York Univ (UK)
CiteSeer, Penn State
City Univ. of New York
Geospatial Environment, UCSD
Drexel University
EOSDIS Distributed Active, NASA Goddard
Georgia Tech
Kentucky State Libraries & Archives
Library of Congress
Los Alamos National Lab
NASA Ames
NASA Goddard Space Flight Center
NCSA Grid Computing
NIH (NCI Center for Bioinformatics)
Penn State University
Pittsburgh Supercomputing Center
Purdue University. Indiana
Stanford University
TACC, University of Texas
Texas A & M
UC Santa Cruz
UCLA
UCSD Neuroscience
University of Maryland
University of Michigan, CAC department
University of New Mexico
University of Washington
University of Wisconsin
USC
Yale University
SDSC SRB Projects (60 million, .5 PB )
Digital Libraries
– UCB, Umich, UCSB, Stanford,CDL
– NSF NSDL - UCAR / DLESE
NASA Information Power Grid
Astronomy
– National Virtual Observatory
– 2MASS Project (2 Micron All Sky Survey)
Particle Physics
– Particle Physics Data Grid (DOE)
– GriPhyN
– SLAC Synchrotron Data Repository
Medicine
– Digital Embryo (NLM)
Earth Systems Sciences
– ESIPS
– LTER
Persistent Archives
– NARA
– LOC
Neuro Science & Molecular Science
– TeleScience/NCMIR, BIRN
– SLAC, AfCS, …
Storage Resource Broker (SRB)
A distributed file system (Data
Grid)
–Client-Server, Server-Server
architecture.
–Abstracts physical

SRB provides the ability to

transparently share data across
remote sites.
–Heterogeneous Resources
–Single sign on
–Single logical file hierarchy
What we are familiar with
What we are not familiar with, yet
 How do the file systems differ?
Logical Abstraction
–Folders are NOT physical
–Files do NOT inherit physical location
–Everything is potentially distributed
Access Control
–Permissions are NOT rwxrwxrwx
–Permissions ARE on a object by object
basis
–Groups and permissions ARE more similar
to NTFS
Domains
–Geographical / logical grouping of users
–Namespace scalability: john@harvard
john@mit
–Also doubles as groups
 Interfaces to the
Storage Resource Broker

inQ – Windows Client

Scommands – UNIX, DOS Command line
Client
Jargon – Java API and GUI
components
mySRB – Web Client
Matrix – WSDL, Data Grid
Workflows
C, C++ – C and C++ API
Python – Python API
Perl – Perl API
Common Scommands (75 total)

Sinit Scp
Senv Smv (logical)
Spwd Sphymove (physical)
Sls Srm
Smkdir
Scd
Srmdir
Sget
Serror
Sput Schmod
Ssh Sexit
mySRB
BIRN Portal (perl based)
NEEScentral Portal (php based)
 Biomedical Informatics Research
Network (BIRN)

Major collaboration with SDSC, several of the

projects’ Co-Investigators and Co-PIs are at
SDSC.

BIRN’s purpose is to provide it’s consortium

of neuroscience laboratories the ability to
share, compute, and collaborate.

The Storage Resource Broker provides the

ability to transparently share data across
remote sites.
The BIRN SRB Data Grid
Doing this “Manually”
The BIRN Data Grid
The grid is in the details
 File Replication
Sls
/home/Demo/SRB-Tutorial/files-2:
Doc.txt

Sls -l
/home/Demo/SRB-Tutorial/files-2:
romanoly 0 z-ucsd-ncmir-nas1 15 2003-07-09-
05.15 Doc.txt
romanoly 1 z-jhu-cis-nas0 15 2003-07-
09-05.16 Doc.txt
romanoly 2 z-stanford-lucas-nas 15 2003-07-09-
05.16 Doc.txt
romanoly 3 z-umn-cmrr-nas0 15 2003-07-09-
05.16 Doc.txt
romanoly 4 z-uci-bic-nas0 15 2003-07-
09-05.17 Doc.txt
 Teragrid SRB
– All Teragrid accounts are given a SDSC SRB Teragrid
account
•The ‘username’ is the same as your SDSC UNIX account
name
•Your SRB ‘domain’ is ‘teragrid’
•You must register your DN string with SDSC’s grid-
mapfile or request a SRB password to activate your SRB
account. Instructions to do so are here:
http://www.sdsc.edu/user_services/srb/account_form.html

– Your ~/.srb/.MdasEnv file OR env variables

mdasCollectionHome '/home/<user-name>.teragrid'
mdasDomainName 'teragrid'
srbUser '<user-name>'
#AUTH_SCHEME 'ENCRYPT1'
AUTH_SCHEME 'GSI_AUTH'
srbHost 'srb.sdsc.edu'
srbPort '7321'
defaultResource 'sfs-tape-tgd'
SERVER_DN '/C=US/O=NPACI/OU=SDSC/CN=Storage Resource
Broker/USERID=srb'
 Scommand Features

Command line interface -> SCRIPTING

Available for all of the most popular UNIX
flavors and DOS
S-commands are the most flexible and powerful
of the clients
They are the fastest, and most reliable
They are multithreaded for big gains in data
flow
They are great for scripts, perl wrappers,
batch jobs, etc…

Installed man pages via “man [Scommand]”

– man Sput
 Scommand Notes

Shelp
–Gives list of commands with brief summary
–“[Scommand] <return>” gives usage info
(usually) or try –h flag
Sinit – establishes a session
Senv – displays connection information
Spwd – display current working directory
Sexit – ends session
Some Public SRB Collections

Southern California Earthquake Center

/home/public.teragrid/SCEC

Two Micron All Sky Survey

/home/public.teragrid/2MASS

The Palomar Digital Sky Survey

/home/public.teragrid/DPOSS
Watch me do a SRB demo
 Thanks!
SRB handles large data and provides the
ability to share and collaborate on
distributed heterogeneous resources.

Questions?

Teragrid SRB userguide:

http://www.sdsc.edu/user_services/srb
SRB website:
www.sdsc.edu/srb
SRB email:
srb@sdsc.edu