Grid Computing Fall 2004 8/31/2004

Paul A. Farrell

Contents

The Grid: Past, Present, Future • The Grid

• Beginnings to the Grid
Fran Berman, Geoffrey Fox, and Tony Hey • A Community Grid Model
• Building Blocks of the Grid
– Networks and Computational “nodes” on the Grid
– Pulling it all Together
– Common Infrastructure: Standards
• Grid Applications and Application Middleware
Grid Computing – Making the Global Infrastructure a Reality
• Futures – Grids on the Horizon
Fran Berman, Geoffrey Fox, and Tony Hey
2002 John Wiley & Sons, Ltd.
Chapter 1

Paul A. Farrell Paul A. Farrell

DiSCoV September 2004 DiSCoV September 2004
Grid Computing 1 Grid Computing 2

The Grid The Grid

• What is the Grid? (cont.)
• What is the Grid?
– The computing and data management infrastructure that will
provide the electronic underpinning for a global society in business,
government, research, science and entertainment
– Integrate networking, communication, computation and information to
provide a virtual platform in the same way

– Fig.2 shows a typical early successful application with information

pipelined through distributed systems (next page)

The Grid infrastructure will provide us with the ability to dynamically link
together resources as an ensemble to support the execution of large-scale,
resource-intensive and distributed applications

Paul A. Farrell Paul A. Farrell

DiSCoV September 2004 DiSCoV September 2004
Grid Computing 3 Grid Computing 4

Dept of Computer Science

Kent State University 1
Grid Computing Fall 2004 8/31/2004
Paul A. Farrell

The Grid Problem Elements of the Problem

• Flexible, secure, coordinated resource sharing among • Resource sharing

dynamic collections of individuals, institutions, and – Computers, storage, sensors, networks, …
resource – Sharing always conditional: issues of trust, policy, negotiation,
From “The Anatomy of the Grid: Enabling Scalable Virtual Organizations”
payment, …
• Enable communities (“virtual organizations”) to share
• Coordinated problem solving
geographically distributed resources as they pursue
– Beyond client-server: distributed data analysis, computation,
common goals -- assuming the absence of… collaboration, …
– central location,
• Dynamic, multi-institutional virtual orgs
– central control,
– Community overlays on classic org structures
– omniscience,
– Large or small, static or dynamic
– existing trust relationships.

Paul A. Farrell Paul A. Farrell

DiSCoV September 2004 DiSCoV September 2004
Grid Computing 5 Grid Computing 6

Beginnings to the Grid Beginnings to the Grid

• Parallel computing in the 1980’s and ’90’s
• Projects after I-WAY
– Focused on providing powerful mechanisms for managing communication
between processors, and development and execution environments for parallel – The Globus and Legion infrastructure
machines – The Condor experimented with high- throughput scheduling
PVM, MPI, HPF, and OpenMP for scalable applications – Mars and Prophet experimented with high- performance scheduling
• The first modern Grid is the I-WAY (SC95) – NWS (Network Weather Service) focused on resource monitoring and prediction
– Aggregate a national distributed testbed with over 17 sites networked together – Storage Resource Broker focused on uniform access to heterogeneous data
by the vBNS resources
– Over 60 applications were developed for the conference – NetSolve and Ninf focused on remote computation via a client- server model
– A rudimentary Grid software infrastructure to provide access, enforce security, • The Grid Forum in the late 1990’s
coordinate resources, and other activities – Open Grid Services Architecture (OGSA) which integrates Globus and Web
– Distributed computing : geographical separation Services approaches
– Grid research : integration and management of software

Paul A. Farrell Paul A. Farrell

DiSCoV September 2004 DiSCoV September 2004
Grid Computing 7 Grid Computing 8

Dept of Computer Science

Kent State University 2
Grid Computing Fall 2004 8/31/2004
Paul A. Farrell

The Grid A Community Grid Model

• The UK e-Science Program
• A layered abstraction of the Grid
– A major initiative developed to promote scientific and data-oriented
Grid application development for both science and industry
– Includes a wide variety for projects such as health, medicine,
genomics, and bioscience, etc.
In the next few years, e-Business, e-Government, e-Science, and e-life
– The link is http://www.escience-grid.org.uk

Paul A. Farrell Paul A. Farrell

DiSCoV September 2004 DiSCoV September 2004
Grid Computing 9 Grid Computing 10

A Community Grid Model A Community Grid Model

• Global Resources • Grid applications
– Such resources include computers, networks, data archives, instruments,
visualization devices, etc.
– To ensure that the Grid presents a robust, stable, usable and useful
– Be distributed, heterogeneous, very different performance, and highly dynamic computational and data management platform to the user
• Common infrastructure • Influence of new devices
– The software services which will represent the Grid as a unified virtual platform and – Sensors, PDAs, wireless as well as global-area networking
provide the target for more focused software and applications
Need to be integrated with the Grid
– Example : NFS’s Middleware Initiative (NMI), OGSA
• User-focused grid middleware, tools, and services Require serious consideration of policies for sharing and using
– To enable applications to use Grid resources by masking some of the complexity
resources
involved in system activities such as authentication, file transfer, etc. Be important to develop Grid social and economic policies
– To connect applications and users with the common Grid infrastructure

Paul A. Farrell Paul A. Farrell

DiSCoV September 2004 DiSCoV September 2004
Grid Computing 11 Grid Computing 12

Dept of Computer Science

Kent State University 3
Grid Computing Fall 2004 8/31/2004
Paul A. Farrell

Building Blocks of the Grid

• Networks
Abilene Network Backbone
– The heart of any Grid is its network- networks link together geographically
distributed resources and allow them to be used collectively to support execution of
a single application
– National network backbone
• In 2002, such national networks exhibit 10 Gigabits/sec backbone performance
in the U.S
• Analogous efforts can be seen in the UK SuperJanet
• CA*net3 from Canarie in Canada and the Asian network APAN
– The ratio
• A typical Grid research environment as a 10:1:0.1 Gigabits/sec ratio
representing national:organization:desktop links
• By 2006, GTRN aims at a 1000:1000:100:10:1 gigabit performance ratio
representing international backbone:national:organization:optical desktop:
Copper desktop
– In the future
• Although network bandwidth will improve, we do not expect latencies to
improve significantly
• A critical area of future work is network quality of service and here progress is
less clear
• wired networks will be further enhanced by continued improvement in wireless
connectivity
Feb 2004 Upgrade to OC-192c 10Gbps links
Paul A. Farrell Paul A. Farrell
DiSCoV September 2004 DiSCoV September 2004
Grid Computing 13 Grid Computing 14

Building Blocks of the Grid Building Blocks of the Grid

• Computational “nodes” on the Grid • Pulling it all Together

– Expect a peak single machine performance of 1 petaflops/sec by around – NASA’s Information Power Grid (IPG)
2010 – DoE’s Science Grid
– ASCI White (LLNL) had peak performance of 12 teraflops/sec – UK e-Science Grid
– The NEC Earth Simulator machine has 640 8-processor nodes and offers – NSF’s TeraGrid
10 terabytes of memory and 700 terabytes of disk space (Mar 2002)
• Will connect the San Diego Supercomputer Center (SDSC), California
– The Fujitsu HPC2500 with up to 16384 processors and 85 teraflops/sec
Institute of Technology, Argonne National Laboratory, and the
peak performance
National Center for Supercomputing Applications (NCSA)
– Complex software environments will be needed to smoothly integrate
resources from PDAs to terascale/petascale resources • Will link the four in a Grid which will comprise in aggregate over .6
Petabyte of on-link disk, over 13 TeraFLOPS compute performance,
and be linked together by a 40Gb/s network

Paul A. Farrell Paul A. Farrell

DiSCoV September 2004 DiSCoV September 2004
Grid Computing 15 Grid Computing 16

Dept of Computer Science

Kent State University 4
Grid Computing Fall 2004 8/31/2004
Paul A. Farrell

Building Blocks of the Grid

Grid Applications and Application
• Common Infrastructure: Standards Middleware
– Both the Internet and the IETF, and the Web and W3C consortium have
defined key standards such as TCP/IP, HTTP, SOAP, XML and now • Commercial Applications
WSDL – Web Services definition language that underlines OGSA – Used in an innovative way in a wide variety of areas including inventory
– The Global Grid Forum is building key Grid-specific standars such as control, enterprise computing, games, etc.
OGSA – The Butterfly Grid and the Everquest muliplayer gaming environment are
current examples of gaming systems using Grid-like environments
– NSF’s Middleware Initiative and the UK’s Grid Core Program are seeking
– The Entropia system of peer-peer or Megacomputing
to extend, standardize and make more robust key pieces of software for
– SETI@HOME
the Grid arsenal such as Globus, Condor, and the NWS
– Climateprediction.com is being developed by the UK e-Science program

Paul A. Farrell Paul A. Farrell

DiSCoV September 2004 DiSCoV September 2004
Grid Computing 17 Grid Computing 18

Grid Applications and Application Online Access to

Middleware Scientific Instruments
• Commercial Applications (cont.) Advanced Photon Source
– End- to - End Automation
– End- to - end Security wide-area
– Virtual Server Hosting dissemination
– Disaster Recovery
– Heterogenerous Workload Management
– End- to - End Systems Management
– Scalable Clustering
– Accessing the Infrastructure real-time archival desktop & VR clients
– “Utility” Computing with shared controls
– Access new capability more quickly collection storage
– Better performance
– Reducing up - front investment
– Gaining expertise not available internally
– Web - based access (portal) for control (programming) of Enterprise fuction

tomographic reconstruction

DiSCoV September 2004

Paul A. Farrell DOE X-ray grand challenge: ANL, USC/ISI, NIST, U.Chicago
Grid Computing 19

Dept of Computer Science

Kent State University 5
Grid Computing Fall 2004 8/31/2004
Paul A. Farrell

Multi-disciplinary Simulations: Aviation Safety

Data Grids for High Energy Physics
Wing Models Image courtesy Harvey Newman, Caltech
~PBytes/sec
•Lift Capabilities 1 TIPS is approximately 25,000
•Drag Capabilities
Stabilizer Models Online System ~100 MBytes/sec SpecInt95 equivalents
Airframe Models •Responsiveness
Offline Processor Farm
There is a “bunch crossing” every 25 nsecs.
~20 TIPS
There are 100 “triggers” per second
~100 MBytes/sec
Each triggered event is ~1 MByte in size
•Deflection capabilities
•Responsiveness ~622 Mbits/sec
Tier 0 CERN Computer Centre
or Air Freight (deprecated)
Crew Capabilities
- accuracy Tier 1
- perception France Regional Germany Regional Italy Regional FermiLab ~4 TIPS
Centre Centre Centre
- stamina
- re-action times ~622 Mbits/sec
- SOP’s
Engine Models
Tier 2 Caltech Tier2 Centre
Tier2 Centre
Tier2 Centre
Tier2 Centre
~1 TIPS ~1 TIPS ~1 TIPS ~1 TIPS ~1 TIPS
~622 Mbits/sec

•Braking performance Institute

Human Models Institute Institute Institute
•Steering capabilities
•Thrust performance ~0.25TIPS Physicists work on analysis “channels”.
•Reverse Thrust performance
•Traction Each institute will have ~10 physicists working on one or more
Physics data cache
•Dampening capabilities
•Responsiveness ~1 MBytes/sec channels; data for these channels should be cached by the
•Fuel Consumption institute server

Source NASA Landing Gear Models Tier 4

Whole system simulations are produced by coupling all of the sub-system simulations Physicist workstations

Network for Earthquake Engineering

Mathematicians Solve NUG30 Simulation
• Looking for the solution to the NUG30
quadratic assignment problem • NEESgrid: US national
• An informal collaboration of infrastructure to couple
mathematicians and computer earthquake engineers with
scientists experimental facilities,
• Condor-G delivered 3.46E8 CPU databases, computers, & each
seconds in 7 days (peak 1009
other
processors) in U.S. and Italy (8 sites)
• On-demand access to
14,5,28,24,1,3,16,15, experiments, data streams,
10,9,21,2,4,29,25,22, computing, archives,
collaboration
13,26,17,30,6,20,19,
NEESgrid: Argonne, Michigan, NCSA,
8,18,7,27,12,11,23
UIUC, USC
MetaNEOS: Argonne, Iowa, Northwestern, Wisconsin

Dept of Computer Science

Kent State University 6
Grid Computing Fall 2004 8/31/2004
Paul A. Farrell

BIRN Network A View of BIRN Federated Data

Access ? Give me an Access Access
Control
index of all DAT-
DB Server
Mouse BIRN User KO Striatum Mouse Storage
DB-A Images DB-B Server

EM Images EM Images

MRI Images Multi TB Disk array

Federated data may

be in a variety of
representations
Access Access
• databases
Meta Data Portal Software
Mouse Catalog Server Server Mouse • image files
DB-C DB-D
• simulation files
BIRN CC Histology • flat text files
2 Ph. Img
•…
IT Infrastructure to hasten the derivation of new understanding and
treatment of disease through use of distributed knowledge

Project Workflows Home Computers

Evaluate AIDS Drugs
• Community =
– 1000s of home computer
users
– Philanthropic computing
vendor (Entropia)
– Research group (Scripps)
• Common goal= advance
AIDS research

Paul A. Farrell
DiSCoV September 2004
Grid Computing 28

Dept of Computer Science

Kent State University 7
Grid Computing Fall 2004 8/31/2004
Paul A. Farrell

Grid Applications and Application

Middleware Futures – Grids on the Horizon
• Application Summary • In the future
– Minimal Communication applications : these include so called – More resources will be linked by more and better networks
“embarrassingly parallel” applications where one divides a problem up into
very many essentially – Sensors, PDAs, health monitors, and other devices will be linked to the
– Staged/linked applications (do part A then do part B) : there include Grid
remote instrument applications where one gets input from instrument at – Grid software will become sophisticated, supporting an unprecedented
site A, compute/analyze data at size B and visualizes at Site C diversity, scale, globalization and adaptation
– Access to resources (get stuff from/do something at site A) : this includes
portals, access mechanisms and environments. Require an immense research, development and deployment effort from
the community
• Next generation Grid applications
– Adaptive applications, Real-time applications, Coordinated applications, Grids are high-capacity, high-capability, persistent, evolutionary, scalable,
Poly-applications (choice of resources for different components) and able to support/promote new applications

Paul A. Farrell Paul A. Farrell

DiSCoV September 2004 DiSCoV September 2004
Grid Computing 29 Grid Computing 30

Futures – Grids on the Horizon Futures – Grids on the Horizon

• Adaptative and Autonomic Computing • Grid Programming Environments
– The infrastructure is largely hidden from the user in the same way as – Robust, useful and usable programming environments will require
individuals generally do not know which power company, transformer, coordinated research in many areas
generator, etc. is being used when they plug their electric appliance into a The GrADS project provides a first example of an integrated approach to
socket the design, development and prototyping of a Grid programming
environment
– “Adaptative Computing”
• “Programming the Grid”
• Allow programs to adapt to the dynamic performance deliberable by
– Preparation fo the individual application nuggets associated with a single
Grid resources
resource
– “Autonomic Computing” – Intergrating the nuggets to form a complete Grid program
• IBM is exploring the concepts of software that is self-optimizing, self- The SQL interface to a database, a parallel image processing algorithm, a
configuring, self-healing, and self-protecting to ensure that software finite element solver
systems are flexible and can adapt to change.

Paul A. Farrell Paul A. Farrell

DiSCoV September 2004 DiSCoV September 2004
Grid Computing 31 Grid Computing 32

Dept of Computer Science

Kent State University 8
Grid Computing Fall 2004 8/31/2004
Paul A. Farrell

Futures – Grids on the Horizon

• New Technologies
– The ubiquitous cell phones and PDAs of today are just the beginning of a
deeper paradigm shift
– It will become important to application developers to intergrate new
devices and new information sources with the Grid
– Sensors and sensor-nets embedded in bridges, roads, clothing, etc. will
provide a immense source of data
• Grid Policy and Grid Economies
– Resource usage and administration must bridge technological, political
and social boundaries, and Grid policies will need to incentivize the
individual to contribute to the success of the group

Paul A. Farrell
DiSCoV September 2004
Grid Computing 33

Dept of Computer Science

Kent State University 9