XCAT3: A Framework For CCA Components As OGSA Services
XCAT3: A Framework For CCA Components As OGSA Services
The Open Grid Service Infrastructure extends the Web Currently, the XCAT3 framework is implemented in
service model by defining a special set of service properties Java, and we plan to implement a corresponding C++ ver-
and behaviors. First, it separates the service naming and ser- sion that is inter-operable with the former.
vice reference. A Grid Service Reference (GSR) is a precise
description of how to reach a service instance on the net- 3.1. General Architecture
work. GSRs can be complete WSDL descriptions of a ser-
vice instance. A Grid Service Handle (GSH), on the other Some of the key features of the XCAT3 architecture are
hand, is an immutable name for a service. The idea is that as follows:
a GSR may change over time as a service is moved or up-
graded. Hence a GSH may be bound to different GSRs over ComponentID: Every component has a unique Com-
time, but the GSH can always be resolved to the official ver- ponentID that can be used to refer to it. CCA defines
sion of the service instance. two operations for the ComponentID: getInstan-
The most important contribution of OGSI is the spec- ceName which returns the name of the component in-
ification and restriction of Grid service behavior through stance, and getSerialization which returns
the definition of a family of standard ports. The most im- the framework specific serialization of the Com-
portant of these is the GridService port. This port provides ponentID. However, we add a few methods to the
dynamic service introspection, which is a common feature XCATComponentID which is an interface that ex-
of component architectures. By invoking queries on the re- tends the ComponentID. These are used internally
quired Grid Service port, a client can discover information by the XCAT3 framework, especially for compo-
such as the other portTypes the service supports, the life- nent composition purposes. The interesting ones are
time of the service instance, and other service-specific in- getPortRef to get a reference for a remote pro-
ternal state data that the service wishes to expose. The in- vides port that can be cached when a connection
formation is conveyed back to the client in the form of XML is made between a local uses and the remote pro-
fragments called Service Data Elements (SDEs). Each SDE vides port, setPortRef to cache a reference for
is described by a Service Data Descriptor (SDD), which de- a remote provides port when a connection is be-
fines the schema and the content of the SDE. ing made between a local uses and the remote
Another important set of portTypes in OGSI involve no- provides port, disconnectProvider to discon-
tification. A client service can subscribe to changes in the nect the remote provides port from a connection
service data of a source service by sending its GSH or GSR with a local uses port (by removing the cached re-
to the source service, via its NotificationSource portType. mote port reference), and disconnectUser to
notify a remote provides port that it has been dis- 3.2. CCA Compatibility
connected from a uses port (for reference counting
purposes). The CCA specification itself is defined in SIDL. The
SIDL describes the various interfaces that a framework
Services: Every component has a Services object that must implement in order to be compliant with CCA. It also
is set using the setServices method via the Com- defines the interfaces that ports and components must im-
ponent interface. The Services object is responsible plement.
for providing methods to register uses ports (reg- We use Babel to generate the Java interfaces from the
isterUsesPort), add provides ports (addPro- SIDL specification in order to be compatible with CCA.
videsPort), fetch a previously register port (get- However, Babel is currently defined for language inter-
Port), get a reference to the ComponentID (get- operability within the same process space, and not for the
ComponentID), etc. distributed object case that we are interested in. Hence, we
needed to strip out code from the stub classes that used Java
Exceptions: CCA defines a set of exceptions that may Native Invocation (JNI) for native calls to other languages,
occur during execution, viz. PortNotDefined, PortAl- and replace it with code that does remote invocations us-
readyDefined, PortNotConnected, BadPortName, etc. ing XSOAP.
XCAT3 creates a Java Exception class for every CCA
defined exception. All exceptions extend from the base
CCAException. All exceptions thrown during commu- 3.3. OGSI Compatibility
nication between components are caught and returned
to the component that initiated the communication. To be compatible with OGSI, we use the GSX toolkit
The exceptions are mapped to SOAP faults on the wire to present both our provides ports, and our components as
and then to corresponding exceptions on receiving end Grid services. We discuss each of the two cases separately
of the initiating component. in the following subsections.
Builder Service: The Builder service is a port imple- 3.3.1. Ports as Grid Services. As per the CCA specifi-
mented by CCA compliant frameworks for composing cation, one component can have more than one instance
components into applications in a standard way. It ex- of a provides port of the same type, where each instance
poses the component creation and composition func- is unique. As an example, consider the case of a remotely
tionalities. Some methods exposed by the Builder ser- accessible electron microscope with multiple guns. In this
vice for component lifecycle purposes are create- case, the microscope itself will be represented as a compo-
Instance for creating an instance of a component, nent, while each of the guns will be represented as a pro-
and destroyInstance for eliminating the compo- vides port. In other words, these ports are envisioned to be
nent instance from the scope of the framework. Since state-ful.
ours is a distributed component framework, the cre- On the other hand, in the Web services world (includ-
ateInstance is capable of creating component instances ing OGSI), if a service has several ports of the same type
on remote locations using the Globus GRAM [7] pro- (with different bindings and addresses), the ports are con-
tocol provided by the Java CoG Kit [21]. Upon suc- sidered semantically equivalent. As a result, the same oper-
cessful creation of a component, a ComponentID is re- ations on different ports of the same type affects the state
turned for the component which can be used to refer to of the service in exactly the same way. Thus, Web service
it in the future. For composition purposes, the Builder ports function as interfaces to the Web service, and can be
services provides methods connect for connecting a inferred to be stateless.
uses port to a provides port, and disconnect for dis- Because of the above semantic difference between CCA
connecting an already existing connection. provides ports and Web service ports, a model where ev-
ery component is mapped to a Web (or Grid) service and
Scripting Interface: For rapid prototyping purposes, every provides port is mapped to a Web service port is too
XCAT3 provides an interface to the Builder service us- restrictive, and is incorrect. Hence, we chose to model ev-
ing Jython scripts. Jython is pure Java implementation ery provides port of a CCA component in XCAT3 as a sep-
of the Python scripting language, and provides an al- arate lightweight Grid service (and hence, a Web service)
most seamless interface to code written in Java. Hence, using GSX. The XSOAP toolkit can be used to create the
exposing the functionality of the Builder service which WSDL definition for these services and these can be pub-
was originally written in Java via Jython was a trivial lished in registries such as UDDI [1] for interested clients.
task. The Jython API provided to the user closely mir- Every provides port could conceivably be implemented
rors the API provided by the Builder service. as a simple Web service, and not a Grid service. However,
in the future, we plan to enable checkpointing and migra- ComponentID as a Grid Service
tion of XCAT3 components. The provision of multiple level
naming for Grid services (and hence, XCAT provides ports) Service Data Elements with Locators
for other Provides Ports
will help us keep our uses-provides connections location-
rt
independent, and help us in enabling migration for individ- ides po
ator fo
r prov
ual components. Additionally, the provides ports are now etrieve
loc
1. R
no different in behavior from other Grid services imple-
Grid
Service
Client 2. Make remote
mented using other frameworks such as the OGSI compli- invocation on
provides port XCAT3
ant Globus Toolkit (GT3). This means that a uses port can Component
be connected to any OGSI compliant Grid service. Thus, ex-
isting Grid services that already provide functionalities that
the users desire can be easily leveraged by simply connect- Other Provides Ports as Grid Services
ing to them via uses ports.
In XCAT3, every port interface extends the XCATPort in- Uses Port as a Grid
Service Client
terface, which extends from gov.cca.Port (which is Babel-
generated) and from XSoapGridServiceInterface which rep- Standard
Grid Service
resents the OGSI GridService port, as provided by GSX.
XCAT3 also provides a basic implementation of a provides
Figure 2. Every XCAT3 component is a Grid
port (BasicPortImpl) that every port implementation should
service. It contains SDEs with locators for all
extend from. This class implements the methods declared in
provides ports, which are also Grid services
the XSoapGridServiceInterface, obviating the need for the
themselves
user to implement (or even be aware of) them. Thus, ev-
ery provides port implemented by a user within the XCAT3
framework is automatically exposed as a Grid service.
of the ComponentID in XCAT3 returns the GSH for the
3.3.2. Components as Grid Services. Every CCA com- component that is valid for all time, and not the GSR which
ponent is a collection of provides ports, along with other may no longer be valid if the component has migrated. The
shared state. It follows that every XCAT3 component is a GSH along with a HandleResolver (which is an OGSI de-
collection of Grid services, which happen to be provides fined port that provides an ability to resolve a GSH into a
ports. We choose to expose every XCAT3 component itself valid GSR) is used by a client-side stub to make sure that
as a Grid service, via its ComponentID. the GSR for the remote component stays valid as long as
The XCATComponentID implementation extends not the component is alive.
only the Babel-generated gov.cca.ComponentID, but also The modeling of an XCAT3 components and ports as
the XSoapGridServiceInterface which makes it a Grid ser- Grid services is illustrated in Figure 2. The OGSI specifica-
vice. When a component is instantiated, the framework tion uses a concept called ServiceGroups to group together
creates a Container for the instance. The container is re- a set of Grid services that are related. We can model a com-
sponsible for managing the component instance. The ponent (using the ComponentID) as a ServiceGroup, and
container adds Service Data Elements to the XCATCompo- plan to do so in the future.
nentID containing the names, GSH and GSR for the pro-
vides ports. Thus, any Grid client can obtain access to the 3.4. Writing Components
provides ports by using this information from the XCAT-
ComponentID. In its current form, the container is very Writing components in the XCAT3 framework consists
basic and performs limited functionality. But in the fu- of writing the port interfaces, implementing the ports, writ-
ture, the container will be responsible for activities such as ing the components, and writing scripts that create com-
checkpointing the state of the component, evaluating per- ponent instances and compose them meaningfully. We de-
formance guarantees or Service Level Agreements (SLA), scribe each of the above steps in this section.
etc.
Since the CCA ComponentID is an object that can be Defining Port Interfaces: In XCAT3, port interfaces are
passed around so that any other user can have access to defined in Java. We don’t use SIDL to describe our port
a particular component, we have to make sure that it can interfaces because currently Babel tools have no no-
provide access to a remote component even if the com- tion of distributed objects. For any Java interface to
ponent has migrated to another resource. In other words, function as a port in XCAT3, it needs to extend from
the ComponentID implementation has to provide location- intf.ports.XCATPort. As explained in Section 3.3, ev-
independence. Hence, the getSerialization method ery port interface extends from XCATPort in order to
be compatible with the OGSI specification. The XCAT-
Port (via its inheritance from XSoapGridServiceInter-
User−specific
face) provides methods to query for Service Data Ele- Framework Services
Components
Jython Scripts
that are added via the definition of the port interface. CCA API GSX
Implementing Components: Every implementa- (Babel generated) (Lightweight OGSI impl)
ently. ers communicate with the Registry with the help of uses
ports. In addition, several uses ports are added dynamically
Scripting an application: Once all the components
to the Master for communicating with the Workers. And fi-
have been written, the Builder service API can be used
nally, the Master also has a Builder service port that is used
from Java to create instances of components, and com-
to make connections between its uses ports and the provides
pose them together to form a distributed application.
ports of the Workers.
However, as we mention in Section 3.1, Jython scripts
A Jython script is used to launch the Master and the pool
can also be written for the same purpose. Currently,
of Worker components. The Jython script also connects the
the Jython API to XCAT3 allows creating instances
Master and the Workers with the Registry with the help of
of components locally and remotely (using GRAM),
uses ports. Once the Workers are instantiated, they register
connecting and disconnecting ports between compo-
Locators (GSH and/or GSR) to themselves with the Reg-
nent instances, querying Service Data Elements from
istry. In addition to the Locators, they also publish Service
components, destroying component instances, and in-
Data Elements periodically containing resource utilization
voking methods on provides ports contained within in-
and capabilities of their locations. In order to choose the
stantiated components.
best Worker for fastest turnaround time, the Master queries
In summary, Figure 3 describes the overall architec- the Registry for Workers with the best availabilties and ca-
ture of XCAT3, as described in the above sections. pabilities, and receives the Locators for the same. It then
uses its Builder service to make a connection with the cho-
4. Typical Scenario sen Worker, and sends out a work packet via the uses port.
Thus, XCAT3 can be used to build a complex distributed
Figure 4 illustrates a typical scenario in which the application by composition of regular CCA components,
XCAT3 framework can be used. It shows how XCAT3 along with Grid services as long as they are compatible
can be used to create a distributed application consist- with OGSI. Even though such a system could conceivably
ing of CCA components and OGSI services. be built using vanilla distributed CCA components or plain
The application consists of a pool of Worker compo- Grid services, a combination of these models helps us use
nents, and a Master component that steers the computation concepts of component composition (via the uses-provides
with the aid of the Workers. Also present in the system is a mechanism) for modular construction of complex applica-
Registry service that is OGSI compliant. The Registry is as- tions in the realm of Grid services. Additionally, the compli-
sumed to be persistent and implemented by a third party. ance with OGSI opens up avenues for reusing hundreds of
ley, Kenneth Chiu, Madhusudhan Govindaraju, and Alek-
Jython Script
Registry Service sander Slominski.
Builder Service Port
Component Launch
Master
Component
Get Best References
Worker
Locators,
Capabilities, [1] Universal Description Discovery and Integra-
Availabilities
Worker
tion of Business for the Web (UDDI), Dec 2003.
Port http://www.uddi.org/specification.html.
[2] B. A. Allan, R. C. Armstrong, A. P. Wolfe, J. Ray, D. E.
Worker Worker Worker
Component Component Component
Bernholdt, and J. A. Kohl. The CCA core specification in
a distributed memory SPMD framework. Concurrency and
Figure 4. Typical scenario for the use of Computation: Practice and Experience 14(5), 2002.
XCAT3 [3] R. Armstrong, D. Gannon, A. Geist, K. Keahey, S. Kohn,
L. McInnes, S. Parker, and B. Smolinski. Toward a Com-
mon Component Architecture for High-Performance Scien-
tific Computing. In 8th IEEE International Symposium on
Grid services and clients available in the fast growing Grid
High Performance Distributed Computation, August 1999.
community (e.g the Registry service that is used in our sce-
[4] D. Bartlett. CORBA Component Model (CCM): Intro-
nario). ducing next generation CORBA, Dec 2003. http://www-
106.ibm.com/developerworks/webservices/library/co-cjct6.
5. Conclusions [5] Center for Applied Scientific Computing
(CASC), LLNL. The Babel Project, Dec 2003.
http://www.llnl.gov/CASC/components/babel.html.
We have presented a distributed software component
[6] K. Chiu, M. Govindaraju, and D. Gannon. The Proteus
framework, XCAT3, that is compatible with the Common Multiprotocol Library. In Supercomputing 2002, November
Component Architecture (CCA) and also compliant with 2002.
the Open Grid Services Infrastructure (OGSI). This allows [7] K. Czajkowski, I. Foster, N. Karonis, C. Kesselman, S. Mar-
the combination of benefits from both models - the ability tin, W. Smith, and S. Tuecke. A resource management ar-
to construct complex distributed applications by composi- chitecture for metacomputing systems. In IPPS/SPDP 98,
tion due to the former, and the ease of use in a Grid-based Workshop on Job Scheduling Strategies for Parallel Process-
environment due to the latter. ing, 1998.
CCA components have two types of ports: Provides [8] N. Elliott, S. Kohn, and B. Smolinski. Language Interop-
which is identical to a Web service port, and Uses which is erability for High-Performance Parallel Scientific Compo-
an external reference from one component to a provides port nents. In International Symposium on Computing in Object-
of another component. Provides ports are modeled as OGSI Oriented Parallel Environments (ISCOPE 1999), September
services in XCAT3, and the component itself is modeled 29 - October 2 1999. San Francisco, CA.
as an OGSI service consisting of a set of OGSI-compliant [9] D. Box et al. Simple Object Access Protocol 1.1, Dec 2003.
http://www.w3.org/TR/SOAP.
provides ports. The Grid Service Handles (GSH) and Refer-
[10] E. Christensen et al. Web Services Description Language
ences (GSR) to the ports contained by the component can be
(WSDL) 1.1, Dec 2003. http://www.w3.org/TR/wsdl.
obtained from the introspection information available from
[11] S. Tuecke et al. Grid Service Specification, April
the component, which is represented as Service Data Ele-
2003. http://www.gridforum.org/ogsi-wg/drafts/draft-ggf-
ments (SDE). ogsi-gridservice-29 2003-04-05.pdf.
In the future, we plan to implement a C++ version of [12] Indiana University Extreme Computing Lab.
XCAT3 that is compatible with the Java version. The C++ Grid Service Extensions (GSX), Dec 2003.
version will use the Proteus Multi-protocol library [6] for http://www.extreme.indiana.edu/xgws/GSX.
better performance for scientific applications. [13] I. Foster and C. Kesselman. The GRID: Blueprint for a New
Computing Infrastructure. Morgan-Kaufmann, 1998.
[14] I. Foster, C. Kesselman, J. Nick, and S. Tuecke. Grid Ser-
6. Acknowledgements vices for Distributed System Integration. Computer 35(6),
2002.
This work is supported by National Science Foundation [15] M. Govindaraju, S. Krishnan, K. Chiu, A. Slominski,
grants EIA-0202048 and NSF-0116050, and the Depart- D. Gannon, and R. Bramley. XCAT 2.0: Design and Imple-
ment of Energy Office of Science SciDAC grants. We also mentation of Component based Web Services. Technical re-
wish to acknowledge insights from our colleagues at the Ex- port, Department of Computer Science, Indiana University,
treme Lab at Indiana University, in specific Randall Bram- June 2002. TR562.
[16] M. Govindaraju, S. Krishnan, K. Chiu, A. Slominski,
D. Gannon, and R. Bramley. Merging the CCA Component
Model with the OGSI Framework. In 3rd IEEE/ACM In-
ternational Symposium on Cluster Computing and the Grid,
May 2003.
[17] C. Johnson and S. Parker. The SCIRun Parallel Scientific
Computing Problem Solving Environment. In 9th SIAM
Conference on Parallel Processing for Scientific Computing,
1999.
[18] R. Monson-Haefel. Enterprise Java Beans. O’Reilly, 1999.
[19] A. Slominski, M. Govindaraju, D. Gannon, and R. Bram-
ley. Design of an XML based Interoperable RMI System
: SoapRMI C++/Java 1.1. In International Conference on
Parallel and Distributed Processing Techniques and Appli-
cations, Las Vegas, Pages 1661-1667, June 25-28 2001.
[20] C. Szyperski. Component Software: Beyond Object-Oriented
Programming. Addison-Wesley, 1998.
[21] G. von Laszewski, J. Gawor, S. Krishnan, and K. Jackson.
Grid Computing: Making the Global Infrastructure a Re-
ality, chapter 25, Commodity Grid Kits - Middleware for
Building Grid Computing Environments. Wiley, 2003.