The OGSA-Basic Execution Services (BES) implementation in UNICORE is a crucial component in the context of job submission and management and in order to understand it we shortly describe other implied technologies. First and foremost, the Job Submission Description Language (JSDL) defines standard XML schema elements for job descriptions that should be executed on computational Grid resources, for instance a simple execution of an application on a parallel resource. However, JSDL is only a language and schema for the standardized description of computational job submissions, but the submit operation itself is represented by the OGF standard interface named as OGSA-BES.

In more detail the OGSA-BES implementation describes a Web services-based interface that is standardized by the OGF and consists also of a well-defined state model of computational jobs. In particular, the interface comprises functionality for the creation, monitoring, and control of computational jobs. Within the OGSA-BES specification such jobs are names as activities described via JSDL documents. Finally, the related HPC Basic Profile (HPC-BP) specifies the usage of the OGSA-BES interface in conjunction with certain extensions for HPC environments to JSDL. By using different security models, the OGSA-BES adoption within UNICORE is also compliant with HPC-BP. Additional security setups might include the use of X.509 proxy certificates for interoperability use cases with technologies that are based on the Grid Security Infrastructure (GSI).

The main functionality of the here described OGSA-BES adoption of UNICORE is to use this service for computational job submissions to UNICORE. This means that jobs that are submitted by end-users are actually executed on the system underneath the interface. In the case of UNICORE, this might be often supercomputers or large clusters. A closer look reveals that the OGSA-BES interface is similar to the Target System Factory (TSF), Target System Service (TSS), and Job Management Service (JMS) of the Unicore Atomic Services (UAS) of UNICORE 6. While the UAS rely on standardized WS-RF compliant message exchanges, the syntax of several UAS operations are proprietary such as the actual submit() operation that takes a JSDL document as parameter.


(high-resolution image)

As shown in the figure, the additional benefit of the OGSA-BES adoption in UNICORE 6 is the provisioning of a standardized syntax of Web service operations that deal with job control and management. It thus basically deprecated the three UNICORE proprietary interfaces TSF, TSS, and JMS. But, the UAS and the OGSA-BES implementation presented here can be used in parallel since they are using internally the same UNICORE backend named as enhanced Network Job Supervisor (XNJS). Hence, the Web service layer is well encapsulated from the lower-level backend execution engine. This backend in turn forwards the job to an interface at the target system and submits it to a resource management system or batch subsystem (e.g. Torque, LoadLeveler, SunGridEngine, etc.).

There is also an OGSA-BES configuration for the UNICORE Command-line Client (UCC) available, which is not yet integrated in the official UCC releases. A work in progress is also the availability of OGSA-BES under the UNICORE Rich Client.

Finally, the UNICORE OGSA-BES interface implementation participates in many interoperability efforts of OGF to check whether it is interoperable with other OGSA-BES adopters. Thus we have been able to contribute to the development of the specification, for instance in the context of the HPC-BP experience document. More recently, the OGSA-BES implementation plays a significant role in different scientific use cases that require the interoperability of numerous e-science infrastructures.

Related publications:

Enhanced Resource Management Capabilities using Standardized Job Management and Data Access Interfaces within UNICORE Grids

M.S. Memon, A.S. Memon, M. Riedel, B. Schuller, D. Mallmann, B. Tweddell, A. Streit, S. van den Berghe, D. Snelling, V. Li, M. Marzolla, P. Andreetto
Proceedings of 3rd Workshop on Scheduling and Resource Management for Parallel and Distributed Systems SRMPDS 2007, ICPADS'07 - The 13th International Conference on Parallel and Distributed Systems, Hsinchu, Taiwan, December, 2007, pp.1-6, DOI 10.1109/ICPADS.2007.4447834, Publisher IEEE Computer Society, ISBN 978-1-4244-1889-3

Open Standards-based Interoperability of Job Submission and Management Interfaces across the Grid Middleware Platforms gLite and UNICORE

M. Marzolla, P. Andreetto, V. Venturi, A. Ferraro, A.S. Memon, M.S. Memon, B. Twedell, M. Riedel, D. Mallmann, A. Streit, S. van de Berghe, V., Li, D. Snelling, K. Stamou, Z.A. Shah, F. Hedman
Proceedings of International Interoperability and Interoperation Workshop (IGIIW) 2007 at 3rd IEEE International Conference on e-Science and Grid Computing, Bangalore, India, December, 2007, IEEE Computer Society, ISBN 0-7695-3064-8, pp. 592 - 599

Web Services Interfaces and Open Standards Integration into the European UNICORE 6 Grid Middleware

M. Riedel, B. Schuller, D. Mallmann, R. Menday, A. Streit, B. Tweddell, M.S. Memon, A.S. Memon, B. Demuth, Th. Lippert, D. Snelling, S. van den Berghe, V. Li, M. Drescher, A. Geiger, G. Ohme, K. Benedyczak, P. Bala, R. Ratering, A. Lukichev
Proceedings of 2007 Middleware for Web Services (MWS 2007) Workshop at 11th International IEEE EDOC Conference "The Enterprise Computing Conference", 2007, Annapolis, Maryland, USA, IEEE Computer Society, pages 57-60, ISBN 978-0-7695-3338-4

Improving e-Science with Interoperability of the e-Infrastructures EGEE and DEISA

M. Riedel, A.S. Memon, M.S. Memon, D. Mallmann, A. Streit, F. Wolf, Th. Lippert, V. Venturi, P. Andreetto, M. Marzolla, A. Ferraro, A. Ghiselli, F. Hedman, Zeeshan A. Shah, J. Salzemann, A. Da Costa, V. Breton, V. Kasam, M. Hofmann-Apitius, D. Snelling, S. van de Berghe, V. Li, S. Brewer, A. Dunlop, N. De Silva
Proceedings of the 31st International Convention MIPRO, Conference on Grid and Visualization Systems (GVS), May 2008, Opatija, Croatia, Croatian Society for Information and Communication Technology, Electronics and Microelectronics, ISBN 978-953-233-036-6, pages 225 - 231

Benchmarking of integrated OGSA-BES with the Grid middleware

F. Hedman, M. Riedel, P. Mucci, G. Netzer, A. Gholami, M. S. Memon, A. S. Memon, Z. A. Shah
Proceedings of 4th UNICORE Summit 2008, Springer LNCS, Euro-Par 2008 Workshop proceedings, to appear

Developer contacts:

Mohammad Shahbaz Memon
Morris Riedel

Status:

The OGSA-BES interface adoption is already available in the UNICORE core bundle. Current research focuses on data transfer technologies (i.e. HPC File Staging Profile) aligned with OGSA-BES compliant job submissions, and a more convenient integration in UNICORE Client releases.

• back to Development

back to top

© UNICORE 2010

Maintained by
Jülich
Supercomputing
Centre