Review of Research Vol:1 Issue:1 /October 2011

ISSN: 2249-894X

Research Papers

COMPONENTS OF META BROKER AND INTEROPERABILITY IN THE GRID

D. RAMYACHITRA and S. POONGODI

Asst.Prof, Department of Computer Science. M.Phil Scholar, Department of Computer Science, Bharathiar University, Coimbatore- 46. poongodimsc@gmail.com

Abstract: Grid computing is a combination of resources of heterogeneous domains to provide remote access and aggregating processing power. The Grid is distributed system with no interactive workloads that
1

involve a large number of files. It is managed by different Meta Brokers, which is managed by particular institution or virtual organizations. This paper describes about the Meta Broker components and the interoperability of various grids through their own resource brokers.

Keywords: Meta Broker, Grid resource, Grid interoperability, Job Execution, Role of Meta broker. INTRODUCTION A grid is a collection of computing resources that are available for an application to perform tasks [1]. These resources are generally heterogeneous, geographically widespread and part of different administrative domains. A grid is an environment where all of the available resources are integrated allowing equal accessibility to them through a standard interface that allows an application to treat them as if they are nodes on a network that have differing capabilities. Grid Meta Broker is a sophisticated decision making thatoperates at different levels of grids. Local Brokers is used at the level of clusters, usually to balance load. Global Brokers are used to map user jobs to resources according to their requirements and properties. Higher level Meta Brokers can be used to select brokers to a specific job. Grid brokering system is used to automate resource selection in grids. The role of grid brokers is to provide an interface for the users to access, accept grid resources and discover resources, find a suitable resource for a job scheduling, submit jobs to resources and provide the output of the jobs to the user.
2

Meta-broker [2] is an abstract scheduling entity responsible for the job scheduling on top of different grid domains. The Meta broker layer functionality deals with the Meta brokering approach which has been also called interoperable grid scheduling. Meta broker is collection of heterogeneous resource information from various brokers. For example when the user request for resource whose specification is OS as Unix, Processor as Intel, Processor clock speed as 1200 MHz, number of processor as 4 and disk size as 1000 MB, initially the resource request is assigned to Meta broker and it checks with the brokers information for the resources availability and assign the user request to the appropriate resources. Meta broker helps user in today grid environment by providing the resources for rent.

Fig.1. General meta-broker architecture [3] Interoperable grid system is a system composed of multiple grids where the interoperation among them has been enabled through the use of common protocols and standards. The importance of interoperability in respect to workflow design is service selection and job execution in the Grid environment. One of the benefits of interoperability is the freedom to select services based on the required functionality rather than on their availability in a specific Grid environment.

ROLE OF META BROKER The Meta broker performs resource discovery, system selection and job execution [4]. We shall discuss about these segments in detail. Resource discovery Resource discovery involves the user selecting a set of resources to investigate. This is generally done in three steps, Authorization filtering - It is generally assumed that a user will know which resources has access to in terms of basic services [5]. Application Requirement Definition - In order to proceed in resource discovery, the user must be able to specify some minimal set of job requirements in order to further filter the set of feasible resources. Minimal Requirement Filtering - Given a set of resources to which a user has access and the minimal set of requirements the job has, the third step in the resource discovery step is to filter out the resources that do not meet the minimal job requirements.
4

System selection Given a group of possible resources or a group of possible resource sets, all of which meet the minimum requirements for the job, a single resource or single resource set must be selected on which to schedule the job. This is generally done in two steps, Gathering Information - In order to make the best possible resource match, a user needs to gather dynamic information about the resources in question. Depending on the application and resource in question, different information may be needed. Select the systems to run on - Given the information gathered by the previous step, a decision of which resource should the user submits a job is made in this step Job execution This is generally done in five steps, Advance Reservation - It may be the case that to make the best use of a given system, part or all of the resources to be reserved in advance [6]. Job Submission - Once resources are chosen the application must be submitted to resources. This may be easy as running a single command or as complicated as running a series of scripts, and may or may not include setup or staging. Task Preparation - The preparation stage may involve setup, claiming a reservation, or other actions needed to prepare the resource to run the application.
5

 Progress Monitoring - Depending on the application and its running time, users may monitor the progress of their application and possibly change their mind about where or how it is executing. Job Completion - When the job is finished, the user needs to be notified. Then the user may need to retrieve files from that resource in order to analyze on the results, break down the environment and remove temporary settings etc.

Fig. 2: Architecture of Brokers Layer

RESOURCE BROKER The term resource broker can [7] only be applied to grid environments where the selection of resources to schedule jobs on is distributed. In a centralized system there exists only one entity that is responsible for scheduling of all jobs, but in a decentralized system the scheduling actions of the RMS are split over multiple entities. The distributed components that perform the scheduling are called resource brokers. An
6

instance of a resource brokering component is associated with each application in the grid environment. Note that the resource brokering component can also be associated with each resource in the system where brokering is performed from the point of view of the resources but it is less common. The first step that a resource broker also referred to as a broker will take is the discovery of available resources that may be used by the application. This is generally performed by interacting with the information services of the grid environment in which the application and associated brokering component is operating. The broker will then select the machines that best fit the needs of the application and submit jobs in the application to those machines. The broker thus handles submission of jobs but not how the job is actually executed on the resource as that is part of the RMS that resides on the resource involved. These actions are referred to as scheduling in the resource broker. Once jobs are being executed or waiting to be executed on resources in the grid, the resource broker for that application monitors the resources and the progression of the jobs. Depending on the policy implemented in the resource broker it may take further action according to the information it receives to either stop a job and start it somewhere else, or migrate that job to a different resource.

ESSENTIAL COMPONENTS OF A META-BROKER IN GRID INTEROPERABILITY

Interoperability problems appeared in several parts of grid middleware [8], including resource management. Since most of the middleware components rely in connection with resource managers, it is crucial to establish interoperation among different grid resource managers. The lack of standards is a problem that cannot be solved in shorter period of time, but there is a correct need for standardization to attempt the interoperability problem. The proper solution for interoperability is to design a system, in which all components know how to communicate with each other. Protocols need to be established and utilized by all components. However, when protocols are diverse by themselves, the classical approach to this problem is to create a superior instance, which has the task to provide an interface between the individual components. This approach is taken in our case to create a new layer in grid resource management called metabrokering. The role of this layer is to utilize the existing, widely used and reliable resource brokers and to manage them transparently. Since most of the users have certificates to access more Grids or Virtual Organizations, they are facing a new problem, which broker has to be chosen for a specific application [8]. Just like users resource brokers also need to choose proper resources within a VO, now they need a meta-brokering service to decide like which broker is the best for them and also to hide the differences of utilizing them.

Fig.3 shows the relevant actors in grid resource management. We need to examine this architecture in order to find the right solution. In the resource management system level standardization could be one solution, since this lower layer is part of the core of every middleware system. Therefore all of them would need to be redesigned to support a common standard. Fig.3: Required components of a General Meta-Broker [9] The component of higher level brokering service is described here [9]. Job Description Language (JDL) JDL service is to offer a uniform way to access various grids; a unified description format is needed to specify all the user requirements. Standardization should be taken into account during the design and development of sustainable solutions. Standards and widespread technologies are used where ever applicable. Though the requirements
9

and the General Meta- Broker Architecture meant to be technology independent they recommend the use of JDL for describing user jobs, and WS-Agreement for handling agreements [11]. Capability Description Language (CDL) Each broker has different set of functionalities which can be specialized in different application-types. In order to store and track these properties, it is required to use a CDL. It should be general enough to include all the service capabilities regarding interfaces, job submission, monitoring and agreements [9]. Regarding CDL we developed a language called BPDL (Broker Property Description Language) [11], which had also incorporated SDL. To provide full support for our Meta brokering approach we have revised and modified BPDL and gathered the scheduling related attributes to MBSDL (Meta Broker Scheduling Description Language). The structure of the new BPDL that we called BPDL 2.0., remains nearly the same, we have only clarified some attributes, added missing ones and separated the scheduling related ones to MBSDL [10]. The updated BPDL version includes BrokerID, Interface to provide metadata about the accessibility and notification, Monitoring, Security, and Middleware kind of middleware, grid or virtual organization the broker can operate, JobType, RemoteFileAccess. They use the Performance Metrics field to store how successfully the broker performed job requests, and how reliable its services are. The Prediction attribute can be used to store predicted data about broker availability and reliability. The MBSDL language can be used to extend BPDL with scheduling related attributes. Since JSDL is also lacking
10

these attributes, MBSDL can also be regarded as a JSDL extension. Its schema contains three fields Constraints that have to be fulfilled during scheduling, Quality of Service requirements, and Policy for scheduling policies. Scheduling Description Language (SDL) Besides CDL and JDL, the scheduling requirements and policies also need to be stored. The users can express their needs by extending the JDL with SDL, and the scheduling policies and methods of the brokers can be stored in this format [9]. The scheduling of jobs, and the description of scheduling requirements, are complex tasks. Scheduling requirements fall into many categories [10]. These include: Temporal scheduling- Job start and end times, validity windows for running a job. Data dependent scheduling -Scheduling jobs based on the expected availability or validity of input data. Workflow dependent scheduling- Scheduling jobs according to the order of jobs in a workflow. In addition, there may be dependencies between these categories for scheduling requirements for a job. For example, there may be a dependency between the defined order of jobs in a workflow and the input data required for the job being scheduled. Due to the complexity of the description of scheduling requirements for a job, the decision has been made to put Scheduling requirements out of
11

scope for JSDL. The importance of a SDL is job submission, management and running the jobs in potentially complex environments, such as Grids. Matchmaking Mechanism This component performs the scheduling of incoming user requests [9]. A proper grid broker which implies a domain, virtual organization or execution environment needs to be selected for a user job taking into account the available scheduling policies. Global Job Identifiers (GJI) It is very important to have unique mapping of user jobs to different grid brokers [9]. An implementation can be done using a single jobID provider for the meta-brokering system or just using each broker system to argument the job identifications. In any case jobIDs must be unique. Security Management (SM) The role of this component is to provide secure access to the interconnected domains [9]. For example, different user certificates, proxies may be accepted in different virtual organizations and grids. Providing a transparent way for users these various proxies also need to be handled by the Meta-Broker. It is obvious that only those grids and brokers can be considered for job submission, to which the user has a valid certificate. Accounting Mechanism The GJI and SM can be a part of a global accounting component [9]. The role of this mechanism is to manage user access by pre-defined policies like submission, negotiation, implementation, charge, payment
12

and reinstallation. Though grid economy is still in a premature state, in future the meta-brokering service should also serve business grids. Agreements Mechanism This component is in connection with the Accounting mechanism. Service Level Agreements (SLA) [9] are planned to be used in future generation grids. The role of this part is to negotiate user requirements, which can also affect scheduling policies. When agreements will be generally accepted and used, this mechanism should be extended to do negotiations with the brokers. Service Level Agreements (SLAs) [11] are powerful instruments for describing all requirements and expectations within such a Grid-based business relationship. Service selection has so far been based on performance and compatibility criteria while neglecting the factor of reliability and risk. Monitoring Mechanism Reliable operation requires global monitoring [9], in terms of the interconnected brokers, reachable domain, grid resources and loads, and local component functionalities. Self-aware and fault tolerant operation need to be provided by the system itself, which needs extensive monitoring. Prediction Mechanism This component is in connection with the monitoring and matchmaking mechanisms [9]. It is necessary to perform calculations of broker availability, service utilization and user request loads to manage with the highly dynamic nature of grids. Addressing and Notification Mechanism
13

This component is responsible for accessing the inter-connected resource brokers, and managing communication including local even and external job state notifications [9]. The goal of our presented meta-brokering approach is to establish a connection between individual Grids domains or Virtual Organizations by managing their resource management systems. The general metabrokering architecture is middleware-independent, therefore implementations of this framework can solve the grid interoperability problem and it can be easily interfaced by application level tools, such as portals or workflow managers. CONCLUSION The architecture of the Grid Meta-Broker and various Meta brokering languages provide higher level resource management by utilizing resource brokers. The establishment of meta-brokering found a new way for interoperability support. In this paper we have discussed about the Meta broker components and its works which deal with interoperable of various grids. In future, this can be simulated in a Grid environment and enhance the function of these components of Meta broker. REFERENCE 1. I. Foster, C. Kesselman, Computational Grids, the Grid: Blueprint for a New Computing Infrastructure, Morgan aufmann, 1998. pp. 15-52. 2. Ivan Rodero, Francesc Guimb, Julita Corbalan, Liana Fong, S. Masoud Sadjadi, Grid broker selection strategies using aggregated resource information, Barcelona Supercomputing Center (BSC-CNS), Spain.
14

3. Kertsz, P. Kacsuk, Grid Meta-Broker Architecture: Towards an Interoperable GridResource Brokering Service, CoreGRID Workshop on Grid Middleware, August 28-29, 2006. 4. Meshkova E, Riihijarvi J, Petrova M and Mahonen P, A Survey on Resource Discovery Mechanisms, Peer-to-Peer and Service Discovery Frameworks, Computer Journal of Networks, vol. 52, no. 2, pp. 2097-2128, 2008. 5. S. Fitzgerald, I. Foster, C. Kesselman, G. von Laszewski, W. Smith, S.Tuecke, A Directory Service for Configuring HighPerformance Distributed Computations, Proc. 6th IEEE Symp on HighPerformance Distributed Computing, pg. 365-375, 1997. 6. Snell and Clement, "Metascheduling Query and Reservation Interface," Global Grid Forum Schedule working Group WD 2.2, April 2000. 7. Krauter, K., Buyya, R., Maheswaran, M., A Taxonomy and Survey of Grid Resource Management Systems. Software: Practice and Experience, (SPE) Journal, ISSN: 0038-0644, Volume 32, Issue 2, 2002, Wiley Press, USA, February 2002. 8. Attila Kertesz, Peter Kacsuk, GMBS: A New Middleware Service for Making Grids Interoperable, Preprint submitted to Future Generation Computer Systems, April 7, 2010 9. A. Kertesz, P. Kacsuk fattila.kertesz, Meta-Brokering requirements and research directions in state-of-the-art Grid Resource Management, Technical University of Catalonia (UPC). 10. http://www.ogf.org/documents/GFD.56.pdf
15

11. A. Kertesz, I. Rodero and F. Guim, Data Model for Describing Grid Resource Broker Capabilities, CoreGRID Workshop on Grid Middleware in conjunction with ISC'07 conference, Dresden, Germany, June 25-26, 2007.

16