CCGRID: A RESOURCE SHARING

INTO ASSOCIATION GRIDS

Amala deepan and Sampath kumar

Department of Computer Science and Engineering

PSG college of Technology
Affiliated to Anna University Coimbatore
Coimbatore
Email :{ deepan7787, samcse}@gmail.com

ABSTRACT: Computational grids are becoming
the computing platform for solving problems in
science, engineering, and commerce. Software to
enable grid computing has been primarily
written for operating systems, thus severely
limiting the ability to effectively utilize the
computing resources of the vast majority of
desktop computers i.e. those running variants of
the Microsoft Windows operating system.
Cluster Computing Grid is a technology to solve
the resource sharing problem with clusters.
Using this technology, we propose to bring the
software resource sharing within cluster along
with web server. The Microsoft .NET
Framework provides a powerful toolset. The
Alchemi grid computing framework was
conceived with the aim of making grid
construction and development of grid software
as easy as possible without sacrificing flexibility,
scalability, reliability and extensibility.
Keywords- CCGrid, FCFS, CAS
1. INTRODUCTION
Computational grids are becoming the computing
platform for solving problems in science,
engineering, and commerce. Software to enable
grid computing has been primarily written for
operating systems, thus severely limiting the ability
to effectively utilize the computing resources of the
vast majority of desktop computers i.e. those
running variants of the Microsoft Windows
operating system.
The Microsoft .NET Framework provides
a powerful toolset that can be leveraged for all of
these, in particular support for remote execution
(via .NET Remoting and web services)
multithreading, security, managed execution and
cross-language development, making it an ideal
platform for grid computing middleware. The
Alchemi grid computing framework was conceived
with the aim of making grid construction and
development of grid software. Alchemi follows the
master-worker parallel programming paradigm in
which a central component dispatches independent
units of parallel execution to workers and manages
them. In Alchemi, this unit of parallel execution is
termed ‘grid thread’ and contains the instructions to
be executed on a grid node, while the central
component is termed ‘Manager’.
2. GRID COMPUTING
A "Grid" is a type of parallel and
distributed system that enables the sharing,
selection, and aggregation of geographically
distributed "autonomous" resources dynamically at
runtime depending on their availability, capability,
performance, cost, and users' quality-of-service
requirements. Grid has ability to share and
aggregate distributed computational capabilities
and delivers them as service.
Grid computing is the application of
several computers to a single problem at the same
time – usually to a scientific or technical problem
that requires a great number of computer
processing cycles or access to large amounts of
data.
Grid computing has emerged as an enabler
for e-Science as it permits the creation of virtual
organizations that bring together communities with
common objectives. Within a community, data
collections are stored or replicated on distributed
resources to enhance storage capability or
efficiency of access.
There is rapidly emerging interest in grid
computing from commercial enterprises. A
Microsoft Windows based grid computing
infrastructure will play a critical role in the
industry-wide adoption of grids due to the large-
scale deployment of Windows within enterprises.
This enables the harnessing of the unused
computational power of desktop PCs and
workstations to create a virtual supercomputing
resource at a fraction of the cost of traditional
supercomputers. There is a distinct lack of service-
oriented architecture-based grid computing
software. To overcome this limitation, we have
developed a Windows-based grid computing
framework called Alchemi implemented on the
Microsoft .NET Platform.
Computational grids are becoming the
computing platform for solving problems in
science, engineering, and commerce. Software to
enable grid computing has been primarily written
for operating systems, thus severely limiting the
ability to effectively utilize the computing
resources of the vast majority of desktop computers
i.e. those running variants of the Microsoft
Windows operating system.
3. CCGrid
Cluster Computing Grid [CCGrid] is a
technology to solve the resource sharing problem
with clusters. Using this technology, we propose to
bring the software resource sharing within cluster
along with web server.
Cluster computing: autonomic computing
techniques can be used for dealing in an integrated
manner with node failures, security and job
migration. Greater significance will be approaches
to optimizing application execution that are
distributed over multiple cluster systems while
maintaining other important properties such as fault
tolerance and secure execution environment. Such
optimization approach relies on the availability of
specialist monitoring tools, which could feed into
execution planning and enactment strategies.
executed on a grid node, while the central
component is termed ‘Manager’.
A ‘grid application’ consists of a number
of related grid threads. Grid applications and grid
threads are exposed to the application developer as
.NET classes / objects via the Alchemi .NET API.
When an application written using this API is
executed, grid thread objects are submitted to the
Alchemi Manager for execution by the grid.
Alternatively, file-based jobs (with related jobs
comprising a task) can be created using an XML
representation to grid-enable legacy applications
for which precompiled executables exist. Jobs can
be submitted via Alchemi Console Interface or
Cross-Platform Manager web service interface,
which in turn convert them into the grid threads
before submitting then to the Manager for
execution by the grid.

4. ALCHEMI OVERVIEW
4.1Overview
Grid computing depends on software to
divide and apportion pieces of a program among
several computers, sometimes up to many
thousands.
Alchemi is a .NET-based framework that A layered architecture for a desktop grid computing
provides the runtime machinery and programming environment.
environment required to construct desktop grids
and develop grid applications. It allows flexible 4.3 Alchemi components
application composition by supporting an object- An Alchemi desktop grid is constructed by
oriented application programming model in deploying a Manager node and deploying one or
addition to a file-based job model. Cross-platform more Executor nodes configured to connect to the
support is provided via a web services interface and Manager. One or more Users can execute their
a flexible execution model supports dedicated and applications on the cluster by connecting to the
non-dedicated (voluntary) execution by grid nodes. Manager. An optional component, the Cross
It can be used to manage a cluster of dedicated or Platform Manager provides a web service interface
non-dedicated compute nodes. The node manager to custom grid middleware.
provides a centralized interface to manage all of the
clients on the grid, which is accessible from
anywhere on the network. m
4.2 Alchemi Desktop Grid Framework
Alchemi follows the master-worker
parallel computing paradigm in which a central
component dispatches independent units of parallel
execution to workers and manages them. In e e e e
Alchemi, this unit of parallel execution is termed
‘grid thread’ and contains the instructions to be
4.3.1 Manager
The Manager provides services associated
with managing execution of grid applications and
their constituent threads. Executors register
themselves with the Manager, which in turn
monitors their status. Threads received from the
User are placed in a pool and scheduled to be
executed on the various available Executors. A
priority for each thread can be explicitly specified
when it is created or submitted. Threads are
scheduled on a Priority and First Come First
Served (FCFS) basis, in that order. The Executors
return completed threads to the Manager which are
subsequently collected by the respective User. The
Manager employs role-based security model for
authentication and authorization of secure
activities. A list of permissions representing
activities that need to be secured is maintained
within the Manager. A list of groups (roles) is also
maintained, each containing a set of permissions.
For any activity that needs to be authorized, the
user or program must supply credentials in a form
of a user name and password and the Manager only
authorizes the activity if the user belongs to a group
that contains the particular permission. Executors
are constantly monitored and threads running on
disconnected Executors are rescheduled.
Additionally, all data is immediately persisted to
disk so that in the event of a crash, the Manager
can be restarted into the pre-crash state.
4.3.2 Executor
The Executor accepts threads from the
Manager and executes them. An Executor can be
configured to be dedicated, meaning the resource is
centrally managed by the Manager, or non-
dedicated, meaning that the resource is managed on
a volunteer basis via a screen saver or explicitly by
the user. For non-dedicated execution, there is one-
way communication between the Executor and the
Manager. In this case, the resource that the
Executor resides on is managed on a volunteer
basis since it requests threads to execute from the
Manager. Where two-way communication is
possible and dedicated execution is desired the
Executor exposes an interface so that the Manager
may communicate with it directly. In this case, the
Manager explicitly instructs the Executor to
execute threads, resulting in centralized
management of the resource where the Executor
resides. Thus, Alchemi’s execution model provides
the dual benefit of: flexible resource management
i.e. centralized management with dedicated
execution vs. decentralized management with non-
dedicated execution; and flexible deployment under
network constraints i.e. the component can be
deployment as nondedicated Where two-way
communication is not desired or not possible (e.g.
when it is behind a firewall or NAT/proxy server).
Thus, dedicated execution is more suitable where
the Manager and Executor are on the same Local
Area Network while non-dedicated execution is
more appropriate when the Manager and Executor
are to be connected over the Internet.
Threads are executed in a sandbox environment
defined by the user. The CAS (Code Access
Security) feature of .NET are used to execute all
threads with the AlchemiGridThread permission set
which can be Specified to a fine-grained level by
the user as part of the .NET Local Security Policy.
All grid threads run in the background with the
lowest priority. Thus any user programs are
unaffected since they have higher priority access to
the CPU over grid threads.
4.3.3 User
Grid applications are executed on the User
node. The API abstracts the implementation of the
grid from the user and is responsible for informing
a variety of services on the user’s behalf such as
submitting an application and its constituent
threads for execution, notifying the user of finished
threads and providing results and notifying the user
of failed threads along with error details.
4.3.4 Cross-Platform Manager
The Cross-Platform Manager is a web
services interface that exposes a portion of the
functionality of the Manager in order to enable
Alchemi to manage the execution of grid jobs (as
opposed to grid applications utilizing the Alchemi
grid thread model). Jobs submitted to the Cross-
Platform Manager are translated into a form that is
accepted by the Manager (i.e. grid threads), which
are then scheduled and executed as normal in the
fashion described above. In addition to support for
the grid-enabling of legacy applications, the Cross-
Platform Manager allows custom grid middleware
to interoperate with and leverage Alchemi on any
platform that supports web services.
5.TESTING
TEST CASE 1: Alchemi MANAGER
Description:
Manager- manages the execution of grid
application with managing thread execution.
INPUT:
Setup Connection
Own Port : 9000
Manager Host :-
Manager Port :0
Storage Configuration
Database type : Microsoft SQL Server 2000
Server : localhost
DB Name : Alchemi
Username : sa
Password :123
Action : start
Expected Output: INPUT:
Attempting to start Manager… [Pi Calculator Grid Application]
Manager started Host [default=localhost] : localhost
Port [default=9000] : 9000
TEST CASE 2: Alchemi EXECUTOR Username [default=user] : user
Description: Password [default=user] : user
Executor-accepts threads from the Manager & Action :enter
executes them.
Expected Output:
INPUT:
Setup Connection
Manager Node
Host/IP Address : local host
Port : 9000
Credentials
Username : executor
Password : executor
Own Node
Id : 0435fd53-f291-41f2-
9ba1-e539eb42cd10
Dedicated : yes
Port : 9001

Action : connect
Expected Output:
Attempting to connect to Manager...
Connected to Manager.

Error:
If the manager not started, 6. CONCLUSION:
Using last verified configuration ... The CCGRID: a web resource sharing into
Attempting to connect to Manager... alchemi grids was designed for reducing the
Error connecting to manager. Could not connect to manual resource sharing. The Alchemi grid
Manager. computing framework was conceived with the aim
of making grid construction and development of
TEST CASE 3: Console Parent Form-[Alchemi grid software. Hence we use the Alchemi toolkit
console] for making the grid cluster. The Microsoft .NET
Description: Framework provides a powerful toolset that can be
Which show the performance of each support for remote execution (via web services)
executor multithreading, security, disconnected data access,
managed execution and cross-language
INPUT: development, making it an ideal platform for grid
Alchemi Grid Connection computing middleware. CCGrid is a technology
Manager to solve the resource sharing problem with clusters.
Host/IP Address : localhost By implementing this project, we aim to
Port : 9000 afford an efficient web resources sharing into
alchemi grids and mainly this reduce the
Credentials installation of the software resources in every
Username : admin system.
Password : admin
Action : Ok 7. REFERENCES:
Expected Output: [1]. Alchemi: A .NET-based Grid Computing
Shows the executors id, port, username, Framework and its Integration into Global Grids
last ping time, connect or not, dedicated or Akshay Luther, Rajkumar Buyya, Rajiv Ranjan,
nondedicated and it’s performance and Srikumar Venugopal
Grid Computing and Distributed Systems (GRIDS)
TEST CASE 4: Application PiCalculator.exe Laboratory
Description: Department of Computer Science and Software
To calculate the Pi value Engineering
The University of Melbourne, Australia
Email:{akshayl, raj, rranjan, srikumar}
@cs.mu.oz.au
Computational grids that couple
geographically distributed resources are becoming
the de-facto computing platform for solving large-
scale problems in science, engineering, and
commerce. Software to enable grid computing has
been primarily written for Unix-class operating
systems, thus severely limiting the ability to
effectively utilize the computing resources of the
vast majority of desktop computers i.e. those
running variants of the Microsoft Windows
operating system. Addressing Windows-based grid
computing is particularly important from the
software industry’s viewpoint where interest in
grids is emerging rapidly. Microsoft’s .NET
Framework has become near-ubiquitous for
implementing commercial distributed systems for
Windows-based platforms, positioning it as the
ideal platform for grid computing in this context. In
this paper we present Alchemi, a .NET-based grid
computing framework that provides the runtime
machinery and programming environment required
to construct desktop grids and develop grid
applications. It allows flexible application
composition by supporting an object-oriented grid
application programming model in addition to a
grid job model. Cross-platform support is provided
via a web services interface and a flexible
execution model supports dedicated and non-
dedicated (voluntary) execution by grid nodes.
[2]. Alchemi: A .NET-based Enterprise Grid
System and Framework
User Guide for Alchemi 1.0
Grid Computing and Distributed Systems (GRIDS)
Laboratory
Dept. of Computer Science and Software
Engineering
The University of Melbourne, Australia
Table of Contents
How Alchemi Works
Installation, Configuration and Operation
Common Requirements
Manager
Role-Based Security
Cross Platform Manager
Executor
Software Development Kit
[3]. Hydrogrid.pdf
An extension to this framework, DIME, makes use
of Alchemi to allow for the parallel execution of
models across a network. It will allow the
distribution of model execution using the Alchemi
API. It allows .NET components to be distributed,
both on local and wide area networks.
Grid enabled proxy for Model Runner,
GridModelRunner, which implements a common
interface, IModelRunner, and inherits from
GridThread. This enables GridModelRunner to be
substituted for the original, non-grid component,
and allows applications to be easily reconfigured
for standalone or grid execution.
[4]. Gridcrypt.pdf
Grid Crypt: High Performance Symmetric Key
Cryptography using Enterprise Grids
Grid Computing and Distributed Systems
Laboratory
Dept. of Computer Science and Software
Engineering
The University of Melbourne, Australia
They develop an application for
symmetric key cryptography using enterprise grid
middleware called Alchemi.
Alchemi is a .NET based grid computing
framework developed at the University of
Melbourne. It is an open source project which
provides middleware for creating an enterprise grid
computing environment by harnessing Windows
machines. Alchemi supports multithreaded parallel
operation in a manner similar to threading in Java
or C#, but with their execution on distributed
resources. The parallelism is realized at thread
level and the programmer has to identify functions
to be parallelized and implement them in the form
of threads. Currently, inter-thread communication
is not supported, so threads must be independent.
Alchemi’s main components
A deployment scheme for Alchemi is shown in
Figure. Its main components are manager and
executor that support a master-worker parallel
model. Alchemi has a number of features that ease
the process of setting up of a grid environment in
an enterprise. The executors can be setup in
dedicated or nondedicated mode on employees’
desktop computers. In non-dedicated mode,
Alchemi has no impact on the workstation as far as
the user is concerned. The Alchemi manager also
requires a Microsoft SQL Server instance, which is
available in most companies.
[5]. Excel Grid: A .NET Plug-in for Outsourcing
Excel Spreadsheet Workload to Enterprise and
Global Grids
Krishna Nadiminti, Yi-Feng Chiu, Nick Teoh,
Akshay Luther, Srikumar Venugopal, Rajkumar
Buyya1
Grid Computing and Distributed Systems
Laboratory and NICTA Victoria Laboratory
Department of Computer Science and Software
Engineering
The University of Melbourne
ICT Building, 111 Barry Street
Carlton, VIC 3053, Australia
Alchemi and .NET framework
Alchemi is a Windows-based desktop
grid computing framework implemented on
Microsoft .NET platform, and developed at the
University of Melbourne as part of the Grid bus
project, which provides the runtime machinery and
the programming environment which is required to
construct desktop grids and develop grid
applications. It supports object oriented grid
application programming and execution of cross
plat form applications via web services. The
Microsoft .NET Framework is an ideal platform for
grid computing middleware because it provides a
powerful toolset that supports security, remote
execution, multithreading, cross-language
development, etc. The key features supported by
Alchemi, are internet-based clustering of desktop
computers without a shared file system, federation
of clusters to create hierarchical, cooperative grids,
dedicated or non-dedicated (voluntary) execution
by clusters and individual nodes, object-oriented
grid thread programming model (fine-grained
abstraction), and a web services interface
supporting a grid job model (coarse-grained
abstraction) for cross-platform interoperability (e.g.
for creating a global and cross-platform grid
environment using a custom resource broker
component).
[6]. Analysis of GRID COMPUTING as IT
APPLIES TO HIGH VOLUME DOCUMENT
PROCESSING AND OCR
By: Dmitri Ilkaev, Stephen Pearson
Abstract: In this paper we analyze the concept of
grid programming as it applies to document
imaging and processing. The paper mostly focuses
on OCR processing mapped to a simple grid
configuration. Alchemi .NET grid framework was
used as the grid engine. Basic dependences of the
OCR engine behavior in the grid configuration are
discussed and the increase of the document
processing rate is demonstrated. Based solely on
the small sample and test regimen employed, we
demonstrate that throughput in excess of 150% can
be achieved even in this CPU intensive process by
employing grid strategies.
We present the basic information about
the technology and the way it works. Alchemi
follows a paradigm in which a central component
dispatches independent units of parallel execution
to workers and manages them. This smallest unit of
parallel execution is a grid thread, which is
conceptually and programmatically similar to a
thread that wraps a "normal" multitasking operating
system thread. A grid application is defined simply
as an application that is to be executed on a grid
and that consists of a number of grid threads. Grid
applications and grid threads are exposed to the
grid application developer via the object-oriented
Alchemi .NET API. Figure below represents the
high level architecture of the Alchemi’s grid
framework.
There are three main components:The
Manager manages the execution of grid
applications and provides services associated with
managing thread execution. The Executors register
themselves with the Manager which in turn keeps
track of their availability. Threads received from
the Owner are placed in a pool and scheduled to be
executed on the various available Executors. A
priority for each thread can be explicitly specified
when it is created within the Owner, but is assigned
the highest priority by default if none is specified.
Threads are scheduled on a Priority and First Come
First Served (FCFS) basis, in that order. The
Executors return completed threads to the Manager
which are subsequently passed on or collected by
the respective Owner.
[7]. Alchemi: A .NET-based Grid Computing
Framework and its Integration into Global
Grids Akshay Luther, Rajkumar Buyya, Rajiv
Ranjan, and Srikumar Venugopal see
http://www.alchemi.net/publications.html
GridbusManual04
The Grid bus Middleware Manual
GRID COMPUTING AND DISTRIBUTED
SYSTEMS LABORATORY
DEPT. OF COMPUTER SCIENCE AND
SOFTWARE ENGINEERING
THE UNIVERSITY OF MELBOURNE,
AUSTRALIA
GRID: A virtual information processing
environment where the user has the ‘illusion’ of a
seamless single-source computing power which is
actually distributed.
Computation = Program + Resource
[8]. Web reference:
http://www.alchemi.net/publications.html
http://www.gridbus.org