An Implementation of Storage Management in

Private Cloud

PROJECT REPORT


Submitted in partial fulfillment for the award of the degree of
MS

I n
Software Engineering


By

MKS SRI HARISH 10mse0116
VENGAT SUDHEENDRA 10mse0337

Under the Guidance of
PROF M LAWANYA SHRI


VIT
U N I V E R S I T Y
(Estd. u/s 3 of UGC Act 1956)
School of Information Technology & Engineering [SITE]

SOFTWARE ENGINEERING DIVISION

NOV 2013











DECLARATION BY THE CANDIDATE


We hereby declare that the project report entitled An Implementation of
Storage Management in Private Cloud submitted by us to Vellore
Institute of Technology, Vellore in partial fulfillment of the requirement for
the award of the degree of M.S.(Software Engineering) is a record of
bonafide project work carried out by us under the guidance of Prof. M
LAWANYA SHRI. We further declare that the work reported in this
project has not been submitted and will not be submitted, either in part or in
full, for the award of any other degree or diploma in this institute or any
other institute or university.

Place: Vellore
Date:
Signature of the Candidates



M.K.S.SRIHARISH, 10mse0116

VENGAT SUDHEENDRA, 10mse0337










VIT
U N I V E R S I T Y
(Estd. u/s 3 of UGC Act 1956)
School of Information Technology & Engineering [SITE]

SOFTWARE ENGINEERING DIVISION

CERTIFICATE

This is to certify that the project report entitled An Implementation of
Storage Management in Private Cloud submitted by MKS SRI
HARISH (10mse0116) and VENGAT SUDHEENDRA (10mse0337) to
Vellore Institute of Technology, Vellore in partial fulfillment of the
requirement for the award of the degree of M.S.(Software Engineering) is
a record of bonafide work carried out by him/her under my guidance. The
project fulfills the requirements as per the regulations of this Institute and in
my opinion meets the necessary standards for submission. The contents of
this report have not been submitted and will not be submitted either in part
or in full, for the award of any other degree or diploma in this institute or
any other institute or university.

Prof. M LAWANYA SHRI
GUIDE
ASST.PROF, SITE


Internal Examiner (s) External Examiner (s)










CERTIFICATE BY THE EXTERNAL GUIDE



This is to certify that the project report entitled Implementation of
storage management using private cloud submitted by MKS Sri
harish(10mse0116) and Vengat Sudheendra(10mse0337) to Vellore
Institute of Technology University, Vellore in partial fulfillment of
the requirement for the award of the degree MS in Software
Engineering is a record of bonafide work carried out by him/her under
my guidance. The project fulfills the requirements as per the
regulations of this Institute and in my opinion meets the necessary
standards for submission. The contents of this report have not been
submitted and will not be submitted either in part or in full, for the
award of any other degree or diploma in this institute or any other
institute or university.



EXTERNAL SUPERVISOR





























ACKNOWLEDGEMENT

We would like to thank the chancellor of Vellore Institute of Technology
University, Dr. G. Viswanathan, for giving us the opportunity to pursue M.S Software
Engineering.

We would like to thank the Director of School of Information Technology and
Engineering Dr. R.Saravanan.

We would also like to thank our Program Manager of M.S Software Engineering,
Prof Jayaram Reddy for giving us adequate guidance to carry out our project.

We would also like to thank our year coordinator of M.S Software Engineering,
Prof Manikandan for giving us adequate support and guidance to carry out our project.


We would like to extend our sincere thanks to Prof. M Lawanya Shri our
internal guide for her timely help and support and for guiding us to complete the project
successfully.


Place : Vellore

Date :
M.K.S. SRIHARISH



VENGAT SUDHEENDRA




















Table of Contents

Chapter No Title Page No

ABSTRACT i

LIST OF TABLES iii

LIST OF FIGURES Iv

LIST OF SYMBOLS v
1. INTRODUCTION 1
1.1 OVERVIEW 1
1.2 PURPOSE 2
1.3 SCOPE 2
1.4 OBJECTIVE 2
2. PROBLEM ANALYSIS 3
2.1 ABSTRACT 3
2.2 DEFINITION OF CONCEPTS 4
2.3 LITERATURE REVIEW 8
3 DESIGN AND ARCHITECTURE 14
3.1 EUCALYPTUS ARCHITECTURE 14
3.2 CLASS DIAGRAM 16
3.3 USE CASE DIAGRAM 17
3.4 ACTIVITY DIAGRAM 18
3.5 SEQUENCE DIAGRAM 19
3.6 EXPERIMENT SET UP 20
4 IMPLEMENTATION 30
4.1 METHODOLOGY USED 30
4.2 IMPLEMENTATION OF THE SYSTEM 31
4.3 TESTING 41
5 CONCLUSION AND FUTURE WORK 50
5.1 CONCLUSION 50
5.2 FUTURE WORK 51
6 REFERENCES 52


i

Abstract

Private cloud is the phrase used to describe a cloud computing
platform that is implemented within the corporate firewall, under the
control of the IT department. Organizations in every industry, regardless
of size or geography are embracing cloud computing as a way to reduce
the complexity and costs associated with traditional IT approaches. The
reality is driven by three related shifts. Customer, employee and partner
expectations are changing as self- service consumption of technology
and services becomes the norm. The economics of computing are
changing as organizations access world-class computing power, now
available anytime, anywhere. Faster delivery of higher-value products
and services is now mandatory to address formidable competition and
escalating customer and shareholder expectations. Storage management
is a general storage industry phrase that is used to describe the tools,
processes, and policies used to manage storage networks and storage
services such as virtualization, replication, mirroring, security,
compression, traffic analysis and other services. The phrase storage
management also encompasses numerous storage technologies including
process automation, real-time infrastructure products and storage
provisioning. Private cloud also called internal cloud is cloud
infrastructure operated solely for a single organization, whether managed
internally or by a third-party and hosted internally or externally. A
private cloud is designed to offer the same features and benefits of public
cloud systems, but removes a number of objections to the cloud
computing model including control over enterprise and customer data,
worries about security, and issues connected to regulatory compliance.
Some of the advantages of cloud storage are: a cloud storage service
automatically transfers files into the cloud; access to unlimited cloud
storage services; ability to synchronize files across different computers;
very easy to share files with others. The proposed model is to build a
private cloud in our infrastructure using eucalyptus software platform
and provide database (include - storage) as a service to the user. Cloud
computing is the use of a networked resources that are on demand which
can be either Server, application, storage, services through a pool of
shared devices that are interconnected to a remote device that offers it
.Private cloud is a solution that has been recently developed for the
access of shared resources solely for a single entity or organization
where the sharing is managed either internally or externally through a
third party client. Private cloud has had its inception to incorporate the
aspects of enhanced security features to a private organizations data and
services. The foundation for cloud is the systematic virtualization of


ii

resources that further proceeded towards the out of box thinking of a
private cloud where each and every organizations resources are
virtualized and shared. The reason for preference of private cloud is the
greater control of the entire stack of resources available. The issue of
storage management in private cloud arises when there is an extensive
demand for resources. In this project, we have implemented an effective
storage management mechanism for uploading and sharing of files on a
storage cloud. This is done by removing the need to transfer files
between users (when files are shared) and to simply grant them privilege
to access those files by using database entries. This not only helps to
manage memory but also to increase security in the cloud server.













































iii

LIST OF TABLES
S.NO Table Name Page No
1 Table 2.1 Front End Requirements 6
2 Table 2.2 Node Requirements 7
3 Table 2.3 Literature Review 8
4 4.3.1 UTC 1 - Test whether the website can be accessed
within the cluster.
41
5 4.3.2 UTC 2Test whether the user is able to register. 42
6 4.3.3 UTC 3Test whether the user is able to upload files. 43
7 4.3.4 UTC 4Test whether another user is able to access
uploaded files.
44
8 4.3.5 UTC 5Test whether the user is able to download files. 46
9 4.3.6 UTC 6Test whether the user is able to share the
upload files.
47
10 4.3.7 UTC 7Test whether the user is able to delete the
uploaded file(s).
49

























iv

LIST OF FIGURES

S.NO Figure Name Page No
1 Fig 3.1 Eucalyptus Architecture 14
2 Fig 3.2 Project System Architecture 14
3 Fig 3.3 Class Diagram 16
4 Fig 3.4 Use Case Diagram 17
5 Fig 3.5 Activity Diagram 18
6 Fig 3.6 Sequence Diagram 19
7 Fig 3.7 Install Screen 20
8 Fig 3.8 Install Screen 2 21
9 Fig 3.9 Install Screen 3 21
10 Fig 3.10 UEC store 24
11 Fig 3.11 UEC Images 25
12 Fig 4.1 Waterfall model 30
13 Fig 4.2 Home Page 31
14 Fig 4.3 Sign up Page 32
15 Fig 4.4 Log in Page 33
16 Fig 4.5 Upload Page 34
17 Fig 4.6 File Successfully Uploaded 35
18 Fig 4.7 Confirm Replace Page 35
19 Fig 4.9 File Index 36
20 Fig 4.10 File Access 37
21 Fig 4.11 File Download 38
22 Fig 4.12 File Share 39
23 Fig 4.13 Shared File 39
24 Fig 4.14 Confirm Deletion 40
25 Fig 4.15 File Deleted 40
26 Fig 4.16 File Uploaded 44
27 Fig 4.17 File Index view 45
28 Fig 4.18 File Share Test 48
29 Fig 4.19 File Share Test 48


v

LIST OF ACRONYMS


Node Controller (NC) - Eucalyptus component that runs on nodes which host the virtual
machines that comprise the cloud. This service is provided by the Ubuntu package eucalyptus-
nc.

S3 - Simple Storage Service. Amazon's pay-by-the-gigabyte persistent storage solution for EC2.

Storage Controller (SC) - Eucalyptus component that manages dynamic block storage services
(EBS). Each 'cluster' in a Eucalyptus installation can have its own Storage Controller. This
component is provided by the eucalyptus-scpackage.

UEC - Ubuntu Enterprise Cloud. Ubuntu's cloud computing solution, based on Eucalyptus.


VM - Virtual Machine.

VT - Virtualization Technology. An optional feature of some modern CPUs, allowing for
accelerated virtual machine hosting.


Walrus - Eucalyptus component that implements the Amazon S3 API, used for storing VM
images and user storage using S3 bucket put/get abstractions.










1

Chapter 1
INTRODUCTION

1.1 Overview

Cloud computing is an expression used to describe a variety of computing concepts
that involve a large number of computers connected through a real-time
communication network such as the Internet In science, cloud computing is a
synonym for distributed computing over a network, and means the ability to run a
program or application on many connected computers at the same time. The phrase
also more commonly refers to network-based services, which appear to be provided
by real server hardware, and are in fact served up by virtual hardware, simulated by
software running on one or more real machines. Such virtual servers do not
physically exist and can therefore be moved around and scaled up (or down) on the
fly without affecting the end user - arguably, rather like a cloud.

The popularity of the term can be attributed to its use in marketing to sell hosted
services in the sense of application service provisioning that run client server
software on a remote location.

Cloud computing is a kind of grid computing, it has evolved by addressing
the QoS (quality of service) and reliability problems. Cloud computing provides the
tools and technologies to build data/compute intensive parallel applications with
much more affordable prices compared to traditional parallel computing techniques.

Private cloud is cloud infrastructure operated solely for a single organization,
whether managed internally or by a third-party and hosted internally or
externally. Undertaking a private cloud project requires a significant level and degree
of engagement to virtualize the business environment, and requires the organization
to re-evaluate decisions about existing resources. When done right, it can improve
business, but every step in the project raises security issues that must be addressed to
prevent serious vulnerabilities. They have attracted criticism because users "still have
to buy, build, and manage them" and thus do not benefit from less hands-on
management, essentially "the economic model that makes cloud computing such an
intriguing concept".






2

1.2 Purpose

The purpose of this project is to implement the storage management mechanism in a
virtual environment in a way to enhance the abilities and capabilities of the cloud
system. The ultimate purpose is to depict the endurance power of the cloud storage
system.

1.3 Scope

The scope of this process is not just limited to cloud storage system but also its
working in different environments where it has been deployed. The cloud storage
mechanism along with its shadow the virtual environment is combined in a
privately created cloud wherein the interaction are not just limited within the cloud
network but also to all the networks interconnected through its IP address.




1.4 Objective

Implement a private cloud and its deployment.

Implement a virtual environment and its interconnections

Implement a workable storage mechanism in the virtual environment

Ensure maintainability of the implemented storage mechanism
























3

CHAPTER 2
PROBLEM ANALYSIS

2.1 Problem Overview

Organizations in every industry, regardless of size or geography are embracing cloud
computing as a way to reduce the complexity and costs associated with traditional IT
approaches. The reality is driven by three related shifts. Customer, employee and
partner expectations are changing as self- service consumption of technology and
services becomes the norm. The economics of computing are changing as
organizations access world-class computing power, now available anytime,
anywhere. Faster delivery of higher-value products and services is now mandatory to
address formidable competition and escalating customer and shareholder
expectations.
Storage management is a general storage industry phrase that is used to describe the
tools, processes, and policies used to manage storage networks and storage services
such as virtualization, replication, mirroring, security, compression, traffic analysis
and other services. The phrase storage management also encompasses numerous
storage technologies including process automation, real-time infrastructure products
and storage provisioning.
Private cloud also called internal cloud is cloud infrastructure operated solely for a
single organization, whether managed internally or by a third-party and hosted
internally or externally. A private cloud is designed to offer the same features and
benefits of public cloud systems, but removes a number of objections to the cloud
computing model including control over enterprise and customer data, worries about
security, and issues connected to regulatory compliance.
Some of the advantages of cloud storage are: a cloud storage service automatically
transfers files into the cloud; access to unlimited cloud storage services; ability to
synchronize files across different computers; very easy to share files with others.
The proposed model is to build a private cloud in our infrastructure using eucalyptus
software platform and provide database (include - storage) as a service to the user
using UEC.












4

2.2 Definition of Concepts

Storage as Service

Virtualization

Eucalyptus software

UEC



2.2.1 Storage as Service
Storage as a service is an architecture model in which a provider provides digital
storage on their own infrastructure. Storage as a service can be implemented as a
business model in which a large service provider rents space in their storage
infrastructure on a subscription basis. The economy of scale in the service provider's
infrastructure theoretically allows them to provide storage much more cost
effectively than most individuals or corporations can provide their own storage,
when total cost is considered is considered. Storage as a Service is often used to
solve offsite backup challenges.

2.2.2 Virtualization
Storage virtualization is part of the software-defined storage layer that must offer
improvements in performance and space efficiency without requiring the purchase of
additional storage hardware.
It must enable rapid provisioning so that high-performance, space-efficient storage
can be spun up as fast as a VM can be spun up today. It must offer a VM-centric
storage management model that is intuitive for virtual administrators who are taking
on more of the storage management tasks in virtual environments. And it must
integrate with the hypervisor platform to leverage familiar, native workflows.
Storage virtualization is a combination of capabilities that provide an abstraction
layer for physical storage resources to be addressed, managed and optimized in a
virtualization deployment.
Storage virtualization technology provides a fundamentally better way to manage
storage resources for virtual infrastructure, giving the ability to:
Significantly improve storage resource utilization and flexibility
Simplify OS patching and driver requirements, regardless of storage topology
Increase application uptime and simplify day-to-day operations
Leverage and complement existing storage infrastructure

2.2.3 Eucalyptus
Eucalyptus is free and open-source computer software for building Amazon Web
Service (AWS)-compatible private and hybrid cloud computing environments
marketed by the company Eucalyptus Systems. Eucalyptus enables pooling compute,


5

storage, and network resources that can be dynamically scaled up or down as
application workloads change.
Eucalyptus has six components:
The Cloud Controller (CLC) is a Java program that offers EC2-compatible
interfaces, as well as a web interface to the outside world. In addition to handling
incoming requests, the CLC acts as the administrative interface for cloud
management and performs high-level resource scheduling and system
accounting. The CLC accepts user API requests from command-line interfaces
like euca2ools or GUI-based tools like the Eucalyptus User Console and
manages the underlying compute, storage, and network resources. Only one CLC
can exist per cloud and it handles authentication, accounting, reporting, and
quote management.
Walrus, also written in Java, is the Eucalyptus equivalent to AWS Simple
Storage Service (S3). Walrus offers persistent storage to all of the virtual
machines in the Eucalyptus cloud and can be used as a simple HTTP
put/get storage as a service solution. There are no data type restrictions for
Walrus, and it can contain images (i.e., the building blocks used to launch virtual
machines), volume snapshots (i.e., point-in-time copies), and application data.
Only one Walrus can exist per cloud.
The Cluster Controller (CC) is written in C and acts as the front end for a cluster
within a Eucalyptus cloud and communicates with the Storage Controller and
Node Controller. It manages instance (i.e., virtual machines) execution and
Service Level Agreements (SLAs) per cluster.
The Storage Controller (SC) is written in Java and is the Eucalyptus equivalent
to AWS EBS. It communicates with the Cluster Controller and Node Controller
and manages Eucalyptus block volumes and snapshots to the instances within its
specific cluster. If an instance requires writing persistent data to memory outside
of the cluster, it would need to write to Walrus, which is available to any instance
in any cluster.
The VMware Broker is an optional component that provides an AWS-compatible
interface for VMware environments and physically runs on the Cluster
Controller. The VMware Broker overlays existing ESX/ESXi hosts and
transforms Eucalyptus Machine Images (EMIs) to VMware virtual disks. The
VMware Broker mediates interactions between the Cluster Controller and
VMware and can connect directly to either ESX/ESXi hosts or to vCenter Server.
The Node Controller (NC)

2.2.4 UEC
Prerequisites
To deploy a minimal cloud infrastructure, youll need at
least two dedicated systems:
A front end.


6

One or more node(s).

The following are recommendations, rather than fixed requirements.
However, our experience in developing this documentation indicated the
following suggestions.
Front End Requirements

Use the following table for a system that will run one or more of:
Cloud Controller (CLC)

Cluster Controller (CC)

Walrus (the S3-like storage service)

Storage Controller (SC)

Eucalyptus - Elastic Utility Computing Architecture for Linking Your Programs To
Useful Systems. An open source project originally from the University of California
at Santa Barbara, now supported by Eucalyptus Systems, a Canonical Partner.

Front-end - Physical machine hosting one (or more) of the high level Eucalyptus
components (cloud, walrus, storage controller, cluster controller).

Node - A node is a physical machine that's capable of running virtual machines,
running a node controller. Within Ubuntu, this generally means that the CPU has
VT extensions, and can run the KVM hypervisor.


Table 2.1 Front End Requirements


7



Node Requirements:

Table 2.2 Node Requirements















8


2.3 Literature Review:
Storage
Management in
Virtualized
Cloud Environment

Sankaran Sivathanu, Ling Liu, Mei
Yiduo, and Xing Pu
In order to enable & support cloud-
based applications in a cost-effective
manner and to lower the barrier to entry
for such applications, a somewhat recent
business model that has emerged is that
of a service provider hosting third-party
applications in a shared data centre.
A key technology that enables such
cost-efficient resource sharing in such
data centers is virtualization; by
providing isolation and separate
protection domains for multiple third
party applications, virtual machines
enable server consolidation, and
consequently significant savings in the
cost of managing such data centres.
A detailed experimental analysis of the
I/O performance observed in a
virtualized environment
through a series of systematic
experiments on the Xen VMM platform
show that the storage I/O performance
observed by an application inside a VM
depends on various aspects such as
which disk partition the virtual machine
uses, the CPU utilization of other VMs
on that system, the nature of I/O
performed by other VMs on the hosted
system and the block size used for
performing disk I/O
As the disk seek distance is vastly
reduced when the virtual disk size is
small, the throughput achieved by a
4GB virtual disk is significantly better
than that of a 40GB disk for the exactly
same workload. When disproportionate
amount of disk space, with respect to
data footprint of the workload, is
allocated for a user, some file system
allocation policies or application
decisions might lead to data being
spread across a larger footprint on disk.
Cloud service providers can improve
their storage efficiency by practicing
thin provisioning of their storage
resources. Customers of cloud storage
can purchase storage based on their


9

expected workload footprint rather than
just the price. This is important because,
customers may tend to purchase more
disk space than what they need,
especially when the price differential is
not significant. It is beneficial to place
sequentially accessed virtual disk in the
outer zone of the disk to achieve better
performance (Eg. VMs that host movies,
large OS distributions, etc.,). When a
cloud service provider needs to place
multiple virtual disks in a single
physical storage, those virtual disks
whose
Most of the infrastructure cloud systems
employ Dynamic Frequency Scaling
technique to conserve power. Therefore,
strategies for co-locating workloads can
be critical for achieving maximum
performance and at the same time
conserving power. Cloud service
providers should avoid co-locating VMs
with two sequential I/O workloads on a
single physical machine, even when
each of them goes to separate physical
disks.
Virtualization overhead is more felt in
sequential workloads accessed through
smaller block sizes than random
workloads. The cloud customers can
estimate the expected performance
based on the characteristics of the
workload they deploy. Cloud service
providers can practice static partitioning
of memory across multiple VMs hosted
in a single physical machine, whenever
strict performance isolation is important
across different customers.
Secure Data storage
and Retrieval in the
Cloud

Vaibhav Khadilkar, Anuj Gupta,
Murat Kantarcioglu, Latifur Khan,
Bhavani Thuraisingham

A new web-based system is created
(Hive Access System) that minimizes
storage costs and provides sufficient
level of security in cloud.

The Hadoop Distributed File System is
combined with Hive to provide a
common storage area for participating
organizations.

The System has a mechanism to load


10

and query shared data securely that is
stored in HDFS using Hive.

It also has an additional layer of security
above HDFS and Hive using a XACML
policy-based mechanism.

The system provides a secure login
feature to users based on a salted hash
technique. When a user logs into our
system, based on the group that the user
belongs to he/she is provided with
different options.

The system also allows new users to
register but only a designated special
user admin can assign these users to
the appropriate groups.

The System is provided with some basic
query rewriting rules that abstract users
from the query language of Hive
(HiveQL). This allows them to enter
regular SQL queries in the web
application which are translated into
HiveQL using the basic rewriting rules.

This system is different from AWS in
the sense that our cloud infrastructure is
completely private versus AWSs
infrastructure that is in the public
domain.
This distinguishing factor makes our
infrastructure trusted over the AWS
infrastructure where data must be stored
in an encrypted format since AWS is in
the public domain. In the future we plan
to extend our work to include both,
public and private clouds.


11


Evolution of Cloud
Storage as Cloud
Computing
I nfrastructure
Service
R. Arokia Paul Rajan, S.
Shanmugapriyaa
Private and Public clouds serve as the
backbone for a variety of different cloud
computing service models.

Currently the industry has been
successfully adopting three common
types of cloud computing service
models.

Infrastructure as a Service (IaaS), is a
service model around servers (compute
power), storage capacity, and network
bandwidth. Examples include Amazon
EC2 and S3, Rackspace, AT&T, and
Verizon.

Platform-as-a-Service (PaaS) provides
an externally managed platform for
building and deploying applications and
services. This model typically provides
development tools such as databases and
development studios for working with
the supplied frameworks, as well as the
infrastructure to host the built
application. Examples include
Force.com, Microsoft Azure, and
Google App Engine.

Software-as-a-Service (SaaS) is simply
having a software system running on a
computer that doesn't belong to the
customer and isn't on the customers
premises. It is based on the concept of
renting an application from a service
provider rather than buying, installing
and running software yourself.

No need to invest any capital on storage
devices.
No need for technical expert to maintain
the storage, backup, replication and
importantly disaster management.
Allowing others to access your data will
result with collaborative working style
instead of individual work.

A Cloud Storage Application
Programming Interface (API) is a
method for access to and utilization of a
cloud storage system. The most


12

common of these kinds are REST
(REpresentational State Transfer)
although there are others, which are
based on SOAP (Simple Object Access
Protocol).

All these APIs are associated with
establishing requests for service via the
Internet. REST is a concept widely
recognized as an approach to "quality"
scalable API design. One of the most
important features of REST is that it is a
"stateless" architecture.

This means that everything needed to
complete the request to the storage
cloud is contained in the request, so that
a session between the requestor and the
storage cloud is not required.
A Secure Model for
Cloud Computing
Based Storage and
Retrieval

Yaga Reddemma, Lingala
Thirupathi , Sreekanth Gunti

The proposed business model separates
data storage service from that of
encryption and decryption service. This
paper uses CRM concept to demonstrate
the new mechanism proposed.

In the proposed system the party that
uses cloud storage services must encrypt
data before sending it to cloud while the
service provider who is responsible for
encryption/decryption must delete data
once encryption/decryption process is
completed.

This separation is required as the cloud
server administrators might have illegal
access to data of the users. To prevent
this, the services such as storage and
encryption/decryption are separated and
moved to different cloud servers.
Generally users use cloud environment
for specific purposes. For instances
SAPs ERP services, Salesforce.coms
CRM service and so on. The data
generated by these operations is saved to
cloud storage. However, this study
advocates an additional cloud server that
takes care of encryption/decryption
activities which are independent of
storage service.


13


First of all users credentials are
authenticated by CRM cloud service.
Once authentication is done user can
access CRM server through which he
performs data retrieval and data storage
operations. As per the users instructions
the CRM cloud service interacts with
storage cloud service and makes data
usage request. Then the storage cloud
service sends encrypted data which is
available in to encryption/decryption
service and requests for decryption. The
encryption/decryption service takes
encrypted data and simply decrypts it
and sends the decrypted data to CRM
cloud service. SSL (Secure Sockets
Layer) is used for encryption and
decryption purposes. The last step is that
the CRM cloud service sends requested
data to end user. Thus secure
communication is taking place across all
components as part of proposed data
retrieval mechanism.

After due authentication, the end users
sends data storage request to CRM
cloud service.
In turn the CRM cloud service sends
unencrypted data to
encryption/decryption cloud service.
The encryption/decryption cloud service
actually encrypts the given content and
sends it to storage cloud service where it
is stored. Then the storage cloud service
sends resultant message to CRM cloud
service, which is then displayed to the
user.








14

Chapter 3
DESIGN AND ARCHITECTURE
3.1 Eucalyptus Architecture

Fig 3.1 Eucalyptus Architecture
3.1.1 System Architecture:

Cluster
Data Store
Server
Node Controller
Node Controller
Private Cloud
Web Interface or EC2 API
Cloud Controller
Walrus Storage Controller
Regular Network
Fig 3.2 Project System Architecture


15

The Node Controller (NC) is part of the Node level of the cloud
computing architecture. It is written in C and hosts the virtual machine
instances and manages the virtual network endpoints. The NC downloads
and caches images from Walrus as well as creates and caches instances.
While there is no theoretical limit to the number of NCs per cluster,
performance limits do exist.
Cluster Controller (CC)
The Cluster Controller (CC) is written in C and acts as the front end for a cluster
within a Eucalyptus cloud and communicates with the Storage Controller (SC) and
Node Controller (NC). The CC manages instance (i.e., virtual machines) execution
and Service Level Agreements (SLAs) per cluster.
Storage Controller (SC)
The Storage Controller (SC) is written in Java and is the Eucalyptus equivalent to
AWS Elastic Block Store (EBS). The SC communicates with the Cluster Controller
(CC) and Node Controller (NC) within the distributed cloud architecture and
manages Eucalyptus block volumes and snapshots to the instances within its specific
cluster. If an instance requires writing persistent data to memory outside of the
cluster, it would need to write to Walrus, which is available to any instance in any
cluster. The SC interfaces with storage systems, including local, NFS, iSCSI, and
SAN.

VMware Broker (Optional)
The VMware Broker is an optional component that provides an AWS-compatible
interface for VMware environments and physically runs on the Cluster Controller
(CC) within the distributed cloud computing architecture. The VMware Broker
overlays existing ESX/ESXi hosts and transforms Eucalyptus Machine Images
(EMIs) to VMware virtual disks. The VMware Broker mediates interactions
between the CC and VMware and can connect directly to either ESX/ESXi hosts or
to vCenter Server.












16

3.2 Class Diagram:

#Verify User()
#Process Command()
+ID : string
+User Name : string
#Password : string
Controller
-Send Requests to Server()
Node
-Store Files()
-Retrieve Files()
#Delete Files()
Server
+Log-in()
#Store Files()
#Share Files()
#Download Files()
#Delete Files()
+ID : string
#Password : string
#File ID : string
User
#Files
Data Store
1
1 0..*
1
1..*
0..*
-Manage Cloud Server()
-Manage Users()
-Manage Nodes()
-Manage Data Store()
Cloud Administrator
1..*
1
1
1..*
+Maintain Nodes()
-Node List : string
Cluster
+Maintain Clusters()
-Cluster List : string
Cloud
-Data Store Information
Storage
+Manage Storage()
Walrus
1 *
1
*
1
*
1
*
1
*

Fig 3.3 Class Diagram



















17

3.3 Use Case Diagram :

User
Server
Storage
Administrator
Store Files
Share Files
*
* *
*
uses
*
*
*
*
Manage Users
*
*
*
*
Store/Retrieve Data
*
*
Manage Storage
1
* *
*
*
1
Intelligent Storage
Read Share
uses
Read-Write Share
uses
Sign up
*
* *
*
Login
*
*
*
*

Fig 3.4 Use Case Diagram















18

3.4 Activity Diagram:

Data Store Server User
Share File
Check Share Type
Display Result
Grant Read Access
Grant Write Access
Read
Read-Write
Forward Result
Grant Read Access

Fig 3.5 Activity Diagram Sharing a File

















19

3.5 Sequence Diagram:
User Server
Data Store
Send Data to be stored
[Success] Send Notification
Display Result
Forward Data
Set Permissions
Intelligent Storage

Fig 3.6 Sequence Diagram File Upload










20

3.6 Experiment Setup:
The Private Cloud is implemented in a virtual environment using VMware.
The Cloud used here is Ubuntu Enterprise Cloud.
The Server includes cloud controller, cluster controller, storage controller,
Walrus Storage service.
The complete installation procedure is very detailed and thus only basic steps
are mentioned here.
Install and Configure the Node Controller(s)
Installing the Eucalyptus Node Service on a running Ubuntu system consists
of doing the following on one or more systems:
1. Install Ubuntu 10.04 Server

Fig 3.7 Install Screen


21


Fig 3.8 Install Screen 2

Fig 3.9 Install Screen 3


22

2. Update to the most current state in the Ubuntu archive:
sudo apt-get update
sudo apt-get dist-upgrade
3. Install the eucalyptus-nc package
sudo apt-get install eucalyptus-nc
4. Install the Node controller on a different Virtual Machine.
5. Repeat steps 2 and 3 for the node controller.
6. Finally, you need to install the Cloud Controller's eucalyptus user's public ssh
key into the Node Controller's eucalyptus user's authorized_keys file. The
easiest way to do this:

1. On the Node Controller, temporarily set a password for
the eucalyptus user:
o sudo password eucalyptus
2. Then, on the Cloud Controller:
o sudo -u eucalyptus ssh-copy-id -i ~eucalyptus/.ssh/id_rsa.pub
eucalyptus@<IP_OF_NODE>
3. You can now remove the password of the eucalyptus account on the
Node:
o sudo password -d eucalyptus
3.6.1 Obtain Credentials
After installing and booting the Cloud Controller, users of the cloud will need to
retrieve their credentials. This can be done either through a web browser, or at the
command line.
From a Web Browser
1. From your web browser (either remotely or on your Ubuntu server) access the
following URL:
https://<cloud-controller-ip-address>:8443/
Important! You must use a secure connection, so make sure you use "https"
not "http" in your URL. You will get a security certificate warning. You will
have to add an exception to view the page. If you do not accept it you will not
be able to view the Eucalyptus configuration page.
2. Use username 'admin' and password 'admin' for the first time login (you will be
prompted to change your password).


23

3. Then follow the on-screen instructions to update the admin password and email
address.
4. Once the first time configuration process is completed, click the 'credentials'
tab located in the top-left portion of the screen.

5. Click the 'Download Credentials' button to get your certificates

6. Save them to ~/.euca


7. Unzip the downloaded zipfile into a safe location (~/.euca)

unzip -d ~/.euca mycreds.zip
From a Command Line

1. Alternatively, if you are on the command line of the Cloud Controller, you can
run:

2. mkdir -p ~/.euca
3. chmod 700 ~/.euca
4. cd ~/.euca
5. sudo euca_conf --get-credentials mycreds.zip
6. unzip mycreds.zip
7. ln -s ~/.euca/eucarc ~/.eucarc

3.6.2 Extracting and Using Your Credentials

Now you will need to setup EC2 API and AMI tools on your server using X.509
certificates.

1. Install the required cloud user tools:

sudo apt-get install euca2ools
2. To validate that everything is working correctly, get the local cluster
availability details:

3. . ~/.euca/eucarc
4. euca-describe-availability-zones verbose
5. AVAILABILITYZONE myowncloud 192.168.1.1
6. AVAILABILITYZONE |- vm types free / max cpu ram disk
7. AVAILABILITYZONE |- m1.small 0004 / 0004 1 192 2
8. AVAILABILITYZONE |- c1.medium 0004 / 0004 1 256 5
9. AVAILABILITYZONE |- m1.large 0002 / 0002 2 512 10
10. AVAILABILITYZONE |- m1.xlarge 0002 / 0002 2 1024 20


24

AVAILABILITYZONE |- c1.xlarge 0001 / 0001 4 2048 20
3.6.3 Install an image from the store
The following is by far the simplest way to install an image. However, advanced
users may be interested in learning how to Bundle their own image.
The simplest way to add an image to UEC is to install it from the Image Store on the
UEC web interface.
1. Access the web interface at the following URL (Make sure you specify https):
https://<cloud-controller-ip-address>:8443/
2. Enter your login and password (if requested, as you may still be logged in from
earlier)
3. Click on the Store tab

Fig 3.10 UEC store
4. Browse available images

5. Click on install for the image you want


Once the image has been downloaded and installed, you can click on "How to run?"
that will be displayed below the image button to view the command to execute to
instantiate (start) this image. The image will also appear on the list given on the
Image tab.


25


Fig 3.11 UEC Images
3.6.4 Run an Image
There are multiple ways to instantiate an image in UEC:
1. Use the command line

2. Use one of the UEC compatible management tools such as Landscape


3. Use the ElasticFox extension to Firefox
Here we will describe the process from the command line:
1. Before running an instance of your image, you should first create a keypair
(ssh key) that you can use to log into your instance as root, once it boots. The
key is stored, so you will only have to do this once. Run the following
command:
2. if [ ! -e ~/.euca/mykey.priv ]; then
3. mkdir -p -m 700 ~/.euca
4. touch ~/.euca/mykey.priv
5. chmod 0600 ~/.euca/mykey.priv
6. euca-add-keypair mykey > ~/.euca/mykey.priv
7. fi
Note: You can call your key whatever you like (in this example, the key is
called 'mykey'), but remember what it is called. If you forget, you can always
run euca-describe-keypairs to get a list of created keys stored in the system.
8. You must make sure to source ~/.euca/eucarc before you run any of the
eucatools. It is probably best to add this to the bottom of your .bashrc script.
9. You must also allow access to port 22 in your instances:
euca-authorize default -P tcp -p 22 -s 0.0.0.0/0
10. Next, you can create instances of your registered image:
euca-run-instances $EMI -k mykey -t m1.small


26

Note: If you receive an error regarding image_id, you may find it by viewing
Images page or click "How to Run" on the Store page to see the sample
command.
11. The first time you run an instance, the system will be setting up caches for the
image from which it will be created. This can often take some time the first
time an instance is run given that VM images are usually quite large. To
monitor the state of your instance, run:
watch -n5 euca-describe-instances
In the output, you should see information about the instance, including its state.
While first-time caching is being performed, the instance's state will be
'pending'.
12. When the instance is fully started, the above state will become 'running'. Look
at the IP address assigned to your instance in the output, then connect to it:
13. IPADDR=$(euca-describe-instances | grep $EMI | grep running | tail -n1 | awk
'{print $4}')
ssh -i ~/.euca/mykey.priv ubuntu@$IPADDR
14. And when you are done with this instance, exit your SSH connection, then
terminate your instance:
15. INSTANCEID=$(euca-describe-instances | grep $EMI | grep running | tail -n1 |
awk '{print $2}')
euca-terminate-instances $INSTANCEID


The Node uses the node controller and the server uses the cloud controller,
cluster controller and the storage controller.

After installing and configuring server and node check the servers IP address
using :

o Ifconfig

Update the server and node using :

o sudo apt-get update
o sudo apt-get u upgrade


27


Test the server-node connection using:

o ping

Install desktop on the node using:
sudo apt-get install ubuntu-desktop

On the nodes browser, connect to the server by giving the address
https://<server-ip>:8443/

Perform first time configuration.
On the server side run :

o sudo vi /usr/lib/python2.6/dist-packages/imagestore/lib/fetch.py

Add the following lines after line 142 :

o curl.setopt(pycurl.SSL_VERIFYPEER, 0)
o curl.setopt(pycurl.SSL_VERIFYHOST, 0)

Save and close the file.
Get certificates for the server using:

o sudo wget P /usr/local/share/ca-certificates/ --no-check-certificate
https://certs.godaddy.com/repository/gd-class2-root.crt
https://certs.godaddy.com/repository/gd_intermediate.crt
https://certs.godaddy.com/repository/gd_cross_intermediate.crt


28


Update server certificates using:

o sudo update-ca-certificates

Get the server credentials using:

o mkdir -p ~/.euca
o chmod 700 ~/.euca
o cd ~/.euca
o sudo euca_conf --get-credentials mycreds.zip
o unzip mycreds.zip
o ln -s ~/.euca/eucarc ~/.eucarc
o cd

Install euca tools using:

o sudo apt-get install euca2ools

Get availability details using:

o . ~/.euca/eucarc
o euca-describe-availability-zones verbose

On the node side, open the server login page; go to the store tab and
download the image suitable for the cloud used.

Create a key-pair using:



29

o if [ ! -e ~/.euca/mykey.priv ];mkdir -p -m 700 ~/.euca
o touch ~/.euca/mykey.priv
o chmod 0600 ~/.euca/mykey.priv
o euca-add-keypair mykey > ~/.euca/mykey.priv fi
Source the eucarc file using:
o . ~/.euca/eucarc

Allow access to port 22 using:
o euca-authorize default -P tcp -p 22 -s 0.0.0.0/0

Run the image using:

euca-run-instances <instance-id> -k mykey -t m1.small
































30

Chapter 4
Implementation

4.1 Methodology Used:

Waterfall Model is adopted as the design process for this project. The
Waterfall development Model is a sequential design processes, in which progress is
seen as flowing steadily downwards (like a waterfall) through the phases of
Requirements Specification, Design, Construction (Implementation or Coding),
Integration, Testing and Debugging (Validation), Installation, and Maintenance.













Fig 4.1 Waterfall Model
The selection of waterfall model has the following reasons:
a) Requirements are clear and unambiguous.
b) Project Reviews are conducted at the end of each phase of the
development.
c) The feedback from the reviews is followed up at the end of each phase
of the development.


Requirements
Design
Construction
Integration
Testing


31

4.2 Implementation of the System:
The project should be developed as a File sharing website on a private cloud
environment. The website is run on the cloud server and thus can be easily accessed
by the nodes in the cluster. The uploaded files are store on the server and share by
using intelligent storage management. The website consists of the following modules
Sign-up module
Log-in module
File upload module
File index module
File access module
File download module
File share module
File delete module
This is the homepage for the project website. Users can sign-up, log-in, learn how to
use the system and contact the administrators using the various links provided at the
bottom of the website.

Fig 4.2 Home Page


32

4.2.1 Sign-up module:
In this module the user registers on the website by giving the appropriate information
such as: User name, Password, Email. When the user clicks the Register button the
server creates a folder for the user where the files uploaded by the user will be saved.


Fig 4.3 Sign up Page





















33

4.2.2 Log-in module:

The user uses this screen to log-in to the website. If the user provides the correct
credentials, they are logged in to the system and their user name is stored in a session
variable. After the user has logged in the Sign up and Login links on the bottom
of the website are replaced by Access Files and Upload Files links which give
access to upload files and allow the user to access the uploaded files respectively.







Fig 4.5 Log-in page
















34

4.2.3 File Upload module:

After the user has logged in to the system they can upload files to the cloud by using
the upload module. In this module the user points to the path of the files and clicks
upload. The server then checks if any file with the same name is already present. If
yes then it checks whether the user has read or read- write permission. If the user has
only read permission then Access Denied message will be displayed. If the user
has read- write permission then an Are you sure? message will be displayed to get
user confirmation before uploading the file to the server. If the file doesnt exist in
the database then the file is simply uploaded. After uploading an entry for the file is
created in the database with the read and write permission set to the user or if file
already exists on the server the database entry is updated.




Fig 4.6 Upload Page




35


Fig 4.7 File successfully uploaded





Fig 4.8 Confirm Replace Page






36

4.2.4 File index module:

After the user has uploaded one or more files, they can access the uploaded files by
using the file index module. When the user clicks the Access files link the server
accesses the file table in the database and finds the files for which the user has read
or write permission and retrieves them. These files are displayed in a table format
with three columns: File Name, Uploaded by and Last Updated. The File names
are displayed as a link and by clicking the links the user can access the respective
files.







Fig 4.9 File Index















37

4.2.5 File Access module:

The user accesses the individual files using the file access module. In this module the
file is retrieved from storage and displayed to the user. If the user has only read
permission then they can only view and download the file. If the user has read and
write permissions then the user can share, download and delete the uploaded file. On
the left hand side of the webpage, information about the file is displayed. This
includes: File name, Uploaded by, File size and Last updated.






Fig 4.10 Access File

















38

4.3.6 File Download module:

The user downloads the file using the download module. In this module the relevant
file is accessed in storage, its data is read and outputted to the user in the form of a
file. The method of download varies based on browser and users softwares.










Fig 4.11 File Download
















39

4.2.7 File Share Module:

The user shares the uploaded file using the file share module. In this module the user
first gives the ID of the user to which file should be shared and then gives the type of
sharing to be done, whether it is a read share or a read-write share. When the user
clicks the share button the server checks the type of share that the user has chosen
and updates the privileges accordingly. There is no real sharing of files done here
(thought that can be easily added), but rather granting permission to access the
uploaded files. The shared files are shown to the appropriate user when they try to
access the uploaded files.

Fig 4.12 File Share


Fig 4.13 Shared File


40

4.2.8 File Delete module:

The user deletes the uploaded files using the delete module. When the user is
viewing the uploaded file, if they click the Delete link then they are forwarded to
the delete module. First the website checks whether the user has read-write privilege
and if yes it then asks the user for confirmation before deleting the file. If the user
confirms deletion the file is deleted and the associated entry in the database table is
removed.



Fig 4.14 Confirm Deletion


Fig 4.15 File Deleted


41

4.3 Testing -Unit Test Case Specification

4.3.1 UTC 1 - Test whether the website can be accessed within the
cluster.
Test Case ID UTC 1
Test Case Description

Test whether the output of the application deployed in
Ubuntu is displayed correctly.
Expected Output The output of the application should be displayed.
Test Setup The application starts running in the



Test Procedure

Step Input Expected Output
1. User types the address in the
browser
The application displays the output in
the browser






















42

4.3.2 UTC 2Test whether the user is able to register


Test Case ID UTC 2
Test Case Description

Test whether the user is able to register to upload a file.
Expected Output The output of the application should display that the user
is successfully registered to upload a file
Test Setup The application deployed takes the values given and
validates the user as a new user.

Test Procedure

Step Input Expected Output
1. User types only the username/only
password or username & password
only and clicks enter.
The application asks for a password
or email id .
2. User types the password and
confirm password differently
The applications displays the output
as passwords dont match
3. The passwords match and user name
and all are specified
The output is displayed as
successfully registered.

















43



4.3.3 UTC 3Test whether the user is able to upload files



Test Case ID UTC 3
Test Case Description

Test whether the user is able to upload a file.
Expected Output The output of the application should display that the user
has successfully uploaded the file
Test Setup The application deployed validates the user and uploads
the file as he wishes.

Test Procedure

Step Input Expected Output
1. User selects the file from the disk
and clicks submit
The application accepts it and loads
the file into the cloud.
2. User selects a very large file The applications displays that very
large files cant be uploaded
3. The user selects an empty file The output is displayed as 0kb file
uploaded.













44


Fig 4.16 File Uploaded








4.3.4 UTC 4Test whether another user is able to access uploaded
files


Test Case ID UTC 4
Test Case Description

Test whether another user is able to see the uploaded file.
Expected Output The output of the application should display that the user
is able to see another
Test Setup The application deployed is able to display the uploaded
files to the user once






45

Test Procedure

Step Input Expected Output
1. User types only the username/only
password and clicks enter.
The application asks for a password.
2. User types the password differently The applications displays the output
as wrng password
3. The user successfully logs in and
clicks access files
The output is displayed with the files
that have been uploaded by him and
those files which have been shared to
him.







Fig 4.17 File Index view









46

4.3.5 UTC 5Test whether the user is able to download files



Test Case ID UTC 5
Test Case Description

Test whether the user is able to download an uploaded
file.
Expected Output The output of the application should display that the user
is able to download and save it in a desired location
Test Setup The application deployed downloads the file uploaded

Test Procedure

Step Input Expected Output
1. User successfully logs in The application displays the
username and he can access or
upload files.
2. User accesses the files stored in the
cloud
The application displays the output as
a table with the list of files that have
been uploaded
3. The user clicks on the file and
selects download
A window pops up with a message
on where to store the file .
















47

4.3.6 UTC 6Test whether the user is able to share the upload files

Test Case ID UTC 6
Test Case Description

Test whether the user is able to share the uploaded file.
Expected Output The output of the application should display that the
users enabled in the share are able to access those files.
Test Setup The application deployed takes the username of those to
whom the files are to be shared.

Test Procedure

Step Input Expected Output
1. User types a valid username and
clicks share
The application asks for the mode of
sharing whether it is read or read
write
2. User types The applications displays the output
as passwords dont match
3. The passwords match and user name
and all are specified
The output is displayed as
successfully registered.











48


Fig 4.18 File Share Test



Fig 4.19 File Share Test 2










49

4.3.7 UTC 7Test whether the user is able to delete the uploaded
file(s)



Test Case ID UTC 7
Test Case Description

Test whether the user is able to delete the uploaded file.
Expected Output The output of the application should display that a file
has been deleted from the storage cloud
Test Setup The application deployed selects the name of the file to
be removed and deletes it accordingly

Test Procedure

Step Input Expected Output
1. A user who has uploaded a file
clicks the access file option
The list of files uploaded are
displayed in a table.
2. The user then clicks the file which
he had uploaded
The output is a new window with 3
options like download , share and
delete.
3. The user clicks the delete option The output is displayed as a pop up
saying that the user is about to delete
the file permanently.
1. The user clicks on submit The file is deleted permanently from
the cloud.












50

Chapter 5
CONCLUSION AND FUTURE WORK

5.1 Conclusion

The website has been successfully developed and deployed in the private cloud
environment. Thus the website provides file upload, file download and file sharing
features and uses intelligent storage management mechanisms to increase security
and the efficiency of memory management. This website can be easily deployed in
any private cloud environment to promote file sharing.


































51


5.2 Future Work:

This can be extended to a large software application which requires
secured file storage, secured file access and a secured file sharing,
uploading facility in a cloud based environment. The application can also
improve the existing storage mechanism facility of any currently
deployed cloud storage application. The application can also be useful
for integrating other cloud storage facility so as to enhance security,
transparency, simplicity for small as well as large files. The application
can be useful for simplifying the cloud storage mechanism by ensuring
that cloud storage need not require an expounding method to provide
such a facility.





























52

References:

[1]. Sankaran Sivathanu, Ling Liu, Mei Yiduo and Xing Pu, Storage
Management in Virtualized Cloud Environment in IEEE 3rd International
Conference on Cloud Computing (CLOUD), 2010, PP 204-211.

[2]. Vaibhav Khadilkar, Anuj Gupta, Murat Kantarcioglu, Latifur Khan,
Bhavani Thuraisingham, Secure Data storage and Retrieval in the Cloud, in
6th International Conference on Collaborative Computing: Networking,
Applications and Worksharing (CollaborateCom), 2010, pp 1-8.

[3]. R. Arokia Paul Rajan, S. Shanmugapriyaa, Evolution of Cloud Storage as
Cloud Computing Infrastructure Service, in arXiv:1308.1303v1 [cs.DC].

[4]. Yaga Reddemma, Lingala Thirupathi , Sreekanth Gunti, A Secure Model
for Cloud Computing Based Storage and Retrieval, in IOSR Journal of
Computer Engineering (IOSRJCE) Volume 6, Issue 1 (Sep-Oct. 2012), pp 01-
05 .

[5]. Cong Wang, Qian Wang, Kui Ren, Ning Cao and Wenjing Lou, Towards
Secure and Dependable Storage Services in Cloud Computing, in IEEE
Transactions on Services Computing, vol. 5, no. 2, pp. 220-232, Second 2012,
doi:10.1109/TSC.2011.24 .

[6]. Cong Wang, Qian Wang, Kui Ren, Wenjing Lou Privacy-Preserving
Public Auditing for Data Storage Security in Cloud Computing, in 2010
Proceedings IEEE INFOCOM, pp 1 - 9.

[7]. James Broberg, Rajkumar Buyya, Zahir Tari, MetaCDN: Harnessing
Storage Clouds for high performance content delivery, in Journal of
Network and Computer Applications, Volume 32, Issue 5, September 2009,
Pages 10121022 .