Augmented Reality Using Android OS

Abstract—
Augmented Reality or AR is an emerging technology in which one’s perception of the real-time environment is
enhanced by superimposing computer-generated information such as graphical, textual, or audio content, as
well as objects onto a display screen. The proposed application is an android mobile based application which
will be compatible with all the existing and upcoming versions of the operating system. The idea is to allow the
user to view the virtual object in the real world using a marker based AR system. The user could provide images
of the object which would be the front, back, top, bottom, left and right side pictures of the object. They will be
placed onto a 3D cube which will make up the complete virtual object. Thus an extended environment will be
created through the amalgamation of real world and generated object and it will appear as though the real-
world object and virtual object coexist within the environment. The advantages of this application as compared
to the already existing 2D application are that it would display object in 3D and enable the user to rotate it
virtually. It is inexpensive as the user need not actually purchase the object to see how it fits in the
environment, instead he can try before the purchase itself.

INTRODUCTION
Augmented reality (AR) is a live, direct or indirect, view of a physical, real-world environment whose elements
are augmented by computer-generated sensory input such as sound, video, graphics or GPS data. Augmented
Reality is a type of virtual reality that aims to duplicate the world's environment in a computer. Virtual reality
(VR) is a virtual space in which players immerse themselves into that space and exceed the bounds of physical
reality . It adds information and meaning to a real object or place. Augmented reality is characterized by the
incorporation of artificial or virtual elements into the physical world as shown by the live feed of the camera, in
real-time. Common types of augmented reality include projection, recognition, location and outline [3].

Projection: It is the most common type of augmented reality, projection uses virtual imagery to augment what
you see live. Some mobile devices can track movements and sounds with a camera and then respond. Virtual or
projection keyboards, which one can project onto to almost any flat surface and use, are examples of augmented
reality devices that use interactive projection.

Recognition: Recognition is a type of augmented reality that uses the recognition of shapes, faces or other real
world items to provide supplementary virtual information to the user in real-time. A handheld device such as a
smart phone with the proper software could use recognition to read product bar codes and provide relevant
information such as reviews and prices or to read faces and then provide links to a person's social networking
profiles.

Outline: Outline is a type of augmented reality that merges the outline of the human body or a part of the
body with virtual materials, allowing the user to pick up and otherwise manipulate objects that do not exist in
reality. One example of this can be found at some museums and science centers in the form of virtual
volleyball. Although the player can stand and move on an actual court, the ball is projected on a wall behind
him, and he can control it with an outline of himself, which is also projected on the wall Using the concept of
 Augmented Reality Using Android OS

augmented reality our project focuses on creating a very useful android mobile based application. The idea is
to allow the user to view the virtual object in the real world. The user could provide images of the object which
would be the front, back, top, bottom, and left and right side pictures of the object. They will be placed onto a
3D cube which will make up the complete virtual object. Thus an extended environment will be created
through the amalgamation of real world and generated object and it will appear as though the real-world
object and virtual object coexist within the environment. In order to use this application, the user will first
need to acquire a marker. A marker is a piece of paper with black and white markings. This is used to display
the augmented object on your mobile phone’s screen. Marker-based augmented reality uses a camera and a
visual marker which determines the centre, orientation, and range of its spherical coordinate system. Once the
marker is present one can view augmented objects. Virtual object interaction is another added feature
wherein the user can rotate or change the orientation of the object according to his requirements

VIRTUAL REALITY
The term "artificial reality", coined by Myron Krueger, has been in use since the 1970s; however, the origin of
the term "virtual reality" can be traced back to the French playwright, poet, actor, and director Antonin
Artaud. Virtual Reality is a computerized simulation of natural or imaginary reality. Often the user of VR is fully
or partially immersed in the environment. Full immersion refers to someone using a machine to shield herself
from the real world. VE is the term used to describe the scene created by any computer program in which the
user plays an interactive role within the context of the computer generated three dimensional world. The user
represents an actor within the system and has an essential presence within the virtual world. Virtual reality
(VR) is a term that applies to computer-simulated environments that can simulate physical presence in places
in the real world, as well as in imaginary worlds.

Advantages: Many different fields can use VR as a way to train students without actually putting anyone in
harm's way. This includes the fields of medicine, law enforcement, architecture and aviation. VR also helps
those that can't get out of the house experience a much fuller life. Doctors are using VR to help reteach muscle
movement such as walking and grabbing as well as smaller physical movements such as pointing. The doctors
use the malleable computerized environments to increase or decrease the motion needed to grab or move an
object. This also helps record exactly how quickly a patient is learning and recovering.
Total immersion within environment.
Increased user presence perception within system.
Facilitated production of an entirely designed environment.
Existing technologies for advanced interaction.
Real-time graphical environment generation possible.

Disadvantages: The hardware needed to create a fully immersed VR experience is still cost prohibitive. The
technology for such an experience is still new and experimental. VR is becoming much more commonplace but
programmers are still grappling with how to interact with virtual environments. The idea of escapism is common
place among those that use VR environments and people often live in the virtual world instead of dealing with
the real one. One worry is that as VR environments become much higher quality and immersive, they will
become attractive to those wishing to escape real life. Another concern is VR training. Training with a VR
environment does not have the same consequences as training and working in the real world. This means that
even if someone does well with simulated tasks in a VR environment, that person might not do well in the real
world.
Not suited to real-world interaction.
Despite advances in technology equipment is still expensive.
Auxiliary senses not stimulated.
Possible lag in motion-display system.
 Augmented Reality Using Android OS

AUGMENTED REALITY
Augmented Reality (AR), also known as Mixed Reality, aims to combine virtual and real scene together to
achieve that virtual ones are belong to the real world. Being characteristic of integration of virtual and real
scene, many applications of Augmented Reality are emerging, such as in field of education, medical treatment
and entertainment

Goals of Augmented Reality:

To challenge the impossible.
To create virtual environment for a more rich user experience.
To integrate it into daily lives to help the masses.
To achieve feats which are limited in real world.
To enhance imagination of youths

PROBLEM STATEMENT
The proposed system aims to provide an environment that will help the users to place artificial 2D as well as
3D objects into real world through the use of AR Markers. The proposed system also allows the user to decide,
where to place the object in real world. Once the object has been placed in the scene, it will be displayed
accurately according to the perspective in the original scene, which is especially challenging in the case of 3D
virtual objects. The proposed system solves the problem of viewpoint tracking and virtual object interaction.
The main advantage of the proposed system is that, it is customer oriented and not product or service
oriented thus allowing the users to augment a product of their wish.

WHY ANDROID OS
Since the advent of 2010 more and more stress has been given on the usage of Free and Open Source
Software (FOSS). Android is leading the current O.S market as shown in figure 5, because it is open source and
developed by a consortium of more than 86 leading M.N.C’s called Open Handset Allowance (O.H.A). Android
also is stated as one the most rapidly growing technologies. More and more applications have been developed
and modified by third party user. Moreover, the Android O.S is user friendly. It has a great performance and
processing power. Thus, the proposed system is being developed for the most rapidly emerging and flexible
O.S- “ANDROID
Augmented Reality Using Android OS

Proposed Architecture Block Diagram

 Augmented Reality Using Android OS

Architecture Block diagram The proposed system is a marker based system and its architecture as shown in
figure 6 contains following modules.
1. Camera
2. Image Capturing Module
3. Image Processing Module
4. Rendering Module
5. Display Screen

1 Camera: A real-world live video is feed as an input from the Android cell phone camera to the Camera
module. Displaying this live feed from the Android cell phone camera is the reality in augmented reality. This
live video stream is given as an input to the Image Capturing Module.

2 Image Capturing Module:

The input to Image Capturing Module is the live video feed from the camera of a mobile device. This module
analyses the camera feed, by analysing each frame in the video. This module generates binary images i.e. a
digital image that has only two possible values for each pixel. Typically the two colours used for a binary image
are black and white. These binary images are provided as an input to Image Processing Module.

3 Image Processing Module:

Inputs to Image Processing Module are the binary images from Image Capturing Module. These binary images
are processed using an image processing technique to detect the AR Marker. Detection of AR Marker is
essential to determine the position, where to place the virtual object. Once the AR Marker is detected, its
location is provided as an input to the Tracking Module.

4 Marker Tracking Module: The tracking module is “the heart” of the augmented reality system; it calculates
the relative pose of the camera in real time. The term pose means the six degrees of freedom (DOF) position,
i.e. the 3D location and 3D orientation of an object. The calculated pose is provided as an input to Rendering
Module.

5 Rendering Module: There are 2 inputs to Rendering Module. First is the calculate pose from the Tracking
Module and other is the Virtual Object to be augmented. The Rendering Module combines the original image
and the virtual components using the calculated pose and renders the augmented image on the display screen
of the mobile device.

SOFTWARE/HARDWARE REQUIREMENTS

Hardware requirements:

Processor : Core 2Duo/Above

Ram : 2 Gb

Hard Disk : 80 Gb.

Input device : Standard Keyboard and Mouse.

Output device : VGA and High Resolution Monitor.

Augmented Reality Using Android OS

Software requirements:

Operating System : Windows Family/Android.

Language : JDK 1.6/Above

Database : MySQL 5.1/Above

Tool : Android Eclipse Sdk

Developing a mobile phone application presents several technological challenges.

Typically the application developed is restricted with the mobile Hardware &
Software platform. The application to be developed has development dependencies
at different levels. Development platforms are the set of Application Programming
Interfaces (API's) that provide access to certain mobile device and network
functionalities that are used by the applications to function on a particular mobile
device. The figure below shows the stacked view of development dependencies of
a Mobile phone application. Developer must be fully aware of the consumer market
available for the specific application and the platform the application is to be fully deployed.
Augmented Reality Using Android OS

A survey conducted by Earl Oliver and David R. Cheriton discusses various mobile platforms
available for application development; the report says the proliferation of the mobile
phones created an opportunity for mobile researchers, but selecting an accurate platform
from the available platforms is the unresolved question.

2.2Android Architecture

Developing a mobile application for native platforms gives access to Device features like
location, communication etc, this access to the primary device features increase the
application performance and increases the security aspects of the application. The
architecture of the android must be clearly understood before developing any application
for android. (Padiadpu, 2008)
Augmented Reality Using Android OS

The bottommost layer of Android architecture is the internal operating system

Linux 2.6 Kernel, which includes all the necessary device drivers like display,
camera, Audio etc., this layer manages memory, performance, network and all the
basic essentials that required for the device functioning. The next layer above the
Linux kernel consists of libraries that act an interface between application
framework and kernel. The application framework layer is entirely written in Java
programming language and includes main components like Activity Manager,
Package Manager, Window Manager, Telephony Manager and Content Provider. All the
applications in built and third party applications run on the top most layer of the overall
architecture. By this constructive architecture and open source developing applications for
Android is a real win-win situation.

Android 1.5 SDK

Augmented Reality Using Android OS

Android 1.5 SDK is a set of development tools to create applications for Android Mobile Platf
orm or Android powered mobile devies. This development kit includes a debugger, libraries,
and handset emulator with proper documentation and tutorials. Eclipse is the officially supp
orted IDE for Android SDK. Android Development Plug-
in (ADT) is needed to run this SDK on Eclipse.

Android Dev Phone

When building mobile phone applications it’s necessary to test on a real mobile phone devic
e before releasing to the users. Android Dev Phone is a SIM and Hardware unlocked phone u
sed by developers to test their applications. This device fully supports Android 1.0 version.

Eclipse

Version 3.4Ganymede

Eclipse is a universal tool kit for software development, it’s a multi language development pl
atform comprising of an Integrated Development Environment (IDE) and plug-
ins to extend the functionality.

Motivation

The main motivation of this application is to learn the mobile application development. I
was always curious to know how things work in mobile. This led me to develop the
Restaurant Finder Android application. There are few Restaurant Finder applications in the
android market of which Yelp is the most popular one. Yelp deals not only with the
restaurants but also many other businesses. It also acts like a social networking application
Augmented Reality Using Android OS

which connects people doing various businesses. I started this application with the intention
of making something simple and only related to Location.

Requirement Analysis

The project involved analyzing the design of few applications so as to make the application
more user friendly. To do so, it was really important to keep the navigations from one
screen to the other well ordered and at the same time reducing the amount of typing the
user needs to do. In order to make the application more accessible, the android version had
to be chosen so that it is compatible with most of the Android devices. Hence Android 2.3.3
Gingerbread version was chosen.

1Requirement Specification

1.1 Functional Requirements

Graphical User interface which the user

 Provide accessibility to the application through Wi-Fi or cellular network

. MySQL that stores the information to be displayed to the user.

Tree Tier Architecture

Three-tier is a client–server architecture in which the user interface, functional process

logic(“Business rules”) computer data storage and data access are developed and
maintained as independent modules, most often on separate platforms. It was developed
by John J. Donovan in Open Environment Corporation(OEC), a tools company he founded in
Cambridge, Massachusetts. The three-tier model is a software architecture and a software
design pattern. Apart from the usual advantages of modular software with well-defined
interfaces, the three-tier architecture is intended to allow any of the three tiers to be
upgraded or replaced independently as requirements or technology change.For eg. a change
of operating system in the presentation tier would only affect the user interface code.

Presentation tier

This is the topmost level of the application. The presentation tier displays information
related to such services as browsing merchandise, purchasing, and shopping cart contents. It
Augmented Reality Using Android OS

communicates with other tiers by outputting results to the browser/client tier and all other
tiers in the network.

Application tier

The logic tier is pulled out from the presentation tier and, as its own layer, it controls an
application’s functionality by performing detailed processing.

Data tier

This tier consists of database servers. Here information is stored and retrieved. This tier
keeps data neutral and independent from application servers or business logic. Giving data
its own tier also improves scalability and performance

SOFTWARE DEVELOPMENT

The ultimate objective of software engineering is to produce good quality maintainable

software within reasonable time frame and at affordable cost. This is achievable only if we
have matured processes to produce it. For a mature process, it should be possible to
determine in advance how much time, cost and effort will be required to produce the final
product. This can only be done using data from past experience, which requires that we must
measure the software process.

A key component of any software development process is the life cycle model on which the
process is based. Life cycle of the software starts from concept exploration and ends at the
retirement of the software.

The system life cycle is:

The period of time that starts when a software product is conceived and ends when the
product is no longer available for use. The software life cycle typically includes a
requirement phase, design phase, implementation phase, test phase, installation and check out
phase, operation and maintenance phase, and sometimes retirement phase.

In this project the Spiral Model has been used

Augmented Reality Using Android OS

Spiral Model:

The problem with traditional software process models is that they do not deal sufficiently
with the uncertainty, which is inherent to software project. Important software projects have
failed because project risks were neglected and nobody was prepared when something
unforeseen happened. Barry Boehm recognized this and tried to incorporate the “Project
Risk” factor into a life cycle mode.

The radial dimension of the model represents the cumulative costs. Each path around the
spiral is indicative of increased costs. The angular dimension represents the progress made in
completing each cycle. Each loop of the spiral from X-axis clockwise through 360 degree
represents one phase. One phase is split roughly into four sectors of major activities:

 Planning: Determination of objectives, alternatives and constraints

 Risk analysis: analyze alternatives and attempts to identify and resolve the risks
involved.
Augmented Reality Using Android OS

 Development: Product development and testing product.

 Assessment: Customer evaluation

An important feature of the spiral model is that each phase is completed with a review by the
people concerned with the project designers and programmers. This review consists of a
review of all the products developed up to that point and includes the plans for the next cycle.
These plans may include a partition of the product in smaller portions for development or
components that are implemented by individual groups or persons, if the plan for the
development fails, then the spiral is terminated .Otherwise with the initiation of new or
modified software.

The advantage of this model is the wide range of option to accommodate the good features of
other life cycle models. It become equivalent to another life cycle model in appropriate
situations. It also incorporates software quality objectives into software development. The
risk analysis and validation steps eliminates errors in the early phases of development.

The spiral model has some difficulties that need to be resolved before it can be a universally
applied life cycle model. These difficulties include lack of explicit process guidance in
determining objectives, constraints, alternatives; relying on risk assessment expertise; and
providing more flexibility than required for many applications.

2.2 SUMMARY

 This deal with developing a good quality software with affordable time and
reasonable cost.
 We have to take a mature process to to determine time and cost required to produce
the project.
 Software development life cycle is the key component of any software development
process. This encomprises a requirement phase, design phase, implementation
phase, test phase, installation and check out phase, operation and maintenance phase
and sometimes retirement phase.
 Here Spiral Model has been used because it includes the Risk Management along
with other phases used with other traditional software.
 Augmented Reality Using Android OS

 Each phase of Spiral model consists of Planning, Risk analysis, Development,

Assessment sectors.
 The problem with this model include lack of explicit process guidance in
determining objectives, constraints ,relying on risk management expertise.

When we receive a request for a new software project from the customer, first of all, we
would like to understand the requirements of the project.
Requirement analysis is very important and essential activity we analyses refine the gathered
requirements in order to make consistent and unambiguous requirements.

In spite of understanding the project we would also analys whether it is feasible in terms of
cost, technical and market. If it is not feasible in any term it will not be fruitful to develop
such a project.
Initially it should also be analysed whether the requirements for the project(hardware and
software) are affordable or feasible.

So before preceding further to the design issues here we wil discuss in some what detail the
requirement analysis and the feasibility study of the project.

3.1 REQUIREMENT ANALYSIS

Requirements analysis in systems engineering and software engineering, encompasses

those tasks that determines the needs or conditions to meet for a new or altered product,
taking account of the possibly conflicting requirements of the various stakeholders, such as
beneficiaries or users.

Requirements analysis is critical to the success of a development project. Requirements

must be documented, actionable, measurable, testable, related to identified business needs or
opportunities, and defined to a level of detail sufficient for system design. Requirements can
be architectural, structural, behavioral, functional, and non-functional.

Conceptually, requirements analysis includes three types of activity:

Eliciting requirements: the task of communicating with customers and users to determine
what their requirements are. This is sometimes also called requirements gathering.
 Augmented Reality Using Android OS

 Analyzing requirements: determining whether the stated requirements are unclear,

incomplete, ambiguous, or contradictory, and then resolving these issues.
 Recording requirements: Requirements might be documented in various forms, such
as natural-language documents, use cases, user stories, or process specifications.

FIG 3.1: REQUIREMENT ANALYSIS STEPS

The various steps of the requirement analysis are shown in figure 3.1:

(i) Draw the context diagram: The context diagram is a simple model that defines
the boundaries and interfaces of the proposed system with the external world. It
identifies the entities outside the proposed system that interact with the system.
(ii) Development of a prototype: One effective way to find out what the customer
really wants is to construct a prototype. Prototype helps the client to visualize the
proposed system and increase the understanding of requirements.
(iii) Model the requirements: This processs is usually consist of various graphical
representation of the functions data entries external entities and relationship
between them. This will help us to find the incorrect inconsistent missing
requirements.
(iv) Finalise the requirements: After modeling the requirements now we finalise the
analysed requirements and next step is to document these requirements

Use case diagram

Augmented Reality Using Android OS
Augmented Reality Using Android OS
Augmented Reality Using Android OS

2 FEASIBILITY STUDY

A feasibility study is conducted to select the best system that meets performance
requirement. This entails an identification description, an evaluation of candidate system and
the selection of best system for the job. The system required performance is defined by a
statement of constraints, the identification of specific system objective and a description of
outputs.

The key consideration in feasibility analysis are:

1. Economic Feasibility
2. Technical Feasibility
3. Operational Feasibility
Augmented Reality Using Android OS

3.2.1 Economical Feasibility

It looks at the financial aspect of the project. It determines whether the management has
enough resources and budget to invest in the proposed system and the estimated time for the
recovery of cost incurred. It also determines whether it is worth while to invest the money in
the proposed project. Economic feasibility is determines by the means of cost benefit
analysis. The proposed system is economically feasible because the cost involved in
purchasing the hardware and the software are within approachable. The personal cost like
salaries of employees hired are also nominal, because working in this system need not
required a highly qualified professional. The operating-environment costs are marginal. The
less time involved also helped in its economical feasibility

3.2.2 Technical Feasibility

It is a measure of the practically of specific technical solution and the availability of

technical resources and expertise.

 The proposed system uses Android as front-end and MY Sql server as back-end tool.
 Oracle is a popular tool used to design and develop database objects such as table
views, indexes.
 The above tools are readily available, easy to work with and widely used for
developing commercial application.

3.2.3 Operational Feasibility

The system will be used if it is developed well then be resistance for users that
undetermined:

 No major training and new skills are required as it is based on SPIRAL model.
 It will help in the time saving and fast processing and dispersal of user request and
application.
 New product will provide all the benefits of present system with better performance.
 User involvement in the building of present system is sought to keep in mind the user
specific requirement and needs.
 User will have control over there own information. Important information such as
pay slip can be generated at the click of a button.
 Augmented Reality Using Android OS

 Faster and systematic processing of user application approval, allocation of IDs

payments, etc. used had greater chances of error due to wrong information entered
mistake.

3.3 SUMMARY

Here we have dealt with the requirement analysis and the feasibility study which gives
the better understanding of the project.
Requirement analysis determines the needs to be fulfilled and what the prepared document
should do after completion.For the better understanding of the requirements we wil draw the
context diagram then build a prototype ,analyse the requirements and lastly finalise them.

In feasibility we analyses the feasibility of the project in terms of economic feasibility

,technical feasibility and operational feasibility.

CONCLUSION AND FUTURE WORK

This paper proposes a marker based augmented reality application using Android operating system which will
help to combine virtual objects with the real environment facilitating various applications as mentioned in this
paper. The main advantage is use of low cost devices as compared to the costly head mounted display devices.
Secondly with the help of this project you need not buy product and then see how it will suit your
environment. In future images of objects from various views can be fetched directly from vendor’s websites;
same could be modelled into a 3D objects and augmented. Also multiple objects will be augmented which is
currently a major challenge.

REFERENCES
[1] http://mobile.tutsplus.com/tutorials/augmented-reality

[2] http://mashable.com/follow/topics/augmented-reality/

[3] http://www.se.rit.edu/~jrv/research/ar/
[4] Rencheng Sun , YiSui , RanLi , Fengjing Shao, The Design of a New Marker in Augmented Reality, 2011
International Conference on Economics and Finance Research , IPEDR vol.4 (2011) © (2011)IACSIT Press,
Singapore/

[5] Ronald T. Azuma, A Survey of Augmented Reality, In Presence: Teleoperators and Virtual Environments 6, 4
(August 1997), 355-385

[6] Si Jung Jun Kim, A User Study Trends in Augmented Reality and Virtual Reality Research, 2012
International Symposium on Ubiquitous Virtual Reality

[7] http://www.pcworld.com/article/253530/top_15_augmented_reality_apps_for_iphone_and_ipad.html

[8] http://en.wikipedia.org/wiki/OpenGL

[9] https://code.google.com/p/andar/
 Augmented Reality Using Android OS

[10] https://code.google.com/p/andar/wiki/HowToBuildApplicationsBasedOnAndAR

[11] http://technoccult.net/archives/2010/01/11/augmented-reality-medical-app/

[12] http://www.studio4d.lt/en/augmented-reality

[13] http://www.iphoneness.com/iphone-apps/best-augmented-reality-iphone-applications/

[14] http://blog.t-immersion.com/2010/09/08/germaine-et-les-martiens-first-augmented-reality-apps-on-iphone-
for-pos-advertising-and-augmented-packaging/