Optical Character Recognition For Smart Phones Powered by Android

Nitin Mishra 105657 M. Tech CS Final Year Supervised by: Prof. C Patvardhan (Department of Electrical Engineering)

1. Project Proposal and Objective

The main Objective of the project is to facilitate OCR facility in Android powered smart phones. Image processing on mobile phones is a new and exciting field with many challenges due to limited hardware, limited CPU utilization, image Quality and connectivity. Phones with high quality cameras, powerful CPUs, and memory storage devices are becoming increasingly common. Most of the available OCRs for smart phones are likely to be remotely executed by the servers which do not face limited memory and limited processing power challenge. Our OCR would have image processing suite as well as OCR engine both to be installed on the smart phone means no server communication, quick and far better processing thus would not just only tackle the limited processing power and limited memory challenge but will also try to provide improved performance.

By applying OCR technologies, scanned or camera captured documents are converted into machine editable soft copies that can be edited, searched, reproduced and transported with ease. For example: A person sitting in a restaurant could easily OCR the menu in his local language to order his favorite dish without any need to know the language of menu. Another example: A foreigner can easily convert the signboard text written in other country language to his own language in order to know where to go from here? and Where is my destination?

3. Literature Reviews
All the flimflams and excitement about the costlier iphones and Blackberrys are vanishing, after the arrival of the most anticipated, open source mobile operating system, the Google Android, which is fated to turn the industry upside down. Android defines certain permissions for certain tasks to be defined in configuration file for kernel security. Despite the growth and popularity for iphones and Blackberrys, it is predicted that, Android will make a history in sales and on acquiring the market down, slicing down the markets of both Symbians and iphones.

2. Motivation behind Project

The next generation of open operating systems wont be on desktops or mainframes but on the small mobile devices we carry every day. The openness of these new environments will lead to new applications and markets and will enable greater integration. As demand grows for mobile phone applications, research in optical Character recognition, a technology well developed for scanned documents, is shifting focus to the recognition of text embedded in digital photographs. Optical character recognition (OCR) is a powerful tool for bringing information from our analog lives into the increasingly digital world. This technology has long seen use in building digital libraries, recognizing text from natural scenes, understanding hand-written office forms, and etc.

4. Why 'Google' Android

When a person boots up his web browser, it's a sure thing, that his home page would be 'Google.com' page, because, everyone who uses Internet relies on Google for its accurate web results. So, whats all about this Google and the Android thing? Simple, it is a revenue Geyser for Google and the sweetest thing is that, its not just for Google, since the Android apps developers are going to be the real beneficiaries. Google's entry in to the Mobile OS sector is a simple strategy to acquire a good piece of the soon-to-boom, Mobile-web Advertising market. The biggest advantage of Android OS over other Mobile OS is its Open source status, and open standards which has created some new business models for thousands of application developers and software development companies. The success of Android mobiles will be like a chain reaction, since people would prefer cost effective devices with smartphone like features rather than costly mobiles with a load of unnecessary features.

6. About Android and its features

Android is an operating system based on Linux with a Java programming interface. It provides tools, e.g. a compiler, debugger and a device emulator as well as its own Java Virtual machine (Dalvik Virtual Machine - DVM). Android is created by the Open Handset Alliance which is lead by Google. Android uses a special virtual machine, e.g. the Dalvik Virtual Machine. Dalvik uses special bytecode. Therefore you cannot run standard Java bytecode on Android. Android provides a tool "dx" which allows converting Java Class files into "dex" (Dalvik Executable) files. Android applications are packed into an .apk (Android Package) file by the program "aapt" (Android Asset Packaging Tool) To simplify development Google provides the Android Development Tools (ADT) for Eclipse. The ADT performs automatically the conversion from class to dex files and creates the apk during deployment.

5. The 2012 Market share forecast of Smart Phone Platforms

According to Gartner, Androids Smartphone market share will grow to 14 Percent from less than 2 percent by 2012, and the Symbians slide will continue, giving way to Android. That kind of historical performance by Android would mirror the Apple phones rule. The first-generation iphone was launched in 2007, in the U.S. and immediately, it took the world by storm. On its release, the iphone grabbed a good 11% of the smartphone market share in the first quarter of 2009 and continued to expand day by day. But the predictions about Androids gathering momentum will overtake Apple in just 2 years. Firstly, Android is going to be bigger in terms of consumer reach than its rival, the iphone OS. Its just because of a true fact that it will be developed and marketed by all the 50 members of OHA, which includes companies like Google, Samsung, HTC, Sony Ericsson, T-Mobile, Motorola, Vodafone, Sprint, China mobile and other world leaders in telecommunication industry. Secondly, Google's support will make everything possible in this Internet era, but up to now, they hadn't started making money from their Android-based activities. Almost all handset vendors are trying to board the Android's bandwagon, while the Google has effectively grown a massive Android developer's community, and also, Android is backed by the Americas largest mobile network carrier, the T-Mobile.

Android supports 2-D and 3-D graphics using the OpenGL libraries and supports data storage in a SQLite database. Every Android applications runs in its own process and under its own userid which is generated automatically by the Android system during deployment. Therefore the application is isolated from other running applications and a misbehaving application cannot easily harm other Android applications.

7. Comparison among Smart Phone Operating System

I. General Android 2.3.4 Beginner Friendly Affordability Completeness II. Hardware Android 2.3.4 Design Variety Performance Battery Life III. Connectivity features Android 2.3.4 Tethering and WiFi Live data Syncing Cloud integration IV. Pros and Cons Android 2.3.4 Pros Cons
Google services and openness Open platform Security issues

iphone OS 4.3.3

Blackberry OS 6 Windows Phone 7

iphone OS 4.3.3

Blackberry OS 6 Windows Phone 7

iphone OS 4.3.3

Blackberry OS 6 Windows Phone 7

iphone OS 4.3.3
Apple device sync Premium Price

Blackberry OS 6
Excellent Security Troublesome update

Windows Phone 7
Native sharing Tied to zune

Source: myphonedeals.co.uk

8. Android Framework and Tools

Android Tools include SDK, ADT for eclipse integration support and an emulator. Lets discuss them one by one in brief:

8.1. Introduction to Android SDK and ADT for Eclipse

The Android SDK archive initially contains only the basic SDK tools. It does not contain an Android platform or any third-party libraries. In fact, it doesn't even have all the tools you need to develop an application. In order to start developing applications, you must install the Platform-tools and at least one version of the Android platform, using the SDK Manager. Platform-tools contains build tools that are periodically updated to support new features in the Android platform (which is why they are separate from basic SDK tools), including adb, dexdump, and others. To install Platform-tools, Android platforms and other add-ons, you must have an Internet connection, so if you plan to use the SDK while offline, please make sure to download the necessary components while online. To start the SDK Manager, please execute the program "SDK Manager.exe". From the command-line you can also directly trigger an update by Executing:

A small drawback with Dalvik is its nonecompatibility with Java SE libraries, Java ME class libraries and Swing Java libraries, while they cannot be and need not to-be run directly on this virtual machine. It uses its own Apache Harmony Java implementation libraries.

9. Android Architecture

10. Building Android Application

The Android tools include an emulator. This emulator behaves like a real Android device in most cases and allows you to test your application without having a real device. You can emulate one or several devices with different configurations. Each configuration is defined via an "Android Virtual Device" (AVD).

tools\Android.bat update sdk 8.2. Advantages of Dalvik Virtual Machine

The Dalvik virtual machine is simple Java interpreter machine, completely optimized for Android platform and which is developed to run on low-end memory mobile devices. One of the prominent aspects in Dalvik is its capability to run along an application compilation enhancing the runtime performance of the applications. Dalvik is not exactly, a Java machine, because Dalvik could not read Java code, but consists its own byte code called dex and so the executable files compacted using Dalvik holds the file type name '.dex'. Google states that the credit for Androids successful development goes to Dalvik VM, because this type of virtual machine, delivers a good performance over various stages of an application runtime environment, conserving more battery-power during long run of an application.

Android allows you to create attributes for resources, e.g. for strings and / or colors. These attributes can be used in your UI definition via XML or in your Java source code. The Android SDK allows defining certain artifacts, e.g. strings and UI's, in two ways, via a rich editor and directly via XML. The following description tries to use the rich UI but for validation lists also the XML. You can switch between both things by clicking on the tab on the lower part of the screen. For example in the Package Explorer select "res/layout/main.xml". Select "res/layout/main.xml" and open the Android editor via double-click. This editor allows you to create the UI via drag and drop or via the XML source code. You can switch between both representations via the tabs at the bottom of the editor. For changing the position and grouping elements you can use the outline view.

Views - The User interface of an Activity is build with widgets classes which inherent from "Android.view.View". The layout of the views is managed by "Android.view.ViewGroups". Services - perform background tasks without providing an UI. They can notify the user via the notification framework in Android. Content Provider - provides data to applications, via a content provider. Your application can share data with other Applications. Android contains a SQLite DB which can serve as data provider. Intents - are asynchronous messages which allow the application to request functionality from other services or activities. An application can call directly a service or activity (explicit intent) or ask the Android system for registered services and applications for intent (implicit intents). For example the application could ask via intent for a contact application. Applications register themselves to intent via an Intent Filter. Intents are a powerful concept as they allow creating loosely coupled applications. Broadcast Receiver - receives system messages and implicit intents, can be used to react to changed conditions in the system. An application can register as a broadcast receiver for certain events and can be started if such an event occurs.

11. Components of Android Application

An Android application consists out of the following parts: Activity - Represents the presentation layer of an Android application, e.g. a screen which the user sees. An Android application can have several activities and it can be switched between them during runtime of the application.

12.1. Tools to be used

Android SDK Android NDK Java Runtime environment 6 Xml Editor Eclipse ADT plugin

12.2. Technologies to be used

Image Processing Library for Android

12.3. Platform to be used

Ubuntu (Linux) for native library compilation Windows for application Building

13.1. Project Implementation Plan

Month May June July August September October November December January February March April Module Analyzing Smart Phone Platforms their future Market Share and performance Matrix Comparison Understanding Android Application Development API interfacing with desktop Framework Design, Available Hardware detection and Camera capture module Available resolution detection, auto focus, flash, auto rotate and zoom support module Save captured image onto SDCARD and Browse picture from gallery GUI module Text Crop module via Generating flexible Crop box for manual text area selection module Linking native image processing libraries with Android tools and performance check module Training and testing Language specific samples on OCR & generating Android compatible library module Compile generated lib file with Dalvik and integrating with translator application module Support for Google map services and performing pre final debugging module Individual module testing, Final Debugging and integration testing module Report Generation and Results to be verified in terms of performance

13.2. Progress Track

Task Android SDK and ADT plugin linking with Eclipse Splash Screen and other activity view Xml Framework Design Camera capture Activity Module Crop, save and browse image from gallery GUI Module Linking and Compiling OCR code on native image processing library module Status Done Done Done Done Under progress

14. Future Scope

Getting registered this application from Google Android Market Provide Availability of application world wide for Android users free of cost Support for different OCR languages Support for Google map shortest distance calculation for destination OCRed from signboard Support for GPS Support for different translation languages Support for social network sites status posting

15.1. Bibliography
From Gizmodo's http://gizmodo.com/5395801/Android-20-reviewalmost-human Learn Google Android http://developer.Android.com/guide/basics/what-isAndroid.html Introduction to Android by Jason Chen who is developer advocate at Google http://www.youtube.com/watch?v=x1ZZ-R3p_w8 What Apple did for smart phones, Google may do for all the rest http://www.economist.com/displaystory.cfm?story_id=1 2304882&fsrc=nwl

16 Glossary
OHA - Open Handset Alliance GSM - Global System for Mobile EDGE - Enhanced Data Rates for GSM Evolution 3G - 3rd Generation mobile phones HSCSD - High Sped Circuit Switched Data HTC - High Tech Corporation OCR Optical Character Reader OEM - Original Equipment Manufacturer OS - Operating System RIM - Research In Motion MPEG- Moving Picture Experts Group AAC - Advanced Audio Coding AMR - Adaptive Multi-Rate (Patented audio data compression) JPEG - Joint Photographic Experts Group PNG - Portable Network Graphics GIF- Graphics Interchange Format VM - Virtual Machine AT&T- American Telegraph & Telephones corporation

15.2. Literature References

[1] Chen, Wei-Chao and Xiong, Yingen and Gao, Jiang and Gelfand, Natasha and Grzeszczuk, Radek. Efficient Extraction of Robust Image Features on Mobile Devices. In Proc. ISMAR 2007, 2007. [2] H. Bay, T. Tuytelaars, and L. Van Gool. SURF: Speeded Up Robust Features. In ECCV (1), pages 404417, 2006. [3] Wagner, Daniel and Langlotz, Tobias and Schmalsteig, Dieter.Robust and Unobtrusive Marker Tracking on Mobile Phones, IEEE International Symposium on Mixed and Augmented Reality 2008 1518 September. Cambridge UK. Page 7 of 34 [4] Wagner, Daniel and Reitmayr, Gerhard and Mulloni, Alessandro and Drummond, Tom and Schmalsteig, Dieter. Pose Tracking from Natural Features on Mobile Phones. IEEE International Symposium on Mixed and Augmented Reality 2008 1518 September, Cambridge UK. [5] Ranganathan, P., Adve, S., and Jouppi, N. Performance of Image and Video Processing with General-Purpose Processors and Media ISA Extensions. IEEE 1999. [6] Meier, Reto. Professional Android Application Development. Wrox Publishing. Nov. 2008.