Android OS Based Real Time Embedded Medical Image Processing

A Project Report
Presented to
The Faculty of Computer Engineering Department
San Jose State University
In Partial Fulfillment
Of the Requirements for the Degree
Master of Science in Computer Engineering

By
Jasti Satish Choudary
December 2010
 Copyright © 2010
Jasti Satish Choudary
ALL RIGHTS RESERVED
APPROVED FOR COMPUTER ENGINEERING DEPARTMENT

Dr. Harry Li, Project Advisor

Dr. Lee Chang, Director, MS in Computer Engineering Program

Dr. Sigurd Meldal, Chair, Computer Engineering Department

 ABSTRACT

Android OS Based Real Time Embedded Medical Image Processing

By Jasti Satish Choudary

Goal of the project is to use any of the Android Operating System running devices like
smartphones, tablet or netbooks to acquire the data from the medical image scanning
devices or raw image data and process it in real time to produce a 3D image to diagnose
any diseases. Since most of the mobile devices now days have very powerfully central
processing unit (CPU), Graphics Processing unit (GPU), and very high resolution
displays, using this hardware very high quality 3D image can be rendered or displayed.
By seeing the images doctors can instantaneously save life and money of the patients.
Additionally if the mobile device has accelerometer, magnetometer, pressure sensor,
proximity, touch and accelerated 3D GPU, application will use hardware capability to
give additional information to the users. This application can send images to any other
android devices or upload it webserver, so that it can be accessed by experts or the end-
users. Data acquisition of the data or the image is done by USB port using the Digital
Imaging and Communications in Medicine (DICOM) protocol between the android and
scanner. Image will be rendered by using OpenGL ES 2.0 hardware 3D graphics
accelerator to Shader, Fragment Shaders and write vertex. Image rendering part of the
programming, analysis and testing can be done on any machine where the Open GL
support is there. Final images can be transferred to webserver using Hypertext Transfer
Protocol (HTTP) or stored on the large Secure Digital (SD) card in JPEG 2000 Format.

iv
 1

ACKNOWLEDGMENTS
 2

Table of Contents
[The content format can be modified with approval by your Project Advisor.
However, a CD/DVD with the softcopy report must be attached to the back cover.
Write the team number on the CD/DVD. Do not seal CD/DVD envelop.]

1 Introduction
1.1 Project goals and objectives
1.2 Project deliverables
1.3 Technology trends
1.4 Market research

2 Requirements
2.1 Functional requirements
2.2 Nonfunctional requirements
2.3 Product requirements analysis
2.4 Project requirements

3 Design
3.1 Architecture
3.2 Structure
3.3 Interfaces

4 Implementation
4.1 Implementation scope
4.2 Implementation coverage
4.3 Develop or adopt decision
4.4 Implementation process
4.5 Implementation resources
4.6 Implementation activities

5 Testing
5.1 Testing scope
5.2 Testing coverage
5.3 Pass/fail criteria
5.4 Testing approach
5.5 Testing resources
5.6 Test cases
5.7 Testing activities
5.8 Test results

6 Schedule
6.1 Planned schedule
6.2 Actual schedule

7 Conclusion
 3

7.1 Learning experience

7.2 Recommendations

Appendix 1 [Title]

Appendix 2 [Title]

Bibliography
 4

List of Figures
 5

List of Tables
 6

Chapter 1 Introduction

1.1 Project goal and objectives.

With the latest developments in gadgets and smartphones, the usage of these
devices is growing in various applications. In recent time the hardware capabilities
of these devices have increased enormously. In addition to their core functionalities
they support huge number of versatile applications using their powerful hardware
capabilities. Taking the advantage of these enhanced hardware capability I propose
my project Android OS Based Real Time Embedded Medical Image Processing
Application. The main goal of my project is to acquire the data from the medical
image scanning devices or raw image data and process it in real time to produce a
3D image using a smart phone device.

The embedded hardware used for this project is Texas Instrument’s OMAP 3530.
Android is used as the operating system for this board.

1.2 Project deliverables

The following items are the deliverable for this project.

1. Plan Report
This report explains the requirements, design, and schedule of the project.

2. Project Report
This report includes the implementation and testing phase of the project and is
delivered at the end of the project.

3. Presentation Slides
Presentation details the main points of the project

4. Prototype
The prototype consists of working hardware model of the project

5. Source code
This is the software used in the project.

1.3 Technology Trends and Market research

In today’s technology world, there are lots of embedded systems every day walk-of
life. These systems use different variety of microprocessor, microcontroller as a
part of the system. Embedded system performs some small pre-defined specific task
in contract to the personal computer or servers. Embedded system can be tiny
 7

system as remote control, cell phone, toys and music players and big as cars,
refrigerator, television sets. Requirements for these systems are very specific, so
that they should be designed keeping in mind these required. Most of the new
embedded system use the Linux as there operating system. As the Linux operating
system is open source and widely adapted system, there are lots of developers
available to help or support the system. Of the available operating systems Android
is very popular and successful operating system

According to the LinuxDevices.com survey of the embedded markets, found out the
Linux is widely adapted operating system on the server, desktop, real time systems
and embedded platform. Below figure shows the details of the Linux operating
system and related tools shipped yearly bases.

Fig 1.3 Embedded Linux OS- Market Trends

Also ARM processor is very widely used on embedded platform according to the
survey. Next 2 years ARM processor would by grow by more than 5 % as the
survey done by the Linuxdevices.com.

1.3.1 Software Platform

Android, the mobile operating system, is centered on a modified version of
the Linux kernel. Google and the other members of the OHA developed today’s
Android. Unit sales for Android OS smartphones ranked first among
all smartphone OS handsets sold in the U.S. in the second and third quarters of
2010, with a third quarter market share of 43.6%.
 8

According to Worldwide Quarterly Mobile Phone Tracker, the smartphone market

of 2014 could see Android with a 25 % market share and a change of over 51.2%
from today’s market.

The other major software component involved in the project is OpenGL (Open
Graphics Library). It is a language defined for writing applications that produces 2
dimensional and 3 dimensional computer graphics. The interface to the OpenGL
has about 250 API’s used to draw complex three-dimensional scenes.

OpenGL's simple objective is to receive primitives such as points; lines etc., and
convert them into pixels. This is done by a graphics pipeline as shown in below
figure. Current version of OpenGL is OpenGL 4.1.
 9

Fig 1.3.1.1 Simplified version of the OpenGL’s Graphics Pipeline Process

The new release OpenGL 4.1is an interface that programs can use to tap directly
into a variety of graphics hardware. It's the 3D interface of choice for many 3D
design applications

Fig 1.3.1.2 OpenGL Innovations

 10

1.3.2 Hardware Platform

Smartphone, today’s mobile phone, has advanced computing ability and
connectivity. These are sophisticated handheld computers unified with a mobile
phone provides a platform for the applications based on Java ME and other
contemporary technologies. This advanced hardware provides capabilities to run
complete operating system and which inturn provides a development base for
application developers. This Personal Pocket Computer with phone features
provides more personal experience to users than a desktop computer.

With increasing markets for these advanced devices, it boats to more

powerful processors, and large memories. This phone from the days with couple of
MHz has grown to today’s GHz processor devices, thus eliminating many
bottlenecks which are limiting the development of many useful applications.

In this project I am using Texas Instruments OMAP 3530. It is an ARM based

Cortex-A8 superscalar core with ~600MHz paired with aTMS320C64x and DSP
~430MHz. Today many smartphones use OMAP i.e., Open Multi Media
Application Platform microprocessors. They include most of Nokia's N-series
range. Phones using OMAP include the N90, N91, N92, N95, N82, E61, E62, E63,
E90, N900, Motorola Droid, Droid X, and Droid 2, and Samsung devices.
Also ARM processor is very widely used on embedded platform according to the
survey. Next 2 years ARM processor would by grow by more than 5 % as the
survey done by the Linuxdevices.com.

Fig1.3.2 Embedded Processor Preference Trends

 11

Chapter 2 Requirements
2.1 Functional requirements

The functional requirement of this project to process the raw image in real time
using a smartphone embedded processor to produce a 3D image. The image will be
rendered by using OpenGL ES 2.0 hardware 3D graphics accelerator to Shader,
Fragment Shaders and write vertex. The requirement here is to program, analyze
and test the Image rendering. Following are functional requirements for the projects
based on

2.1.1 Hardware
 Provide embedded hardware platform for porting Android OS on it.
 Selecting the hardware which have DSP processor on it for processing 3D
graphics
 Provide a prototype affordable for academic projects.

2.1.2 Software
 Building Android OS for porting OpenGL applications.
 Programming image rendering aspect.
 Building a webserver to store the 3D images.

2.2 Project Requirements

The resources such as hardware platforms, software components and tools sets
needed for the projects are listed in the below table.

Item Quantity
Hardware OMAP 3530 Beagle Board 1
Server to Store the 3D Images 1
Laptop/Desktop to build the S/W
Software Android Operating System to port on H/W board
Oracle JDeveloper IDE to develop OpenGL Code
Linux OS to build Android & OpenGL Software
Knowledge OpenGL
Java Programming
Software Development Life Cycle
Embedded Systems Development
Misc., Keyboard and Mouse with USB Hub for Beagle Board 1
Monitor with HDMI I/P and cable 1
SD Card to store the Android OS 1
Table 2.2 Project Requirements
 12

Chapter 3 Design
3.1 Architecture
[Describe the architectural solution for your project. This section must include
textual description accompanied with figures.]

3.2 Structure
[Describe the structure of your design with components. This section must include
textural description accompanied with figures, e.g., component diagrams, bill of
materials diagrams, block diagrams, etc.]

3.3 Interfaces
[Describe interfaces to other systems, subsystems, and users. Also include industry
standards applicable to the project. This section must include textual description
accompanied with figures.]
 13

Chapter 4 Implementation
4.1 Implementation scope

End goal of the project is to provide medical except or doctors and end user i.e
patient’s high resolution 3D images of the target part of the body. This system
consists of acquisition of data from scanner, processing of this data and rendering of
the data on to the display or finally processed data. Medical imaging device acquires
data in industry standard format JPEG 2000 and transferred to the Android device
using DICOM protocol. Data which is transferred from the scanner or raw data
through USB or Wi-Fi is decompressed and then rendered on the screen using 3D
hardware acceleration using open GL 2.0 ES API provided by android platform.
Finally this processed data can stored in the SD card or uploaded to the web server,
so that we viewed by any web browser.

Fig 4.1 Implementation details

 14

4.1.1 Acquisition
Data is scanned using various types of devices like Computed Axial Tomography
(CAT or CT) or Magnetic resonance imaging (MRI) in JPEG 2000 format. JPEG
2000 is the industry standard of get data from medical image scanners. JPEG 2000
has ability to store data in lossless format in the interested area of the image. Users
can chose which part of the image can be lossless, So that full details of the image
can rendered in certain part of the area. Since all part of the image is not
compressed, usually JPEG 2000 files are huge; all the metadata where to compress
the image is stored in the header of the file. This JPEG 2000 can be transferred to
the android device though USB, Wi-Fi (TCP/IP) or Bluetooth. Data is transferred
to the android device using DICOM protocol over any of the interfaces.

Fig 4.1.1 Data Acquisition

4.1.2 Image Rendering

This is most critical part of the system, as the data comes as various speeds from
different type of the devices, need to pre-processing of data. Data coming from
various devices is in JPEG 200 format and using DICOM protocol. This module has
to receive data at different transfer rates and decompress the data using DSP engine
if present on the android platform. After decompress of data, raw data is presented
to 3D accelerating engine using the Open GL 2.0 ES API’s which are supported on
the Android platform. Beagle board has very powerfully DSP, which can be used
for decompressing/decoding of the data.

Fig 4.1.2 Image Rendering

 15

4.1.3 Post- processing

Images which are displayed on screen can be stored multimedia card (MMC) or
secured digital card (SD) and also can also uploaded to the webserver. When the
data is stored on the MMC or SD card, data is in a format where 3D images can be
displayed which using much of the hardware processing as battery life of the
devices is also important. Also when data has to be uploaded to webserver, data is
format where is least processing is done on the Android platform. All the data
decoding or processing of the data is done on the Webserver. 3D Images will be in
HTML5 format, which can be seen by all modern browsers like Internet Explorer 9
or Google’s chrome.

Fig 4.1.3 Post Processing

4.2 Implementation coverage

As explained in the above section, this project consists of 3 modules a) Acquisition

b) Image rendering in 3D using hardware acceleration c) Post–processing

Table below gives description of various modules which are implemented and
which are not. Also these modules are classified in 2 different type hardware and
software.

Type Module Implemented Reason

Hardware Acquisition, getting the data No MRI or CT scanner is
from the hardware the MRI or not available; also this is
CT scanner. very expensive
equipment.
Hardware Use Wi-FI/Bluetooth module No Data which is coming
from these modules can
be simulated to pre-
defined data.
Software Drivers to get data from USB/ No Already open source
 16

Bluetooth module drivers are available for

USB/Bluetooth modules.
Also due time
constraints didn’t allow
the development of this
module.
Software 3D image rendering on Yes This is core part of the
android mobile device project which displays
3D images on the screen
Hardware Beagle board which can run Yes Run the OpenGL 2.0 ES
the Android platform software.
Software Post processing Software No Standard free software
can be customizing to
achieve this. Also the
time didn’t allow do this
module.
 17

Chapter 5 Testing
5.1 Testing scope
[Describe system and its components from the structural aspects of testing. This
section should include the granularity of components and their dependencies. This
section must include textual description accompanied with figures and/or tables.]

5.2 Testing coverage

[Describe the features and combinations of features to be tested from the functional
aspects of testing. Describe the features that are not to be tested and provide the
reasons for not testing them. This section must include textual description
accompanied with figures and/or tables.]

5.3 Pass/fail criteria

[Describe the pass/fail criteria of the test plan at subsystem and component levels.
This section must include textual description accompanied with figures and/or
tables.]

5.4 Testing approach

[Describe the approach to the testing process. This section includes the reasons for
the selected system integration testing strategy. This section also includes the
dependencies between the individual tests and their involvement in the integration
test. This section must include failure analysis techniques. This section must
include textual description accompanied with figures and/or tables.]

5.5 Testing resources

[Describe the resources that are needed for testing. This should include the
physical characteristics of the facilities such as hardware, software, special test
tools, lab spaces, etc. This section must include textual description accompanied
with figures and/or tables.]

5.6 Test cases

[Describe the methodology of generating test cases, the document format of Test
Case Specification, and the document format of Test Incident Report. This section
may include some use cases. This section must include textual description
accompanied with tables.]

5.7 Testing activities

[Describe the staffing, responsibilities, training, risks, and contingencies. Schedule
will be included in Chapter 6 not here. This section must include textual description
accompanied with figures and/or tables.]

5.8 Test results

 18

[Describe the results of various tests and analyze the results with respect to the
requirements stated in Chapter 2. This section must include textual description
accompanied with figures and/or tables.]
 19

Chapter 6 Schedule
6.1 Planned schedule

In this project, I am using Critical Path Method and Gantt Charts to represent my
plan and schedule of the project. Gantt chart also called as bar char explains the
schedules of different tasks and activities in the projects. It also helps in
coordinating the tasks among different activities. Gantt chart is 2 dimensional view
in which horizontal axis represents the timespan of the project for all the activities
listed in the vertical axis. This project is categorized into 5 phases. First phase, the
design phase, depending on our project goals and objectives I identified the software
and hardware requirements. In second phase, Coding, I build the software platform
for the project and on top of which I started coding some of the requirements of the
project. In next phase, Testing, I tested the software built for better standards. In last
phase, Integrating, I combined different programmed module on one single
hardware platform to meet the objectives of the project. The Gantt chart below
shows these different stages in detail.

Fig. 6.1.1 Gantt Chart

Critial Path Method is a program managemnet tool which is used while designing
and scheduling different activites in a project. In a way it is similar to Gantt chart. In
additions to those features this tool, CPM, can also reslove the dependencies among
the activities. The above table decipates these different activites in the project and
the below table shows the dependiences among these activites and will give the
estimates of the time that this project needs to complete. Optimized activity
schedules for the project is in fig 6.2. The approximate project completion according
to the CPM is around 12 weeks. The software used for this project estimates is “The
Managemant Scientist”
 20

Table 6.1.1 Activity table

Fig 6.1.2 Optimal CPM chart

 21

6.2 Actual schedule

The below are the actual schedules compared to the proposed estimates.

Sep 2010 Oct 2010 Nov 2010

ID Task Name Start Finish Duration
8/29 9/5 9/12 9/19 9/26 10/3 10/10 10/17 10/24 10/31 11/7 11/14 11/21

1 Design Work 8/30/2010 9/24/2010 20d

2 Identify Potential Problems in Design 9/24/2010 9/24/2010 1d

3 Coding 9/27/2010 10/28/2010 24d

4 Testing the Code 10/28/2010 11/3/2010 5d

5 Integrating the Modules Into System 11/3/2010 11/15/2010 9d

6 System Testing 11/15/2010 11/22/2010 6d

7 Implementation & Testing documentation 8/30/2010 11/25/2010 64d

8 Documnetation Integration 11/25/2010 12/1/2010 5d

Fig 6.2.1 Gantt chart

Fig 6.2.2 Activity Schedule

22
 23

Chapter 7 Conclusion

7.1 Learning experience

[Explain the achievement of project goals and objective; describe lessons learned
with respect to academic and industrial values]

7.2 Recommendations
[Describe possible project enhancement ideas; propose follow-up topics for future
projects]
 24

Appendix 1 [Title]
[Optional]
 25

Bibliography
[List the sources referenced in the report.]