MoMM 2009 Short Papers

Proceedings of MoMM2009

Color Quick Response Code for Mobile Content Distribution

Kris Antoni Hadiputra Nurwono
BINUS International BINUS Business School Hang Lekir 1 no.6, Kebayoran Baru Jakarta, Indonesia +62-21-7202222

Raymondus Kosala
BINUS International BINUS Business School Hang Lekir 1 no.6, Kebayoran Baru Jakarta, Indonesia +62-21-7202222

ksakti@hotmail.com rkosala@binus.edu Contact Author: rkosala@binus.edu ABSTRACT

In this paper, we propose Color Quick Response Code or CQR Code for short, which is a Matrix code that has been extended into a 3D barcode system. Matrix code itself is a 2-dimensional barcode that appears to be a 2D image that consists of usually black and white dots that can be printed or displayed on a screen. CQR Code is an extended version of the popular QR Code. 3D barcodes contains data not only in its x and y axis, but also in depth. By stacking QR Codes we can create a new barcode system that can store more data than other existing barcode systems. Through this barcode system, we can embed data that normally could not be done using existing barcode systems. From the tests that had been conducted, currently the CQR Code is capable of storing data up to 9KB with 3 layers and 8 colors. The CQR Code can be read with 40% - 100% chance of success using a multi-format mobile phone barcode reader prototype. We have also implemented a simple color correction algorithm into the barcode reader to ensure its accuracy. In conclusion, CQR Code has the potential for future developments for it to be a simple, sophisticated, and easy to use barcode system that can store relatively large amounts of data. for mobile contents are continuing to increase. Distributing these mobile contents can be in many forms, from just a simple text messaging using SMS, image transfer through MMS, to internet using GPRS or WAP. Even with all of these technological advancement, printed media such as newspaper, magazines, and posters still remains the most common source of information. It already has a wide range of distribution networks, and also easy and cheap to produce. The idea is to combine both, the old and the new. Using printed media (although not limited to) as a way to distribute mobile contents by using matrix code. Matrix code is a 2-dimensional barcode that appears to be a 2D image that consists of usually black and white dots that can be printed or displayed on a screen. This image actually contains an encrypted data that can be decoded through a computer system. All the user need to do is snap a picture of the matrix code and a software will decode the image and extract its hidden content. By using this method, companies can give a sample of their digital product along with its printed advertisement. Many different types of data can be encrypted into the matrix code, such as a wallpaper image, music file, video file, a web link, or even a simple text. Actually, matrix code had been implemented in Japan in October 1997, by a company named Denso Wave. The matrix code was called the quick response (QR) Code. It was initially used for tracking parts in vehicle manufacturing. Up until now, QR codes can only stores data up to 2,953 bytes which is very small [2], and can only store limited data (usually text or web address). Nowadays, our mobile phones camera could have focus up to 3 mega pixel, or more, which means we can store much more data in to the matrix code than it was before. When it was first built, the matrix code only consists of black and white dots to make it easier to distinguish between each pixel. In this paper, we propose Color Quick Response Code or CQR Code for short, which is a Matrix code that has been extended into a 3D barcode system. By implementing color patterns (instead of just black and white), utilizing mobile phone cameras, and data compression, we can add more data into the matrix code. This paper also contains procedures to encode a CQR Code and also procedures to read it using a mobile phone barcode reader application. CQR Code is an extended version of the popular QR Code. 3D barcodes contains data not only in its x and y axis, but also in depth. By stacking the QR Codes we can create a new barcode

Categories and Subject Descriptors

E.2 DATA STORAGE REPRESENTATIONS

General Terms
Algorithms, Performance, Design, Experimentation

Keywords
Mobile content distribution, 3D barcode, Quick Response (QR) code, mobile barcode reader

1. INTRODUCTION
Mobile phones, PDA, and handheld devices alike have taken an essential part in our modern world, and so the trend and demand
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. MoMM2009, December 1416, 2009, Kuala Lumpur, Malaysia. Copyright 2009 ACM 978-1-60558-659-5/09/0012...$10.00.

267

Proceedings of MoMM2009

MoMM 2009 Short Papers

system that can store more data than the existing QR barcode systems. Through the CQR barcode system, we can embed data that normally could not be done using existing barcode systems.

x x x

1.1 Related work

Paper Memory Code System or PM Code [1] for short, is another 3D barcode system currently being developed by Content Idea of Asia Co. Ltd., a Japanese mobile developer company. PM Code is based on QR Code. The idea is to stack QR Codes into one integrated 3D barcode. However, we could not find any academic papers related to this system. Another related work is ColorCodeTM by Jeong, et al. [4]. It also provides means to distribute mobile content. However, the ColorCodeTM does not carry content inside the code, aside from being an index, and it requires a server to deliver the content.

Position / Finder Pattern is used to help detect the codes position when decoding. Alignment Pattern is used to find the correct angle when decoding Timing pattern is used to help determine a symbol's coordinate in decoder application.

The version of a QR Code is determined by its size. The smallest QR Code, a 21 x 21 matrix, is known as Version 1. Every version has four module differences with its next or previous versions, so Version 2 is a 25 x 25 matrix, Version 3 is a 29 x 29 matrix, and so on. Currently there are 40 versions of QR Code available with a 177 x 177 matrix as the largest version.

2. THE QUICK RESPONSE (QR) CODE

QR Code that stands for Quick Response Code System is a type of 2D barcode that has become very popular all over the world, mostly in Japan. QR Code was first developed by a Japanese company called Denso Wave in 1994. It was originally developed, patented and owned by TOYOTA subsidiary Denso Wave for car parts management. Even so, Denso Wave stated in their website that the specification has been approved as an AIM (Association for Automatic Identification and Mobility) and ISO standard. [3]. However, the term QR Code itself is trademarked. Figure 1 shows a comparison between a 2D Barcode (in this case, QR Code) and a 1D Barcode. As you can see from the comparison, QR Code contains information in both the vertical and horizontal directions, whereas a bar code contains data in one direction only. With this design, QR Code can contain considerably greater volume of information than a singledimension Barcode.

Figure 2 Structure of a QR Code Image taken from www.wikipedia.com To ensure successful data read, an error correction system is integrated. This error correction system uses Reed-Solomon error correction algorithm [7].

3. THE COLOR QR (CQR) CODE

Colored Quick Response Code System or CQR Code for short, is an expanded version of QR Code. The idea is to develop a similar code system as Paper Memory (PM) Code and create a working encoder / decoder prototype to be implemented on a mobile phone. By using a mobile phone and a decoder, users can snap pictures of the 3D barcode and extract data from it.

QR Code (2D Code) Contains data Contains no data

Bar Code

Contains data

Contains data

Figure 1 QR Code (left) compared to 1D Barcode (right) Images are taken from www.wikipedia.com We can see from Figure 2 that a QR Code consists of several parts: x x x Version Information information regarding the QR Code itself, its version, and size. Format Information information of error correcting level and mask pattern (described later) are stored. Data and Error Correction Keys this is where the encoded data is stored, error correction patterns are also embedded into the data.

Figure 3 Color QR Code (3D barcode) concept

268

MoMM 2009 Short Papers

Proceedings of MoMM2009

The concept, which is shown in Figure 3, shows that by stacking QR Codes, CQR Code can store more data than other existing code systems. The target is to create a printable code system where we can embed mobile contents into it. Therefore, internet connectivity is no longer required for users to download mobile contents. Even so, CQR Code can also be combined with internet connectivity for other purposes. In this paper, we demonstrated our proposed method by implementing a prototype system that combines three different colors / layers. However, our method can use more layers to increase the data capacity.

colors on points where the layers meet. We could use an image editing software for this purpose. 5. We now open Adobe Photoshop, and create a new RGB canvas. We create 4 separate layers, each containing a QR Code and one filled with black color. If we are successful, we will see something like in Figure 5.

3.1 CQR Code Design (3D Barcode)

CQR Code is an extended barcode format using QR Code standards. QR Code itself is already a robust system, the only limitation is its data capacity. Our 3D barcode does not physically have a 3D form, in this case 3-dimension (3D) means the number of dimensions used to store data. CQR Code stores data in x-axis, y-axis, and color. Color represents depth or layers. CQR Code contains stacked QR Codes, in order to tell each layer apart we assign color codes into each layer.

3.1.1 Encoding CQR Code

To create or encode a CQR Code is fairly easy, all the user need is a QR Code Encoder that can easily found online (e.g. Kaywa Website [5], Swetakes QR Code Encoder [6]) and an Image Editing Software (e.g. Adobe Photoshop). The process will demonstrated step by step below. We will be using Adobe Photoshop to stack the three (3) layers, so we will explain the steps that are done using the software. These steps may not be available if using other software: 1. First, we divide the data that we want to encode into 3 parts. Each part will be encoded into a different layer of the CQR Code. If we are going to encode a file or a binary data, we need to get its binary data and divide them into 3 parts. Please remember that QR Codes can only encode 8-bit data so we need to divide them proportionately. Then we encode each part into a QR Code. We can choose what ever size and error correction level that we want. Please keep in mind the size that we choose because we have to apply the same size to all 3 QR Codes, although the error correction level can be different in each QR Code. After all 3 QR Codes have been generated, now we need to stack them. But before that, we need to assign a different color for each layer. Since we are only using 3 layers in this prototype, the first portion of data or, in other words, the first QR Code barcode should be assigned with Red color. The second layer should be Green, and the last should be Blue. After all layers have been assigned a color, we can now stack them. Even so, we could not just stack them together, because the later layers of the barcode will be occluded by the first layer. We need to combine the 6.

Figure 4 Individual Layers

As you can see, the later layers are occluded by the top layer and the background is still black, this is not the result that we wanted. To get the result that we wanted, we need to combine the colors on points where the layer meets. This can be achieved by adding Difference effect on the layers. If we are successful, you will get something like in Figure 6.

2.

3.

Figure 5 Adding Difference Effect on the Layers

7.

4.

Now we invert the image, and we will have the final result, as shown in Figure 7.

269

Proceedings of MoMM2009

MoMM 2009 Short Papers

3.2 Barcode Reader - General Steps (J2ME variant)

There are five steps in general in order to capture data from a printed 3D barcode using the mobile barcode decoder application. x x Figure 6 Example of Final Color QR Code x x x Start Application Capture an image of the barcode by using the camera on your mobile phone The application will then try to detect if a barcode is present If barcode is detected then decode the barcode Show the result. Result can be a simple text, a file, url address, etc.

3.1.2 Encoding: Pseudo code

Currently we have not created a system to auto-generate a CQR Code. However, we can follow the following pseudo code to build a CQR Code auto-generator: x x x x Get input data from user. Data can be numeric, alphanumeric, binary, and even KANJI. Calculate data length and divide them into 3 parts. Generate QR Codes for each data parts. Assign Color to each layer. Layer 1 is Red, so every black point of the generated QR Code for Layer 1 is changed to Red. The RGB value of Black is (0,0,0) or #000000, changed to (255,0,0) or #FF0000 for Red. Repeat the process to all layers and colors. Create a new byte array and check every pixel of the image and check if a layer is meets with another layer. If yes, add their RGB values. For example: Layer Red and Green meets at pixel position (10,25). We add Red RGB value #FF0000 and Green RGB value #00FF00 so the new RGB value for that pixel is (255,255,0) or #FFFF00. Then, we invert this value using a logical NOT, so the final value is #0000FF. Then store the final value into the new byte array. Save byte data and create a new image file.

These steps are very simple and easy for any user to understand. Even so, the J2ME variant has its limitations due to hardware limitations in mobile devices, such as a very small memory.

4. EXPERIMENT AND RESULTS

The tools that we used to develop the mobile Barcode Reader are: x x Programming Tools : NetBeans 6.0, Java, Apache Ant APIs : Java SDK 1.6.4, J2ME, Java Wireless Toolkit, ZXing 0.6

We tested the prototype accuracy as follows. Accuracy testing was done through unit testing in a controlled environment. Figure 8 shows the testing environment and Table 1 shows the test case.

3.1.3 Decoding CQR Code

To decode CQR Code is to separate the layers and read each layer. The process needed to decode each layer is basically the same as decoding QR Codes. Decoding Pseudo code: x x x x x x x x x Get captured image Retrieve byte data Read each pixel and retrieve RGB value Extract RGB layers Decode each layer using QR Code Decoder Append the result Detect result type If result is text then Show result Else, show result and show available actions (example: result contains URL, available action is to access URL) Figure 87 Test Environment Illustration The testing was done inside a room with only 1 light source, and the light source was a table lamp with interchangeable light bulbs. The light source was placed next to the paper, with a distance of 25 cm from the paper. The lights were pointed diagonally to prevent shadows overlaying the paper. The mobile phone was placed on top of the paper with the camera pointing down with approximately 90 degrees angle. The mobile phone was distanced from the paper. During unit testing we used Sony Ericsson K810i, which has a 3 megapixel camera with autofocus feature.

270

MoMM 2009 Short Papers

Proceedings of MoMM2009

Table 1. 3D barcode Accuracy Test Case 3D barcode Accuracy Test Case Test Case Accuracy test under controlled environment with 25 or 40 Watt Soft White light bulb and plain paper User has already opened the application Mobile Device is placed 15cm and 20cm above the barcode x The barcode is printed on a plain A4 paper using HP psc2300 printer x Take picture of barcode x Wait for result x Test 10 times x Calculate success ratio Shows the success read ratio x x

When capturing the image in a dark environment or with minimal light, the barcode reader seems to be giving a system error in some cases.
120% 100% 80% 60% 40% 20% 0% < 20 Watt 25 Watt 30-35 Watt 40 Watt

Precondition

15 cm 20 cm

Test Steps

Expected Result

Figure 8 Success ratio to the amount of lights

Table 2. Test Results Barcode Success Reads 15 cm: 0 20 cm: 0 Failed Reads 10 10 Lights Less than 25 Watt

5. CONCLUSION
CQR Code is a new system that has a lot of potentials. CQR Code can be utilized for mass mobile content distribution and can be integrated with printed media. If we are able to boost its data capacity and accuracy, the system can revolutionize how mobile contents are distributed. Current CQR Code data capacity (9KB) is not sufficient enough according to user standards and for the barcode to be internet independent. However, the data capacity can be improved by adding more layers into the barcode system. Barcode Reader accuracy should also be improved to meet user standards. To overcome this problem, better color correction algorithm can be implemented to improve accuracy. This color correction algorithm might also be useful for tackling the problem of discoloration and color changes of the color barcode by external factors.

15 cm: 4 20 cm: 7 Data : Alphanumeric Result : kris antoni hebat sekali hehehe Barcode Format: CQRCode - 3D Barcode 15 cm: 5 20 cm: 7

6 3 5 3

25 Watt Soft White 30 - 35 Watt Soft White

15 cm: 10 20 cm: 10

0 0

40 Watt Soft White

6. REFERENCES
[1] Content Idea of Asia Co. Ltd. PM Code. CI-A.co.jp. [Online] [Cited: March 13, 2008.] http://cia.co.jp/pm/pm_eng.pdf. [2] Denso Wave Incorporated. About QR Code. QR Code. [Online] [Cited: April 10, 2008.] http://www.qrcode.com/. [3] Denso Wave Incorporated. History of QR Code Standards. Denso Wave QR Code. [Online] [Cited: February 25, 2008.] http://www.denso-wave.com/qrcode/qrstandard-e.html. [4] Jeong, C., Lee, N.K., Shin, E.D., Han, T.D.. A Design of ColorCode System for Name Card Information, KISS HCI, 9(1), 2000. [5] Kaywa. The Kaywa Reader. Kaywa Reader Website. [Online] [Cited: April 27, 2008.] http://reader.kaywa.com/. [6] Swetake, Y. How to Create QR Code. Swetake. [Online] [Cited: April 11, 2008.] www.swetake.com/qr/qr1_en.html. [7] Wicker, S. B. and Bhargava, V. K., "Reed-Solomon Codes and Their Applications", IEEE Press, Piscataway, NJ, 1983.

Table 2 shows the test results and Figure 9 summarizes them. Under different light conditions in the controlled environment, the barcode can be read with an average of 63% accuracy, with results ranging from 40% to 100%. In a sufficiently bright environment, which is 40 Watts or above, the prototype can decode the barcode perfectly. Thus, we can conclude that in the outdoor environment under a daylight, the prototype would work well. From this test we can also conclude that distance influences the success rate. If the camera is too far or too close it will not work properly. In our case, we are using a mobile phone with autofocus enabled. In our experiment, the reason that the success rate in 15 cm is lower than the success rate in 20 cm may be due to the the camera reading that is out of the focus because is it too close to the barcode. Even so distance may vary depending on the size of the barcode and the focal point (focus point) of the camera. In order to get the best results, the user should also capture the barcode from a 90 degree angle (or as close to 90 degrees as possible) from the surface of the paper where the barcode is contained.

271