8th ICCIT 2005

Islamic University of Technology (IUT), 28-30 December 2005

Bengali Handwritten Bank check Recognition Using

Automatic Extraction of the User–Entered Data
Md. Rezaul Hoque Khan, Gahangir Hossain †
Department of Electrical & Electronics Engineering,
Chittagong University of Engineering & technology, Chittagong-4349, Bangladesh.
†
Department of Computer Science & Engineering,
Bangladesg University of Engineering & technology, Bangladesh.
sohagiut@yahoo.com , ghcsecuet@yahoo.com

Abstract their respective values into the computer. Accordingly,

This paper deals with methodology proposes to
recognize a bank check written in Bengali using
automatic extraction of the user–entered Data. Despite
its apparent simplicity, a check is a very complex
document. It integrates pictorial images (colored
layout), many pre-printed components (logos,
guidelines, labels of data-entry fields…) as well as
handwritten components (signature, date, issuing place,
literal and courtesy amount …). In addition, these items
of information do not have a normalized position and
their structuring varies according to countries and
institutions. A template has to design for extracting the
user–entered items of any bankcheck, no mattering
which financial institution has issued it. The Extracted
data then will recognize and verify with database.
Keywords: Extraction, Recognition, Verification.
Bankcheck.
I. INTRODUCTION
the field of automatic check processing has witnessed
sustained interest for a long time. The performance in
handwriting recognition is greatly improved by
constraining the writing, which addresses the problem
of segmentation and makes the people write more
carefully. A system that is able to read checks
automatically would be very helpful, especially if it is
fast and accurate. Even if misclassification occurs, the
mistake could potentially be detected during the
recognition process; however it is more desirable that
the system rejects a check in case of doubt so that it can
be directed to manual processing from the beginning.
The organization of this paper is as follows. Section 2
gives a brief description of bank checks. Section 3
provides system overview. Section 4 describes, step by
step, the extraction of user-entered data and the
elimination of non–interested information. Section 5
represents process of recognition of data. The data are
then verified by section 6. Finally, the discussion and
conclusion is stated in the last section.

Hand writing recognition has become exceedingly

popular in the past few years. Most applications belong II. DESCRIPTION OF BANKCHECKS
to the off-line recognition category that corresponds to
already written documents acquired by a scanner or a
camera. One of the most representative examples of this
category is bank check processing. The account number
is printed on the checks in magnetic ink and is the only
fields that can be processed automatically with near-
perfect accuracy by magnetic ink character recognition
(MICR) systems. The other fields may be handwritten,
typed, or printed; they contain the name of the recipient,
the date, the amount to be paid (textual format), the
courtesy amount (numerical format) and the signature of Fig. 1: A sample of Bangladeshi bank check (Sonali
the person who wrote the check. Bank)
The official value of the check is the amount written in
words; this field of the check is called “literal amount". Check No Account No
The amount written in numbers is supposed to be for Name and Address of
courtesy purposes only and is therefore called “courtesy the Financial Institute Date
amount". The information contained in a check is Payee’s name
frequently handwritten, specially considering that most
Literal amount
of the checks that were written by computer systems
Courtesy amount
have been gradually replaced by newer methods of
electronic payment. Handwritten text and numbers are Signature
difficult to read by automatic systems (and sometimes
even for humans); so check processing normally Fig. 2: A bankcheck layout description
involves manual reading of the checks and keying in

ISBN 984-32-2873-1 ICCIT 2005 650

In Bangladesh each financial institution is responsible
for designing and providing bankchecks to its
customers. The financial institutions must follow some
general standard rules stated by the Bangladesh Bank to
manufacture checks. These standards determine the
elements in a check, their location and size as well as
specification for the paper. In spite of this
standardization, the financial institutions are free to
customize some parts of the bankchecks such as the
background pattern. The background pattern is
generally used to personalize the bankchecks, and they
can assume different colors and have imprinted pictures
and drawings. Other variations may occur among the
bankchecks issued by different financial institutions
such as different character fonts, special symbols, logos,
lines, etc. Figure 1 shows a sample of a Bangladeshi
bankcheck. Figure 2 shows the distribution of the
information that the bankchecks may have, in
accordance with the standards of Sonali Bank,
Bangladesh.
III. SYSTEM OVERVIEW
The system that we have proposed for automatic
processing bankchecks is composed by four main
modules: data acquisition, image processing, data
recognition and database as shown in Figure 3.
R eal B an k ch eck
O p tical S can n er
Im ag e P ro cessin g
D ate
L iteral am o u n t
C o u rtesy am o u n t
S ig n atu re
Fig. 3: System Overview
The data acquisition module includes two devices, an
optical scanner and a MICR scanner. Actually there are
many devices that include both an optical and a MICR
scanner. The optical scanner provides a digitized image
of the bankchecks with 200 dpi spatial resolution and
256 gray levels for the image-processing module, while
the MICR scanner reads account number. The image
processing module will compose of several algorithms
651
for extracting the user– entered data: position evaluation
for adjusting the skew angle and evaluating the
horizontal and the vertical position of the scanned
bankcheck images; items extraction for extracting the
user–entered data, background elimination for
eliminating the background pattern present on the
bankcheck images; baseline erasing and character
elimination for eliminating the vertical and horizontal
baseline, and the character string printed below the
signature baseline. Finally, tracing recovering for
recovering some pixels that were lost during the item
extraction and baseline elimination. Data recognition
module will present different architecture to recognize
courtesy amount, literal amount and method for
signature verification. The database proposes to design
to store financial institution information, the
background pattern samples, customer’s information,
bankcheck layout information, and other parameters
needed for bankcheck processing.
IV. USER–ENTERED DATA EXTRACTION
In spite of intensive research efforts, the degree of
automation in processing bankchecks is still very
limited. Recent papers have addressed the problem of
automatic processing of bankchecks [1],[2],[4].
Currently, two strategies are used for extracting the
user–entered data: thresholding techniques and image
subtraction. Different thresholding techniques have been
suggested to isolate the user–entered data from the
bankchecks [2],[3],[7]. These techniques have shown
good results only if the bankchecks do not have
complex background patterns. If these techniques were
applied to bankchecks in which the background pattern
has colorful pictures and draws, it would be very
difficult to find a threshold value to segment the
background from the other elements. Furthermore, it
would be very difficult to segment the printed
information from the user–entered data since they can
have similar gray levels. On the other hand, the
techniques based on image subtraction have shown
more robustness to segment the user–entered data.
Okada–Shridhar (1997) [7] have proposed a method for
extracting the user– entered data from American
bankchecks that have colorful pictures on the
background pattern
Here in this paper we propose an approach
where the information in which we are not interested
will eliminate in a stage before the information
extraction. The main idea is to handle only sub–images
that contain the user–entered data, said, the courtesy
amount, the literal amount, the date, the payee’s name,
and the signature. After digitizing the bank check image
is position–adjust and a template will use to extract the
areas where the user–entered data is supposed to appear.
For each of the five resulting sub–images, the
corresponding background pattern will subtract from a
sample stored in a database. The baselines present in the
sub–images will detect by using an algorithm based on
the projection profiles. The baselines detect will erase
by substituting the corresponding positions with white
pixels. The character strings present below the baseline
dedicate for signature will eliminate by other
morphological subtraction operation between the
corresponding sub–image and a generated binary image
which contains every make–up character string to be
eliminated. In this new approach we propose to include
a tracing recovering algorithm that will recover some
parts of the user–entered data that can be lost during the
baseline erasing. This novel approach assumes that a
sample of every background pattern is available in a
database.
which will being processed. The basic template is
presented in Fig. 4. The information present into the
white areas will maintain while the information that will
coincide with the black areas will eliminate. By
applying the template we will be able to eliminate the
redundant information, that is, the information in which
we are not interest, and segment the different user–
entered data. Figure 5 shows a resulted image after
applying the template operation. The output of the item
extraction algorithm are sub–images representing the
user–entered items: digit amount, worded amount,
payees’ name, date and signature.

A. Item Extraction
Due to the standardization of the layout structure of
bankchecks, it is reasonable to design a template for
extracting the interested data from any bankcheck, no
mattering that customer has filled–in it or which
financial institution has issued it, only using a prior
knowledge about the domain. As we are only interested
on the user–entered data, the template must include
every area where this data is supposed to appear into the Fig, 4: The template
bankchecks. The other areas can be eliminated without
compromising the understanding of the document.
Nevertheless, these areas are not located exactly at the
same position for every bankcheck. There are small
position variations, depending on the financial
institution that has issued the bankcheck. Thus, the
template must be adapted for these small variations in
order to avoid selecting wrong areas, what can cause the
loss or the degradation of the interested data. From a
basic template, we propose to construct a database with
the possible positions of the interested data. During the
processing, the database will access and the convenient Fig. 5: The extracted areas
parameters will select to adjust the template, according
to the financial institution that will issue the bankcheck

V. DATA RECOGNITION strings into individual digits is a challenging task

because of connected and overlapping digits, broken
A. Courtesy Amount recognition digits, and digits that are physically connected to pieces
Our proposed courtesy amount format is of strokes from neighboring digits. Our proposed
architecture shown in figure 6 involves four stages:
1 2 0 0 0 0
segmentation of the string into individual digits,
In the case of checks, the segmentation of unconstrained normalization, recognition of each character using a
neural network classifier, and syntactic verification.

Segmentation and Recognition

Post-Processing
Divide Slant Correction Neural Network
Image of
String into
Courtesy Digits Size Normalization
Amount
Thickness
Normalization

Fig. 6: Key steps in reading the courtesy amount.

652
Neural network based recognition
While template matching, structural analysis and neural
networks have been very popular classification
techniques for character recognition, neural networks
are now increasingly used in handwriting recognition
Fig. 7: General scheme of Recognition Module.
B. Literal Amount recognition
One of the most difficult tasks in automatic check
processing is the literal amount recognition. Indeed,
writing found on checks certainly covers all the possible
and conceivable writing styles. In addition, these entries
are far from being the best writing samples that their
authors could produce: in addition to the fact that a
writer is rarely happy to fill a check, he is not
necessarily well placed to do it. For these reasons,
researchers try to use all available contextual knowledge
and to combine several complementary approaches to
achieve highly reliable results in literal amount
recognition. Figure 8 shows general architecture for
literal amount recognition of our proposed system.
653
applications. In our endeavor, the recognition module
will implement as an array of four neural networks work
in parallel. The proposed recognition procedure is
shown in Figure 7.
Fig. 8: General architecture of the proposed system
C. Signature verification
Although handwritten signatures are by no means the
most reliable means of personal identification, it
remains one of the most widely acceptable means of
personal identification. It is also non intrusive,
inexpensive and one of the most commonly used
personal identification systems. As by this time
extensive work have been done on this signature
verification field, and benefit comes because there is no
language boundary to apply any suggested method to
apply for other language. In our bankcheck processing
we found that An Off-Line Signature Verification
System Using Hidden Markov Model and Cross-
Validation [5] would be most significant where the
mean error rate is less then 1.5%.
VI. DATABASE
A database proposes to design to store images,
parameters and data that will be used during the
bankcheck processing steps. The images are related to
the samples of the background patterns from the
bankchecks. Several parameters will store in the
database such as the parameters will use to adjust the
template. The data represents the information printed on
the bankchecks, such as customer's name, agency’s
address, account number, etc. The information stored in
the database must be indexed using adequate keywords
that will provide a unique identification for each
account. The keywords will use for indexing the
information in the database will compose of the three
fields present on the bankchecks, as follow:
bank agency account
number number number
To reduce the complexity of the database, it will have to
design to have three levels: bank–level, agency–level
and account–level. Into the bank–level will store the
background pattern samples, and the parameters use to
adjust the template, according to the financial institution
characteristics. The data which is printed on the
bankchecks regarding bank’s agency name and its
address will include into the agency–level, and it has to
store as ASCII text. In this level is also including a list
of the agency’s customers and his/her account numbers.
The account–level will contain the personal information
about every customer and the data that is printed on the
bankchecks, such as the customer’s name, his/her
personal information, etc. This data will also store as
ASCII text. The size of the database is directly related
to the number of the financial institutions, agencies and
accounts.
VII. DISCUSSION AND CONCLUSION
While extensive efforts have already been devoted to
Latin and oriental check processing systems, to the best
of our knowledge, no attempts have been made towards
an Bengali Handwritten check processing system. This
is probably due to the lack of supporting infrastructure
required to conduct, develop, and compare such systems
in order to advance towards systems operating on real
databases. Our system will address a complete solution
for extracting and identifying the information from
bankchecks. The other approaches focus mainly the
extraction of the digit and worded amounts, and
sometimes the extraction of the date [8], [10], [11].
Neither of them has proposed a solution for extracting
all the items of the bankchecks. By applying our
proposed system, an automatic bankcheck processing
system for payment might be feasible for practical
applications
REFERENCE
[1] L. Koerich, L. L. Lee, “Automatic Extraction of
Filled–in Information from Bankchecks Based on
Prior Knowledge About Layout Structure”, First
Brazilian Symposium on Document Image Analysis
(1997), 322–333.
[2] G. Dimauro, S. Impedovo, G. Pirlo, A. Salzo,
“Bankcheck Recognition Systems: Re-Engineering
the Design Process”, III Int. Workshop on Frontiers
in Handwriting Recognition (1996), 419–426.
654
[3] G. Dimauro, S. Impedovo, G. Pirlo, A. Salzo,
“Automatic Bankcheck Processing: A New
Engineered System”, Int. Journal of Pattern
Recognition and Artificial Intelligence 11 (1997),
467–504.
[4] J. E. B. Santos, F. Bortolozzi, R. Sabourin, “A
Simple Methodology to Bank Cheque
Segmentation”, First Brazilian Symposium on
Document Image Analysis (1997), 334–343.
[5] E.J.R. Justino, A. El Yacoubi, F. Bortolozzi, R.
Sabourin, “An Off-Line Signature Verification
System Using Hidden Markov Model and Cross-
Validation” XIII Brizilian Symposium on
Computer Graphics and Image Processing
(SIBGRAPI'00) p. 105
[6] S. Knerr, V. Anisimov, O. Baret, N. Gorski, D.
Price, J. C. Simon, “The A2iA Intercheque System:
Courtesy amount and Legal Amount Recognition
for French Checks”, Int. Journal of Pattern
Recognition and Artificial Intelligence 11 (1997),
505–548.
[7] M. Okada, M. Shridhar, “Extraction of User
Entered Components from a Personal Bankcheck
Using Morphological Subtraction”, Int. Journal of
Pattern Recognition and Artificial Intelligence 11
(1997), 699–715.
[8] A. Bishnu and B. B. Chaudhuri, “Segmentation of
Bangla handwritten text into characters by
recursive contour following”, Proc. 5th ICDAR,
pp.402-405, 1999.
[9] Yi-Kai Chen and Jhing-Fa Wang, “Segmentation of
Single-or Multiple-Touching Handwritten Numeral
String Using Background and Foreground
Analysis”, IEEE PAMI vol.22, 1304-1317, 2000.
[10] U. Pal, Sagarika Datta “Segmentation of Bangla
Unconstrained Handwritten Text”, ICDAR 2003,
1128-1132
[11] U. Pal, B. B. Chaudhuri “Automatic Recognition of
Unconstrained Off-Line Bangla Handwritten
Numerals”, ICMI 2000, 371-378