Robust Zero-Watermarking of Images using Neural Network

Santosh Chapaneri
Department of Electrical and Computer Engineering University of Arizona Tucson, Arizona - 85719, USA Email: santoshc@ece.arizona.edu

Radhika Chapaneri
Thakur College of Engineering and Technology University of Mumbai Kandivali, Mumbai - 400101, India Email: radhikachapaneri@gmail.com

Abstract Watermarking is an effective tool for copyright protection of owners intellectual property. This paper proposes a zero watermark scheme using neural networks due to their high fault tolerance capabilities. The watermark is a meaningful binary image and is not embedded in the cover image because of the zero watermark scheme. As a result, there is no visual degradation in watermarked image. Watermark is rst scrambled to avoid illegal recovery by attacker. Back Propagation Neural Net is used to learn the relevant features of cover image using chaotic maps for random pixel selection. A secret key is obtained by the algorithm which can be used to recover the watermark. The proposed scheme is robust against various attacks on image.

I. I NTRODUCTION As an effective method for copyright protection and content integrity verication of owners intellectual property, digital watermarking has been a very active area in the eld of multimedia security. There are broadly three types of watermarking techniques. The rst is visible embedded watermark with the advantage that it can be easily identied, but it is easier to remove using image processing techniques. The second is invisible embedded watermark, with embedding done either in spatial domain or in frequency domain. The third is zero watermark scheme which is the focus of this paper. The embedding techniques denitely results in visual quality degradation to some extent. To maintain the quality, zero watermark schemes are effective. In such schemes, the watermark is usually extracted from the image instead of embedding into it. The extracted zero watermark is then used for copyright demonstration. II. R ELATED W ORK Schyndel et al. [1] proposed a technique based on manipulating the bit plane of LSB in the spatial domain, but

it is not robust enough. In the frequency domain, Deng et al. [2] proposed a method of embedding watermark in DC component of transformed blocks. Hsieh et al. [3] proposed a watermarking method based on the qualied signicant wavelet tree (QSWT) where the embedding scheme takes the relationships of DWT coefcients and spatial information into consideration. Neural Networks were suggested as alternative approaches due to their high fault tolerance and potential for adaptive training. Davis et al. [4] proposed a method based on neural networks to nd maximum strength watermarks for DWT coefcients. Chun et al. [5] presented three-layer neural networks to determine the strength of watermarking for DCT coefcients of the image. Zhang et al. [6] proposed a RBF neural network to achieve maximum strength watermark as per the frequency components of cover image. Zhou et al. [8] used Support Vector Regression for embedding watermark in the cover image. The above mentioned techniques requires complex embedding and extracting procedures and results in visual quality degradation. To overcome this, zero watermark schemes were proposed in [9], [10] and [11]. The basic idea is to identify relevant features from the cover image and use these features for extraction of watermark. Hwang et al. [9] proposed a spatial domain technique where the features are extracted using variance metrics and XOR operations. Alam et al. [11] used back progagation neural net for learning the features. Our scheme follows closely on this idea. Chang et al. [10] presented a novel Full Counter Propagation neural net, as a combination of Kohonen and Grossberg layers with capacity of bi-directional mapping. This is a Near-Zero Watermark scheme and gives acceptable visual quality.

III. R ELATED T HEORIES A. Arnold Transform

B. Chaos Map Recently, chaotic maps have been used for digital watermarking to enhance the security [12]. The most attractive features of chaos in watermarking are its sensitivity to initial conditions and outspreading of information over the entire space, thus satisfying Shannons requirement of confusion and diffusion [14]. Behavior in chaotic systems is aperiodic so that no variable describing the state of the system undergoes a regular repetition of values. Logistic map is the simplest chaotic map and is described as:

The watermark used in this paper is not a pseudorandom binary sequence, rather it is a meaningful binary two-dimensional image. The watermark is scrambled rst to improve the robustness against attacks. Scrambling means that every pixels in the image change locations, but the sum of pixels remain invariable. The commonly used scrambling methods are Arnold Transform, IFS model, Magic Square, Gray Code Transform and so on. We use Arnold Transform [7] as it is intuitive, simple, periodic and easy to use. The two-dimensional automorphism Arnold Transform Xk+1 = Xk (1 Xk ) (3) is given by x = (x + y) mod(w) (1) where 0 < 4 and 0 X 1. When 3.569 < 4, the map is in chaotic state. The logistic map is recursive, y = (x + p y) mod(h) (2) meaning that the second term is a function of the rst, where (x , y ) = transformed location, (x, y) = original the third a function of the second and so on. All the location, p = seed of permutation, w = width and h sequences generated by this logistic map are extremely = height of image. Transforming repeatedly for the sensitive to initial conditions in the chaotic state, such image generates different results until reaching back the that two logistic sequences generated from different original image due to the periodic property of transform. initial conditions are uncorrelated statistically. The following gure shows effect of the transform:
C. BPNN The Back Propagation Neural Network (BPNN) is a kind of supervised learning network used widely. The basic principle involves the use of steepest gradient descent method for convergence during training of the network [15]. The BPNN architecture for a 1 N 1 network is as shown:

Fig. 1.

Scrambling of Watermark
Fig. 2. Architecture of BPNN

The period of repetition is different depending on size of image. For example, with a 256 by 256 image, the original image repeats after 25 iterations. The advantage of scrambling the watermark is that even if some attacker can detect the watermark signal, he cannot recover it without the scrambling algorithm. Thus, the secrecy and security of watermark is strengthened.

There are three layers including input, hidden and output layers. The hidden and output layers consists of neurons with weighted synapses connecting inputhidden-output layers as shown. Each neuron represents an activation function, which is typically a sigmoid function due to its differentiability. The training

algorithm of BPNN is as follows:

netj (t) =
i

A. Secret Key Generation (4) (5) 1) Consider the cover image as I with 8 bpp grayscale of width Iw and height Ih , 0 I(i, j) 255 where 0 i < Iw and 0 j < Ih . A binary digit watermark image is represented as W of width Ww and height Wh , W (i, j) (0, 1) where 0 i < Ww and 0 j < Wh . 2) Apply Arnold Transform on W to obtain the scrambled watermark W using equation (1) and (2). Convert the resulting watermark to onedimensional sequence as follows:
W = (W0 , W1 , . . . , Wk , . . . , Ww Wh 1) (8)

wij zi (t) j

zj (t + 1) = fact (netj (t))

where
j = index of current unit i = index of predecessor of current unit j fact (x) = activation function = 1/(1 + exp(x)) netj (t) = activation of unit j j = threshold/bias of unit j wij = weight of the link from unit i to unit j zj (t) = output of unit j in iteration t

The derivative of sigmoid activation function fact (x) = fact (x)(1 fact (x)) is required in the training procedure to modify the weight wij . In each iteration, delta rule is used to update the weights by computing the required change wij as follows:
fact (netj ) (tj zj) f (net ) j act
k k

wij = j zi

(6)

if unit j is an output unit if unit j is a hidden unit (7)

j =

wkj

where = learning factor (constant)

tj = target output value of unit j k = index of successor unit zj = output value of predecessor unit j .

Initially, the weights are assigned randomly usually between (0,1). After training, we can use the input vectors with wij to predict the corresponding output vectors. IV. P ROPOSED S CHEME In this section, we discuss our proposed scheme with an algorithm. Arnold Transform is used to scramble the watermark in pre-processing stage. A chaotic map is used to determine randomly the pixel locations in cover image. A BPNN model is established to learn the features at these pixel locations. A binary pattern is generated which is used with the scrambled watermark using XOR operation to determine the secret key, which can be used for copyright demonstration.

where Wk = W (i, j), k = j + (i Ww ), 0 i < Ww and 0 j < Wh . 3) Next perform random pixel selections from the cover image depending on the size of watermark sequence. We use logistic map as dened in (3) to generate a sequence of length Ww Wh . Divide the cover image into non-overlapping 8 8 blocks in scan-line order. For convenience, consider cover image of 256 gray levels and size 256 256. On dividing, we label the blocks from 1 to 1024. Using an initial value K (K = X1 ) for the logistic map, generate a random chaotic sequence X of 1024 elements. Multiply each element by 1024 and round towards innity. Thus, we obtain a sequence of random values X whose range is [1, 1024] in the integer domain. These values indicate the block numbers in cover image. The selected pixel locations are then: Sk = I(Xk , [1], [1]) where Xk is the block number and 0 k < Ww Wh . We consider pixels having 3 3 neighborhood so as to model the BPNN. For each block location Xk in the cover image, select the second row and second column location pixel to avoid any border pixels. 4) To establish the neural network model, an 8 10 1 back propagation neural network is selected to determine the relationships between selected pixels and their 3 3 neighborhoods. The BPNN model consists of 8 nodes in input layer, 10 nodes in hidden layer and 1 node as output. The input of training pattern is (px1,y1 , px1,y , px1,y+1 , px,y1 , px,y+1 , px+1,y1 , px+1,y , px+1,y+1 ) and the desired output is px,y for all selected pixels (x, y) Sk . Here, pi,j is the intensity value of the pixel location (i, j) divided by 255 to constrain the input and output of BPNN to be in the interval (0, 1), since we use

sigmoid as the activation function. The training goal is Mean Absolute Error M AE < T /255 where T is threshold, user parameter. 5) Generate a binary pattern after training of BPNN. Suppose pi,j is the desired output value of BPNN and pi,j is the corresponding actual output value. Then a binary sequence is generated as:
bi,j = 1 if pi,j > pi,j 0 otherwise

V. E XPERIMENTAL R ESULTS In our experiments, standard grayscale cover images (Lena, Baboon, Pepper,...) were used and several kinds of image processing attacks were performed on the watermarked image using Adobe Photoshop to evaluate the performance of our scheme. The watermark used is the binary image logo of character A with size 32 32. Due to space limitations, only the results with Lena are presented, but similar results are obtained with other standard cover images. Peak Signal to Noise Ratio (PSNR) is used to assess the differences in watermarked image quality. A larger value of PSNR means there is little difference between the original image and the processed image.
P SN R(dB) = 10 log10
2 where e is dened as 2 e = 2 Ipeak 2 e

(9)

where 0 i < Ww and 0 j < Wh . 6) Compute the secret key by applying XOR operation to the binary pattern obtained and the watermark as:
KEYi,j = bi,j Wi,j

(10)

(13)

where 0 i < Ww and 0 j < Wh . The set of parameters Q = (p, K, KEY ) and the trained neural network BPNN are registered at a copyright authentication center to prevent malicious attackers from forging an illegal neural network with their own watermarks. Q includes the seed of permutation p, the initial value for logistic map K and the secret key (KEY ). B. Watermark Recovery Watermark can be extracted using three steps: pixel selection, generating binary pattern and watermark recovery, with steps similar as above. Using K , determine the random pixel locations as in step (3). Thereafter, use the trained BPNN to calculate the output values qi,j for all selected pixels. Generate a binary pattern by comparing with the desired pixel values:
bi,j = 1 if qi,j > qi,j 0 otherwise

1 Iw Ih

Iw

Ih

(I(i, j) I (i, j))2

(14)

i=1 j=1

where Iw Ih is the size of cover image, I(i, j) is the grayscale value of (i, j)th pixel of cover image and I (i, j) is the grayscale value of (i, j)th pixel of 2 (attacked) watermarked image. Ipeak denotes the squared peak value of cover image. Normalized Correlation (NC) is used to evaluate the performance of recovered watermark and is dened as follows:
NC =
Ww i=1 Ww i=1 Wh j=1 W (i, j) W (i, j) Wh j=1 W (i, j) W (i, j)

(15)

(11)

where 0 i < Ww and 0 j < Wh . Next, apply XOR operation to binary pattern b and secret key KEY to compute the scrambled recovered watermark as:
Wi,j = bi,j KEYi,j

(12)

where 0 i < Ww and 0 j < Wh . Unscramble W using Arnold Transform with p as the seed to obtain the extracted watermark. With the recovered watermark, one may verify whether the test image is a copy of the cover image or not and thus the potential piracy can be detected.

where Ww Wh is the size of watermark image, W (i, j) is the binary value of (i, j)th pixel of watermark image and W (i, j) is the binary value of (i, j)th pixel of extracted watermark, N C (0, 1). The higher PSNR and NC is, the more similar watermarked image and watermarks are. We choose p = 2, K = 0.1564, = 3.7 and T = 16 empirically in our experiments. Figure (3) shows a) original Lena image, b) original Baboon image, c) original watermark, d) watermark extracted from (a) and e) watermark extracted from (b). Note that since we perform zero-watermarking, the watermarked image is same as cover image with no visual quality degradation. The NC obtained in the attack free case is shown in table I. Figure (4) shows extracted watermarks after following attacks on image: increased luminance by 50%, increased

of watermark image plays an important role in the performance. Higher the size, more better will be the performance but this will increase the storage capacity of the secret key. Furthermore, the scheme is authentic in the sense that if a different cover image is used to recover the watermark with the trained neural network, then the result would be different from the desired watermark. The scrambling of watermark with Arnold Transform helps in recovering the watermark even after severe attacks like cropping.
Fig. 3. Attack Free Recovery

VI. C ONCLUSION In digital image watermarking methods, there are two main requirements: Imperceptibility and Robustness. The embedding techniques degrade the visual quality of images more or less. The zero-watermark scheme eliminates this weakness by extracting relevant features from the cover image instead of embedding watermark into it; thus the visual quality of the cover image remains invariant. Since no information is embedded in cover image, it is possible that an attacker may embed unwanted/illegal information into the cover image and claim copyright. To avoid this, the zero-watermark scheme must be registered at a copyright authentication center. This is in accordance with the ideas presented in [13]. Thus, the scheme can be used for copyright violation detection. We have proposed a neural network based zero watermark scheme using Arnold transform for higher robustness and Chaotic maps for random pixel selection. BPNN model along with XOR operation are used for features extraction. The extracted watermark is highly correlated to original watermark under various attacks.

Fig. 4.

Watermarks after Attacks

contrast by 50% and cropping 1/4th of the image. A detailed analysis of attacks on Lena image is presented in table II. The results suggests the robustness of our scheme under various image processing attacks on the watermarked image. In all cases, NC is higher than 0.85, indicating a good correlation between the extracted watermark and the original watermark. Also, the scheme does not degrade the visual quality of watermarked image due to zero-watermarking. The robustness of the scheme is related to neural network training threshold value. Training with T = 16 shows a superior performance compared to T = 5, but inferior performance compared to T = 64. Also, the size
TABLE I NC OF WATERMARK IN ATTACK FREE CASE Image Lena Baboon NC 0.9935 0.9961

TABLE II PSNR OF I MAGE AND NC OF WATERMARK UNDER VARIOUS ATTACKS ON L ENA Attack JPEG(80%Q) Luminance(+50%) Contrast(+50%) Sharpen Blur Median Filter (3 3) Crop(25%) Mosaic NC 0.9863 0.9968 0.9777 0.9931 0.9421 0.9512 0.8981 0.8672 PSNR (dB) 39.084 13.857 16.357 32.186 31.472 27.925 11.374 20.822

R EFERENCES
[1] R.G.Van Schyndel, A.Z. Trikel and C.F. Osborne, A digital watemark, Proc. IEEE Int. Conf. Image Processing, 1994, vol. 2, pp 86-92 [2] Fengsen Deng and Bingxi Wang, A Novel Technique for Robust Image Watermarking in the DCT domain, Proc. Int. Conf. Neural Networks and Signal Processing, 2003, Vol. 2, pp 15251528 [3] Ming-Shing Hsieh, Din-Chang Tseng and Yong-Huai Huang, Hiding digital watermarks using multiresolution wavelet transform, IEEE Trans. Industrial Electronics, Vol. 48, Oct 2001, pp 875-882 [4] K.J Davis and K. Najarian, Maximizing strength of Digital Watermarks using Neural Networks, Proc. International Joint Conf Neural Networks, 2001, Vol 4, pp. 2893-2898 [5] Shi-chun Mei, Reb-hong Li, H. Dang and Yun-Kuan Wang, Decision of image watermarking strength based on articial neural networks, Proc 9th International Conf Neural Information Processing, 2002, Vol 5, pp. 2430-2434 [6] Zhang Zhi-Ming, Li Rong-Yan, Wang Lei, Adaptive Watermark scheme with RBF Neural Networks, Proc. Int. Conf. Neural Networks and Signal Processing, 2003, Vol. 2, pp 1517-1520 [7] Quan Liu and Xuemei Jiang, Design and Realization of a meaningful digital watermarking algorithm based on RBF Neural Network, IEEE Int. Conf. Neural Networks and Brain, 2005, Vol. 1, pp 214-218 [8] Chun-hua Li, Zheng Lu, Ke Zhou, An Image Watermarking Technique based on Support Vector Regression, IEEE International Symposium on Communication and Information Technology, Oct 2005, Vol 1, pp 183-186 [9] C. Chang, K. Hwang and M. Hwang, A block based digital watermarks for copy protection of images, 5th Asia-Pacic Conf. Commun. 4th Optoelectron. Commun. Conf. 1999, pp 977980 [10] Chuan Chang and Sheng Su, A Neural-Network-Based Robust Watermarking Scheme, IEEE International Conference on Systems, Man and Cybernetics, Oct 2005, Vol 3, pp 2482-2487 [11] Jun Sang and Mohammad Alam, A Neural Network based Lossless Digital Image Watermarking in the Spatial Domain, Int. Symposium on Neural Networks 2005, Lecture Notes in Computer Science 3497, pp 772-776 [12] D. Zhao, G. Chen and W. Liu, A chaos based robust wavelet domain watermarking algorithm, Chaos, Solitons and Fractals Vol 22, 2004, pp 47-54 [13] A. Perrig and A. Willmott, Digital Image Watermarking in the Real World, refer http://citeseer.ist.psu.edu/perrig98digital.html [14] W. Stallings, Cryptography and Network Security: Principles and Practice 3/E, Prentice Hall, 2003 [15] S. Haykin, Neural Networks: A Comprehensive Foundation, Prentice Hall, ISBN 0132733501, 1998 [16] V. Vapnik, The Nature of Statistical Learning Theory, Springer-Verlag, New York, 2000