A ROBUST CONTENT BASED DIGITAL SIGNATURE FOR IMAGE AUTHENTICATION

Marc Schneider and Shih-Fu Chang

Columbia University
Image and Advanced Television Laboratory
Room 801 Schapiro Research Building
530 West 120th Street
New York, NY 10027-6699
USA
E-mail: {mars, sfchang}@ctr.columbia.edu

Abstract
A methodology for designing content based digital signatures
which can be used to authenticate images is presented. A continu-
ous measure of authenticity is presented which forms the basis of
this methodology. Using this methodology signature systems can be
designed which allow certain types of image modification (e.g.
lossy compression) but which prevent other types of manipulation.
Some experience with content based signatures is also presented.
The idea of signature based authentication is extended to video, and
a system to generate signatures for video sequences is presented.
This signature also allows smaller segments of the secured video to
be verified as unmanipulated.
1.0 Motivation
Powerful, and easy to use image manipulation software has made it
possible to alter digital images. It has been suggested that the
authenticity of digital images can be preserved by having a camera
“sign” the image using a digital signature. [1] However, applying a
signature scheme directly to the image has some drawbacks. For
many applications, image compression is desired to reduce trans-
mission bandwidth, storage space, etc. Authenticity, the ability to
detect image manipulation, is also desired. These two function are
at odds with each other since lossy compression is a form of manip-
ulation. Our goal is to develop a way to be able to prove some form
of authenticity, while still allowing desired forms of manipulation,
such as lossy compression. Ideally, a robust signature scheme
should not declare an image modified under these circumstances.
Digital signatures can be used for more than just image authentica-
tion. In particular when combined with secure timestamp, a digital
signature can be used as a proof of first authorship.
A watermark, on the other hand, is a code secretly embedded into
the image. The watermark allows for verification of the origin of an
image. However, a watermark alone is not enough to prove first
authorship, since an image could be marked with multiple water-
marks. It has also been pointed out [6] that digital watermarks are
not well suited to protecting the authenticity of an image.
3.0 Content Based Signatures
The key to developing a robust digital signature for images is to
examine what the digital signature should protect. Ideally the signa-
ture should protect the message conveyed by the content of the
image, and not the particular representation of that content. Thus
the robust signature can be used to verify the authenticity of an
image which has been modified by processing that does not affect
the content of the image. Examples of this type of processing are
removal of noise or lossy compression. However, manipulation of
the image which changes the content, such as removal of a person
from a scene, can still be detected by the use of this signature.
Additionally, the use of a content based signature fits well with
other content based image processing, such as content based coding
and queries. By using the same content for both the signature and
the compression algorithm, the signature will be able to authenti-
cate images highly compressed using content based coding. With
content based queries a signature can be the basis of a query.

2.0 Previous Work 4.0 Authenticity and Feature Selection

Previous work on image authentication falls into two groups, digital
signatures[1] and digital watermarks[3]. A digital signature is based
upon the idea of public key encryption. A private key is used to
encrypt a hashed version of the image. This encrypted file then
forms a unique “signature” for the image since only the entity sign-
ing the image has knowledge of the private key used. An associated
public key can be used to decrypt the signature. The image under
question can be hashed using the same hashing function as used
originally. If these hashes match then the image is authenticated.
Often people think of authenticity as a binary quantity, either an
image is authentic or it is not authentic. However, this not always
what people want when they are concerned with detecting image
manipulation. We propose a continuous interpretation of authentic.
An image which is bit by bit identical to the original image is con-
sidered completely authentic (authenticity measure of 1.0). An
image which has nothing in common with the original image would
be considered unauthentic (authenticity measure of 0.0). All other
images would be partially authentic. Partially authentic is a loosely
defined concept and measurement of the authenticity is subjective,
and changes from application domain to application domain. One
way of thinking of this authenticity measure is as an authenticity
vs. modification curve (see Figure 1). For example a curve could Original Content Hash for Encryption Content
Data Based
be drawn relating authenticity to the bit rate of a compressed Image Extraction + Digital
image. Thus for each different type of modification there would Reduction Signature
Io Co = fc(Io) Ho = fh(Co) S
be a corresponding curve. The old concept of authenticity would
be represent as a Dirac delta function or a unit step function for all
of the possible types of modification.
Private Key
Kpr
JPEG Compression
Authentication

Figure 2. Generating a Content Based Signature

Binary Cropping
Authentication encrypted. The hash Ho is then encrypted using the private key
Kpr of the signing entity to produce the final signature S. To verify
Modification
C o = f c(I o)
Figure 1. Authenticity vs. Modification Curve
H o = f h(C o)
S = H o ⊕ K pr
Since authenticity is a subjective quantity, it is difficult to use
directly as the basis of an authenticity verification system. We the authenticity of an questionable image It, the signature is
therefore need an approximation to authenticity which is analyti- decrypted using the public key Kpu and is compared to the hashed
cal and can be computed from an image. The approach we are tak- content extracted from the questionable image. If the distance
ing is to define a concept call feature authenticity Af, which is one between the feature vectors is less then a threshold value tau, then
minus the normalized distance between a feature vector computed the questionable image is declared unmanipulated. This procedure
for the original image, Io and the same feature vector computed is shown in figure 3.
from the image whose authenticity is to be measured, Im. The key C t = f c( I t )
A f = 1 – feature(I o) – feature(I m) normalized
H t = f h(C t)
H o = S ⊕ K pu
is to find a set of features such that the feature authenticity closely
approximates the image authenticity curves for the allowable H o – H t ≤ τ ⇒ authentic
forms of modification (e.g. lossy compression). Additionally, the
feature authenticity curves for undesired forms of modification Test Content Hash for
Image Extraction Data
should be significantly below the curves for allowable forms of Reduction Compare
manipulation. Using the continuous measure of feature authentic- It Ct =fc(It) Ht = fh(Ct)
ity, a minimum acceptable authenticity can be defined. This can Decryption
be defined directly, or defined in terms of some acceptable amount Public Key
of manipulation. For example, the minimum acceptable authentic- Kpu -
ity can be defined in terms of maximum compression ratio. This
minimum authenticity becomes a constraint on the optimization
of the feature set. Once acceptable forms of manipulation are Content
specified (e.g. compression, noise reduction, etc.) and the unac- Based
Digital
ceptable forms are specified (e.g. cropping, cut and paste, etc.) the Signature
optimal set of features can be found. The goal is to have the S
authenticity vs. modification curve have a gentle slope for desired Figure 3. Verifying a Content Based Signature
type of manipulation, and to have a very steep slope for the undes-
ired forms of manipulation.
The threshold value can be set by examining the amount of error
introduce into an image by lossy compression. The difference
5.0 Generating and Verifying a Content Based between the image content hash at a target compression rate and
Signature the original image content hash can be used to set the threshold
value. Note: If the threshold value is non zero, a cryptographically
The general procedure for generating a content based signature is based hashing function can not be used, since there is no signifi-
diagramed in figure 2. First, the content of interested Co is cance to closeness once a cryptographic hash has been applied to
extracted from the image Io to be signed, using an extraction func- the content. A non cryptographic hashing function can be used to
tion fc. The content is then possibly hashed, using a hash function reduce the size of the signature, however this will typically
fh, to reduce the amount of data. This may be necessary since the weaken the signature.
size of the signature is dependent upon the amount of data
It should be noted that a cryptographic hash can be used as the con-
tent extraction function. This content is of course not related to the
way visual information is processed. Thus operations such as lossy
compression as well as most other forms of image modification will
have very steep authentication vs. modification curves. This is of
course the signature scheme proposed in [1].
The problem is now one of finding a set of features which ade-
quately describe the content of an image. Several different features
can be used such as edge information, DCT coefficients, and color
or intensity histograms.
We examined using the intensity histogram to sign the image. How-
ever, the histogram of the entire image itself is not very useful,
since it contains no spatial information about the image intensities.
Thus the images were divided into blocks and the intensity histo-
gram for each block was computed separately. This allows some
spatial information to be incorporated into the signature since the
location of these blocks are fixed. Further spatial information can
be incorporated by using a variable size block. Starting with small
blocks, the histograms can be combined to form the histogram of a
larger block. This can be used to produce blocks of different sizes
for different parts of the image, allowing fine details to be protected
by a small block size, and larger regions to be protected by a larger
block size.
The distance function for detecting content changes is a subtle and
challenging issue. The euclidean distance between intensity histo-
grams was used as a measure of the content of the image. This per-
formed well in detecting modification of the image. The amount of
lossy compression which could be applied to the image and not pro-
Figure 4. Original Image
Figure 6. Manipulated Image
Strip on Fireman’s Jacket Removed
duce a false positive was limited to at most 4:1. If we used a
reduced distance function, then the maximum permissible compres-
sion ratio is increased. However, this increased robustness is
achieved at the cost of sensitivity to subtle image manipulation. For
example, It was found that the mean of the intensity histograms was
a useful measure for detecting image content manipulation. Several
different images were signed using the block average intensity tech-
nique. These images were then altered, typical altered test images
are shown in figures 5 and 6, and the signatures were used to suc-
cessfully detect the image manipulation. Note that the white boxes
were added to highlight the modified regions and did not appear in
the original test images. The original image was also compressed
using JPEG compression. The signature system was not triggered
even at high compression ratios of 14:1. As opposed to the euclid-
ean distance using the full histogram, the maximum compression
ratio is increased, but we can clearly see the trade-off.
Content based signatures for images can also be used to have an
author sign an image. A typical use of this would be proof of first
authorship. For this application the signature would have to be pro-
cessed by a secure timestamp server. Additionally, it would be
desirable to have the signature embedded into the image for this
application. The signature should travel with the image so that
authorship can readily be confirmed. It should be embedded so that
even if the image is converted from one format to another the signa-
ture will remain with the image.
Embedding the signature into the image brings forth an additional
issue, development of an embedding process which does not effect
the signature verification process, since the embedding processes
manipulates the image data. Information embedding in images is a
Figure 5. JPEG
Compressed 14:1
Figure 7 Manipulated Image
Fire Hose Removed
generalization of the image watermark problem. One possible
approach to embed a watermark into an image is to code the sig-
nature such that it resembles quantization noise and embed this in
the image.[7] Another technique embeds a message into an image
in the frequency domain. The image is broken into 8x8 blocks and
a Discrete Cosine Transform is performed on each of the blocks.
The signature is embedded by modifying the middle frequency
coefficients and transforming the block back to the spatial
domain.[8] This technique could easily be used with a signature
based on the DC component, since this is unaffected by the
embedding process. A third method also embeds a message in the
frequency domain, however, here the image is treated as a noisy
communication channel. The watermark is transmitted in this
channel using a spread spectrum technique. [9]
6.0 Authentication of Video
It is also possible to extend authentication systems to video. There
are two additional problems that need to be addressed when deal-
ing with video sequences. The first is that of maintaining frame
order integrity. The still image authentication techniques can be
applied to each frame of the video sequence, but an addition sig-
nature must be provided to insure that the frames aren’t reordered.
The second problem is that we would like an unmodified clip
from the larger sequence to be detected as unmodified. Here we
present an extension to the original trustworthy camera[1]. A
cryptographic hash is applied to each frame of the video. The
hashes are ordered according to the corresponding frame order.
This ordered sequence of hashes is then itself hashed (See Figure
8). All of these hashes are then encrypted using a public key cryp-
tographic system to prevent forging of the hashes. To verify the
authenticity of the entire video sequence only the second level
hash is needed. The first level hashes are generated from the video
sequence to be checked. These are then used to compute the sec-
ond level hash. This second level hash is then compared to the
original second level hash. To verify a subsection of the video
sequence, first level hashes are generated from the subsection.
Missing hashes are supplied from the signature. The second level
hash can then be generated and checked. This system can be used
to protect any group of pictures, not just video. For example, it
could be used to protect the authenticity and order of multiple
slices of MRI data.
Video Sequence
A Hash for Every Frame
First Level Hash
Hash Generated from
All First Level hashes
Second Level Hash
Figure 8. Two Level Video Hashing
This idea can be extended to provide a more flexible method of
verifying the authenticity of still pictures. In order to do this we
can break the still picture into blocks. A hash is generated for each
of the blocks. The blocks are then ordered, for instance by scan-
ning across the rows. The sequence of hashes is then hashed to
generate a signature which can protect the order. This image sig-
nature can be used to verify the authenticity of sections of cropped
images. The blocks not effected by the cropping can be verified
using the hashes for those blocks.
7.0 Contribution and Conclusion
The contributions of this work are the idea of using the image
content to form a signature which can be used to protect the
authenticity of images and survive acceptable compression. We
also proposed a method for the extension of the authentication
system to video. We have also presented a methodology for deter-
mining the set of features which can be used to approximate the
image authenticity. We have also presented a method to extend
digital signatures to video sequences, which can also be used to
enhance the robustness of signatures for still images.
8.0 References
[1] Friedman, Gary L., “The Trustworthy Digital Camera: Restor-
ing Credibility to the Photographic Image”, IEEE Transactions on
Consumer Electronics, vol. 39 no. 4, November 1993, pp. 905-
910.
[2] Netravali, Arun N. and Haskell, Barry G.“Digital Pictures:
Representation and Compression”, New York, NY, Plenum Press,
1988.
[3] Walton, Steve, “Image Authentication for a Slippery New
Age”, Dr. Dobb’s Journal, April 1995, pp. 18-26
[4] Stinson, Douglas R., “Cryptography: Theory and Practice”,
Boca Raton, FL, CRC Press, 1995.
[5] Wallace, G.K., “The JPEG still picture compression stan-
dard.”, Communications of the ACM, vol. 34, no. 4, April 1991,
pp. 30-40.
[6] Macq, B.M. and Quisquater, J.J., “Cryptology for Digital TV
Broadcasting”, Proceedings of the IEEE, vol. 83, no. 6, June
1995.
[7] Matsui, Kineo and Tanaka, Kiyoshi, “Video-Steganography:
How to Secretly Embed a Signature in a Picture”, IMA Intellec-
tual Property Project Proceedings, vol. 1, pp. 187-206, 1994.
[8] Zhao, Jian and Koch, Eckhard, “Embedding Robust Label into
Images for Copyright Protection”, Proc. of the International Con-
gress on Intellectual Property Rights for Specialized Information,
Knowledge and New Technologies., Vienna, Austria, August 21-
25, 1995.
[9] Cox, I.J., Kilian, J., Leighton, T. and Shamoon, T., “Secure
Spread Spectrum Watermarking for Multimedia”, NEC Research
Institute Technical Report 95-10, 1995.