Survey on Methods for Preventing Privacy Leakage

From Photos in Social Networks

Aditya Rao and Karthik Sankar Ram1

Abstract Currently, online privacy with respect to

images is very rudimentary on online social networks
(OSNs). Recent security issues like privacy infringement,
unfair character assessment (due to inappropriate photos
of a person found online) are highly correlated to
the person0 s information available on OSNs. Although
many privacy protection methods are gathering steam
nowadays, in this paper we focus on the ones which are
mainly related to images and videos. We also include
aspects of Facebook0 s artificial intelligence (AI) software
which, they claim, prevents online embarrassment.

I. INTRODUCTION
Online social networks like Myspace, Orkut,
Facebook and Google+ have inadvertently ushered
in unfavorable elements like online bullying, online fraud and privacy infringement. With over a
billion active users[1], Facebook is arguably the
most popular site for users to share and store moments of their personal life. When a user0 s personal
life starts hampering his/her professional life, the
implications can be dire. Many recent stories[2,3]
reveal that employers have fired employees based
on information found online. A recent survey by
Microsoft [4] found that over 79% of recruiters
used information found on OSNs to access candidates. The survey, conducted by Cross-tab Marketing Services,[5] found that 70 percent of the
recruiting professionals in the U.S.A had rejected
job candidates based on the candidate0 s online
profile.
These unfavorable elements are a major cause of
distress and embarrassment for the users of OSNs.
There are various approaches to this, ranging from
blurring the faces in the image to distorting the image itself, if classified as potentially embarrassing.
We look at one approach from each of blurring the
image, reconfirmation, image distortion.
1

Both Aditya and Karthik are Masters students in the Department

of Computer Science at the University of California, Los Angeles

II. B LURRING THE IMAGE : FACE /O FF :

P REVENTING P RIVACY L EAKAGE F ROM
P HOTOS IN S OCIAL N ETWORKS[6]
Most users on Facebook are unconcerned or
unaware about their privacy settings. Generally,
images posted online end up being shared with
a much wider audience than intended. Privacy
settings, although effective to a certain degree, are
ironically very inefficient at managing the privacy
when a user has a popular friend, who shares/
likes/ comments on the images posted by the user.
Here, a popular friend is a friend who has more
than 500 friends.
The paper explains the ineffectiveness of Facebooks privacy settings while going through various
scenarios with two users Alice and Bob.
A. Scenario 1
Alice and Bob are colleagues. One night they go
to a party and Alice happens to click a few pictures
which portray Bob in embarrassing situations. If
Alice uploads these images on Facebook, then
other colleagues of Alice and Bob can see the
images which depict Bob in very bad light, thus
tarnishing Bob0 s character. Even if Bob has very
strict privacy settings, it is overridden by Alice0 s
privacy settings as she0 s the uploader of the image.
Two sub-cases arise here:
1) Alice also has strict privacy settings which
would still mean that unless Bob0 s and
Alice0 s friend lists are exactly the same, the
uploaded image ends up being visible to
Alice0 s friends who are not friends of Bob,
thus resulting in loss of privacy for Bob.
2) The second sub-case is a more serious one.
If Alice has privacy settings set to public,
then the image would be visible to over 1
billion users of Facebook! There are several

Fig. 1.

Propagation of an uploaded image[6]

instances of embarrassing images going viral

causing a lot of distress to the uploader/
person(s) in the uploaded image.

B. Scenario 2
Alice and Bob are potential candidates for a
promotion. Now Alice has a few pictures that she
had clicked from an event, which portray Bob in
bad light. Alice now wants to upload the image
onto Facebook, knowing that Charlie, who is the
boss of Alice and Bob, and a common friend of
both, would be able to see the image. If Alice does
upload the image, Bob could potentially not get
the promotion or, worse still, Bob might be passed
over for promotion in the coming years too, based
on this one image.
Figure 1 visualizes the propagation of an uploaded image, considering a Facebook user as an
example.

C. Proposed Solution
The authors of the paper have proposed and
implemented a simple three step method to help
protect the privacy of OSN users. The steps are
enumerated below:
1) Use a reliant face detection algorithm to recognize the faces in an image: This happens
once an uploader uploads an image onto the
OSN. The faces are compared to friends,
friends of friends and so on, subsequently
moving outward for each iteration in a graph
structure similar to the one shown above.
2) Template generation: Every recognized user
gets a notification and can set privacy settings for each photo. Default settings can
also be used. Users will be asked to confirm
if the recognized face is indeed him/her and
based on this a template photo is generated.
A template photo is made of the uploaded
image and F layers, where F is the number
of faces recognized. Each layer has a tuple

Fig. 2.

Visualization of the revised access control model[6]

<p,f> where p is the photo and f is each

individual face in the photo.
3) Template rendering: When a photo is to be
viewed by a friend/ OSN user, the access
control matrix (see Figure 2) is used. It
has the value of p,f for each user set, so
that the photo is generated dynamically and
presented to the friend/ OSN user.
Figure 3 gives an overview of the enumerated
steps.
D. Disadvantages
While this paper provides a very convenient
approach to tackle privacy issues, it would fail
when the user uploads images with celebrities/
famous personalities. The overhead becomes too
much for this system to function at an appreciable
speed. Moreover, chances are that some of the
user0 s friends may have uploaded photos in which
he/she consistently tags himself/herself wrongly.
For eg., some users who might be super fans of
Roger Federer might tag themselves over Federer0 s
photos. The system fails in such cases in the initial
step itself as it fails to notify the actual user.

III. R ECONFIRMATION : FACEBOOK

DEVELOPING AI TO AVOID EMBARRASSING
PHOTOS FROM BEING UPLOADED [7]
Facebook was in the news during Dec 2014,
for the fact that FAIR(Facebook Artificial Intelligence Research) was developing a tool to identify
drunken/embarrassing photos. Facebook said that
this tool would prompt the user to reconsider
uploading an embarrassing photo by prompting the
user through messages such as Are you sure your
boss would want to see this?. The key part is how
the identification would be done. This is a hard
computer vision problem which involves identifying the state a user is in in a photo. Machine
learning approaches require each user to provide
Facebook with a set of embarrassing pictures so
that the tool can identify such pictures to a certain
extent. But such approaches are highly impractical
considering Facebook0 s active user base.
IV. I MAGE D ISTORTION : S ECURE JPEG
S CRAMBLING ENABLING P RIVACY IN P HOTO
S HARING[8]
Unrestrained online photo or multimedia sharing
has raised serious privacy concerns, especially after reports of citizens surveillance by governmental

Fig. 3.

Overview of the entire process[6]

agencies and scandalous leakage of private photos

from prominent photo sharing sites or online cloud
services. Most solutions allow users to control
either who can access the shared photos or for
how long they can be accessed. In contrast, in
[8], the authors take a structured privacy by design approach to the problem of online photo
privacy protection. This paper proposes a privacypreserving photo sharing architecture based on
a secure JPEG scrambling algorithm capable of
protecting the privacy of multiple users involved
in a photo.

Fig. 4.

Processes of JPEG Encoding and Decoding[8]

A. Proposed Method
The paper proposes a multi-region selective
JPEG scrambling scheme to protect visual privacy
in photo for multiple users. In such a scrambling
scheme, one can scramble multiple regions of
interest (ROIs) with arbitrary shapes in an images,
using one or different secret keys. Each scrambled
region is assigned with an ID and the descrambler can selectively descramble the regions using
corresponding scrambling keys. The scrambling
of the JPEG data is achieved by modifying the
signs of the quantized discrete cosine transform
(DCT) coefficients corresponding to the defined
ROIs. Scrambling and descrambling processes can
be done in not only JPEG encoding and decoding
respectively, but JPEG transcoding. Scrambling a
JPEG image in transcoding mode ensures a loss-

less reconstruction of the original image because

transcoding process does not involve quantization and requantization operations and therefore
does not affect the DCT coefficients outside the
scrambled regions. Scrambling and descrambling
in JPEG encoding/decoding and transcoding are
illustrated by Figure 4.
1) Region Selection and Key Preparation: First
of all, one can select several regions in an image,
by providing a mask matrix M, non-zero elements
of which indicate the 16 x 16 Minimum Coded
Unit (MCU) blocks of the image to be scrambled.
The scrambled regions are restricted to match the
MCU blocks boundaries. In this matrix, except
for zero-valued elements, the value n of non-

Fig. 5.

Processes of JPEG Encoding and Decoding[8]

zero element is referred to as the ID of each

scrambled region. We note each scrambled region
as ROIn . For each ROIn , a secret scrambling key
kn and a scrambling strength level ln which lies in
the range of 1 to 4 are defined. The scrambling
key can be any value or sequence set by user.
The scrambling strength is subdivided into four
levels: 1-low, 2-medium, 3-high, and 4-ultra-high,
meaning of which is shown in Figure 5. Therefore,
a mask matrix M with regions ID n defined, a
set of secret scrambling keys kn (n = 1, 2, 3), and
a corresponding set of scrambling strength levels
ln (n = 1, 2, 3) constitute the parameters to scramble
an image.
2) DCT Coefficient Manipulation: In the next
step, DCT coefficients corresponding to the defined scrambled regions are modified, according to
the scrambling key and strength level. A pseudo
random number generator (PRNG) initialized by
a seed value is used to drive the scrambling
process, where we simply use the scrambling key
as the seed. For low-level scrambling, only AC
coefficients of all YUV components are modified;
for medium-level scrambling, both DC and AC
coefficients of only luminance (Y) component are
modified; while for high-level scrambling, both
DC and AC coefficients of all YUV components
are changed. In case a stronger scrambling is
needed (ultra-high-level), we can further scramble
the DC coefficients, by performing a bitwise XOR

operation between each DC value and a pseudorandom number with the same length in bits as
the DC coefficient. Figure 5 illustrates scrambled
versions of an image with different strength levels.
3) Scrambling Information Insertion: Once an
image is scrambled, information about the scrambled regions is inserted in one or more application
markers (APPn) in JPEG file header. The information to be inserted includes the elements of the
mask matrix M and the scrambling strength, former of which records the location, shape and IDs
of the scrambled regions. Therefore, the scrambled
image is JPEG-compliant and can be viewed by
a typical JPEG decoder. However, to descramble
and view the original image, a special descrambler
(decoder or transcoder) and the correct scrambling
key(s) are needed. Intuitively, descrambling process simply reverses the scrambling processes described above. Given a region ID, the descrambler
can extract corresponding scrambling strength and
region location and shape from APPn markers of
JPEG header. As long as a correct scrambling
key is provided, the region can be recovered. The
process of a multi-region scrambling and selective
descrambling is illustrated in Figure 6.
V. CONCLUSIONS
While there are people so concerned about privacy that they do not post any pictures of themselves or their loved ones on social media or even
the Internet, this is not the case with a majority

Fig. 6.

Multi-region scrambling and selective descrambling[8]

of the online community. Cybercrime is growing

more and more prevalent with the advancement in
technology. Hence, better and efficient methods to
protect user privacy like those discussed in our
survey paper must be adopted by leading social
media platforms like Facebook and Twitter in order
to prevent misuse of their service and unnecessary
embarrassment of their users.
ACKNOWLEDGMENT
Both of us would like to thank our professor,
Dr. Miodrag Potkonjak, for helping us choose
our topic for this survey paper and for giving us
valuable feedback after our class presentation.
R EFERENCES
[1] Statista, Number of monthly active users of Facebook
(http://www.statista.com/statistics/264810/number-ofmonthly-active-facebook-users-worldwide/).
[2] M. Mitchell, Georgia man loses job, upsets
mother
after
Facebook
post,
Fox
5
News
(http://www.fox5atlanta.com/news/30427294-story).
[3] D. Love, 17 people who were fired for using Facebook,
Business Insider (http://www.businessinsider.com/facebookfired-2011-5).

[4] Cross-tab
and
Microsoft
Research,
Online
Reputation
in
a
Connected
World
(http://www.slideshare.net/nickbraak/online-reputation-ina-connected-world).
[5] S.J.
Johnston,
Microsoft
Survey:
Online
Reputation
Counts
(http://www.internetnews.com/webcontent/article.php/3861241/Microsoft+
Survey+Online+Reputation+Counts.htm).
[6] P. Ilia, I. Polakis, E. Athanasopoulos, F. Maggi, and S.
Ioannidis, Face/Off: Preventing Privacy Leakage From Photos
in Social Networks. ACM Conference on Computer and
Communications Security. pg 781-792. 2015.
[7] C.
Bryant,
Facebooks
A.I.
To
Help
You
Eliminate
Embarrassing
Photos,
Newsy
(http://www.newsy.com/videos/facebook-s-a-i-to-help-youeliminate-embarrassing-photos/)
[8] L. Yuan, P. Korshunov and T. Ebrahimi, Secure JPEG Scrambling enabling Privacy in Photo Sharing. Workshop on Deidentification for privacy protection in multimedia, Ljubljana,
Slovenia, 2015.
[9] A. Acquisti and C. M. Fong. An experiment in hiring discrimination via online social networks, 2013.