This article has been accepted for inclusion in a future issue of this journal.
Content is final as presented, with the exception of pagination.
IEEE TRANSACTIONS ON COMPUTATIONAL SOCIAL SYSTEMS
Synthetic Social Media Data Generation
Yalin E. Sagduyu , Senior Member, IEEE, Alexander Grushin , and Yi Shi , Senior Member, IEEE
Abstract— This paper presents a novel system, synthetic high-
fidelity social media data generator (SHIELD), for generating
the synthetic social media data. SHIELD jointly generates time-
varying, directed and weighted interaction graph structures and
topic-driven text features similar to the input social media data.
A synthetic interaction graph is generated by a social network
model to minimize the distance to real graph and is enhanced
by adding various patterns, such as anomalies and information
cascades, interaction types, and temporal dynamics. A synthetic
text generator based on the n-gram Markov model is trained
under each topic identified by topic modeling. Synthetic text
and graph structures are combined through the assignment of
synthetic social media entities. Extensive performance evaluation
via a graph and text analysis is provided to demonstrate the
statistical fidelity of large-scale synthetic data generated by
SHIELD. A data evaluation exercise with human participants is
executed to identify how difficult it is for a human to distinguish
between tweets that were generated by SHIELD and tweets that
were posted by real users. Experimental results followed by a
statistical significance analysis showed that human participants
cannot reliably distinguish between real and synthetic tweets.
Index Terms— Computational test, dK-2 distance, experiment,
graph, graph fitting, interactions, latent Dirichlet alloca-
tion (LDA), social media, social network, synthetic data, text,
topic modeling, topics.
I. I NTRODUCTION
S OCIAL media has been growing at a fast pace, with
various microblogging and social networking services
generating large-scale data. As an example, Twitter has around
336 million monthly active users (reported for the first quarter
of 2018 [1]). Social media provides rich data feeds that can be
utilized for different purposes, such as marketing and social
event prediction. It has changed the way that people interact
with each other and respond to events. However, the full
capture and understanding of social media is largely missing
due to the growing number and type of data feeds with
complex interactions among social network actors. In this
context, social media analytics (including graph, machine
learning, and natural language processing algorithms) play a
critical role in extracting, capturing, and assessing information
in social media.
In turn, research and development progress in social media
analytics depends on the availability of social media data
Manuscript received September 11, 2017; revised March 23, 2018 and
June 7, 2018; accepted June 10, 2018. This work was supported by the
Defense Advanced Research Projects Agency (DARPA) under Contract
W31P4Q-13-C-0055. The views, opinions, and/or findings expressed are those
of the author and should not be interpreted as representing the official
views or policies of the Department of Defense or the U.S. Government.
(Corresponding author: Yi Shi.)
The authors are with Intelligent Automation, Inc., Rockville, MD
20855 USA (e-mail: ysagduyu@i-a-i.com; agrushin@i-a-i.com; yshi@
i-a-i.com).
Digital Object Identifier 10.1109/TCSS.2018.2854668
2329-924X © 2018 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission.
See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
1
to model and analyze the social behavior. Existing small
and static data sets cannot keep up with growing social
media feeds. Social media data are available through either
public application programming interfaces (APIs) or paid data
services. However, there are privacy limitations on collect-
ing, sharing, or distributing new social media data (such as
Facebook or Twitter data). These limitations may slow down
the progress in social network research and social media
analytics by granting data access only to a small group of
researchers and preventing full data disclosure that would be
necessary to verify and further improve the results reported
in research findings. Therefore, it is important to generate
large-scale synthetic data sets that reflect real-word data sets
in terms of statistical or application properties and can be
shared with others without privacy concerns because of their
anonymized nature. In addition, synthetic data can be used
for generating and analyzing the large-scale and high-fidelity
behavior of networks of social bots (automated social media
posting programs, such as spammers) and for marketing,
advertisement, and generating buzz in social media. These
synthetic data that are needed for these purposes should be
large-scale, automatically generated, and high fidelity in terms
of similarity to real data.
A. Novel Contributions
This paper presents the synthetic high-fidelity social media
data generator (SHIELD) system for generating large volumes
of synthetic data in social media, such as from microblogging
(e.g., Twitter) or social networking (e.g., Facebook) services.
SHIELD integrates synthetic graph structures and synthetic
text features (while preserving different graph and text pat-
terns, such as anomalies, temporal dynamics, information
cascades, and topics) to produce large-scale and anonymized
social media data that is similar to input social media (in terms
of statistical and application-level properties). SHIELD gen-
erates the underlying graph structure and the text content
jointly by fitting to the graph, text, statistical, and temporal
characteristics of the input data generated by real social
media, such as data from microblogging or social networking
services. Computational tests demonstrate that SHIELD can
generate high-fidelity data that is statistically similar to real
data (although realizations are widely different), whereas a
data evaluation exercise with human participants demonstrates
that participants cannot reliably distinguish synthetic data
generated by SHIELD from real data.
Our novel contributions are given as follows.
1) To generate synthetic graph structures, a social inter-
action graph is generated from social interactions in
the given social media data set [2]. Users and other
social media entities (such as hashtags in Twitter) are
 This article has been accepted for inclusion in a future issue of this journal. Content is final as presented, with the exception of pagination.
2 IEEE TRANSACTIONS ON COMPUTATIONAL SOCIAL SYSTEMS
assigned to vertices in the interaction graph and social
media interactions between two entities (e.g., mention,
reply, and retweet in Twitter) are assigned to edges.
Then, a synthetic graph similar to the given interaction
graph is generated by minimizing the distance (statis-
tical or application-dependent) between the input and
synthetic graphs. Synthetic patterns (such as anomalies
in the form of high-degree vertices, hubs, and cliques)
are added to the synthetic graph. The synthetic graph
is generated with multiple connected components and
attributes (such as group memberships) and can be
fitted to composite social network models and temporal
dynamics in the input data.
2) The textual content is extracted from the given social
media data set and synthetically generated by dividing
data based on topics, training an n-gram model (Markov
chain) for each topic, sampling text data from the
models, and filtering synthetic social media posts with
identical text or with grammar mistakes. Social media
entities (e.g., hashtags and hyperlinks in tweets) are
added with the same distributions as in the input data.
3) Large-scale synthetic social media data sets are gen-
erated by combining the synthetic graph and synthetic
textual content. Graph and textual properties of social
media entities (such as hashtags in Twitter) in the input
interaction graph and the synthetic graph are matched
with each other, and the text is selected for the most
dominant topic assigned to some localized subset of
vertices in the synthetic graph. Synthetic data generation
is scaled up beyond the size of input social media
data by sampling vertices and edges according to the
fitted social network model and sampling text and social
media entities according to their statistical properties
in the input social media data. Synthetic data genera-
tion is parallelized by dividing the input social media
data into multiple sets by the posting time, generating
synthetic social media for each set, and combining data
sets through common usernames and social media text
entities.
4) We performed extensive tests to evaluate the synthetic
data generated by SHIELD as follows.
a) First, we performed computational experiments
where we collected a large set of data from Twitter,
generated a large corresponding set of synthetic
tweets, and compared the two data sets in terms of
graph and text properties. The analysis showed that
synthetic graph and text structures are statistically
similar to their real counterparts, but that their
realizations are different.
b) Second, we carried out a data evaluation exercise
with human participants to measure how difficult
it is for the participants to distinguish between
tweets that were generated by SHIELD and tweets
that were posted by real users. Participants are
given a set of tweets and asked to determine
whether each tweet was generated by a human
(real) or a machine (synthetic). After classifying
the tweets, participants complete a brief follow-up
survey about which features they used to decide
whether tweets are real or synthetic. We received a
Human Research Protection Office (HRPO) Con-
currence Memorandum with the exempt determi-
nation for the protocol “SHIELD Data Collection
Plan.” Experimental results followed by the sta-
tistical significance analysis showed that human
participants cannot reliably distinguish between
real and synthetic tweets.
SHIELD is particularly useful in generating data to be
used for various purposes, including developing and testing
new social media analytics, generating or analyzing social bot
network behavior and campaigns in social media, and sharing
test data with others without privacy concerns.
B. Related Work
Synthetic graphs can be generated according to various
social network models, such as Forest Fire [3], Nearest Neigh-
bor [4], Random Walk [4], Barabasi–Albert [5], Octopus [6],
Kronecker [7] and their tensor follow-ups [8], Random Typing
[9], dK-2 [10] and dK-2.5 [11], or by other graph gen-
eration algorithms [12], [13]. For social network models,
graph parameters (in parametric models) can be selected to
make the synthetic graph statistically similar to a real graph,
as discussed in [14].
On the other hand, synthetic text can be generated separately
via natural language generation processes such as n-gram
models [15]. Recently, machine learning, e.g., the generative
adversarial network (GAN) [16], has been applied to build
generator models for synthetic data, such as images [17],
topological features of graphs [18], and text [19]. However,
the scalability and the need for fine-tuning the underlying deep
neural networks for the generator and discriminator models in
the GAN remain as challenges.
From a more practical application perspective, there have
been efforts to generate chat bots [20], [21] that can produce
synthetic natural language for a conversation. For example,
a chat bot is developed in [20] for the Twitter social network
for entertainment and viral advertising, and an agent model is
developed in [21] to generate humorous sentences and recog-
nize humoristic expressions introduced by the user during the
dialog. Chat bots have been used for many applications, such
as speech recognition [22], authorship recognition [23], and
generating conversations [24]. SHIELD provides a systematic
and scalable solution to generate synthetic data that can
be used to drive not only individual but also coordinated
groups of chat bots. SHIELD’s major advantage is jointly
generating synthetic graph and text features along with social
media entities that involve highly complex structures similar
(statistically and at the application level) to the real social
media data.
The rest of this paper is organized as follows. Section II
describes the SHIELD system that consists of synthetic graph
generation (see Section II-A), synthetic text generation (see
Section II-B), and combination of synthetic text and graph
structures (see Section II-C). Section III provides a computa-
tional fidelity analysis of synthetic graph (see Section III-B)
 This article has been accepted for inclusion in a future issue of this journal. Content is final as presented, with the exception of pagination.
SAGDUYU et al.: SYNTHETIC SOCIAL MEDIA DATA GENERATION
Fig. 1. Main steps of SHIELD.
and text (see Section III-C) properties. Section IV provides
experimentation results based on a data evaluation exercise
with human participants. Section V concludes this paper.
II. S YSTEM D ESCRIPTION
Fig. 1 shows three main steps of SHIELD in generating
synthetic social media data: Step 1a generates synthetic graph
structures, Step 1b generates synthetic text, and Step 2 com-
bines synthetic text and graph structures (through synthetic
social media entities). Step 1a and Step 1b can be run in
parallel, followed by Step 2. We will discuss each step in detail
in Sections II-A–II-C. Each step applies state-of-the-art tech-
niques with analytical foundations. SHIELD combines these
steps in a unique way to jointly generate synthetic graph and
text data for social media. SHIELD can be applied to social
media data from different platforms. While we use Facebook
and Twitter data for numerical results, the underlying graph
and text generation procedures of SHIELD are generic enough
to extend to different platforms.
A. Synthetic Graph Generation
Methods for generating synthetic graph structures G  for
social media are shown in Fig. 2.
1) Interaction Graph Generation: An interaction graph G
is extracted from the input (real) social media data. An interac-
tion graph represents how social network actors interact with
each other [25], [26]. Entities and their interactions in social
media are identified, and an interaction graph is built with a
vertex set V , including entities, an edge set E representing
interactions, and an attribute set A, which includes both
vertex (entity) attributes and edge (interaction) attributes.
This framework can be applied to any social media platform.
In the following, we provide examples.
1) Twitter: The vertex set V includes users and hashtags,
while the edge set E includes replies, retweets, and
mentions (of both users and hashtags).
2) Instagram: The vertex set V includes users and tags,
while the edge set E includes post comments and
mentions in post captions or post comments.
3) Facebook: The vertex set V includes users, while
the edge set E includes wall posts and photograph
comments.
In the attribute set A for each platform, user and hashtag/
tag information, such as language and location for a user,
3
Fig. 2. Step 1a: generate synthetic graph structures.
is defined as vertex attributes, and edge attributes are defined
by the number of interactions between two vertices.
In this section, we will use Facebook data (corresponding
to a graph of 10k vertices) in all the presented examples
to demonstrate the effectiveness of individual graph gener-
ation components of SHIELD. In this data set, interactions
correspond to wall posts or photograph comment activities
(captured from Facebook for the New York region [14]) and
build the edges in the interaction graph. We will switch to
a larger Twitter data set (corresponding to a graph of more
than 75k vertices) in Section III when we evaluate the overall
system performance.
2) Synthetic Graph Generation With Distance
Minimization: A synthetic interaction graph G  is generated
via some social graph model, minimizing the distance
between the real and synthetic interaction graphs.
SHIELD uses various social graph models that can generate
a large graph with a small number of parameters.
1) Graph Generation With Statistical Distance: SHIELD
generates the synthetic graph by minimizing the sta-
tistical graph distance from the input graph, e.g., the
dK-2 distance [10] between joint degree distributions.
The dK-2 distance has been used in [14] to cali-
brate synthetic graph generation with respect to input
real graphs. Under the dK-2 distance, the number of
edges (n i j for the real graph and n i j for the syn-
thetic graph) is identified connecting a vertex with
degree i to a vertex with degree  j . The distance is then
defined as dist(G, G  ( p)) = ( i, j (n i j − n i j )2 )1/2 . The
dK-2 distance refers to the difference of joint degree dis-
tributions and provides more fidelity than other measures
listed in Table I, as it captures higher order statistics.
SHIELD supports both parametric and nonparametric
social network models.
a) Parametric Models: The parametric models include
Forest Fire [3], Nearest Neighbor [4], Random
Walk [4], Barabasi–Albert [5], Octopus [6], and
Kronecker [8]. For example, the Forest Fire
model has two parameters (connection probabil-
ity and average number of iterations per vertex)
to select. The parameters are tuned such that
the distance dist(G, G  ( p)) between the real and
synthetic interaction graphs is minimized over
 This article has been accepted for inclusion in a future issue of this journal. Content is final as presented, with the exception of pagination.

4 IEEE TRANSACTIONS ON COMPUTATIONAL SOCIAL SYSTEMS

TABLE I
S TATISTICAL F IDELITY E VALUATION

parameters p, i.e., the synthetic graph G  ( p∗ ) is

generated with the optimal selection of parameters
p∗ = argmin p dist(G, G  ( p)).
b) Nonparametric Models: There are also nonpara-
metric graph models, such as the dK-2 model [10]
that minimizes the dK-2 distance or the dK-2.5
model [11] that minimizes the dK-2 distance while
aiming to preserve the clustering coefficient. These
models can be readily applied without a need to
select parameters.
Ultimately, the best social network model (with the
Fig. 3. Influence spread performance under different social network models.
best parameters, if any) that minimizes the distance is
selected, and the synthetic interaction graph is generated
A synthetic graph with r × n vertices (smaller than the
according to the selected social network model (with the
target output graph), where n is the number of vertices
best set of parameters, if any).
in the input graph, is generated by the first model. Then,
Example: Table I evaluates different graph statistics
the second graph model is used to add more vertices and
(including the dK-2 distance) for various social network
edges to this small graph such that a synthetic graph with
models and compares the performance of these social
a total of n vertices is obtained. This approach can be
network models in terms of graph properties. Nearest
readily extended to cascades of network models.
Neighbor achieves the smallest dK-2 distance (708)
Example: Fig. 4 shows the dK-2 distance when Nearest
among all the parametric models.
Neighbor and Forest Fire are combined with different
2) Graph Generation With Application-Level Distance:
fractions. The distance is reduced to 679 when, first,
SHIELD can also generate the synthetic graph by mini-
the Nearest Neighbor graph is generated with r = 0.8
mizing the distance in terms of application performance
and then the Forest Fire graph is added.
from the input graph. For example, consider influence
spread [28] for an application-based distance. The num- 3) Generation of Patterns in Synthetic Graphs: A synthetic
ber of vertices (n k for the real graph and n k for the graph with patterns is generated in SHIELD. Social networks
synthetic graph) that can be influenced by some given typically provide smoothed graph output, which might not
number of seeds (say k) is computed. The distance is capture various patterns, such as anomalies. After we generate
then defined as dist(G, G  ( p)) = |n k − n k |. a synthetic interaction graph, we further add these patterns in
Example: Consider influence spread which starts with a the synthetic interaction graph. Here, an anomaly corresponds
number of seeds that are selected from the vertices with to a local structure that is significantly different from other
the highest degrees. A vertex is influenced if the ratio parts of a graph, e.g., a high-degree vertex or a high-degree
of the influenced neighbors exceeds a fixed threshold. vertex with low-degree neighbors, such as a hub or a large-
Suppose the influence threshold is 0.5, and 30 seeds size clique. Such a structure can exist in the real interaction
(vertices) are used. The number of influenced vertices is graph but may not be generated by the social graph models.
shown in Fig. 3 for different social network models. The SHIELD identifies patterns in the real graph analyzed and then
performance under the influence and dK-2 distances is generates similar structures in the synthetic interaction graph
shown in Table II. as follows.
3) Graph Generation According to Composite Social Net- 1) All vertices in these patterns and all associated edges
work Models: A synthetic graph can be generated in are removed from the input graph. Denote the graph of
SHIELD by fitting to composite social network models, removed patterns as G̃ and the remaining graph as Ĝ.
which are models that combine multiple social network Each vertex i in G̃ has a vertex property ci , which is
models. A synthetic graph can be generated by two the number of edges connecting vertex i and a vertex
graph models with a parameter r ∈ (0, 1) as follows. in Ĝ.
 This article has been accepted for inclusion in a future issue of this journal. Content is final as presented, with the exception of pagination.
SAGDUYU et al.: SYNTHETIC SOCIAL MEDIA DATA GENERATION
TABLE II
A PPLICATION -L EVEL F IDELITY E VALUATION
Fig. 4. Graph generation under a composite social network model.
2) A synthetic graph is generated for graph Ĝ.
3) G̃ is reattached to this synthetic graph Ĝ by adding ci
edges for each vertex i .
For high-degree anomalies, such as hubs, the following
alternative procedure can also be applied.
1) For each vertex v in the real graph that has a degree
greater than a threshold, one of the higher degree
vertices v  in the synthetic graph is selected.
2) Some topologically proximate, but nonneighboring
vertices are rewired to become neighbors of v  , until
the degree of v  becomes close to the degree of v.
This procedure preserves the number of edges and the con-
nectivity of the graph while generating high-degree vertices.
Example (Consider Two Types of Patterns):
1) Degree Anomaly: Any vertex with degree greater than t.
2) Hub Anomaly: Any vertex with greater than t1 neigh-
bors, each with degree less than or equal to t2 .
Consider three alternative methods. Method 1 fits to the input
graph (irrespective of patterns/anomalies), whereas Method 2
fits to the input graph and then adds patterns. Method 3 (used
by SHIELD) removes patterns from the input graph, fits to the
input graph, and then adds the patterns to the synthetic graph.
Table III shows the dK-2 distance and the number of patterns
in the input graph and the synthetic graph (generated with
Nearest Neighbor) for t = 100, t1 = 20, and t2 = 5. Method 1
cannot match patterns, while Methods 2 and 3 preserve the
patterns. In addition, Method 3 reduces the dK-2 distance.
4) Generation of Multiple Connected Components in Syn-
thetic Graphs: A synthetic graph with multiple connected
components is generated in SHIELD. In general, a giant
connected component can be found in a social graph, and
remaining components are relatively smaller [29]. Some social
graph models generate one connected graph while the input
(real) graph may be disconnected. To preserve the same
5
connectivity as in the real graph, the entire synthetic graph
should not be generated at once. Instead, one or more of the
largest components is fitted, while the remainder of the input
graph is returned unchanged. For example, components may
be fitted one by one in nonincreasing order of component size
(the number of vertices in a component) and terminated once
most (e.g., 90%) of the input graph is fitted.
5) Generation of Attributes in Synthetic Graphs: A syn-
thetic graph with attributes is generated in SHIELD. Social
graph models, such as Nearest Neighbor and Forest Fire,
typically generate a synthetic graph by adding vertices one
by one and adding associated edges when a vertex is added
to the graph. Vertex attributes, such as group membership, are
then assigned based on the group distribution of vertices in the
input data, and the edge attributes are added based on the joint
group distribution in the input data. That is, suppose in the
input data, pi is the number of vertices in group i divided by
the number of all vertices. Then, a new vertex in the synthetic
graph is assigned as a member of group i with probability pi .
Suppose a vertex in group i is added in the synthetic graph
and given that one vertex of an edge is in group i in the
input data, pi j is the number of edges with another vertex in
group j divided by the number of all edges with a vertex in
group i . Then, with probability pi j , the newly added vertex is
connected to a vertex in group j .
Example: Consider the simple method that generates first
the vertices and edges, and then separately generates attributes
according to prior distributions of attributes. Suppose the
vertex attribute is 0 or 1, depending on whether the number
of friends of a user is greater than 200 or not. Define
the ratio of edges ai j for the real graph and ai j for the
synthetic graph, connecting a vertex with attribute i to a
vertex with attribute j . The attribute distance is defined as
dist(G, G  ( p)) = ( i, j (ai j − ai j )2 )1/2 and computed as
0.904769 for a synthetic graph generated with Nearest Neigh-
bor. On the other hand, by jointly generating graph interactions
and attributes (as carried out in SHIELD), this distance is
reduced to 0.175193.
6) Generation of Interaction Types in Synthetic Graphs:
The edges of the synthetic graph are probabilistically
labeled as representing interaction types (e.g., retweets,
replies, and mentions in Twitter) and their numbers. Define
a = tail(e) and b = head(e) as the tail and head of
edge e = (a, b), and deg(x) as vertex x’s degree. Define
Pcat(e),deg(tail(e)),deg(head(e)) (w1 , w2 , . . . , wn ) as the joint dis-
tribution of weights w1 , w2 , . . . , wn (corresponding to the
number of interactions of type 1, 2, . . . , n) conditioned on the
edge being of category cat(e) (e.g., user–user versus user-
hashtag) with degrees deg(tail(e)) and deg(head(e)). This
distribution is computed from the input graph G, and we use
 This article has been accepted for inclusion in a future issue of this journal. Content is final as presented, with the exception of pagination.
6 IEEE TRANSACTIONS ON COMPUTATIONAL SOCIAL SYSTEMS
TABLE III
G ENERATION OF S YNTHETIC PATTERNS
TABLE IV
T EMPORAL G RAPH G ENERATION
it to probabilistically assign edge weights w1 , w2 , . . . , wn to
edges in the synthetic graph G  . If the synthetic graph G 
has an edge with degrees x  and y  for which no distribution
exists in the real graph G, then we instead use an existing
joint distribution Pcat(e),x,y to assign weights, where (x, y) is
selected to be as close as possible to (x  , y  ).
7) Generation of Temporal Dynamics in Synthetic Graphs:
A synthetic graph with temporal dynamics is generated
in SHIELD. Denote G 1 , G 2 , . . . as a sequence of input
graphs. There are overlapping structures among these graphs,
i.e., n i,i+1 is the number of vertices in G i ∩ G i+1 . A sequence
of synthetic graphs is generated by Method 4 in SHIELD as
follows.
1) Generate the first synthetic graph S1 for G 1 .
2) Given Si and n i,i+1 , randomly keep n i,i+1 vertices in Si
and edges among them.
3) Apply a graph model to add more vertices and edges to
this small graph such that a synthetic graph Si+1 with
the same number of vertices as G i+1 is obtained.
Method 4 of SHIELD preserves temporal correlation in terms
of Jaccard index of edge sets across various time instances.
The Jaccard index across intervals i and i + 1 is |E i ∩ E i+1 | /
|E i ∪ E i+1 |, where |E| denotes the size of set E.
Example: Table IV shows the dK-2 distance and the Jaccard
index (averaged over time instances) of synthetic graphs gen-
erated by Method 4 and the alternative method (Method 5) that
fits each graph G 1 , G 2 , . . . independently. Time is partitioned
into three equal intervals, and Nearest Neighbor is used to
generate synthetic graphs for each partition. SHIELD reduces
the dK-2 distance and preserves the temporal correlation in
terms of average Jaccard index.
8) Complexity Reduction in Synthetic Graph Generation:
The time complexity of graph fitting is mainly caused by the
need to generate the same size graph as the original graph
for each set of parameters and then measure the distance
between the two graphs. To reduce the time complexity of
graph fitting, a smaller synthetic graph can be generated, and
the distance between that graph and the input graph can be
measured by an extended dK-2 distance defined as follows.
Suppose two graphs G and Ĝ are compared, where the number
Fig. 5. Run time as a function of graph compression size m.
of vertices in G and Ĝ is n and m, respectively, and n ≥ m.
For i, j ∈ [1, m], denote ei j as the number of edges between
a vertex with degree i and a vertex with degree j in G and êi j
as the number of edges between a vertex withdegree i and a
m m
vertex with degree j in Ĝ. mDefine p ij = (e ij / i=1 j =i ei j )
and p̂i j = (êi j / m ê
j =i i j ). The new dK-2 distance
 m i=1
( m i=1 j =i ( pi j − p̂i j ) )
2 1/2 is scaled up by multiplying with
the number of edges in graph G.
Example: We apply this new distance definition to synthetic
graphs generated from the input graph with n = 10 000 ver-
tices and study how the run time and dK-2 distance change
with m. The results are reported in Figs. 5 and 6 that show the
run time and the dK-2 distance, respectively, as a function of
graph compression size m. Fig. 5 shows that by using a small
m value, the run time for both models can be significantly
decreased. On the other hand, the dK-2 distance achieved by
the fitted graph does not increase much when we decrease
m from 10 000 to 5000, as shown in Fig. 6. In this case,
the distance increases only by 16% for Nearest Neighbor
and by 26% for Forest Fire, while the run time is reduced
by 59% for Nearest Neighbor and by 47% for Forest Fire.
We also note that during the graph fitting process, it is not
the absolute dK-2 distance that matters, but how this distance
changes depending on which parameter values are used (with
the distance-minimizing parameters ultimately selected).
9) Synthetic Graph Generation From Sampled Data: It is
possible that the input graph is incomplete, i.e., there may be
missing vertices or edges. One potential reason is that data
collection through public APIs has strict rate limitations [30].
For example, keyword search though Twitter public API allows
180 requests per 15 min for a user or 400 requests per minute
for an app. Another option is to use Twitter Gardenhose, which
provides only 10% of all Twitter data, compared with Twitter
Firehose, which provides 100% of all Twitter data. This raises
 This article has been accepted for inclusion in a future issue of this journal. Content is final as presented, with the exception of pagination.
SAGDUYU et al.: SYNTHETIC SOCIAL MEDIA DATA GENERATION
Fig. 6. dK-2 distance as a function of graph compression size m.
Fig. 7. Step 1b: generate synthetic text.
the question of whether it is sufficient to use a limited resource
as input and how representative these input data are when we
generate synthetic graphs out of it. To answer this question,
we construct the following experiment with 10k tweets and
compare two scenarios in the following.
1) Take 100% of available tweets as input, synthetically
generate a graph, and compare it with the input graph.
2) Take 10% of available tweets as input, synthetically
generate a graph corresponding to the 100% data size,
and compare it with the input graph.
Results show that the change in the dK-2 distance between
the two scenarios is less than 3% (we observed small changes
in other data sets), implying that it is feasible to generate a
synthetic graph from the sampled data with high fidelity.
B. Synthetic Text Generation
Methods for generating synthetic text for social media are
shown in Fig. 7.
1) Topic Modeling and Markov Chain Construction for Text
Distributions: The textual content of social media is modeled
as a set of posts P, where each post Pi = (w1 , w2 , . . . , wm ) ∈
P is a sequence of terms wk . Here, wk is an ordinary
word or a special social media entity, such as a username,
hashtag, or hyperlink. Before the textual content of synthetic
social media posts P  is generated, real social media posts P
are classified according to the dominant topic T (Pi ) of the
post Pi . Topics are identified by applying the latent Dirich-
let allocation (LDA) algorithm [31] to the text of the real posts
after removing stopwords (stopwords are common words such
as “the” in English that are considered to be not necessarily
significant for natural language processing purposes). Other
topic modeling techniques, such as latent semantic analy-
sis [32] and probabilistic latent semantic analysis [33], may be
7
applied here as an alternative. LDA models each post Pi ∈ P
as a mixture of some fixed set of topics {T1 , T2 , . . . Tn }, where
each topic is a vector in the space of the terms wk . The
dominant topic T (Pi ) is Tdi , where di = argmax j pi j and pi j
is the probability of topic T j for post Pi . All posts where some
topic Td dominates (in the mixture) are placed into the same
class d. Synthetic textual content is then generated separately
for each class/topic d.
To generate content for posts belonging to some class d,
part-of-speech tagging [34] is first applied to find the gram-
matical category g(wk ) for each term wk in the post. These
categories include both standard parts-of-speech, such as noun,
verb, and adjective, as well as social media entities, such as
usernames, hashtags, and hyperlinks.
From this tagged set of posts, a Markov chain Md (more
specifically, an n-gram model) is constructed. For example,
the posts “I like dogs” and “I like cats” will result in the
following second-order Markov chain (3-gram).
1) α ⇒ [I/pronoun, I/pronoun].
2) (α, I/pronoun) ⇒ [like/verb, like/verb].
3) (I/pronoun, like/verb) ⇒ [dogs/noun, cats/noun].
4) (I/pronoun, like/verb) ⇒ [dogs/noun, cats/noun].
5) (like/verb, dogs/noun) ⇒ ω.
6) (like/verb, cats/noun) ⇒ ω.
Here, α and ω indicate the beginning and the end of a post,
respectively.
2) Generation of Synthetic Text From Markov Chains
According to Underlying Topics: From each Markov
chain Md , synthetic posts Pi = (w1 , w2 , . . . , wm ) are sto-


chastically generated, where each wk  in Pi is a term from
the original set of posts P. Notably, the word frequencies
and transition probabilities in the synthetic posts reflect the
word frequencies and transition probabilities in the real posts.
Furthermore, because generation is performed separately for
each LDA topic Td , the resulting text is less likely to contain
incoherent sentences that begin with one topic and end with
another. Finally, because part-of-speech tagging is performed,
the generation process is more likely to preserve the gram-
matical rules inherent in the real posts. To further improve
the quality of synthetic tweet text (grammar and originality),
the following postprocessing steps are applied.
1) Any synthetic post Pi whose grammar significantly
violates the grammatical rules found in the original set
of posts P is automatically identified and removed.
2) The synthetic posts and real posts are ensured to contain
distinct text, by removing any synthetic post whose
textual content is too similar to the textual content of
some real post. A synthetic post Pi can be identified
to be too similar to a real post Pi if the sequence of
terms in Pi (excluding social media entities, such as
usernames, hashtags, and hyperlinks) is contained as a
subsequence within the terms of Pi .
An alternative method to generate synthetic data is to apply
the GAN, as discussed in Section I.
Synthetic text generation has high complexity and is the
bottleneck of synthetic social media data generation. For
instance, starting from 100k real tweets, it takes approximately
 This article has been accepted for inclusion in a future issue of this journal. Content is final as presented, with the exception of pagination.
8 IEEE TRANSACTIONS ON COMPUTATIONAL SOCIAL SYSTEMS
3 to 4 times longer to build the Markov chains and generate
synthetic text from them compared with generating a synthetic
interaction graph (with hubs). We will discuss how to reduce
complexity in Section II-C.
At this point, the usernames, hashtags, and hyperlinks in the
synthetic posts P  are the same ones as in the real posts P.
Hyperlinks can be randomly replaced with other hyperlinks
that exist in some other set of social media posts Q different
from P but thematically related. For example, for Twitter,
hyperlinks are generated by collecting a separate set of real
tweets Q through Twitter API [35], using keywords similar
to those that were used to obtain the input data set P. The
generation of synthetic users and hashtags is described in the
next section.
C. Combination of Synthetic Graph and Text Structures
Methods for generating synthetic social media entities and
combining the synthetic graph with the synthetic text are
shown in Fig. 8.
1) Generation of Synthetic Social Media Entities: The gen-
erated synthetic social media posts P  are connected with the
synthetic graph G  by replacing social media entities (such
as usernames and hashtags) in posts Pi ∈ P  with the labels
from the vertices V  and edges E  of G  . It is therefore first
necessary to generate these labels. If v  ∈ G  is a user vertex,
then it is assigned a random string of characters u(v  ) as the
synthetic username. If v is a hashtag vertex, then it is assigned
a synthetic hashtag as follows. For each hashtag vertex v in the
real graph, a procedure attempts to find a hashtag vertex v  in
the synthetic graph, such that v and v  have a similar degree.
Then, the vertex v  is labeled with the most common dominant
topic T (v  ) = Td among all real posts P that mention v (recall
that each post has been labeled with its dominant topic Td via
LDA). The hashtag name h(v  ) for v  is generated by randomly
choosing and combining one or two LDA terms from this
topic, where the probability of choosing a particular term wk
is proportional to qd,k , which we define as the probability of
term wk for topic T (v  ) = Td (as computed by LDA).
2) Generation of Synthetic Social Media Posts: A fully
synthetic post Pi is generated as follows. For each iteration,
some synthetic edge e = (u  , v  ) ∈ E  is selected. One vertex
of e (without loss of generality, call it u  ) is the user who
created the post. Now, let S  (u  ) be a subset of vertices that
are adjacent to u  , randomly drawn from the distribution of
social media entities in the real set of posts P (for example,
if posts in P contain two user mentions on the average, then
the average number of user vertices in S  (u  ) will be 2). A post
in Pi ∈ P  is selected whose entities as closely as possible
match the entities in S  (u  ). For example, if S  (u  ) contains
one user vertex s1 and two hashtag vertices s2 and s3 , then we
select Pi such that one of its terms (call it w1 ) is a username
and two of its terms (w2 and w3 ) are hashtag names. Then,
terms in Pi are replaced with vertex labels as appropriate,
i.e., w1 is replaced with u(s1 ), and w2 and w3 are replaced
with h(s2 ) and h(s3 ), respectively.
The real usernames and hashtags in post Pi are thus
replaced with synthetic usernames and hashtags, yielding a
Fig. 8. Step 2: combine synthetic text and graph structures (through synthetic
social media entities).
fully synthetic post Pi . The above-mentioned process is then
repeated for different edges of G  to generate a final set
of fully synthetic posts P  . Note that after each iteration,
the selected post Pi is removed from P  , to avoid unnecessary
duplication in the generated posts. In order to be able to select
an appropriate post Pi (with username counts and hashtag
counts that match the user vertex counts and hashtag vertex
counts in S  (u  ), respectively) in every iteration, P  is typically
overgenerated.
Notably, in the above-mentioned procedure, for cases where
S  (u  ) contains hashtag vertices, it is attempted to select
Pi such that its dominant topic T (Pi ) matches the most
common topic found among the hashtag vertices. In the
above-mentioned example, Pi is picked, such that T (Pi ) =
T (s2 ) or T (Pi ) = T (s3 ) (if the hashtag vertices have different
topics assigned to them, then the tie is broken arbitrarily).
Because topics T (v  ) were assigned to vertices based on their
degree (as described earlier), the synthetic hashtag assignment
process thus bridges content and topology. Furthermore, if the
edge (u  , s1 ) is a reply (i.e., u  replies to s1 ), then we also
generate the original post of a reply, as described earlier, but
using a subset of vertices S  (s1 ) and selecting a post Pi such
that T (Pi ) = T (Pi ) (i.e., the reply and the original post
should have the same LDA topic). Finally, if (u  , s1 ) is a
retweet, then we simply replicate the generated retweet Pi
to generate the original tweet Pi .
3) Generation of Synthetic Information Cascades: Synthetic
forms of information cascades are added. As an example,
information cascades can be defined for Twitter as follows. If a
user u 1 replies to another user u 2 and mentions a hashtag #h,
then user u 1 is influenced by user u 2 on the topic of #h.
For a given topic, we may find some long influence paths
in a social graph. However, in a synthetic graph, there may
be only a few influence paths. Long influence paths can be
identified in the input graph, and then, the number of long
influence paths in the synthetic graph can be checked to see
how many influence paths should be added. An influence path
can be added by adding influenced users one by one. For
synthetic Twitter data, suppose that the goal is to add a user u 1
influenced by user u 2 on the topic of #h. We then need to add
“@u 2 ” at the beginning of one of tweets from u 1 and add
“#h” in this tweet.
4) Generation of Other Synthetic Social Media Attributes:
Other synthetic attributes, such as locations and geo-
coordinates, can be generated for social media posts, based on
 This article has been accepted for inclusion in a future issue of this journal. Content is final as presented, with the exception of pagination.
SAGDUYU et al.: SYNTHETIC SOCIAL MEDIA DATA GENERATION
the correlation between locations of real posts (where avail-
able) and LDA topics of corresponding posts. For example,
consider the generation of synthetic locations. The coupling
between content and locations is preserved since the subjects
of the tweets are correlated in general with locations. Given
a number of real tweets with locations and synthetic tweets
without locations, the synthetic location generator executes
the following steps to generate the locations for the synthetic
tweets.
1) Clean all synthetic and real tweets by removing hash-
tags, mentions, and hyperlinks (URLs).
2) Apply LDA to all tweets and compute the topic per tweet
distribution (namely the LDA scores).
3) For each tweet, find the tweet’s dominant topic
(as defined in Section II-B1).
4) Collect the location information (latitude and longitude)
for real tweets.
5) Divide the real tweets’ locations into small geographic
bins and compute the probability of the real tweet
location falling in one of those bins, given a dominant
topic.
6) Generate synthetic tweets’ locations in small geographic
bins for a given dominant topic, from the probability
distribution defined in the previous step.
Real tweets with no location are assigned location (0, 0). This
particular (0, 0) location is assigned to synthetic tweets with
the same ratio as for input tweets.
5) Scalable Generation of Synthetic Social Media Data:
To improve scalability, input tweets are divided into multiple
sets T1 , T2 , . . . by the posting time. The above-mentioned
generation procedure is applied to obtain a set of synthetic
tweets Si for each set Ti . Then, user names and hashtags are
updated in Si+1 based on Si and the correlation between Ti+1
and Ti (measured by the Jaccard index |Ti ∩ Ti+1 |/|Ti ∪ Ti+1 |)
such that the same time correlation can be observed among
synthetic and original sets. Synthetic data generation can
be parallelized by dividing the input social media data into
multiple sets by the posting time, generating synthetic social
media for each set, and combining data sets through common
usernames and social media text entities.
III. C OMPUTATIONAL E VALUATION OF S YNTHETIC
S OCIAL M EDIA DATA
In this section, we describe a computational experiment
where we collected a large set of Twitter data, generated
a large set of synthetic tweets, and compared the two data
sets in terms of graph and text properties. In the following,
we describe the details of the data collection and generation
procedure and present the comparison measures and results.
A. Experimental Procedures
To generate synthetic tweets, real tweets were first collected
on the following keywords: “mlb,” “baseball,” “rockies,” and
“mets” using Twitter API [35]. The collection was done at
the end of July 2016. The first 120 000 collected tweets were
used. These tweets were split into 12 subsets of 10 000 tweets;
synthetic tweet generation was performed separately for each
subset; the resultant synthetic tweets were then combined into
9
a full set of synthetic tweets, from which a full synthetic
interaction graph was extracted. We used the following Forest
Fire, Nearest Neighbor, and dk-2.5 algorithms for fitting the
graph.
1) Forest Fire [3] reproduces dynamic ranges in graph
density and represents increasing density and decreasing
diameter over time. The graph grows with each new
vertex connecting to (“burning”) existing vertices. After
a new vertex connects to an existing vertex, it randomly
connects to some of the neighbors of the vertex. There
are two parameters p1 and p2 ; Once a vertex is burned,
its neighbor will be burned with probability p1 , and this
process is limited to its p2 -hop neighbors.
2) Nearest Neighbor [4] emulates similarity and
transitivity-based growth. The model is based on
the observation that two people sharing a common
friend are more likely to become friends. Each new
vertex added to the graph is connected to a random
existing vertex. Additionally, random pairs of 2-hop
neighbors around the new vertex are connected. There
are two parameters p1 and p2 . The probability p1
determines at each step if a new vertex is added or if
a pair of 2-hop neighbors is connected. To reduce the
power law exponent, each time a new vertex is added,
we also connect p2 pairs of existing vertices randomly
chosen from the graph.
3) dK-2.5 [11] is an extension to the dK-2 method [10],
which matches joint degree distributions px,y (the proba-
bility that a vertex with degree x is connected to a vertex
with degree y) of the synthetic graph to those of the real
graph. dK-2 achieves only small clustering coefficient.
dK-2.5 improves the clustering coefficient by rewiring
(adding and removing) edges while keeping joint degree
distributions. There is no parameter associated with
dK2.5.
Note that while the parameter p2 in Forest Fire and Nearest
Neighbor algorithms is discrete, dK-2 distance minimization
can be further refined by instead using a floating number x.y,
where x is the integer part of a decimal number p2 and y is
the fractional part. Then, with probability y, p2 is chosen as
x + 1 and with probability 1 − y, p2 is chosen as x.
For the parametric models, the objective for graph fitting
is to minimize the dK-2 distance. Each connected component
was fitted separately, with the smallest components not fit-
ted. Hubs were added through the vertex rewiring procedure
described in Section II-A3, which was applied to every vertex
with degree 50 or greater. For each subset of real tweets, text
was generated along four LDA topics, with 400 000 posts
produced (overgenerated before tying them to the graph).
These posts were combined with the synthetic graph to gen-
erate 30 000 synthetic tweets. Prior to combining the subsets
of synthetic tweets, we automatically filtered out those that
contained profane or potentially offensive language, or those
that were too similar to some real tweet.
B. Synthetic Graph Properties
For each subset i of 10 000 real tweets, we compared the
real interaction graph G i with the synthetic fitted graph G i ; the
 This article has been accepted for inclusion in a future issue of this journal. Content is final as presented, with the exception of pagination.
10
TABLE V
S TATISTICAL G RAPH C OMPARISON M EASURES
TABLE VI
O PTIMAL PARAMETER VALUES
same was done for the real interaction graph G corresponding
to the entire set of 120 000 real tweets and the synthetic
interaction graph G  that was extracted from the combined set
of synthetic tweets. In comparing each pair of graphs (G i , G i ),
our goal is twofold: 1) we wish to show that statistically,
G i and G i have similar properties and 2) in terms of concrete
topological features, we wish to show that the graphs are
different (i.e., are significantly nonisomorphic).
For the statistical comparison, we use the aforementioned
dK-2 distance dist(G i , G i ) as the primary measure. We also
count the number of vertices N(G i ) and N(G i ) in the real
and synthetic graphs, respectively, as well as the number of
edges E(G i ) and E(G i ) in these graphs. Table V provides
these measures for the Forest Fire, Nearest Neighbor, and
dK-2.5 fitting algorithms; the last row provides them for
the full 120 000 tweet real and synthetic graphs, while the
previous rows correspond to each subset of 10 000 tweets.
Table VI gives the best graph fitting parameters found by the
algorithm for the largest component in the graph (note that the
dK-2.5 algorithm has no parameters).
By design, for Forest Fire and Nearest Neighbor, each
10 000 tweet synthetic graph has the same number of vertices
as the corresponding real graph; while the number of edges
is not the same for G i and G i , the number is reasonably
close, implying a similar average degree; dK-2 distances, while
IEEE TRANSACTIONS ON COMPUTATIONAL SOCIAL SYSTEMS
nonzero, are relatively small (compared with the total number
of edges), suggesting statistical similarity between real and
synthetic graphs. On the other hand, for the dK-2.5 algorithm,
the number of edges does not change during graph generation,
while the number of vertices does. By design, dK-2 distances
are 0 for all 10k tweet graph pairs, though the distance is
positive for the two full graphs, due to the effects of the tweet
combination procedure.
It is important to note that the statistical differences do
not necessarily imply that the graphs have very different
concrete topologies; for example, consider a complete graph
of k vertices (with an edge between each pair of vertices), and
another complete graph of k − 1 vertices. The dK-2 distance
between the two graphs will be very high; however, the graphs
will have k −1 vertices and (k −1)(k −2)/2 edges in common;
in fact, the latter graph will be isomorphic to a subgraph of the
former. To determine the “amount” of isomorphism between
two graphs G i and G i , we must find a bijection m from the
vertices of G i to the vertices of G i , such that we maximize
the number of edges (u, v) in G i that have a corresponding
edge (m(u), m(v)) in G i . Unfortunately, finding this bijection
is a difficult problem; although some approximations have
been developed, at present, no known algorithm exists that
will solve it optimally in a tractable amount of time for large
graphs. We thus measure the concrete difference between the
graphs empirically over a number of trials, where each trial
potentially uses a different bijection, which it creates with a
different set of random choices, as follows.
1) For both G i and G i , order the vertices from “highest”
to “lowest,” according to the following criteria. First,
define deg0 (v) = deg(v) to be the degree of vertex v,
and degk (v) to be the degree of v’s kth neighbor, where
the neighbors are ordered by degree, from highest to
lowest (with ties broken randomly). In other words,
when k = 1, degk (v) is the degree of v’s highest degree
neighbor, and when k = deg(v), degk (v) is the degree of
v’s lowest degree neighbor. Now, the order of two ver-
tices u and v is determined by the following algorithm.
a) For k from 0 to deg(u), do the following.
i) If degk (u) > degk (v), exit and return u > v.
 This article has been accepted for inclusion in a future issue of this journal. Content is final as presented, with the exception of pagination.
SAGDUYU et al.: SYNTHETIC SOCIAL MEDIA DATA GENERATION
TABLE VII
G RAPH R EALIZATION C OMPARISON M EASURES
ii) If degk (u) < degk (v), exit and return u < v.
b) If the loop executed all deg(u) + 1 iterations
without exiting, return u = v and the order of
the two vertices is determined randomly during the
sorting process.
2) Given the ordered (according to the above-mentioned
criteria) lists of vertices V (G i ) and V (G i ) from
G i and G i , respectively, randomly remove N(G i ) −
N(G i ) vertices from V (G i ) if N(G i ) > N(G i ) (i.e.,
if |V (G i )| > |V (G i )|), to make the number of vertices
equal; similarly, if N(G i ) > N(G i ), then randomly
remove N(G i ) − N(G i ) vertices from V (G i ).
3) Define a bijection m using the following algorithm.
a) For k from 1 to |V (G i )|, do the following.
i) Define m(V (G i )[k]) = V (G i )[k].
4) Count the number C of edges (u, v) in G i that have a
corresponding edge (m(u), m(v)) in G i , and compute
the edge match score M(G i , G i ) = 2C/(E(G i ) +
E(G i )).
We compute the expected edge match score E[M(G i , G i )]
between G i and G i over 100 trials. However, because the
bijections (used in each trial) are likely not optimal, a low
score does not necessarily imply a low number of matching
edges. Thus, for comparison, we also computed E[M(G i , G i )]
and E[M(G i , G i )]. All these measures are shown in Table VII
for Forest Fire, Nearest Neighbor, and dK-2.5. The mea-
sures E[M(G i , G i )] and E[M(G i , G i )], though not equal
to 1 (which would be the case with an optimal bijection,
since a graph is isomorphic to itself), are much higher than
E[M(G i , G i )]; in all cases, the difference is statistically
significant, according to a two-sample two-tail t-test assuming
unequal variances ( p 0.01). This empirically suggests that
G i and G i are significantly nonisomorphic. Note that 100 trials
were used, which should be sufficient if the random function
is, indeed, random. However, the random function provided in
a programming language is not truly random, and its output
is determined by seed numbers. We used the pseudorandom
generator of Python that uses the Mersenne Twister as the core
generator and produces 53-bit precision floats with a period
11
of 219937 − 1 [36]. While a long period is not a guarantee of
quality in a random number generator, it is long enough to
provide diverse random numbers to SHIELD.
C. Synthetic Text Properties
In addition to comparing the graphs, we also compared the
textual content of each subset of 10 000 real tweet posts P
with textual content from the corresponding set of synthetic
tweet posts P  and performed a similar comparison for the full
set of 120 000 real tweets and the combined set of synthetic
tweets. Again, our goal is to demonstrate both statistical
similarity and a difference in concrete features. For the former,
we use the well-known cosine distance similarity, which we
compute as follows.
1) Remove all user mentions, hashtags, and hyperlinks
from each tweet in P and P  .
2) For P and P  , define vectors T (P) and T (P  ), respec-
tively, of term frequency counts (where a term is a
word or any other string that is separated by whitespace
but does not contain whitespace within it). It is ensured
that T (P) and T (P  ) have the same length (correspond-
ing to the size of the vocabulary common to both sets
of tweets), and that for any k, T (P)[k] and T (P  )[k]
correspond to the frequency count for the same term.
3) Compute the cosine similarity as dcos(P, P  ) = T (P)×
T (P  )/(|T (P)||T (P  )|), where × is the vector product.
As Table VIII shows, the cosine similarity values are
relatively large for text (where the underlying graph is syn-
thetically generated with Forest Fire, Nearest Neighbor, and
dK-2.5). This is not surprising, given that a Markov chain
is used to generate the synthetic text; since Markov chains
are designed to replicate the statistical distribution of n-grams
(sequences of n terms), they also replicate the statistical
distribution of individual terms. At the same time, as discussed
in Section II-B, we wish to avoid situations where the sequence
of terms (which comprises a synthetic tweet) is identical to
the sequence of terms comprising some real tweet or appears
as a subsequence within it (e.g., if there is a real tweet that
says “It is cold today,” we wish to avoid synthetic tweets,
 This article has been accepted for inclusion in a future issue of this journal. Content is final as presented, with the exception of pagination.
12
TABLE VIII
T EXT C OMPARISON M EASURES
such as “cold today” or “It is cold”). As Table VIII shows,
the proportion R(P, P  ) of such tweets is quite small, and as
we stated earlier, they are filtered out and do not appear in the
final set of tweets.
Next, we evaluate the semantic features of synthetic data in
terms of sentiments. For that purpose, we classify tweets into
two different classes (sentiments) “positive” and “negative.”
We use word distributions (i.e., the number of times a given
word appears in each tweet) as features. Training data are col-
lected through associations with emoticons in tweets according
to the procedure outlined in [37]. We use the Naive Bayes
Classifier for illustration purposes (more advanced classifiers
can be used for sentiment analysis and the class “neutral”
could be added). We compare sentiments of real tweets with
those of synthetic tweets generated with the Nearest Neighbor
method; 63% of real tweets are positive, whereas 55% of
synthetic tweets are positive. This implies that the synthetic
data generation maintains the general trend of sentiments.
Importantly, the ultimate test of synthetic text quality
is whether it appears indistinguishable (from real text) to
humans. In Section IV, we demonstrate that this is, indeed,
often the case.
IV. DATA E VALUATION E XERCISE W ITH H UMAN
PARTICIPANTS AS A “A M INI T URING T EST ”
The underlying goal of the experiment is to evaluate the
content generation capabilities of the SHIELD system, which
generates the synthetic Twitter data. The capabilities are
effective if it is difficult for a human to distinguish between
tweets that were generated by SHIELD and tweets that were
posted by real users. Thus, our experiment can be viewed
as a simplified, noninteractive version of the Turing test,
where participants are asked to determine whether a given
tweet was generated by a human or a machine. We prepared
the plan/protocol for a data evaluation exercise with human
participants, obtained the necessary approvals (we received
an HRPO Concurrence Memorandum with the exempt deter-
mination for the Protocol, “SHIELD Data Collection Plan”),
IEEE TRANSACTIONS ON COMPUTATIONAL SOCIAL SYSTEMS
recruited participants, collected consent forms, and executed
the experiment.
A. Methods
1) Data: For our experiment, we drew from two data sets:
one real and one synthetic. The data sets are described in
Section III-A; the synthetic data set was generated with the
Forest Fire algorithm. For the purposes of a data evaluation
exercise with human participants, we anonymized the tweets
by replacing user, hashtag, and URL mentions with the generic
strings @user, #hashtag, and http://url, respectively. We addi-
tionally converted all alphabetical characters to lowercase and
removed all retweets. After the two data sets were processed
in this manner, 50 tweets were randomly selected from the
real data set, and 50 were selected from the synthetic data set,
ensuring that no chosen tweet contains offensive content or too
little content. The two subsets were combined and shuffled,
and each participant was then presented with the same list
of 100 tweets.
2) Participants: 25 Intelligent Automation, Inc. employees
were selected as the participants for the experiment. They have
college and/or graduate degrees, are either native or fluent
English speakers, and include frequent, occasional, and non-
Twitter users.
3) Experiment: Each participant was sent the list of tweets
(as generated earlier) via e-mail and asked to determine
whether each tweet was generated by a human (real) or a
machine (synthetic), according to his/her best judgment. The
participants were not told the proportion of real versus syn-
thetic tweets in the list. No strict deadline or time limit
was imposed, although 10–15 min was typically given as a
guideline for the amount of time needed to process the list
of tweets. Results were collected from all participants, and
for each participant, we computed the false negative rate (the
number of synthetic tweets that were identified as real by
the participant) and the false positive rate (the number of
real tweets that the participant identified as synthetic). After
rating the tweets, each participant is asked to complete a brief
follow-up survey on the features (listed in the first column
 This article has been accepted for inclusion in a future issue of this journal. Content is final as presented, with the exception of pagination.
SAGDUYU et al.: SYNTHETIC SOCIAL MEDIA DATA GENERATION
TABLE IX
C LASSIFICATION E RROR PER PARTICIPANT
Fig. 9. Distribution of classification error by participant.
of Tables X and XI) that he/she used to determine whether a
tweet is real or synthetic and to state his/her level of experience
with Twitter. Again, each participant was able to complete
the survey at his/her convenience, with no hard time limit
imposed.
B. Results
1) Individual Classification Performance: In Table IX,
we present the overall mean false negative and false positive
error rates, along with standard deviations, while in Fig. 9,
we show how these error rates are distributed across all
participants. Both Fig. 9 and Table IX suggest that it is much
more common for a participant to misclassify a synthetic
tweet as real, rather than the other way around. A two-sample,
two-tail pairwise t-test shows that the difference is significant
( p = 1.5 × 10−2 ; this is the probability that the difference
is due to random chance). This validates our choice of real
tweets, as it indicates that most errors arise not because the
real tweets are of a low quality (and appear to be synthetic),
but rather, due to SHIELD’s content generation capabilities
(such that generated tweets appear to be real). We see that
on average, SHIELD is able to deceive participants 52.7%
of the time. Using a one-sample, one-tail t-test, we can claim
that this misclassification rate is significantly greater than 44%
( p = 1.5 × 10−2 ), i.e., it is likely that more than 44% of
SHIELD’s tweets will be classified as real by the average
participant.
2) Group Classification Performance: Given the above-
obtained results, it is natural to ask whether different par-
ticipants tend to misclassify different subsets of synthetic
tweets, or whether certain tweets are much more susceptible to
misclassification than others. In the latter case, such synthetic
tweets would be difficult to detect even through “wisdom of
the crowd” approaches, where multiple participants vote on
whether a given tweet is real or synthetic. This approach was
13
Fig. 10. Distribution of classification error by tweet.
applied for the detection of synthetic social media profiles
(Facebook and Renren) in [27].
To determine whether highly misclassified tweets exist,
we “invert” our previous analysis: instead of gauging how well
individual participants perform on the provided list of tweets,
and we measure how well each tweet is classified by the entire
set participants. In Fig. 10, we show how misclassification
rates are distributed across all tweets. As expected, synthetic
tweets tend to have much higher classification errors. Most
notably, 28 of the 50 synthetic tweets (i.e., 56%) present a
problem for the majority. A one-sample, one-tail significance
test for a population proportion shows that in general, if the
25 participants were to collectively classify other tweets,
the misclassification rate (where 13 or more participants make
an error) will be greater than 41% ( p = 1.6 × 10−2 ).
Synthetic tweets that are classified as synthetic and real by
the largest number of participants are as follows.
1) Synthetic tweet “ihip mlb baseball mouse ny new
york yankees players names 100% cotton cool a
http://url1 via @user1” is classified as synthetic by 88%
of participants.
2) Tweet “@user2 make it a rocket launch instead of a
game homie i keep it real.httr” is synthetic. However,
88% of participants classified it as real.
We capture temporal dynamics first as cascades in graph
models and then through the assignment of the same topic
to replies that capture temporal dynamics in text sequences.
For a retweet, we ensure that its content is the same as that
of the original tweet. Therefore, temporal text dynamics are
captured through topics in both replies and retweets. Some
examples of synthetic retweets and replies are given in the
following.
1) “RT @user4: @user2 make it a rocket launch instead
of a game homie i keep it real.httr” is a retweet of the
above-mentioned second tweet.
2) “RE @user5: angels 2 red sox vs kansas city royals
baseball card 0q3 http://url1” is a reply to “30 minutes
smoke em if you want to go to another baseball game
with aluminum bats.”
3) Classification Criteria: As mentioned earlier, we con-
ducted a survey to determine what tweet features are typi-
cally used by participants to determine whether a tweet is
 This article has been accepted for inclusion in a future issue of this journal. Content is final as presented, with the exception of pagination.
14
TABLE X
F REQUENCY AND U TILITY OF T WEET F EATURES A SSOCIATED
W ITH R EAL T WEETS
real or synthetic. We analyzed the survey results to deter-
mine not only which features tend to be used more or less
frequently but also, to determine whether a given feature is
beneficial or detrimental to accurate classification. Specifically,
in Table X (see Table XI), the second column (Number of
Participants) reports the number of participants who said that
the feature is indicative of real (synthetic) tweets, while last
two columns report the false negative rate (FN) and the false
positive rate (FP), averaged only across those individuals who
selected the feature. For example, 20 participants stated that
they considered tweet readability when making their decisions;
on average, these 20 participants misclassified 52.9% of the
synthetic tweets as real and 36.7% real tweets as synthetic; at
the same time, in Table XI, three participants stated that tweet
readability is, in fact, indicative of synthetic tweets. In some
cases, these averages (outlined in bold in Tables X and XI)
are lower than the averages over all participants (52.7% and
38.4%, as reported in Table IX).
For the purposes of detecting synthetic tweets (i.e., reducing
the false negative rate), several features appear to be useful
(false negative error rate under 40%): similarity of the tweet
to other tweets in the data set, the presence of common
idioms/phrases, short tweet length, and descriptions of past
experiences (e.g., of what the user did). However, because few
participants used these features in a beneficial way (e.g., only
three considered similarity of the tweet to other tweets in the
data set to be indicative of real tweets), and because the benefit
IEEE TRANSACTIONS ON COMPUTATIONAL SOCIAL SYSTEMS
TABLE XI
F REQUENCY AND U TILITY OF T WEET F EATURES A SSOCIATED
W ITH S YNTHETIC T WEETS
was limited (false negative error rates were still over 30% in
all cases), we cannot claim that the difference in performance
is statistically significant. Thus, we cannot conclusively show
that any feature stands out very strongly as a clear indicator of
whether a tweet is real or synthetic. Even if we use all these
features to build a classifier, the overall predictive power is
uncertain and may be small.
As a complement to the above-mentioned analysis, we con-
sidered a feature that was not present in the survey, namely,
the sentiment of a tweet (positive versus negative). We man-
ually classified the tweets according to sentiment, with
33 tweets labeled as negative and the remaining 67 labeled as
positive. We further determined that 16 of the negative tweets
were real, while 17 were synthetic; in other words, sentiment
distributions were very similar among real and synthetic
tweets. Furthermore, we sorted the tweets by classification
error (see Fig. 10) and split them into two halves. We then
found that 17 of the negative tweets were in the lower half,
while 16 were in the upper half; thus, positive and negative
tweets presented approximately the same level of difficulty for
the human participants. From these results, we can conclude
that tweet sentiment is not a reliable feature for distinguishing
between real and synthetic tweets.
Finally, we performed an analysis to determine whether
experience with Twitter had an effect on the participants’
performance; results are shown in Table XII. Although the
participants who use Twitter perform somewhat better at
 This article has been accepted for inclusion in a future issue of this journal. Content is final as presented, with the exception of pagination.
SAGDUYU et al.: SYNTHETIC SOCIAL MEDIA DATA GENERATION
TABLE XII
T WITTER E XPERIENCE AND I TS E FFECT ON C LASSIFICATION
identifying synthetic tweets than the participants who do not
use it at all, the difference is not highly significant ( p = 0.10
for detecting synthetic tweets and p = 0.19 for detecting real
tweets, under a two-sample, two-tail t-test assuming unequal
variances). Even nonusers of Twitter are often exposed to
tweets, e.g., in news articles, which may explain why Twitter
users did not have a significant advantage when performing
the experiment.
V. C ONCLUSION
This paper presented the SHIELD system, which generates
the synthetic social media data, including rich text and graph
structures and dynamics. SHIELD jointly generates time-
varying, directed and weighted interaction graph structures and
topic-driven text features similar to the input social media data.
Various properties, such as anomalies, temporal dynamics,
and information cascades, are added synthetically to match
those in real data. Computational results and a data evaluation
exercise with human participants show that SHIELD generates
high-fidelity synthetic data (statistically similar to real data
and difficult to distinguish by humans from real data). The
synthetic data generated by SHIELD does not duplicate real
data and is suitable for evaluating new social media analytics
while avoiding the violation of social media users’ privacy.
ACKNOWLEDGMENT
The authors would like to thank the Defense Advanced
Research Projects Agency for their support. They would also
like to thank Prof. B. Zhao from the University of Chicago
for his guidance on graph models and initial code for graph
fitting. Distribution Statement A (Approved for Public Release,
Distribution Unlimited).
R EFERENCES
[1] Number of Monthly Active Twitter Users. Accessed: Jul. 18, 2018.
[Online]. Available: https://www.statista.com/statistics/282087/number-
of-monthly-active-twitter-users
[2] J. Jiang et al., “Understanding latent interactions in online social
networks,” ACM Trans. Web, vol. 7, no. 4, pp. 1–39, 2013.
[3] J. Leskovec, J. Kleinberg, and C. Faloutsos, “Graphs over time: Densi-
fication laws, shrinking diameters and possible explanations,” in Proc.
ACM SIGKDD, 2005, pp. 177–187.
[4] A. Vazquez, “Growing network with local rules: Preferential attachment,
clustering hierarchy, and degree correlations,” Phys. Rev. E, Stat. Phys.
Plasmas Fluids Relat. Interdiscip. Top., vol. 67, p. 056104, Jun. 2003.
[5] A.-L. Barabási and R. Albert, “Emergence of scaling in random net-
works,” Science, vol. 286, no. 5439, pp. 509–512, 1999.
[6] H. Inaltekin, M. Chiang, and H. V. Poor, “Delay of social search on
small-world graphs,” J. Math. Sociol., vol. 38, no. 1, pp. 1–46, 2014.
[7] J. Leskovec, D. Chakrabarti, J. Kleinberg, and C. Faloutsos, “Real-
istic, mathematically tractable graph generation and evolution, using
Kronecker multiplication,” in Proc. PKDD, Porto, Portugal, 2005,
pp. 133–145.
15
[8] L. Akoglu, M. McGlohon, and C. Faloutsos, “RTM: Laws and a
recursive generator for weighted time-evolving graphs,” in Proc. 8th
IEEE Int. Conf. Data Mining, Dec. 2008, pp. 701–706.
[9] L. Akoglu and C. Faloutsos, “RTG: A recursive realistic graph generator
using random typing,” in Proc. PKDD, 2009, pp. 13–28.
[10] P. Mahadevan, D. Krioukov, K. Fall, and A. Vahdat, “Systematic
topology analysis and generation using degree correlations,” in Proc.
SIGCOMM, 2006, pp. 135–146.
[11] M. Gjoka, M. Kurant, and A. Markopoulou, “2.5K-graphs: From sam-
pling to generation,” in Proc. IEEE INFOCOM, Turin, Italy, Apr. 2013,
pp. 1968–1976.
[12] D. F. Nettleton, “A synthetic data generator for online social network
graphs,” Soc. Netw. Anal. Mining, vol. 6, p. 44, Dec. 2016.
[13] P. J. Lin et al., “Development of a synthetic data set generator for build-
ing and testing information discovery systems,” in Proc. IEEE ITNG,
Las Vegas, NV, USA, Apr. 2006, pp. 707–712.
[14] A. Sala, L. Cao, C. Wilson, R. Zablit, H. Zheng, and B. Y. Zhao,
“Measurement-calibrated graph models for social network experiments,”
in Proc. WWW, Raleigh, NC, USA, 2010, pp. 861–870.
[15] E. Reiter and R. Dale, Building Natural Language Generation Systems.
Cambridge, U.K.: Cambridge Univ. Press, 2000.
[16] I. Goodfellow et al., “Generative adversarial nets,” in Proc. Adv. Neural
Inf. Process. Syst., 2014, pp. 2672–2680.
[17] V. Dumoulin et al. (2016). “Adversarially learned inference.” [Online].
Available: https://arxiv.org/abs/1606.00704
[18] W. Liu, P.-Y. Chen, H. Cooper, M. H. Oh, S. Yeung, and T. Suzumura.
(2017). “Can GAN learn topological features of a graph?” [Online].
Available: https://arxiv.org/abs/1707.06197
[19] S. Rajeswar, S. Subramanian, F. Dutil, C. Pal, and A. Courville.
(2017). “Adversarial generation of natural language.” [Online]. Avail-
able: https://arxiv.org/abs/1705.10929
[20] S. M. Rodrigo and J. G. F. Abraham, “Development and implementation
of a chat bot in a social network,” in Proc. IEEE ITNG, Apr. 2012,
pp. 751–755.
[21] G. Pilato, A. Augello, G. Vassallo, and S. Gaglio, “EHeBby:
An evocative humorist chat-bot,” J. Mobile Inf. Syst., vol. 4, no. 3,
pp. 165–181, 2008.
[22] A. Santangelo, A. Augello, A. Gentile, G. Pilato, and S. Gaglio,
“A chat-bot based multimodal virtual guide for cultural heritage tours,”
in Proc. PSC, 2006, pp. 114–120.
[23] N. Ali, M. Hindi, and R. V. Yampolskiy, “Evaluation of authorship attri-
bution software on a chat bot corpus,” in Proc. IEEE ICAT, Oct. 2011,
pp. 1–6.
[24] T. Holtgraves and T.-L. Han, “A procedure for studying online conversa-
tional processing using a chat bot,” Behav. Res. Methods, vol. 39, no. 1,
pp. 156–163, 2007.
[25] C. Wilson, B. Boe, A. Sala, K. P. N. Puttaswamy, and B. Y. Zhao, “User
interactions in social networks and their implications,” in Proc. EuroSys,
2009, pp. 205–218.
[26] Z. Yang, J. Xue, C. Wilson, B. Y. Zhao, and Y. Dai, “Uncovering user
interaction dynamics in online social networks,” in Proc. ICWSM, 2015,
pp. 698–701.
[27] G. Wang et al., “Social Turing tests: Crowdsourcing sybil detection,” in
Proc. NDSS, 2012.
[28] D. Kempe, J. Kleinberg, and E. Tardos, “Maximizing the spread
of influence through a social network,” in Proc. SIGKDD, 2003,
pp. 137–146.
[29] H. Kwak, C. Lee, H. Park, and S. Moon, “What is Twitter, a social
network or a news media?” in Proc. WWW, 2010, pp. 591–600.
[30] Twitter API Rate Limits. Accessed : Jul. 18, 2018. [Online]. Available:
https://dev.twitter.com/rest/public/rate-limits
[31] D. M. Blei, A. Y. Ng, and M. I. Jordan, “Latent Dirichlet allocation,”
J. Mach. Learn. Res., vol. 3, nos. 4–5, pp. 993–1022, 2003.
[32] S. C. Deerwester et al., “Computer information retrieval using latent
semantic structure,” U.S. Patent 4 839 853 A, Jun. 13, 1989.
[33] T. Hofmann, “Probabilistic latent semantic analysis,” in Proc. Conf.
Uncertainty Artif. Intell., Jul. 1999, pp. 289–296.
[34] D. Jurafsky and J. H. Martin, Speech and Language Processing.
Englewood Cliffs, NJ, USA: Prentice Hall, 2000.
[35] Twitter API. Accessed: Jul. 18, 2018. [Online]. Available:
https://dev.twitter.com/rest/public
[36] Random Function. Accessed: Jul. 18, 2018. [Online]. Available:
https://docs.python.org/3.1/library/random.html
 This article has been accepted for inclusion in a future issue of this journal. Content is final as presented, with the exception of pagination.

16 IEEE TRANSACTIONS ON COMPUTATIONAL SOCIAL SYSTEMS

[37] A. Go, R. Bhayani, and L. Huang, “Twitter sentiment classifica-
tion using distant supervision,” Stanford Univ., Stanford, CA, USA,
Tech. Rep. CS224N, vol. 1, p. 12, 2009.
Yalin E. Sagduyu (S’02–M’08–SM’15) received the
Ph.D. degree in electrical and computer engineering
from the University of Maryland, College Park, MD,
USA.
He is currently the Associate Director of
Networks and Security at Intelligent Automation,
Inc., Rockville, MD, USA. His current research
interests include machine learning, graph analytics,
networks, and communications.
Dr. Sagduyu chaired workshops at the ACM
MobiCom, the IEEE CNS, and the IEEE ICNP,
served as the Track Chair at the IEEE PIMRC 2014, and served in the
Organizing Committee of the IEEE GLOBECOM 2016.
Yi Shi (S’02–M’08–SM’13) received the Ph.D.
degree from Virginia Tech, Blacksburg, VA, USA.
He is currently a Senior Research Scientist at Intel-
ligent Automation, Inc., Rockville, MD, USA. His
current research interests include algorithm design,
optimization, machine learning, communication net-
works, and social networks.
Dr. Shi was a recipient of the IEEE INFOCOM
2008 Best Paper Award and the IEEE INFOCOM
2011 Best Paper Award Runner-Up. He has been a
TPC Chair for IEEE and ACM workshops. He has
been an Editor of the IEEE C OMMUNICATIONS S URVEYS AND T UTORIALS .

Alexander Grushin received the Ph.D. degree in

computer science from the University of Maryland,
College Park, MD, USA.
He was with Quantum Leap Innovations, Newark,
DE, USA, where he developed and evaluated opti-
mization algorithms for constrained problems. He is
currently a Senior Research Scientist at Intelligent
Automation, Inc., Rockville, MD, USA, where he
has been involved in machine learning, network sci-
ence, optimization, multiagent systems, social media
analytics, air traffic analysis and modeling, and the
control of robot swarms.