Journal of the American Statistical Association

ISSN: 0162-1459 (Print) 1537-274X (Online) Journal homepage: https://www.tandfonline.com/loi/uasa20

Clustering Huge Number of Financial Time Series:

A Panel Data Approach With High-Dimensional
Predictors and Factor Structures

Tomohiro Ando & Jushan Bai

To cite this article: Tomohiro Ando & Jushan Bai (2017) Clustering Huge Number of
Financial Time Series: A Panel Data Approach With High-Dimensional Predictors and
Factor Structures, Journal of the American Statistical Association, 112:519, 1182-1198, DOI:
10.1080/01621459.2016.1195743

To link to this article: https://doi.org/10.1080/01621459.2016.1195743

View supplementary material Accepted author version posted online: 10

Jun 2016.
Published online: 25 Apr 2017.

Submit your article to this journal Article views: 1890

View related articles View Crossmark data

Citing articles: 18 View citing articles

Full Terms & Conditions of access and use can be found at

https://www.tandfonline.com/action/journalInformation?journalCode=uasa20
JOURNAL OF THE AMERICAN STATISTICAL ASSOCIATION
, VOL. , NO. , –, Theory and Methods
http://dx.doi.org/./..

Clustering Huge Number of Financial Time Series: A Panel Data Approach With
High-Dimensional Predictors and Factor Structures
Tomohiro Andoa and Jushan Baib,c
a
Melbourne Business School, University of Melbourne, Carlton, Australia; b Department of Economics, Columbia University, New York, NY; c School of
Finance, Nankai University, Tianjin, China

ABSTRACT ARTICLE HISTORY

This article introduces a new procedure for clustering a large number of financial time series based on Received August 
high-dimensional panel data with grouped factor structures. The proposed method attempts to capture Revised May 
the level of similarity of each of the time series based on sensitivity to observable factors as well as to KEYWORDS
the unobservable factor structure. The proposed method allows for correlations between observable and Clustering; Factor structure;
unobservable factors and also allows for cross-sectional and serial dependence and heteroscedasticities Heterogeneous panel; Lasso;
in the error structure, which are common in financial markets. In addition, theoretical properties are estab- Serial and cross-sectional
lished for the procedure. We apply the method to analyze the returns for over 6000 international stocks error correlations
from over 100 financial markets. The empirical analysis quantifies the extent to which the U.S. subprime
crisis spilled over to the global financial markets. Furthermore, we find that nominal classifications based
on either listed market, industry, country or region are insufficient to characterize the heterogeneity of the
global financial markets. Supplementary materials for this article are available online.

1. Introduction diversification approach and its alternatives—such as country-,

industry-, and market-based diversification strategies, to name a
The U.S. subprime crisis of 2007, which was triggered by the
few—is also an important issue for international portfolio man-
collapse of the U.S. housing market, subsequently spilled over
agers. Moreover, it is sometimes difficult to assign a nationality
to the entire U.S. and the European financial markets, result-
to a multinational company. In other words, industry classifica-
ing in bankruptcies, forced mergers, and bailouts for many large
tion can be a subjective undertaking, particularly for large con-
firms.1 These financial shocks further spread to the global finan-
glomerates.
cial markets, and led to massive declines in worldwide asset
Motivated by these important issues, this study analyzes a
values.
large number of financial industry stock returns, that is, over
Thus, identifying the sources of the co-movement of inter-
6000 returns from more than 100 international stock mar-
national stock returns is one of the most important issues in
kets. In particular, we seek to answer the following empirical
finance. Portfolio managers explore investment opportunities
questions:
not only in applicable domestic markets but also in foreign
1. How many groups are there among the large number of
financial markets. In the field of asset pricing, researchers have
assets returns?
been searching for those factors that explain the cross-sectional
2. In each group, how many group-specific common factors
variations in global stock returns (see, e.g., Griffin 2002; Hou
are there that explain the cross-sectional and time-series
et al. 2011 and references therein). Previous studies that have
variations in stock returns?
attempted to identify the factors influencing the determina-
3. Do the co-movements within a market, industry, coun-
tion of international stock returns have arrived at mixed results.
try, or region constitute the only sources of cross-
Fama and French (1998) emphasized a more globally integrated
sectional and time series variations in the stock markets?
market, while Griffin (2002) argued that only local, country-
4. What are the different characteristics of the markets that
specific factors matter in explaining global stock returns. This
can be observed during the recent financial crisis?
paper attempts to address these important questions by intro-
To address these questions, we introduce a new procedure
ducing a new statistical modeling procedure for building an
for clustering a large number of financial time series based on
empirical asset pricing model.
high-dimensional panel data with grouped factor structures.
An internationally diversified portfolio often requires that
Clustering is based on similarity measures using past stock
the most influential factors in a variety of regions, countries,
returns. In particular, we introduce panel data models with
markets and industries be assessed and evaluated (see, e.g., Hes-
heterogeneity, and these models have many attractive features.
ton and Rouwenhorst 1994). Thus, comparing a geographic

CONTACT Tomohiro Ando T.Ando@mbs.edu Melbourne Business School, University of Melbourne,  Leicester Street, Carlton , Australia.
Color versions of one or more of the figures in the article can be found online at www.tandfonline.com/r/JASA.
Supplementary materials for this article are available online. Please go to www.tandfonline.com/r/JASA.
 The firms affected include AIG, Bear Stearns, BNP Paribas, Fannie Mae, Freddie Mac, Lehman Brothers, and Merrill Lynch.

©  American Statistical Association


First, heterogeneity is captured by using a factor error structure
and heterogeneous regression coefficients. Second, our method
allows for a large number of observable factors, while the set
of relevant observable factors are selected automatically. Third,
observable factors can be correlated with unobservable factors
or factor loadings or both. Fourth, the group membership of
each unit is unknown, and these memberships will be estimated
from historical stock returns. Finally, the number of groups
remains unknown and is to be determined using a novel model
selection criterion. We note that our asymptotic theory is devel-
oped for the optimal solution, which is obtained by minimizing
the penalized objective function. In terms of computation in
practice, the exact optimal solution can be time consuming due
to the nature of the NP-hard problem. We, therefore, consider
an algorithm that quickly searches approximate solutions.
Our empirical analysis indicates that the country-specific fac-
tor is one of the sources of co-movement in the cross-sectional
and time-series variations of stock returns. This result is con-
sistent with that of Fama and French (2012), who reported that
global models fare poorly, while local versions of their three- and
four-factor models for each of four regions—North America,
Europe, Japan, and Asia Pacific—capture local average returns
rather well.
The remainder of this article is organized as follows. In the
next section, we introduce asset pricing model and its assump-
tions. We also briefly review the related literature. Section 3 lays
out the proposed statistical framework, discusses the parameter
estimation, and explains how the best model is chosen to cap-
ture the underlying market structures. Section 4 establishes new
asymptotic results, including the consistency of the proposed
estimator and its asymptotic behaviors. A number of theoreti-
cal results are established. Additional theoretical results are pro-
vided in the Appendix. The Appendix also contains simulation
results that demonstrate that the proposed method works well.
Section 5 describes the dataset. Empirical results are given in
Section 6. Finally, Section 7 provides some concluding remarks.
2. Asset Pricing Model and Related Literature
2.1. Asset Pricing Model
Let t = 1, . . . , T be the time index and i = 1, . . . , N be the
index of financial asset. Let S be the number of asset groups
(which is unknown, finite and independent of N and T ), and
let G = {g1 , . . . , gN } denote the group membership such that
gi ∈ {1, . . . , S}. A distinctive feature of the model is that group
membership is not specified. Let N j be the number of cross-
sectional assets within group G j ( j = 1, . . . , S) such that N =
S
j=1 N j . To capture the underlying market characteristics, we
assume that the return of the ith asset, observed at time t, yit , is
expressed as
yit = xit βi + f c,t λc,i + f gi ,t λgi ,i + εi,t ,
i = 1, . . . , N, t = 1, . . . , T, (1)
where xit is a pi × 1 vector of observable factors, f c,t is an
r × 1 vector of unobservable common factors that affect the
returns of all securities in all groups, λc,i is the corresponding
factor loading; f gi ,t is an rgi × 1 vector of unobservable group-
specific factors that affect the returns only of asset group gi ,
JOURNAL OF THE AMERICAN STATISTICAL ASSOCIATION 1183
λgi ,i is the unknown sensitivity to unobservable group-specific
factors, and εit is the asset-specific error. The pi × 1 vector βi
represents the unknown sensitivity to the explanatory variables
(observable factors). It is assumed that εit is independent of f c,t ,
f gi ,t and xit . Depending on applications, the unobserved fac-
tor components may be specified as an exact dynamic factor
model, as a static approximate factor model, or as a special case
of the generalized dynamic factor model. Technical assump-
tions are specified in Section 2.3. The factor structures in the
model (1) considered here is similar to Hallin and Liška (2011)
and Wang (2010). However, the group membership here is
unknown.
The unobservable term, f c,t λc,i + f gi ,t λgi ,i + εi,t , is typi-
cally treated as the overall error term that embodies cross-
sectional/serially dependence and heteroscedasticity.
However, ignoring the unobservable factor structure,
f c,t λc,i + f gi ,t λgi ,i , in general, does not work due to the endo-
geneity problem (the regressors are correlated with the factors
and factor loadings). Under such circumstances, the depen-
dency between the regressors and unobservable factor struc-
tures should be captured simultaneously. Our model building
procedure takes this important issue into account.
When building the model, there are two issues to address.
One is how to estimate the unknown parameters, includ-
ing the regression coefficients {β1 , . . . , βN }, the unobserv-
able factor structure {Fc , F1 , . . . ., FS }, its corresponding load-
ings {c , 1 , . . . ., S }, and the unknown group membership of
each of the N units. Here, Fc = ( f c,1 , f c,2 , . . . , f c,T ) is the T ×
r matrix of common factors, c = (λc,1 , . . . , λc,N ) is the N ×
r matrix of factor loadings for Fc , Fj = ( f j,1 , f j,2 , . . . , f j,T )
is the T × r j matrix of factors for group G j , and  j =
(λ j,1 , . . . , λ j,N j ) is the N j × r j matrix of factor loadings for
group G j . These quantities can be determined under the given
values of the number of groups S and the dimension of the unob-
servable factors structure. Given S and the dimension of fac-
tor structure for each group, our method jointly estimates the
optimal grouping of the cross-sectional stocks, the regression
coefficients, the factors and the factor loadings. This part of the
modeling is called the estimation problem. To improve the speed
of computation, the shrinkage method is incorporated into the
estimation algorithm.
The other issue is determining the number of groups (S) and
the dimension of factors in each group. We will also determine
the relevant explanatory variables (observable factors). We refer
to this part of the modeling as the model selection problem.
2.2. Related Literature
Our modeling procedure and empirical investigation are related
to several disparate strands in the statistics and finance literature.
In terms of methodology, we propose a new modeling procedure
for asset pricing. The proposed statistical inference procedure
is a combination of high-dimensional grouped factor analysis
and shrinkage methods. In particular, a large panel data model
is considered for cases in which both the number of stocks, N,
and the length of time periods, T , are large.
Factor models have attracted substantial research interest
in recent years. In the econometric and statistical literature,
a number of studies have been devoted to factor models that
analyze high-dimensional data, including a dynamic exact
1184 T. ANDO AND J. BAI
factor model (Geweke 1977; Sargent and Sims 1977), a static
approximate factor model (Chamberlain and Rothschild 1983),
and a generalized dynamic factor model (Forni et al. 2000;
Forni and Lippi 2001; Amengual and Watson 2007; Hallin and
Liska 2007), among others.
Previous studies have considered panel data models with fac-
tor error structures to address cross-sectional and serial depen-
dence, including (Pesaran 2006; Bai 2009). Bai (2009) esti-
mated panel data models with interactive effects, permitting
the predictor to be correlated with unobserved heterogene-
ity. These papers considered homogeneous regression coeffi-
cients over cross-sectional units, which is somewhat restrictive.
Ando and Bai (2015) relaxed Bai’s (2009) model to allow for
heterogeneous regression coefficients that vary for each stock.
In addition, some previous studies featured “grouped” factor
structures with known group membership, including (Diebold,
Li, and Yue 2008; Kose, Otrok, and Whiteman 2008; Wang
2010; Hallin and Liška 2011; Moench and Ng 2011; Moench,
Ng, and Potter 2012). Under “grouped” factor structures, each
group is subject to its own unobservable factors that vary
by group. In the context of asset pricing models, each asset
group is exposed to unobservable factors, which are group-
specific. Although these methods are useful under known group
memberships, the group membership of each unit is often
unknown.
Some studies consider unknown group memberships with-
out factor structures, for example, Lin and Ng (2012), Su, Shi,
and Phillips (2016) and Sun (2005). The model by Bonhomme
and Manresa (2015) may be considered as a special factor model
with a single factor and known factor loadings being one. Ando
and Bai (2016) considered unknown group memberships with
factor structure under the common or group-heterogeneous
coefficients, in which the slope parameters are either the same or
vary only across the groups. Although their procedure captures
underlying market structure well, group-heterogeneous coef-
ficients remain restrictive. In fact, these authors applied their
method to the analysis of the two Chinese mainland stock mar-
kets – the Shanghai and Shenzhen stock exchanges – and found
that group-heterogeneous coefficients are acceptable for some
groups, whereas other groups show that group-heterogeneous
coefficients are a too strong assumption. In this article, we
allow heterogeneous regression coefficients that vary for each
stock.
Recently, various types of shrinkage methods have been pro-
posed, including the lasso method (Tibshirani 1996) and its vari-
ants (Zou 2006; Yuan and Lin 2006; Park and Casella 2008),
least-angle regression (Efron et al. 2004), elastic net (Zou and
Hastie 2005), the smoothly clipped absolute deviation approach
(SCAD; Fan and Li 2001), the minimax concave penalty method
(MCP; Zhang 2010), and the Dantzig selector (Candes and
Tao 2007), among many others. As with these studies, we aim
to select an appropriate set of observable factors among the
huge number of possible variables. More specifically, we use the
smoothly clipped absolute deviation (SCAD) penalty approach
(Fan and Li 2001). Thus, the nonzero coefficients are estimated
as if the zero coefficients were known and were imposed (the
so called “oracle property”). This result is obtained despite the
existence of many unobservable factors.
In empirical finance, identifying the sources of international
stock returns’ co-movements is of central importance. Some
studies are based on factors that can explain the cross-sectional
variation in global stock returns (see, e.g., Griffin 2002; Hou,
Karolyi, and Kho 2011; and references therein). There is a sub-
stantial body of literature that has attempted to identify the
influential factors that determine international stock returns,
but these studies have yielded only mixed results. Griffin (2002)
argued that country-specific factors are important to explain-
ing global stock returns and voiced doubts about the benefits
of extending the Fama and French (1993) three-factor model to
a global context. By contrast, Fama and French (1998) demon-
strated the applicability of the global version of multifactor
models. In regards to the influences of country and industry
factors, a number of studies emphasize the dominance of the
country factor over the industry factor (Heston and Rouwen-
horst 1994, 1995; Beckers, Connor, and Curds 1996; Griffin
and Karolyi 1998; Kuo and Satchell 2001), whereas Roll (1992)
reported that industry factors are the most important. Baca,
Garbe, and Weiss (2000) and Cavaglia, Brightman, and Aked
(2000) argued that the relative influence of the country fac-
tor and the industry factor depend on the time period. In
this article, we investigate this important issue by analyzing
the impact of the U.S. financial crisis on international financial
markets.
Our theoretical and empirical contributions are summarized
as follows. First, the model to be introduced in the next sec-
tion is new and very general. Under unknown group mem-
bership, the model allows heterogeneous regression coefficients
that vary with each stock (asset-dependence coefficients). More-
over, the number of regressors can increase as the size of the
panel increases. In the context of a cross sectional regres-
sion, Fan and Peng (2004) and Lam and Fan (2008) con-
sidered the case of increasing number of regressors. How-
ever, this is the first study to consider a divergent number of
regressors under the panel data models with a grouped factor
structure.
Second, a number of theoretical results—including consis-
tency, asymptotic normality, oracle property, and model selec-
tion consistency—are established. Because of the more gen-
eral model structure, establishing the inferential theory requires
nontrivial arguments. Although Ando and Bai (2015) and Ando
and Bai (2014) considered heterogeneous regression coeffi-
cients, they assumed that group membership is known and
also that the number of regressors is fixed. In contrast, the
group membership is unknown in this article. While Ando
and Bai (2016) considered unknown group membership, the
regression coefficients in their model only vary over the groups,
and the number of regressors is fixed. Here in this arti-
cle, the group membership is unknown, the regression coeffi-
cients are asset dependent, and the number of regressors can
increase with the sample size. We provide a theoretical anal-
ysis for these results. To our knowledge, this is the first study
that investigates a divergent number of regressors for “het-
erogeneous” regression coefficients in panel data with a factor
structure.
This article also makes an empirical contribution in analyz-
ing the recent U.S. financial crisis (Longstaff 2010; Diebold and
 JOURNAL OF THE AMERICAN STATISTICAL ASSOCIATION 1185

Yilmaz 2014 and so on). The results provide insightful informa- Assumption D: Observable Factors.
tion on the grouping of financial assets and its evolution prior to (D1): The vector of predictor xit satisfies
and during the financial crisis. max1≤i≤N T −1 Xi 2 = O p (N α ) with α < 1/8. We
also assume N/T 2 → 0.
(D2): Define MFc ,Fj = I − Fc (Fc Fc )−1 Fc − Fj (Fj Fj )−1 Fj . Let
2.3. Assumptions Xi,βi0 =0 be the submatrix of Xi , corresponding to the
columns of nonzero elements of the true parameter
Here, we state the assumptions and then provide comments con-
vector β0i . We use qi to denote the number of nonzero
cerning these assumptions. Throughout, the norm of matrix A
elements of β0i . Suppose that the ith financial asset
is defined as A = [tr(A A)]1/2 , where “tr” denotes the trace of
belongs to the gth group (i.e., gi0 = g). We assume the
a square matrix. The equation an = O(bn ) states that the deter-
qi × qi matrix
ministic sequence an is at most of order bn , cn = O p (dn ) states
that the random variable cn is at most of order dn in probability, 1  
and cn = o p (dn ) is of smaller order in probability. All asymp- Xi,β 0 =0 MFc0 ,Fg0 Xi,βi0 =0
T i
totic results are obtained under N, T → ∞. Restrictions on the
relative rates of convergence of N and T are specified in later is positive definite.
sections. (D3): Define Ai = T1 Xi MFc ,Fgi Xi , Ci = (Cci , Cgi ),
The true regression coefficient is denoted by β0i . Further,
Fc = ( f 0c,1 , . . . , f 0c,T ) and λ0c,i are the true common factor and
0  
Bci Bcgi
its factor loading of individual i, and Fg0i0 = ( f 0g0 ,1 , . . . , f 0g0 ,T ) Bi = ,
i i Bcgi Bgi
and λ0g0 ,i are the true factor and factor loading of individual i
i
with true group membership gi0 . with
  
Assumption A: Common and Group-Specific Factors. The com- Bci = λ0c,i λ0c,i ⊗ I, Bgi = (λ0g0 ,i λ0g0 ,i ) ⊗ I,
T 
mon factors satisfy E f 0c,t 4 < ∞ and T −1 t=1 f 0c,t f 0c,t →
i i
 
Fc as T → ∞, where Fc is an r × r positive definite matrix. Bcgi = λ0c,i λ0g0 ,i ⊗ I,
i
The group-specific factors satisfy E f 0j,t 4 < ∞ j = 1, . . . , S.
T 1  1 
Furthermore, T −1 t=1

f 0j,t f 0j,t → Fj as T → ∞, where Fj Cci = √ λ0c,i ⊗ Xi MFc ,Fgi , Cgi = √ λ0g0 ,i ⊗ Xi MFc ,Fgi .
T T i
is an r j × r j positive definite matrix. Although correlations
between f 0j,t and f 0k,t ( j = k) are allowed, they are not perfectly Let A be the collection of (Fc , Fg ) such that A =
correlated. Also, we assume orthogonality between the com- {(Fc , Fg ) : Fc Fc /T = I, Fg Fg /T = I}. We assume, for j =
T 
mon and group-specific factors T1 t=1 f 0c,t f 0j,t = 0 for j = 1, . . . , S,
1, . . . , S.
1 
Assumption B: Factor Loadings. inf Fc ,Fj ∈A Ei (Fc , Fj ) is positive definite,
N
i;gi0 = j
(B1): The factor loading matrix for the common factors
4
0c = [λ0c,1 , . . . , λ0c,N ] satisfies Eλ0c,i  < ∞ and
 where Ei (Fc , Fj ) = Bi − Ci A− −
i Ci and Ai is a generalized
N −1 0c 0c − c  → 0 as N → ∞, where
inverse of Ai .
c is an r × r positive definite matrix. The fac-
tor loading matrix for the group-specific factors Assumption E: Number of Units in Each Group. All units are
4
0j = [λ0j,1 , . . . , λ0j,N j ] satisfies Eλ0gi ,i  < ∞ and divided into a finite number of groups S, each containing N j

N −1
j  j  j −  j  → 0 as N j → ∞, where  j
0 0 units, such that 0 < a < N j /N < ā < 1, which implies that the
is an r j × r j positive definite matrix, j = 1, . . . , S. We number of units in the jth group increases as the total number
also assume that λ0g0 ,i  > 0. of units N grows.
i
 Some comments on the assumptions are provided. Assump-
(B2): For each i and j, f 0j,t λ0g0 ,i is strongly mixing processes
i tions A and B are usual and imply the existence of r common
with mixing coefficients that satisfy r(t ) ≤ exp(−a1t b1 ) factors and r j group-specific factors, j = 1, . . . , S. The last part

and with tail probability P(| f 0j,t λ0gi ,i | > z) ≤ exp{1 − of the assumption A assumes that the common factors f 0c,t and
(z/b2 ) }, where a1 , a2 , b1 and b2 are positive constants.
a2
the group-specific factors f 0g,t are orthogonal. This assumption
is needed to separately identify the common and the group-
Assumption C: Error Terms. specific factors (Wang 2010). In Assumption C, heteroscedas-
(C1): E[εit ] = 0, var(εit ) = σi2 , and εit is independent over i ticity is allowed. Although it is outside the scope of this arti-
and over t. cle, the errors are also allowed to have cross-sectional correla-
(C2): A positive constant, C < ∞, exists such that E[|εit |8 ] < tion, serial correlation, or both. This allows us to address var-
C for all i and t. ious types of dependency. However, it will require more tech-
(C3): εit is independent of xks , λ0c, , λ0g, , f 0c,s and f 0j,s ( j = nical conditions such as those in Bai (2009), thus its discussion
1, . . . , S) for all i, k, , t, s. is omitted. Assumption D1 requires some moment conditions
1186 T. ANDO AND J. BAI
on the observable factors. The observable factors can be corre-
lated with group-specific factors, factor loadings or both. The
number of cross-sectional units N can be much greater than the
number of time periods T . In practice, most of the applications
are carried out in the case of N > T , as the number of assets N
is much larger than the length of the time series. However, N
should grow less than T 2 and especially N = O(exp(T )) is not
allowed. The true number of groups, S, is assumed to be finite
and independent of N and T . Assumption D2 is analogous to
the full rank condition in standard linear regression models and
that is made in Ando and Bai (2015). Assumption D3 is simi-
lar to a condition used in Bai (2009), where only a single group
exists. The assumption is used for proof of consistency when
factor and factor loadings are also estimated. Note that if Ai =
0, then Ci must be zero because Xi MFc ,Fg = 0. As a result, the
term Ci A−  −  −
i Ci becomes Ci Ai Ci = 0. Thus, Ci Ai Ci is well defined
even if Ai = 0. This assumption is also used in Ando and Bai
(2015).
3. Model Building
In this section, we describe our modeling framework, which
involves identifying the number of groups and group-specific
factors, and estimating model parameters from a large panel
data. The goal of our procedure is to identify the underlying
market structure.
3.1. Estimation
Given the number of groups S, the number of common fac-
tors r, the number of factors in group r j ( j = 1, 2, . . . , S),
and the size of penalty κi in pi (βi ) = pκi ,γ (βi ), the estimator
ˆ c, 
{β̂1 , . . . , β̂N , Ĝ, F̂c , F̂1 , . . . , F̂S ,  ˆ 1, . . . , 
ˆ S } is defined as the
minimizer of
L(β1 , .., βN , G, Fc , F1 , . . . , FS , c , 1 , . . . , S )
N
N
= yi − Xi βi − Fc λc,i − Fgi λgi ,i  + T 2
pi (βi ), (2)
i=1 i=1
subject to normalization restrictions on Fc and Fj ( j = 1, . . . , S),
to be discussed below. Here, c = (λc,1 , . . . , λc,N ) is the N ×
r factor loading matrix for the common factors Fj ,  j =
(λ j,1 , . . . , λ j,N j ) is the N j × r j factor loading matrix ( j =
1, . . . , S) for the group-specific factors Fj (Connor and Kora-
jzcyk 1986; Bai and Ng 2002; Stock and Watson 2002). The first
term is the fitness term and the second term is the penalty.
For the penalty function pi (βi ) in (2), we can consider the
ridge penalty, the lasso penalty (Tibshirani 1996) and its vari-
ants (Yuan and Lin 2006; Zou 2006), the elastic net (Zou and
Hastie 2005), the minimax concave penalty (Zhang 2010) the
SCAD penalty of Fan and Li (2001), and so on. To explain the
estimation procedure, we here use the SCAD penalty of Fan and
Li (2001), which is given by
pj
pi (βi ) ≡ pκi ,γ (βi ) = pκi ,γ (|βi, j |)
j=1
with
pκi ,γ (|βi, j |)
⎧
⎪ κi |βi, j | (|βi, j | ≤ κi )
⎪
⎪
⎪
⎪  2 
⎪
⎨ γ κi |βi, j | − 0.5 βi, j + κi
2
= (κi < |βi, j | ≤ γ κi )
⎪ γ −1
⎪
⎪
⎪
⎪ κi (γ 2 − 1)
2
⎪
⎩ (γ κi < |βi, j |)
2(γ − 1)
for κi > 0 and γ > 2. Following Fan and Li (2001), we use the
value γ = 3.7, which minimizes a Bayesian risk criteria for the
regression coefficients. The regularization parameter κi controls
the size of the penalty and varies over the cross-sectional asset.
In contrast to the previous studies where the common regular-
ization parameter is employed for each of the cross-sectional
units (e.g., Ando and Bai 2015), this article allows the flex-
ibility to the regularization parameter. This flexibility makes
sense because some assets may be subject to a small number
of factors, whereas the other group may be influenced by a
large number of factors. To our best knowledge, this is the first
panel study that allows the regularization parameter to vary
over the cross-sectional units. Later in this section, we pro-
vide an algorithm that searches the best values of the regular-
ization parameters {κ1 , . . . , κN } over a prespecified candidate
space.
To obtain the minimizer of L(β1 , . . . , βN , G, Fc , F1 , . . . , FS ,
c , 1 , . . . , S ), we can use an iterative scheme. Given the
group membership G, the common factor structures Fc λc,i , and
the group-specific factor structures Fj λgi ,i , we define the vari-
able y∗i = yi − Fc λc,i − Fgi λgi ,i for i = 1, . . . , N. Then, the objec-
tive function for βi can be viewed as y∗i − Xi βi 2 + T pκi ,γ (βi ).
Thus, the estimator of βi can be obtained by the SCAD approach.
Given the group membership G, the common factor
structures Fc λc,i , and the value of the regression coefficient
β1 , . . . , βN , we define the variable Z j = (z j,1 , . . . , z j,N j ) with
z j,i = yi − Xi βi − Fc λc,i for gi = j. Then, model (1) reduces to
z j,i = Fj λgi ,i + εi . Because this implies that matrix Z j has a pure
factor structure, we can use the previously established factor
analysis methods. We can obtain the principal components’ esti- √
mate of Fj , subject to the normalization Fj Fj /T = Ir j , is T
times the eigenvectors corresponding to the r j largest eigenval-
ues of the T × T matrix Zj Z j . Given F̂j , the factor loading matrix
can be obtained as  ˆ j = Z j F̂j /T . See also Bai and Ng (2002, pp.
197–198), Connor and Korajzcyk (1986), and Stock and Watson
(2002).
Given the group membership G, the group-specific factor
structures Fgi λgi ,i , and the value of the regression coefficient
β1 , . . . , βN , we define the variable Zc = (z c,1 , . . . , z c,N ) with
z c,i = yi − Xi βi − Fgi λgi ,i for gi = j. The model (1) reduces to
z c,i = Fc λc,i + εi . The principal components’ √ estimate of Fc sub-
ject to the normalization Fc Fc /T = Ir , is T times the eigen-
vectors corresponding to the r largest eigenvalues of the T × T
matrix Zc Zc . Given F̂c , the factor loading matrix can be obtained
as  ˆ c = Zc F̂c /T .
For any given values of βi , Fc λc,i and Fgi λgi ,i ( j = 1, . . . , S),
the optimal assignment for each individual unit is given

as gi∗ = argmin j∈{1,...,S} yi − Xi βi − Fc λc,i − Fj λ j,i 2 . The final
estimated individual membership satisfies
ĝi = argmin j∈{1,...,S} yi − Xi β̂i − F̂c λ̂c,i − F̂j λ̂ j,i 2 , (3)
which minimizes the sum of squared residuals among the S pos-
sible groups. Here λ̂ j,i = F̂j (yi − Xi β̂i − F̂c λ̂c,i )/T .
Because the estimates of β1 , . . . , βN , {Fc , c }, {Fj ,  j ; j =
1, . . . , S}, and G ∈ {g1 , . . . , gN } depend on one another, we
update the set of parameters sequentially. Moreover, we have
to employ this strategy to capture the dependency between the
regressors and unobservable factor structures simultaneously
due to the endogeneity problem.
Estimation Algorithm
Step 1. Fix κ1 ,...,κN , r, {r1 , . . . , rS } and S. Initialize the unknown
parameters β1(0) , . . . , βN(0) , {Fc(0) , c(0) }, {Fj(0) , (0) j ; j =
1, . . . , S}, G(0) ∈ {g1(0) , . . . , GN(0) }.
Step 2. Given the values of β1 , . . . , βN , {Fc , c }, and
{Fj ,  j ; j = 1, . . . , S}, update gi for i = 1, . . . , N based
on (3).
Step 3. Given the values of {Fc , c }, {Fj ,  j ; j = 1, . . . , S} and
G, update βi for i = 1, . . . , N.
Step 4. Given the values of β1 , . . . , βN , {Fj ,  j ; j = 1, . . . , S}
and G, update {Fc , c }.
Step 5. Given the values of β1 , . . . , βN , {Fc , c } and G, update
{Fj ,  j } for j = 1, . . . , S.
Step 6. Repeat Steps 2 ∼ 5 until convergence. Then we obtain the
ˆ c, 
ˆ 1, . . . , 
ˆ S }.
estimators {β̂1 , . . . , β̂N , Ĝ, F̂c , F̂1 , . . . , F̂S , 
Remark 1. In Step 1, starting values are needed. To obtain initial
group membership G(0) , we use the well-known K-means algo-
rithm (Forgy 1965) that divides the dataset {yi ; i = 1, . . . , N}
into S clusters that correspond to the number of groups. An ini-
tial estimate of βi(0) (i = 1, . . . , N) is obtained via the SCAD
approach by ignoring the factor structures. Given G(0) and βi(0)
(i = 1, . . . , N), the common factor structure {Fc(0) , c(0) } is then
obtained. Finally, given the values of β (0) , {Fc(0) , c(0) } and G(0) ,
we obtain the starting values {Fj(0) , (0)j } for j = 1, . . . , S by the
principal components.
Remark 2. Our asymptotic theory is developed for the opti-
ˆ c, 
mal solution {β̂1 , . . . , β̂N , Ĝ, F̂c , F̂1 , . . . , F̂S ,  ˆ 1, . . . , 
ˆ S }. In
terms of computation in practice, the exact optimal solution can
be time consuming because the possible combinations of the
group membership G is large. Approximate solutions are rel-
atively quick to obtain. Our computation is approximate, as is
mostly done in practice for clustering analysis. Group member-
ship G is updated sequentially in the algorithm instead of brute-
force enumeration.
Remark 3. In a simulation study, we also consider the standard
lasso
pκi (βi, j ) = κi |βi, j |
and the minimax concave penalty (MCP)

κi βi, j − βi,2 j /(2γ ) (βi, j ≤ γ κi )
pκi ,γ (βi, j ) =
κi2 γ /2 (βi, j > γ κi )
JOURNAL OF THE AMERICAN STATISTICAL ASSOCIATION 1187
for γ > 1. Breheny and Huang (2011) suggested that MCP and
SCAD are worthwhile alternatives to the lasso. The supplemen-
tal document contains more details for comparison.
Remark 4. As discussed by Bai and Ng (2002, p.198), we can con-
sider either one of the following two procedures when estimat-
ing the factor structure. To explain the idea, we focus on extract-
ing the common factor structure from the matrix Zc . In proce-
dure 1, we first estimate Fc from the T × T matrix Zc Zc to obtain
F̂c subject to the normalization of Fc Fc /T = I. Then, the corre-
sponding factor loading is obtained  ˆ c = Zc F̂c /T . As an alter-
native procedure, we can first extract eigenvectors of the N × N
matrix Zc Zc to obtain the estimate ˜ c subject to the normaliza-
tion of c c /N = I. This normalization implies F̃c = Zc 
 ˜ c /N.
Even when N > T , the spiked eigenvalues and related eigen-
vectors are consistently estimated (Fan, Liao, and Mincheva
2013). As suggested by Bai and Ng (2002, p.198), the first pro-
cedure is computationally less intensive when T < N, which is
the case in our application. The second procedure is preferred
when N < T because the computation is less costly. Thus, the
size of panel will be useful in determining the procedure for
extracting factor structures. Either procedure produces the same
common components (the multiplication of factor and factor
loadings).
3.2. Model Selection
In practice, however, the number of groups, S, and the num-
ber of common factors, r, the number of group-specific fac-
tors, {r1 , . . . , rS }, are unknown. Moreover, we have to select
the size of the regularization parameters such that the rele-
vant observable factors are included, while excluding irrelevant
observable factors. We propose a new criterion to select these
quantities.
PICC (S, k, k1 , . . . , kS , κ1 , . . . , κN )
1  2
S
 
= yi − Xi β̂i − F̂c λ̂c,i − F̂ĝi λ̂ĝi ,i 
NT j=1 i;g = j
i
1 2
N
+C × σ̂ log(T ) p̂i
N i=1
 
2 T +N
+C × k × σ̂ log (T N )
TN
G  
T + Nj  
+ C × k j × σ̂ 2 log T N j , (4)
j=1
T Nj
where p̂i is the number of nonzero elements of β̂i , C is some posi-
1 N T
tive constant and σ̂ 2 is an estimate of NT i=1 t=1 E(εit ). Note
2
that the effect of the regularization parameters κi (i = 1, . . . , N)
is measured through p̂i . Too large of a regularization parame-
ter will lead p̂i = 0 for i = 1, . . . , N, while too small of a regu-
larization parameter causes an over-fitting problem. In the next
section, we show that our proposed panel information criterion,
PIC, can select the true nonzero regression coefficients. By min-
imizing PIC, we can choose the number of groups S, the num-
ber of common factors k, the number of group-specific factors
1188 T. ANDO AND J. BAI
k j ( j = 1, . . . , S), and the size of the regularization parameters
κ1 , . . . , κN .
The criterion has incorporated the procedure of Hallin and
Liška (2007), that is, the criterion (4) has the theoretical prop-
erty that a penalty function on the number of common fac-
tors k × σ̂ 2 ( TT+NN
) log(T N) leads to a consistent estimate of the
true number of common factors, r, even when multiplying the
penalty by some positive constant C. However, for given finite N
and T , the value of C affects the model selection result. A very
large value of C over penalizes the number of common factors
and vise versa. Similar arguments also apply to the penalty func-
tions on the dimension of regression coefficients as well as to the
number of group-specific factors.
To optimize the value of C, we use the suggestion of
Hallin and Liška (2007) and Alessi, Barigozzi, and Capasso
(2010). We investigate the asymptotic behavior of the selected
number of groups S, number of common factors r, and
number of group-specific factors k1 , . . . , kS , from consid-
ering subsamples of sizes (N (a) , T (a) ) with a = 1, . . . , A
such that 0 < N (1) < N (2) < · · · < N (A) = N and 0 < T (1) <
T (2) < · · · < T (A) = T For any (N (a) , T (a) ) and C, we can
compute the number of groups SC (N (a) , T (a) ), number of
common factors rC (N (a) , T (a) ), and number of group-specific
factors kC1 (N (a) , T (a) ), . . . , kCS (N (a) , T (a) ). Hallin and Liška
(2007) pointed out that between the extreme small values
and too large values, there exist a range of moderate val-
ues of C such that the selected model is a stable function
of the subsample size (N (a) , T (a) ). They measured the sta-
bility with respect to sample size by the empirical variance
of the selected values of SC (N (a) , T (a) ), rC (N (a) , T (a) ), and
kC1 (N (a) , T (a) ), . . . , kCS (N (a) , T (a) ). In our case, it is measured
by
 2
1 C (a) (a)
A A
−1 (b) (b)
VC2 = r (N , T ) − A r (N , T )
C
A a=1 b=1
⎡  2 ⎤
Smax A A
⎣ 1
+ rC (N (a) , T (a) ) − A−1 rCj (N (b) , T (b) ) ⎦ ,
A a=1 j
j=1 b=1
(5)
where the first term measures the variability of selected common
factors, and the second term measures the variability of selected
number of groups as well as the number of group-specific fac-
tors.
Under the given value of C, N (a) , T (a) , the following provides
a model search algorithm.
Model Search Algorithm
Step 1. Prepare the candidate values of regularization parame-
ters {κ1 , . . . , κN }, the number of groups S and the numbers
of group-specific factors {k1 , . . . , kS }.
Step 2. Fix S and initialize the values of regularization parame-
ters {κ1 , . . . , κN }, the numbers of common factors k, and the
numbers of group-specific factors {k1 , . . . , kS }.
Step 3. Given the current values of S, k and {k1 , . . . , kS }, optimize
each of the regularization parameters, κi (i = 1, . . . , N) by
minimizing PIC.
Step 4. Given the values of S, the numbers of group-specific fac-
tors {k1 , . . . , kS } and κi (i = 1, . . . , N), optimize the num-
bers of common factors k by minimizing PIC.
Step 5. Given the values of S, the numbers of common factors
k, and κi (i = 1, . . . , N), optimize the numbers of group-
specific factors {k1 , . . . , kS } by minimizing PIC.
Step 6. Repeat Steps 3 and 5 until convergence. Then, store the
value of PIC.
Step 7. Change the value of S and implement Steps 2 ∼ 6.
Step 8. Select the best model based on the stored values of PIC.
Using the above algorithm, we calculate the stability measure
VC2 in (5). The final model is obtained under the optimized C,
which is a moderate value such that the selected model is stable.
Simulation results show that the proposed algorithm perform
well. For more details, see supplementary materials.
4. Theoretical Analysis
In this section, we consider the asymptotic analysis. In partic-
ular, we derive the asymptotic properties of the proposed esti-
mator and show that the proposed estimator is consistent, as N
and T go to infinity simultaneously. We also develop the variable
selection consistency of the proposed estimator for the regres-
sion coefficients. Ando and Bai (2015) established an oracle
property for the finite parameter case, under the known group
membership. In the context of cross-sectional regression, Fan
and Li (2001) demonstrated that penalized likelihood estima-
tors based on SCAD are asymptotically as efficient as the ora-
cle estimator. In this article, the group membership is unknown
and thus the establishment of variable selection consistency is a
challenge.
We also consider the situation in which the number of
predictors tends to infinity. In Appendix B, we provide the
consistency of the estimated regression coefficients, the con-
sistency of the estimated group membership, and the vari-
able selection consistency under the diverging number of
parameters.
We use Fc0 , {Fj0 , j = 1, . . . , S} to denote the true parameter
values of the common and group-specific factors from the data-
generating process. As T increases, the number of elements in Fc
and Fj ( j = 1, . . . , S) is also increasing. We first show that the
estimated factors are consistent in the sense of some averaged
norm, which will be specified below. First, we have the following
theorem.
Theorem 1. Consistency. Under Assumptions A–E, κ =
max{κ1 , . . . , κN } → 0 and T × κi → ∞ (i = 1, . . . , N) as
T, N → ∞. The estimators F̂c and {F̂j , j = 1, . . . , S} are
consistent in the sense of the following norm
T −1 F̂c − Fc0 Hc 2 = o p (1), T −1 F̂j − Fj0 H j 2 = o p (1),
j = 1, . . . , S, (6)

where Hc−1 = Vc,NT (Fc0 F̂c /T )−1 (0c 0c /N)−1 , H −1
j =

V j,N j T (Fj0 F̂j /T )−1 (0j 0j /N j )−1 , and Vc,NT and V j,N j T
satisfy

 
1
N

(y − Xi β̂i − F̂ĝi λ̂ĝi ,i )(yi − Xi β̂i − F̂ĝi λ̂ĝi ,i ) F̂c
NT i=1 i
= F̂cVc,NT ,
⎡ ⎤
Nj
⎣ 1 (yi − Xi β̂i − F̂c λ̂c,i )(yi − Xi β̂i − F̂c λ̂c,i ) ⎦ F̂j
NjT
i;ĝi = j
= F̂jV j,N j T .
The following theorem shows that the estimated group mem-
bership converges to the true group membership as T and N
grow.
Theorem 2. Consistency of the estimator of group membership.
Suppose that the assumptions in Theorem 1 hold. Then, for all
τ > 0 and T, N → ∞, we have
 
 
P  0
sup ĝi − g > 0 = o(1) + o(N/T τ ).
i
i∈{1,...,N}
Theorem 2 implies that if for some b > 0, N/T b → 0, as both
N and T tend to infinity simultaneously, the true group mem-
bership gi0 and the proposed group membership estimator ĝi
are asymptotically equivalent. Theorem 2 is similar to a result
obtained by Bonhomme and Manresa (2015) and Ando and Bai
(2016). But these studies do not allow heterogeneous regression
coefficients.
Next, we establish the variable selection consistency of the
estimated regression coefficients. Let β0i = (βi10  , βi20  ) be the
 
true parameter value, and β̂i = (β̂i1 , β̂i2 ) be the correspond-
ing parameter estimate. Without loss of generality, we assume
that βi20 = 0. We also assume that the dimension of βi10 is small
(uniformly bounded over i) but the dimension of βi20 can be
large. We show that the estimator possesses the sparsity property,
β̂i2 = 0. We denote β̂i1 as the parameter estimate of nonzero true
coefficients βi10 .
Before we provide the theorem that establishes the oracle
property of our estimator, we introduce the following assump-
tion.
Assumption F: Regression Coefficients. Each element of βi10
satisfies
min |βi10,k |/κi → ∞ as T → ∞
for k = 1, . . . , qi with qi being the dimension of βi10 . This
assumption is required for obtaining the variable selection con-
sistency.
Theorem 3. Variable selection consistency. Suppose that the
assumptions A∼F hold. Let β̂i2 be the parameter estimate of
zero true coefficients βi20 . The following variable selection con-
sistency holds:
P(β̂i2 = 0) → 1, N, T → ∞.
Also, the parameter estimate of nonzero true coefficients βi10 ,
β̂i1 , is consistent, βi10 − β̂i1  → 0.
In Theorems 1–3, we assumed that the number of observable
factors is fixed. We note that similar results still hold even when
JOURNAL OF THE AMERICAN STATISTICAL ASSOCIATION 1189
Table . Timeline of the U.S. subprime crisis, summarized from Reuters and Federal
Reserve of St. Louis.
Time Events
Late  The U.S. housing market slows down and delinquency
rates on U.S. subprime mortgages are on the rise.
February  HSBC announces that it must manage bad debts in U.S.
subprime lending portfolios.
April  California’s New Century Financial Corp files for Chapter 
bankruptcy protection.
June  Standard and Poor’s and Moody’s Investor Services
downgrad over  bonds backed by subprime
mortgages.
Two Bear Stearns funds must manage losses after making
bad bets on securities backed by subprime loans.
S&P slashes ratings on some top-rated mortgage bonds by
eight notches.
August  American Home Mortgage Investment Corporation files for
Chapter  bankruptcy protection.
BNP Paribas freezes $. billion worth of funds by citing
subprime problems.
Barclays Bank borrows  million pounds from Bank of
England’s standing lending facility.
October  UBS indicates it will write down $. billion of assets.
Merrill Lynch announces $. billion of losses and
writedowns in CDOs, subprime and leveraged loans in
Q.
Citigroup announces $. billion of losses and writedowns
on subprime-related debt and loans in Q.
November  Citigroup announces a further $- billion of writedowns.
March  JP Morgan acquires Bear Stearns in rescue.
July  Federal Reserve Bank of New York is authorized to lend to
the Federal National Mortgage Association (Fannie Mae)
and the Federal Home Loan Mortgage Corporation
(Freddie Mac).
September Federal Housing Finance Agency (FHFA) places Fannie Mae
 and Freddie Mac in government conservatorship.
Bank of America announces its intent to purchase Merrill
Lynch & Co. for $ billion.
Lehman Brothers files for Chapter  bankruptcy protection.
Federal Reserve Bank of New York is authorized to lend up
to $ billion to American International Group.
FOMC expands/authorizes swap lines with the Bank of
Japan, Bank of England, and Bank of Canada.
FOMC establishes new swap lines with the Reserve Bank of
Australia, the Sveriges Riksbank, the Danmarks
National bank and the Norges Bank.
FOMC increases swap lines with the European Central Bank
and the Swiss National Bank.
FDIC announces that Citigroup will purchase the banking
operations of Wachovia Corporation.
October  The U.S. Congress passes the Emergency Economic
Stabilization Act to establish the $ billion Troubled
Asset Relief Program.
December  The U.S. Treasury purchases preferred stock ($. billion)
from  U.S. banks under the Capital Purchase Program.
May  Freddie Mac reports a first quarter  loss of $. billion.
October  The Dow Jones Industrial Average closes above , for
the first time since October , .
the number of observable factors goes to infinity, as N, T → ∞.
All theoretical proofs relating to these arguments are provided
in Appendix B.
Finally, we must determine the number of groups, S, the
number of group-specific factors, k j ( j = 1, . . . , S), and the size
of the regularization parameters, κ1 ,...,κN . The following theo-
rem justifies the use of our proposed PIC in (4) for selecting
these quantities.
Theorem 4. Consistent model selection. Suppose that
the assumptions of Theorem 3 hold. Moreover, the
difference between the diverging speed of N and T are not
so extreme such that (T + N)/(T N) log(T N ) → 0 and
1190 T. ANDO AND J. BAI

Table . Distributions of the number of listed financial firms. Table . Distributions by market and industry: Whole period: July , , to Novem-
ber , .

Period  Period  Period  Period  Period  Whole

Life Non Life Financial
Amman       Bank Insurance Insurance Services Real Estate
Australian      
Bangkok       Amman     
Berlin       Australian     
BSE Ltd       Bangkok     
Dhaka       Berlin     
Euronext.liffe Paris       BSE Ltd     
Frankfurt       Dhaka     
Hong Kong       Euronext.liffe Paris     
Indonesia       Frankfurt     
Karachi       Hong Kong     
Korea Stock Exchange       Indonesia     
Kuala Lumpur       Karachi     
Kuwait City       Korea Stock Exchange     
London       Kuala Lumpur     
NASDAQ       Kuwait City     
National India       London     
NYSE       NASDAQ     
Non NASDAQ OTC       National India     
PSE       NYSE     
Shanghai       Non NASDAQ OTC     
Shenzen       PSE     
SIX Swiss       Shanghai     
Stuttgart       Shenzen     
Taiwan       SIX Swiss     
Tel Aviv       Stuttgart     
Thailand       Taiwan     
Tokyo Stock Exchange       Tel Aviv     
Toronto       Thailand     
TSX Ventures       Tokyo Stock Exchange     
XETRA       Toronto     
Others       TSX Ventures     
Total       XETRA     
Others     
NOTE: Period : July , , to December , . Period : July , , to December
, . Period : February , , to August , . Period : October , , NOTE: NYSE: New York Stock Exchange; PSE: Philippine Stock Exchange.
to March , . Period : May , , to November , . Whole period: July
, , to November , . NYSE: New York Stock Exchange. PSE: Philippine
Stock Exchange. returns because the worldwide financial markets do not have
the same trading hours. For example, the business hours of the
min{T, N} × (T + N)/(T N) log(T N ) → ∞. Then, the pro- New York Stock Exchange (NYSE) and stock markets in East
posed PIC provides a consistent estimation of the true number Asia (Tokyo, Hong Kong, Shanghai, etc.) do not overlap at all.
of groups, the true number of group-specific factors, and the set Notably, the performance of financial markets in the Asia Pacific
of true observable factors. region may influence the financial markets in North America.
Conversely, the U.S. stock exchange often influences the next
Note that the conditions (T + N)/(T N) log(T N ) → 0 and day’s performance of the Tokyo Stock Exchange, as described in
min{T, N} × (T + N)/(T N) log(T N ) → ∞ are not strong. As Ohno and Ando (2015). Following Forbes and Rigobon (2002),
discussed in Bai and Ng (2002), examples such as N = exp(T ) we calculate stock returns in U.S. dollars. To study the dynamic
or T = exp(N) are rare situations that will violate the condi- characteristics of the worldwide stock market during the sub-
tions. prime financial crisis, we analyze the following five periods, in
addition to the whole period (July 1, 2006, to November 31,
2009).
5. Data
Period 1: July 1, 2006, to December 31, 2006
The data employed in this article cover publicly traded firms Period 2: July 1, 2007, to December 31, 2007
and firms traded at over-the-counter trading markets. The firms Period 3: February 1, 2008, to August 31, 2008
in our dataset belong to the following industries: Banking, Period 4: October 1, 2008, to March 31, 2009
Life Insurance, Nonlife Insurance, Financial Services, and Real Period 5: May 1, 2009, to November 31, 2009
Estate Investment and Services. All data are collected from the
Datastream database, and we followed the industry assignment Based on the information summarized by Reuters and Fed-
for each firm from this database. eral Reserve Bank of St. Louis, Table 1 provides a timeline
We analyze the stock returns of over 6000 firms from more of the U.S. subprime crisis. Longstaff (2010) investigated the
than 100 financial markets. Following Forbes and Rigobon pricing of subprime asset-backed collateralized debt obligations
(2002), market returns are calculated as the rolling average, two- and contagion effects arising from the U.S. subprime market
day returns of each of the firms. We use the two-day average in a worldwide framework. In that study, the sample period is
 JOURNAL OF THE AMERICAN STATISTICAL ASSOCIATION 1191

Figure . Correlation matrix of the set of observable factors (see the text for explanation). Period : July , , to December , . Period : July , , to December ,
. Period : February , , to August , . Period : October , , to March , . Period : May , , to November , .

divided into three distinct periods: the 2006 pre-crisis period,
the 2007 subprime-crisis period, and the 2008 global financial
crisis period. During Period 1, particularly during late 2006, the
U.S. housing markets had peaked, and delinquency rates for sub-
prime mortgages were on the rise, setting up the subprime cri-
sis. Period 1, Period 2, and Period 3 are considered the pre-crisis
period, the subprime-crisis period, and the global financial crisis
period, respectively. As shown in Table 1, conditions worsened
during Period 4, including the Lehman Brothers bankruptcy. So
Period 4 is also a global financial crisis period. In October 2009,
the Dow Jones Industrial Average closed above 10,000 for the
first time since October 3, 2008. Thus, Period 5 contains the
recovery of the U.S. financial markets.
Stocks with missing returns are excluded from our analy-
sis. In addition, stocks with no variation at all were deleted
from our sample. This operation leads to the final sample
for each period as follows, Period 1: N = 6066 firms, Period
2: N = 6100 firms, Period 3: N = 6087 firms, Period 4: N =
6010 firms and Period 5: N = 6003 firms, respectively. For
the whole period, there are N = 5813 firms without missing
values. Table 2 shows the distribution of our sample stocks
across the markets. Financial markets with fewer than 50 stocks
in our sample are merged together and denoted as “Others”
in Table 2. Table 3 presents the distribution of our sample
stocks across markets and industries during the whole period.
The distribution of our sample stocks across markets and
1192 T. ANDO AND J. BAI

Table . Size of each cluster.

Period  Period  Period  Period  Period  Whole

N1      

N2      
N3      
N4      
N5      
N6      
N7      
N8     
N9  
N10  
N11  
N12 
N13 

NOTE: Period : July , , to December , . Period : July , , to December
, . Period : February , , to August , . Period : October , ,
to March , . Period : May , , to November , . Whole period: July
, , to November , .

Figure . The behavior of VC2 as a function of C under the Period . 6. An Empirical Analysis
We estimate the model parameters in (1) by minimizing
the objective function. Then, we apply the proposed model-
industries for the other periods are similar to Table 3 and thus selection criterion, PIC, to simultaneously select the number
omitted. of groups, S, the number of group-specific pervasive factors,
For each of the 31 markets (with more than 50 stocks), we and the size of the regularization parameters, {κ1 , . . . , κN }. We
compute the average return. Then, the lagged average returns set the maximum number of groups to Smax = 30. The possi-
xlag,t = (xlag,1t , xlag,2t , . . . , xlag,31t ) are used as the predictors xit . ble number of group-specific pervasive factors r j ranges from
Also, their interactions (products) are also added to the pre- 0 to 16. Possible candidates for the regularization parameter, κi
dictors. Thus, the dimension of the observable factors xit is i = 1, . . . , N are κi = 101−(k−1)/2 with k = 1, . . . , 11. To deter-
pi = 31 + (31 × 30)/2 = 496, which is more than three times mine the value of C in PIC, we prepared its candidate values
larger than the length of time series T for Period 1–Period 5. as C = 0.1 × k for k = 1, . . . , 20. When we calculate the VC2
Figure 1 shows the correlation matrix of the set of 31 lagged aver- score in (5), we prepared the subsamples of sizes (N (a) , T (a) ) =
age returns xlag,t , and the magnitude of correlation increases as (N − a × 100, T − a × 10) with a = 0, 1, . . . , 5. We then opti-
time passes. In particular, the correlation in Period 4 exhibits the mized the value of C by minimizing VC2 .
highest dependency among the five periods. The magnitudes of Figure 2 shows the behavior of VC2 as a function of C under the
correlation decrease in Period 5. Period 1. Noting that too small C (i.e., C) leads VC2 ≈ 0 (because
the maximum model will be identified) and too large C leads
also VC2 ≈ 0 (because the model with no factor structure will
Table . Selected number of groups, the number of common factors and the num- be identified), Figure 2 indicates that the stable range of VC2 is
ber of group-specific factors.
around C = [0.9, 1.4]. Similarly we can identify the stable range
of VC2 for the other periods. Using the value of C that achieves
Period  Period  Period  Period  Period  Whole
the minimum value of VC2 under the stable range, the proposed
S       criterion PIC selects the best model among the set of candidate
r       models.
r1      
r2      
r3      
r4       6.1. Grouping Results
r5      
r6       The estimated number of groups, the number of common fac-
r7       tors, and the number of group-specific factors for each of the
r8      periods are summarized in Table 4. In this table, the number of
r9  
r10   groups in Period 1 is determined to be S = 7 because it achieved
r11   the smallest value of the proposed model-selection criterion,
r12  PIC, which suggests that there are approximately S = 7 asset
r13 
R1 . . . . . . groups in Period 1. The table shows that the number of groups
R2 is increasing as time goes by. In addition,
. . . . . .
 the total number of
NOTE: Period : July , , to December , . Period : July , , to December
group-specific unobservable factors ( 7j=1 r j ) in Period 1 is
, . Period : February , , to August , . Period : October , , much smaller than that in other periods, which implies that the
to March , . Period : May , , to November , . Whole period: July degree of market heterogeneity has increased during the finan-
, , to November , . R1 and R2 in () measure how much variations left
cial crisis. Thus, investors’ behaviors may be more varied due to
in the error term ε̂it = yit − xit β̂i − f̂ c λ̂c,i − f̂ ĝ λ̂g ,i .
i i the increase in the degree of uncertainty of future events. The
 JOURNAL OF THE AMERICAN STATISTICAL ASSOCIATION 1193

Figure . Distribution of firms in each of the sectors. An (i, j)th element denotes % of firms in industry i such that they belong to jth group. Period : July , , to
December , . Period : July , , to December , . Period : February , , to August , . Period : October , , to March , . Period : May ,
, to November , .

empirical results also show that the number of factors varies where R2 adjusts the contribution by the observable factors.
across groups. There exist one common factors in each period. Table 7 also reports the values of R1 and R2 for each period. It can
To explore the economic meanings of the constructed common be seen that Period 4 has larger R1 values among the five subperi-
factors, we regress the extracted common factor on Thomson ods, while R2 achieves the smallest. This implies that the relative
Reuters Global Financial Index. We found that the factor is relat- explanatory power of unobservable factor structure increased in
ing to the index; the estimated regression coefficients are statis- Period 4. We can also see that R1 and R2 under the whole period
tically significant at the 1% level. is much larger than the other subperiods. One of the possible
We investigated how much variation is left in the error term reasons may be that the number of groups S is changing over
ε̂it = yit − xit β̂i − f̂ c λ̂c,i − f̂ ĝi λ̂ĝi ,i . Here, we reported the fol-
the subperiods, a single common factor structure is too restric-
lowing two ratios: tive for the entire period.
The size of the groups is summarized in Table 5. In Period 1,
Group G7 is subject to a total of two factors. In contrast, Group
N T N T G5 is subject to one group-specific factor. Notably, the size of G5
t=1 (ε̂it ) t=1 (ε̂it )
2 2
R1 = N T
i=1
, and R2 = N Ti=1
, is more than 1,100 and is two times larger than G7. This implies
i=1 y
t=1 it
2
i=1 (y
t=1 it − x β̂
it i ) 2 that Group G7’s degree of heterogeneity is much larger than that
(7) of G5. Such contrast can also be observed in Period 2. Group G3
1194 T. ANDO AND J. BAI

Figure . Distribution of firms in each of the stock exchanges An (i, j)th element denotes % of firms listed in a stock exchange i such that they belong to jth group. Period
: July , , to December , . Period : July , , to December , . Period : February , , to August , . Period : October , , to March , .
Period : May , , to November , .

is subject to a single group-specific factor, while Group G8 is
subject to three group-specific factors.
Because the industry classifications and listed markets are
known, a two-way table of the estimated group membership
ĝi against these classifications is investigated. Figure 3 shows
the distribution of the firms in each sector. An (i, j)th element
denotes % of firms in industry i such that they belong to jth
group. More specifically, let ni, j denote the number of firms that
belongs to sector i and to group j. Then, the (i, j)th element

mi j is calculated as mi j = ni, j /{ Sk=1 ni,k }. In Period 1 and
Period 2, Banking and Insurance (Life, Non-life) sectors seem
to be in large clusters. However, due to the subprime crisis,
stocks in the real estate investment sector diverged more in
Period 3. In particular, we find a large cluster in Period 4,
whereas we found a large cluster in the Banking, Life insur-
ance and Real estate sectors in Period 5. These investigations
imply that the industry factor by itself is an important factor
pre-financial crisis or during the financial crisis, but industry
may not matter after a large shock. This finding, derived form
our general procedure, is a useful insight for institutional
investors.
Figure 4 shows the distribution of the firms in each of the
stock exchanges. An (i, j)th element denotes % of firms listed at
stock exchange i such that they belong to the jth group. We can
thus make the following observations. First, the magnitudes of
similarity between the New York Stock Exchange and the NAS-
DAQ are stable over the periods. In contrast, non-NASDAQ
OTC market exhibits different behavior. This implies that the
 JOURNAL OF THE AMERICAN STATISTICAL ASSOCIATION 1195

Table . Factor risk premiums for the common and group-specific factors.

Period  Period  Period  Period  Period  Whole

fc fg fc fg fc fg fc fg fc fg fc fg

G1 / / / / / / / / / / / /
G2 / / / / / / / / / / / /
G3 / / / / / / / / / / / /
G4 / / / / / / / / / / / /
G5 / / / / / / / / / / / /
G6 / / / / / / / / / / / /
G7 / / / / / / / / / / / /
G8 / / / / / / / / / /
G9 / / / /
G10 / / / /
G11 / / / /
G11 / /

ˆ ν +
For each group,we run the following cross-sectional regression: r̂ j = ν0, j  +  ˆ ν + ξ , ( j = 1, 2, . . . , S). Details on this model are described in Section ..
c, j c, j j g, j j
The number of common and group-specific factors that are priced at the significance level α = 0.05. For integers a∗ and b∗ , the ratio a∗ /b∗ means that a∗ factors are
∗
priced among a set of b factors.

investor’s behaviors at the New York Stock Exchange and the
NASDAQ are different from those at non-NASDAQ OTC mar-
ket. Thus, investors should consider such market characteristics
although all three markets are located in the U.S. Second, an
increase in the dissimilarity of the Shanghai and Shenzen stock
exchanges from the rest of the exchanges is observed after Period
3. Third, the stocks listed on the Tokyo stock exchange tend to be
in the same group, even in Period 1, which indicates that it is dif-
ficult to diversity the portfolio risk that consists purely of stocks
listed on the Tokyo stock exchange. Portfolio managers who are
willing to diversify their portfolio should consider taking posi-
tions in stocks in different exchanges. These results imply that
country and listed markets are important factors to be consid-
ered. However, we also note that these factors are not able to fully
capture the underlying market behavior.
When we compare the distribution of firms in each of the
stock exchanges in Figure 4, Period 4 indicates co-movements
owing to financial crisis because many stocks in different stock
exchanges congregate in the same group G4. In fact, G4 is the
biggest cluster in Table 5, which indicates stronger comovements
as a result of the financial crisis. On the other hand, this phe-
nomenon is relatively much weaker in the other subperiods.
Thus, our estimation result is consistent with the idea that in
crisis co-movements are stronger.
We also implemented Fishers exact test for the independence
between our grouping results and these two nominal classifica-
tions. These two null hypothesis of independence were rejected
for all periods. Although this rejection implies that the our
grouping results are relating to industry and listed exchange,
the nominal classifications are not sufficient to capture the com-
plicated market characteristics. In summary, these investiga-
tions imply that while industry, market, country, and region are
sources of the co-movement of the cross-sectional and time-
series variations in stock returns, they are not the only sources
of the co-movement.
6.2. Price of Risk
In the APT framework, the expected returns on assets are
approximatively linear in their sensitivities to the factors
E[r] = ν0 + γ  ν, where ν0 is a constant, ν is a vector of factor risk
premiums, and γ is a vector of factor sensitivities. In our model,
factor sensitivities correspond to the regression coefficients
βi , and factor loadings λc,i and λgi ,i for i = 1, . . . , N. Here,
we partition the excess returns into the identified groups and
investigate the subset pricing relations based on the Fama and
MacBeth (1973) approach. This subgroup two-stage approach
was also used in Goyal et al. (2008) and Ando and Bai (2015),
for example.
Through the model building process, we have already esti-
mated the matrix of factor sensitivities  ˆ c (common factors),
ˆ
 j (group-specific factors with respect to jth group). Following
Ando and Bai (2015), we then run the following cross-sectional
regression for each group:
ˆ c, j νc, j + 
r̂ j = ν0, j 1 +  ˆ j νg, j + ξ j , ( j = 1, 2, . . . , S),
where 1 is a vector of ones, ξ j is a vector of pricing errors,
ˆ c, j corresponds to the sub-element of  ˆ c associated with
the jth group, r̂ j is a vector of average excess returns, which
are observable-risk adjusted, that is, for the ith security,
T
T −1 t=1 (yit − xit β̂i ) is used. Table 6 reports the results of this
cross-sectional regression. The estimates for the risk premium
on the common and group-specific factors are statistically sig-
nificant in each group. Almost all factors seem to be priced. This
indicates that our method extracted factors that are priced.
One of the main characteristics of the proposed method is
selecting the set of relevant observable factors. Figure 5 pro-
vides the histogram of the percentages (%) of nonzero esti-
mated regression coefficients for each of the observable factors
N
i=1 I(β̂ik = 0)/N for k = 1, . . . , 496. That is, the histogram
is based on 496 values of percentages. Because the histograms
under the Periods 2, 3, and 5 are similar to that under Period 1,
these figures are omitted. We can see that each of the observable
factors are relevant for at most 5% of stocks’ returns for Period 1.
The relevance is further reduced for the whole period, implying
that the sensitivities to the observable factor are likely to be time
varying (because the percentage becomes smaller under longer
time series T ).
1196 T. ANDO AND J. BAI


Figure . Left column: Histogram of the percentages (%) of nonzero estimated regression coefficients for each of the observable factors Ni=1 I(β̂ik = 0)/N for k =
1, . . . , 496. Right column: Histogram of the p-values of price of risk. Period : May , , to December , , Period : September , , to March , , Whole
May , , to December , .

It is also interesting to see whether these selected observable
factors are priced in the cross-section of asset returns. Similar to
the above analysis, we run the following cross-sectional regres-
sion for the observable factors:
ˆ β νβ ,
r̂ = ν0 1 + 
where 1 is a vector of ones,  ˆ β is the matrix of sensitiv-
ities to the observable factors and r̂ is a vector of average
excess returns, which are unobservable common/group-specific
T
factors adjusted, that is for the ith security, T −1 t=1 (yit −
 
f̂ c,t λ̂c,i − f̂ ĝi ,t λ̂ĝi ,i ) is used. Figure 5 also provides the histogram
of the p-values of estimated νβ . Again, the histograms under the
Periods 2, 3, and 5 are similar to that under Period 1, and thus
omitted. When we set the critical level as α = 0.05, more than
50% of observable factors are priced under the Period 1. This
indicates that our method detected relevant observable factors
for explaining stock returns. In contrast, 5% of observable fac-
tors are priced under the whole period. As shown in Figure 5,
our proposed method estimated almost all of regression coef-
ficients as zero. This indicates that most of observable factors
are priced in a short time period, while it may not be true
in the whole sample period. In contrast, the extracted unob-
servable factor structures are priced even for the whole sam-
ple period. Thus, the unobservable factor structure, f c,t λc,i +
f gi ,t λgi ,i , plays an important role in modeling asset returns.

6.3. Robustness Check
Finally, the five periods split the sample into pre-crisis, and
during-crisis subsets. Different specifications can be used for
these subperiods. However, we note that the different sub-period
specifications still lead to similar results.
To treat the differences in international market trading hours,
we used the rolling two-day average of returns. It is also pos-
sible to implement the proposed modeling procedure for daily
returns instead of a two-day rolling average. Again, similar
results are obtained. Thus our results are robust in different sub-
period specifications and in using either daily returns or two-day
averages.
7. Conclusion
This article proposed a novel and a general approach that simul-
taneously implements the following features: (1) detecting a set
of relevant observable factors, (2) extracting unobservable com-
mon and group-specific factors, (3) automatically determining
the number of groups, and (4) clustering a huge number of
assets.
To study the global financial crisis caused by the collapse
of the U.S. subprime mortgage market, we analyzed the daily
stock returns of several industries related to financial services
for over 6000 stocks from more than 70 countries and over 100
financial markets. We found that the number of groups dur-
ing the financial crisis is much larger than during the precrisis
period. We also found that industry, market, country, and region
are sources—but not the only sources—of the co-movement of
cross-sectional and time-series variations in stock returns dur-
ing the financial crisis and that other sources of co-movement
extend beyond these usual classifications.
Although grouping stocks based on nominal classifications
(industry, market, country, and region) is convenient and sim-
ple, the market structure is not simple enough for portfo-
lios to be well diversified based on these nominal classifica-
tions alone. We recommend that investors looking for global
investment opportunities consider diversifying their portfolios
broadly based on our grouping results. Our empirical findings
may be useful for paired trading, valuation of firms, etc.
This article analyzes stocks relating to the financial industry.
Our method is also applicable to stocks of other industries. In
addition, the proposed methods can be applied to the analy-
sis of international bond markets and to the analysis of high-
frequency trading data. Because the proposed method can han-
dle hundreds and thousands of asset returns simultaneously, the
scope of its applicability is wide.
Supplementary Materials
The supplementary materials contain the proofs of all theorems
and additional theoretical results. Simulation results are also
provided.
Acknowledgment
The authors thank the Co-editor, the associate editor and two anonymous
reviewers for constructive and helpful comments that improved the quality
JOURNAL OF THE AMERICAN STATISTICAL ASSOCIATION 1197
of the article considerably. The authors also thank comments from partici-
pants at the 28th Australasian Finance and Banking Conference.
Funding
Ando’s research is supported by Research Grant from Melbourne Business
School, and Bai’s research is supported by the National Science Foundation
(SES1357198)
References
Alessi, L., Barigozzi, M., and Capasso, M. (2010), “Improved Penalization
for Determining The Number of Factors in Approximate Static Factor
Models,” Statistics and Probability Letters, 80, 1806–1813. [1188]
Amengual, D., and Watson, M. W. (2007), “Consistent Estimation of the
Number of Dynamic Factors in a Large N and T Panel,” Journal of Busi-
ness and Economic Statistics, 25, 91–96. [1184]
Ando, T., and Bai, J. (2014), “Selecting the Regularization Parameters in
High-Dimensional Panel Data Models: Consistency and Efficiency,”
Econometric Reviews, forthcoming. [1184]
——— (2015), “Asset Pricing With a General Multifactor Structure,” Journal
of Financial Econometrics, 13, 556–604. [1184,1186,1188,1195]
——— (2016), “Panel Data Models With Grouped Factor Structure Under
Unknown Group Membership,” Journal of Applied Econometrics, 136,
163–191. [1184,1189]
Bai, J. (2009), “Panel Data Models With Interactive Fixed Effects,” Econo-
metrica, 77, 1229–1279. [1184,1185]
Bai, J., and Ng, S. (2002), “Determining the Number of Factors in Approxi-
mate Factor Models,” Econometrica, 70, 191–221. [1186,1187,1190]
Baca, S. P., Garbe, B. L., and Weiss, R. A. (2000), “The Rise of Sector Effects
in Major Equity Markets,” Financial Analysts Journal, 56, 34–40. [1184]
Beckers, S., Connor, G., and Curds, R. (1996), “National Versus Global
Influences on Equity Returns,” Financial Analysts Journal, 52, 31–39.
[1184]
Bonhomme, S., and Manresa, E. (2015), “Grouped Patterns of Heterogene-
ity in Panel Data,” Econometrica, 83, 1147–1184. [1184,1189]
Breheny, P. and Huang, J. (2011), “Coordinate Descent Algorithms for Non-
convex Penalized Regression, With Applications to Biological Feature
Selection,” Annals of Applied Statistics, 5, 232–253. [1187]
Candes, E., and Tao, T. (2007), “The Dantzig Selector: Statistical Estimation
When p is Much Larger Than n,” Annals of Statistics, 35, 2313–2351.
[1184]
Cavaglia, S., Brightman, C., and Aked, M. (2000), “The Increasing Impor-
tance of Industry Factors,” Financial Analysts Journal, 56, 41–54. [1184]
Chamberlain, G., and M. Rothschild., (1983), “Arbitrage, Factor Structure
and Mean-Variance Analysis in Large Asset Markets,” Econometrica,
51, 1305–1324. [1184]
Connor, G., and Korajzcyk, R. (1986), “Performance Measurement With
the Arbitrage Pricing Theory: A New Framework for Analysis,” Journal
of Financial Economics, 15, 373–394. [1186]
Diebold, F., Li, C., and Yue, V. (2008), “Global Yield Curve Dynamics and
Interactions: A Dynamic Nelson-Siegel Approach,” Journal of Econo-
metrics, 146, 315–363. [1184]
Diebold, F. X., and Yilmaz, K. (2014), “On the Network Topology of
Variance Decompositions: Measuring the Connectedness of Financial
Firms,” Journal of Econometrics, 182, 119–134. [1185]
Efron, B., Hastie, T., Johnstone, I., and Tibshirani, R. (2004), “Least Angle
Regression,” Annals of Statistics, 32, 407–499. [1184]
Fama, E. F., and French, K. R. (1993), “Common Factors in the Returns on
Stocks and Bonds,” Journal of Financial Economics, 33, 3–56. [1184]
——— (1998), “Value Versus Growth: The International Evidence,” Journal
of Finance, 53, 1975–1999. [1182,1184]
——— (2012), “Size, Value, and Momentum in International Stock Returns,”
Journal of Finance, 105, 457–472. [1183]
Fama, E. F., and MacBeth, J. D. (1973), “Risk, Return, and Equilibrium:
Empirical Tests,” Journal of Political Economy, 81, 607–636. [1195]
Fan, J., and Li, R. (2001), “Variable Selection via Nonconcave Penalized
Likelihood and Its Oracle Properties,” Journal of the American Statis-
tical Association, 96, 1348–1361. [1184,1186,1188]
1198 T. ANDO AND J. BAI
Fan, J., and Peng, H. (2004), “Nonconcave Penalized Likelihood With
a Diverging Number of Parameters,” Annals of Statistics, 32, 928–
961. [1184]
Fan, J., Liao, Y., and Mincheva, M. (2013), “Large Covariance Estimation
by Thresholding Principal Orthogonal Complements,” Journal of the
Royal Statistical Society, Series B, 75, 603–680. [1187]
Forbes, K. J., and Rigobon, R. (2002), “No Contagion, Only Interdepen-
dence: Measuring Stock Market Comovements,” Journal of Finance, 57,
2223–2261. [1190]
Forgy, E. W. (1965), “Cluster Analysis of Multivariate Data: Efficienty vs.
Interpretability of Classifications,” Biometrics, 21, 768–769. [1187]
Forni, M., Hallin, M., Lippi, M., and Reichlin, L. (2000), “The Generalized
Dynamic Factor Model: Identification and Estimation,” Review of Eco-
nomics and Statistics, 82, 540–554. [1184]
Forni, M., and Lippi, M. (2001), “The Generalized Factor Model: Represen-
tation Theory,” Econometric Theory, 17, 1113–1141. [1184]
Geweke, J. (1977), “The Dynamic Factor Analysis of Economic Time
Series,” in Latent Variables in Socio-Economic Models, eds. D. J.
Aigner, A. S. Goldberger, Amsterdam: North-Holland, pp. 365–
383. [1184]
Griffin, J. M. (2002), “Are the Fama and French Factors Global or Country
Specific?,” Review of Financial Studies, 15, 783–803. [1182,1184]
Griffin, J. M., and Karolyi, G. A. (1998), “Another Look at the Role of the
Industrial Structure of Markets for International Diversification Strate-
gies,” Journal of Financial Economics, 50, 351–373. [1184]
Hallin, M., and R. Liška, R. (2007), “The Generalized Dynamic Factor
Model: Determining the Number of Factors,” Journal of the American
Statistical Association, 102, 603–617. [1188]
——— (2011), “Dynamic Factors in the Presence of Blocks,” Journal of
Econometrics, 163, 29–41. [1183,1184]
Heston, S. L., and Rouwenhorst, K. G. (1994), “Does Industrial Structure
Explain the Benefits of Industrial Diversification,” Journal of Financial
Economics, 36, 3–27. [1182,1184]
——— (1995), “Industry and Country Effects in International Stock
Returns,” Journal of Portfolio Management, 21, 53–58. [1184]
Hou, K., Karolyi, G. A., and Kho, B.-C. (2011), “What Factors Drive
Global Stock Returns,” Review of Financial Studies, 24, 2527–
2574. [1182,1184]
Kose, A., Otrok, C., and Whiteman, C. (2008), “Understanding the Evo-
lution of World Business Cycles,” International Economic Review, 75,
110–130. [1184]
Kuo, W., and Satchell, S. E. (2001), “Global Equity Styles and Indus-
try Effects: The Pre-Eminence of Value Relative to Size,” Journal
of International Financial Markets, Institutions and Money, 11, 1–
28. [1184]
Lam, C., and Fan, J. (2008), “Profile-Kernel Likelihood Inference With
Diverging Number of Parameters,” Annals of Statistics, 36, 2232–2260.
[1184]
Lin, C., and Ng, S. (2012), “Estimation of Panel Data Models With Param-
eter Heterogeneity When Group Membership is Unknown,” Journal of
Econometric Methods, 1, 42–55. [1184]
Longstaff, F. A. (2010), “The Subprime Credit Crisis and Contagion
in Financial Markets,” Journal of Financial Economics, 97, 436–450.
[1184,1190]
Moench, E., and Ng, S. (2011), “A Factor Analysis of Housing Market
Dynamics in the U.S. and the Regions,” Econometrics Journal, 14, 1–24.
[1184]
Moench, E., Ng, S., and Potter, S. (2012), “Dynamic Hierarchical Factor
Models,” Review of Economics and Statistics, forthcoming. Available at
Staff Reports 412, Federal Reserve Bank of New York. [1184]
Ohno, S., and Ando, T. (2015), “Stock Return Predictability: A Factor-
Augmented Predictive Regression System With Shrinkage Method,”
Econometric Reviews, forthcoming. [1190]
Park, T., and Casella, G. (2008), “The Bayesian Lasso,” Journal of the Amer-
ican Statistical Association, 103, 681–686. [1184]
Pesaran, M. H. (2006), “Estimation and Inference in Large Heterogeneous
Panels With a Multifactor Error Structure,” Econometrica, 74, 967–
1012. [1184]
Roll, R. (1992), “Industrial Structure and the Comparative Behavior of
International Stock Market Indices,” Journal of Finance, 47, 3–42.
[1184]
Sargent, T. J., and Sims, C. A. (1977), “Business Cycle Modeling Without
Pretending to Have too Much a Priori Economic Theory,” in New Meth-
ods in Business Cycle Research, eds. C. Sims, et al., Minneapolis, MN:
Federal Reserve Bank of Minneapolis. [1184]
Stock, J. H., and Watson, M. W. (2002), “Forecasting Using Principal Com-
ponents From a Large Number of Observable Factors,” Journal of the
American Statistical Association, 97, 1167–1179. [1186]
Su, L., Shi, Z., and Phillips, P. (2016), “Identifying Latent Structures in Panel
Data,” Econometrica, 84, 2215–2264. [1184]
Sun, Y. X. (2005), “Estimation and Inference in Panel Structure Models,”
Working paper, San Diego: Department of Economics, University of
California. [1184]
Tibshirani, R. (1996), “Regression Shrinkage and Selection via the
Lasso,” Journal of the Royal Statistical Society, Series B, 58, 267–288.
[1184,1186]
Wang, P. (2010), “Large Dimensional Factor Models With a Multi-Level
Factor Structure,” Working paper, Department of Economics, HKUST.
[1183,1184,1185]
Yuan, M., and Lin, Y. (2006), “Model Selection and Estimation in Regression
With Grouped Variables,” Journal of the Royal Statistical Society, B68,
49–67. [1184,1186]
Zhang, C. H. (2010), “Nearly Unbiased Variable Selection Under Minimax
Concave Penalty,” Annals of Statistics, 38, 894–942. [1184,1186]
Zou, H. (2006), “The Adaptive Lasso and Its Oracle Properties,” Journal of
the American Statistical Association, 101, 1418–1429. [1184,1186]
Zou, H., and Hastie, T. (2005), “Regularization and Variable Selection via
the Elastic Net,” Journal of the Royal Statistical Society, Series B, 67,
301–320. [1184,1186]