Broadband and Unemployment: Analysis of Cross-sectional Data for U.S.

Counties
Krishna Jayakar Penn State University Eun-A Park University of New Haven Paper presented at the Telecommunications Policy Research Conference 2013

Abstract This paper examines the effect of broadband availability on the unemployment rate at the county level in the United States. Using data on broadband availability from the National Broadband Map and census data on unemployment for all 3000-plus counties in the United States, an econometric model is estimated that isolates the effect of broadband deployment after controlling for other factors. The results indicate that broadband availability had a significantly reduced county unemployment in 2012, as well as reduced increases in county unemployment over the 2008 to 2012 period. 1. Introduction In 2007-08, a major crisis broke out in the U.S. economy precipitated by a collapse in the housing market, leading to fears of a total failure of the financial system, a liquidity crisis and widespread unemployment. Though fears of financial collapse have more or less abated, and the housing and stock markets have more or less recovered, unemployment continues to be a persistent problem. From approximately 4.7% in January 2007 before the crisis, the national unemployment rate peaked at 10% in October 2009; though it has recovered somewhat since then, the rate remains at 7.4% in July 2013, at the time of this writing. 1 Job creation remains

Bureau of Labor Statistics [BLS], Labor Force Statistics from the Current Population Survey. (Washington, DC: U.S. Department of Labor, 2013). http://data.bls.gov/timeseries/LNS14000000

TPRC, 2013, p. 1

anemic, and the unemployment rate for many demographic groups, such as African-American men, remains unconscionably high. Seeking a path out of the financial crisis, Congress in February, 2009, passed the American Recovery and Reinvestment Act (ARRA), with three broad goals: create new jobs and save existing ones, spur economic activity and invest in long-term growth; and foster accountability and transparency in government spending. Significant components of ARRA were the Broadband Technology Opportunities Program (BTOP) and the Broadband Initiatives Program (BIP), both aiming to increase broadband deployment in the United States. The BTOP made available grants for deploying broadband infrastructure in unserved and underserved areas in the United States, enhancing broadband capabilities at public computer centers, and promoting sustainable broadband adoption projects, 2 while the BIP extend[ed] loans, grants and loan/grant combinations to facilitate broadband deployment in rural areas. 3 The National Telecommunications and Information Administration (NTIA), and the Department of Agriculture's Rural Utilities Service (RUS), entrusted with the implementation of BTOP and BIP respectively, completed the identification of grant recipients by September 2010.4 Congresss commitment of ARRA funds to broadband infrastructure investments is supported by an extensive academic literature identifying a positive relationship between telecommunications, and specifically broadband deployment, and economic activity and growth.5

Federal Communications Commission [FCC], Connecting America: The national broadband plan (Washington, D.C.: FCC, 2010), 139. http://download.broadband.gov/plan/national-broadband-plan.pdf 3 Ibid, 139. 4 National Telecommunications and Information Administration (NTIA), The Broadband Technology Opportunities Program: Expanding broadband access and adoption in communities across America: Overview of grant awards (Washington, DC: NTIA, 2010). http://www.ntia.doc.gov/report/2010/expanding-broadband-access-and-adoptioncommunities-across-america-overview-grant-awards 5 Robert Crandall, William Lehr, and Robert Litan, The effects of broadband deployment on output and employment: A cross-sectional analysis of U.S. data, Issues in Economic Policy No.5 (Washington, DC: The Brookings Institution: 2007), http://www.brookings.edu/papers/2007/06labor_crandall.aspx; Jed Kolko, Does broadband boost local economic development? (San Francisco, CA: Public Policy Institute of California: 2010),

TPRC, 2013, p. 2

Based on this literature, broadband deployment may reasonably be expected to advance the second of ARRAs three goals, namely spurring economic activity and long term growth. However, the evidence that broadband deployment will help with ARRAs first goal of creating new jobs and saving existing ones is much more tenuous and contradictory. On the one hand, the increased economic activity resulting from broadband deployment may create new jobs. On the other hand, broadband technologies may improve productivity reducing the need for labor. One researcher, for example, has shown that though broadband contributes to the creation of jobs, it may not significantly affect the unemployment rate because workers are mobile, and relocate or commute to places with higher labor demand.6 On the contrary, others have found that broadband deployment has a significant effect on the employment rate, but the effect is stronger in counties with a higher percentage of college graduates and in industries that employ a more educated workforce.7 The effect of broadband availability on employment is an unresolved question, which requires further investigation. This paper seeks to analyze this question. Past attempts at investigating the relationship between broadband and economic growth were complicated by the lack of adequate data on broadband deployment and availability. Previously accessible data on broadband availability, such as the FCCs ZIP Code-based broadband availability data have been heavily criticized.8 Others used the 2007 Computer and

http://www.ppic.org/main/publication.asp?i=866; Pantelis Koutroumpis, The economic impact of broadband on growth: A simultaneous approach. Telecommunications Policy 33 (2009): 471 -485; Hal Singer and Jeffrey West, Economic effects of broadband infrastructure deployment and tax incentives for broadband deployment. (Dallas, TX: Fiber To The Home Council: 2010), http://www.neofiber.net/Articles/Economic_Effects_of_FTTH.pdf 6 Kolko, Does broadband boost local economic development 7 Hilal Atasoy, The effects of broadband internet expansion on labor market outcomes, 2011. http://papers.ssrn.com/sol3/papers.cfm?abstract_id=1890709 8 Tony Grubesic, Spatial data constraints: Implications for measuring broadband, Telecommunications Policy 32 (2008): 490-502; Kolko, Does broadband boost local economic development; James Preiger and Wei-Min Hu, The broadband digital divide and the nexus of race, competition, and quality, Information Economics and Policy 20, no. 2 (2008): 150-167.

TPRC, 2013, p. 3

Internet Use Supplement of the Census Bureaus Current Population Survey,9 but this data too is more than five years old and out-of-date. In this context, the newly available National Broadband Map (NBM) data present an attractive opportunity to test the relationship between broadband availability and employment. Authorized by Congress through ARRA,10 the NTIA created the NBM in collaboration with the FCC, using data reported by broadband providers in all 50 states and the District of Columbia. The first publicly searchable database of about 25 million records was made available in February 2011, with updates every six months: the latest installment is dated to December 2012. Though researchers have pointed out problems such as incomplete information and lack of pricing data,11 and measurement errors and selection bias,12 the database remains a major advance over previously available sources. Thus, the NBM provides researchers an unprecedented opportunity to test out the relationship between broadband deployment and economic growth and employment generation. In this paper, we test the hypothesis that counties with better broadband availability, will have lower unemployment rates and smaller increases in unemployment over the period of the crisis. Our analysis will therefore help to evaluate the effectiveness of broadband investments is achieving one of ARRAs major policy goals, namely employment generation. The results will enable policymakers to make or reject, depending on the results, the economic case for continued investments in broadband deployment. In view of the contradictory findings in the literature, this paper has the theoretical rationale of empirically testing the hypothesized relationship between broadband deployment and economic growth and employment generation.
9

T. Randolph Beard, George Ford and Richard Saba, Internet use and job search, Phoenix Center Policy Bulletin No. 39. (Phoenix Center: Washington, DC, 2010). 10 American Reinvestment and Recovery Act, Title VI, Section 6001(l) 11 Tony Grubesic, The U.S. National Broadband Map: Data limitations and implications, Telecommunications Policy 36 (2012): 113-126. 12 George Ford, Challenges in using the National Broadband Maps data. Phoenix Center Policy Bulletin No. 27. (Phoenix Center: Washington, DC, 2011).

TPRC, 2013, p. 4

The paper structured in this manner. In the section following immediately below, we examine the literature on telecommunications and broadband deployment on economic growth and employment generation and frame the hypotheses to be tested based on the literature. Thereafter, we describe our data sources and sampling methodology. Section Four presents statistical analyses, followed by conclusions and discussion in the last Section Five.

2. Broadband deployment, economic growth and employment The relationship between telecommunications and economic growth has been long recognized in the economics literature, ever since Jipps pioneering work found a positive correlation between telephone density in a country and per capita Gross Domestic Product (GDP).13 The relationship was subsequently confirmed by other scholars.14 By substituting for other production input and reducing transaction costs, telecommunications contributes to economic growth. Growth in turn makes more investment capital available for

telecommunications development and also contributes to demand by increasing household income. Several econometric studies of network penetration have used economic growth (percentage change in GDP) to predict teledensity, and vice versa. 15 Others use variations of GDP or GDP per capita: for example, statewide average personal disposable income.16 Though these studies have established a correlation between telecommunications and economic growth, a causal connection has been harder to prove due to simultaneity bias and

13 14

A. Jipp, Wealth of nations and telephone density, Telecommunications Journal ( 1963): 199-201. Raymond Duch, Privatizing the economy: Telecommunications policy in comparative perspective. (Ann Arbor, MI: University of Michigan Press, 1991); Andrew P. Hardy, The role of the telephone in economic development, Telecommunications Policy 4 (1980): 278-286; Robert J. Saunders, Jeremy Warford and Bjorn Wellenius, Telecommunications and economic development (Baltimore, MD: Johns Hopkins University Press, 1994). 15 James Foreman-Peck, Competition and performance in the U.K. telecommunications industry, Telecommunications Policy 9 (1985): 215-228; Scott J. Wallsten, An econometric analysis of telecom competition, privatization and regulation in Africa and Latin America, Journal of Industrial Economics 49, no. 1 (2001): 1-19. 16 Brooks Albery, What level of dialtone penetration constitutes universal service? Telecommunications Policy, 19 (1995): 365-380.

TPRC, 2013, p. 5

spurious correlation.17 More recent studies have utilized a number of econometric techniques to eliminate many of these problems. 18 For example, Cronin and his colleagues showed that appropriately lagged telecommunications penetration variables are causally linked to economic growth, and vice versa.19 The presence of a correlation between telecommunications deployment and economic growth is now taken virtually for granted. A similar relationship between telecommunications or broadband deployment and employment has been more difficult to establish. If some authors found significant positive effects,20 others found non-significant or even negative outcomes.21 Among the studies that support a significant positive relationship, one found that a 1 percent increase in broadband penetration will increase private, nonfarm employment by 293,200 jobs annually, based on analysis of state-level data for three years, 2003-2005. 22 Using broadband penetration per capita as the main independent variable, and with a number of controls in place (state tax levels, union membership, education level, hourly wages, mean temperature), they estimated the year-over-year growth rates in private nonfarm employment and
17 18

Kotroumpis. Nina Czernich, Oliver Falck, Tobias Kretschmer, and Ludger Woessmann, Broadband infrastructure and economic growth. CESIFO Working Paper No. 2861, Category 6: Fiscal Policy, Macroeconomics and Growth. (Munich, Germany: CESIFO: 2009). http://www.CESifo-group.org/wp; Anusua Datta, and Sumit Agarwal, Telecommunications and economic growth: A panel data approach, Applied Eco nomics 36, no. 15, (2004): 16491654; Kolko, Does broadband boost local economic development; Gary Madden, and Scott Savage, CEE telecommunications investment and economic growth Information Economics and Policy 10, (1998): 173-195. 19 Francis Cronin, Edwin Parker, Elizabeth Colleran, and Mark Gold, Telecommunications infrastructure and economic growth: An analysis of causality, Telecommunications Policy 15, (1991): 529 -535; Francis Cronin, Edwin Parker, Elizabeth Colleran, and Mark Gold, Telecommunications infrastructure investments and economic development, Telecommunications Policy 17, (1993): 415 -430. 20 Robert Crandall, William Lehr, and Robert Litan, The effects of broadband deployment on output and employment: A cross-sectional analysis of U.S. data. Issues in Economic Policy No.5 (Washington, DC: The Brookings Institution, 2007). http://www.brookings.edu/papers/2007/06labor_crandall.aspx; Singer and West; Krishna Jayakar and Eun-A Park, Broadband availability and employment: An analysis of county level data from the National Broadband Map. Journal of Information Policy, vol. 3 (2013): 181-200. 21 Martin Fornefeld, Gilles Delaunay, Dieter Elixmann, The impact of broadband on growth and productivity . (Brussels, Belgium: European Commission, Directorate General of Information, Media and Society, 2008). http://breitbandinitiative.de/wp/wp-content/uploads/2009/04/2008_micus-studie-broadbandeu_long.pdf; David Shideler, Narine Badasyan, and Laura Taylor, The economic impact of broadband deployment in Kentucky, Federal Reserve Bank of St. Louis Regional Economic Development 3, no. 2 (2007): 88-118. 22 Crandall, Lehr and Litan.

TPRC, 2013, p. 6

state gross domestic product.23 The results for employment were positive and significant, while those for GDP were positive but not significant. Interestingly, these researchers also reported results disaggregated by industry sectors (2-digit SIC codes).24 Broadband penetration affected employment growth positively and significantly for manufacturing; finance and insurance; educational services; healthcare and social assistance; and accommodation and food services. The effect on GDP growth was positive and significant for finance and insurance; real estate rental and leasing; administrative support, waste management and remedial services; educational services; and other services (SIC code 81). Only for finance and insurance and educational services was the effect positive and significant for both employment growth and GDP growth. Also, interestingly, coefficients were negative for a few industrial sectors: management of companies and enterprises; and arts, entertainment and recreation. Another study commissioned by the industry group Fiber-To-The-Home North America25 found that if current-generation broadband access26 were made available to all households in the United States by 2015 under a national broadband plan, it would add $38 billion in output to annual GDP, and create almost 40,000 jobs per year. If instead, next-generation broadband networks27 were deployed to 80 percent of homes by 2015, the total incremental gains would be $198 billion in annual output, and more than 250,000 jobs per year. Singer and West estimate these two alternative plans to cost additional investments of $14 billion and $63 billion beyond

23 24

Ibid. Ibid. 25 Hal Singer and Jeffrey West, Economic Effects of Broadband Infrastructure Deployment and Tax Incentives for Broadband Deployment, white paper, Fiber-To-The-Home Council, Mar. 2, 2010, accessed Apr. 23, 2013, http://www.neofiber.net/Articles/Economic_Effects_of_FTTH.pdf. 26 Ibid, 3. Current generation broadband access is defined as (3 megabits per second (Mbps) downstream, 768 kilobits per second (Kbps) upstream). 27 Ibid., 3. Next generation broadband networks are expected to deliver peakperiod speeds of at least 50 Mbps downstream and 20 Mbps upstream.

TPRC, 2013, p. 7

the projected investments expected without a national broadband plan. 28 In arriving at these estimates, the authors calculated both direct and indirect impacts. Direct impacts involve the number of jobs that will be created through increased capital investments in equipment, and through increased employment for installation, etc. Indirect effects were expected to be realized through increased productivity as a consequence of broadband deployment, and increased demand for other goods and services. Using the first installment of NBM data (February 2011), Jayakar and Park studied the effect of broadband deployment on county-level employment in eight states, utilizing methods similar to those proposed for this paper. 29 Combining broadband data from the NBM and unemployment statistics from the Bureau of Labor Statistics, econometric models were run that controlled for the effects of other variables such as education, regional effects and the presence of minorities. The paper found a small but significant positive effect of broadband availability on county employmentit concluded that increasing the national average of households with at least 3 Mbps in download speed (that stood at 93% at the time the research was conducted) to 100% through broadband investments would reduce unemployment by approximately 0.49%. However, a problem with this research paper was that the authors used a restricted set of states chosen to represent rich and poor states by per capita GDP and urban and rural states, in a 2x2 matrix. Two states were chosen from each cell of the matrix (rich-urban states, rich-rural states, poor-urban states and poor-rural states) for a sample of eight states. It is possible that some of these states (e.g. Alaska) were not truly representative of the group they were chosen to represent

28

Ibid., 10. Normal capital expenditures over the 5 -year period (without any national broadband plan) are expected to be $37 billion in current generation networks, and $11 billion in next generation networks. A national broadband plan to extend current generation broadband networks to all households is expected to cost $51 billion (an increment over projections of $14 billion); a target of 80 percent penetration for next generation networks will cost a total capital expenditure of $74 billion (an increment of $63 billion over projections). 29 Jayakar and Park.

TPRC, 2013, p. 8

(rich, rural states). Therefore, the present study proposes to replicate the same method used by Jayakar and Park, but utilizing data from all 3000-odd counties in the United States. The mostly optimistic findings cited above are not uniformly supported by other studies. Some have argued that ICT-enabled productivity gains may actually lead to job losses in the short term.30 A company improves its processes to increase its employees efficiency. After the change has been made the company is able to produce more with the same personnel, or produce the same with fewer personnel. 31 However, over the long term, as the benefits of process improvements percolate through the economy, the demand for labor may increase. Other studies have found that broadband deployment resulted in overall employment growth, but the impact on specific industrial sectors varied.32 One study found that broadband deployment has a significant effect on the employment rate, but the effect is stronger in counties with a higher percentage of college graduates and in industries that employ a more educated workforce.33 Similarly complex relationships between broadband deployment and employment growth and GDP growth have been reported by others as well.34 Unlike other studies that use data at the state level, these authors used census-block level data aggregated to the county level, generated from the Connect Kentucky program that sought to encourage full broadband deployment in Kentucky by the year 2007: their data therefore was limited to one state. Their regressions examined the impact of broadband penetration on employment growth in 20 industrial sectors (by 2-digit NAICS codes). They found that broadband had a positive and significant impact on total employment (across all sectors) and on employment in mining, construction, information, and administrative support, waste management and remedial services. Positive and significant
30 31

Fornefeld, Delaunay, and Elixmann. Ibid, 96. 32 Crandall, Lehr and Litan; Shideler, Badasyan and Taylor. 33 Atasoy. 34 Shideler, Badasyan and Taylor.

TPRC, 2013, p. 9

coefficients were observed in some models (but not all) for arts, entertainment and recreation; real estate rental and leasing; and other services. Positive but non-significant coefficients were observed for retail trade; professional, scientific and technical services; and healthcare and social assistance. Negative and significant coefficients were observed for only one industrial sector, accommodation and food services. A review of the available literature allows some interesting points to be made. First, broadband deployment affects different industrial sectors differently. At least in some limited cases, there is potential for job losses as well. Researchers have speculated that this may be the result of substituting broadband technologies for less productive workers.
35

Another

explanation put forward is that, in sectors such as travel, accommodations and food service, broadband has allowed the creation of new online services such as hotel bookings and air reservations, which replace traditional in-person services.36 Broadband also makes it possible to replace local jobs with identical services provided by personnel from across the county, state or even national borders (e.g. call centers). While the net impact of broadband on job creation is likely to be positive, the specific impact might be a redistribution of some jobs between industrial sectors, or sometimes the loss of jobs to distant locations. Scholars have discussed the different processes by which firms may realize efficiencies through optimizing the locus of their production activities. 37 Outsourcing occurs when a company entrusts part of its production or distribution activities to another company, usually but not always based in the same geographical location. The incentives to seek maximal cost efficiencies by going farther and farther afield may be increased because of the availability of broadband. Off-shoring thus involves a companys decision to move some activity to another
35 36

Shideler, Badasyan and Taylor, 117. Ibid. 37 Fornefeld, Delaunay and Elixmann.

TPRC, 2013, p. 10

country with a labor cost advantage. The off-shore facility may still be owned by the parent company, in which the operation is off-shored but not outsourced. Business-process outsourcing (BPO) involves the outsourcing of some of a companys knowledge-intensive activities, such as CRM, legal services, or accounting to another company. Originally intended to achieve cost advantages, BPO activities are now pursued to secure competencies for a company that it cannot or does not want, to make available internally. All these processes are strongly related to the availability of high-quality broadband connectivity and advanced telecommunications services. While firms may gain productivity from these processes, the impact on job creation is not always positive. Thus, the consensus from the studies on productivity and employment seems to be that the effect of productivity growth on job creation is a function of the nature of a firms economic activity, as well as local economic conditions including the educational level of the local community. However, research has indicated that even if an industrial sector were to lose jobs as a result of productivity gains, wage rates are likely to improve for the remaining workers. 38 The long run prospects are that that industrial sector will emerge with better-educated and higher paid workers, and a more knowledge-intensive production mode. Based on this review of the literature, a number of hypotheses were framed as given below, to be tested with data from the NBM. But before proceeding further, we have to justify our choice of the unit of analysis. Several choices for unit of analysis were available in the literature, ranging from the state, down to county and census block. In this paper, hypotheses were framed with the county as the unit of analysis, since it is small enough to capture local variations but large enough to constitute a reasonable labor market. One study using network analysis utilized data on in-commuting
38

Shideler, Badasyan and Taylor.

TPRC, 2013, p. 11

(workers commuting into a county) and out-commuting (workers commuting out of a county) to identify the limits of labor markets.39 It found that relatively few counties, notably those in the New York City; Washington, DC; Atlanta; and Minneapolis-St. Paul areas, had very high levels of in-commuting with a corresponding number of out-commuting counties located in the peripheries of these urban agglomerations. Outside of these regions, and especially in the west and the midwest, the incidence of in- and out-commuting is very low, suggesting that the county constitutes a reasonable boundary of the labor market. The county as unit of analysis is a common choice in the literature, with several previous studies of unemployment being based at the county level.40 Finally, the data we collected from the U.S. Bureau of Labor Statistics show that there are wide variations across states in unemployment rates. Aggregating to the state level ignores these regional variations in employment. First, we investigate the effect of broadband deployment on the unemployment rate. As discussed previously, there is no consensus on this effect in the literature, with some researchers finding a significant positive effect 41 and others a non-significant or even negative effect. 42 Accordingly, the following hypothesis is framed. H1: Broadband availability has no significant effect on county unemployment rate. In addition to the unemployment rate at a point in time, it is also worth investigating the dynamic effect of broadband deployment on unemployment changes during a recession. It might be argued that irrespective of initial conditions, broadband deployment may enable a location to mitigate the employment effects of a recession; on the other hand, broadband (and ICTs in
39

. Stephan J Goetz, Yicheol Han, Jill L. Findeis, and Kathryn J. Brasier, U.S. commuting networks and economic growth: measure and implications for spatial policy, Growth and Change 41(2) (2010): 276 -302. 40 Cronin, Parker, Colleran, and Gold, 1993; Xiaobing Shuai, Who benefits from job creation at county level? An analysis of leakage and spillover of new employment opportunities in Virginia, Business Economics 45 (2010): 38 48; Dustin L. Sweet, Forecasting county-level unemployment accounting for spatial correlation, Doctoral dissertation (2011), University of Missouri, Columbia. 41 Crandall, Lehr and Litan; Singer and West. 42 Fornefeld, Delaunay, and Elixmann; Shideler, Badasyan and Taylor.

TPRC, 2013, p. 12

general) may enable firms to realize productivity gains, allowing them to reduce employment during economic downturns.43 A second null hypothesis is therefore framed as follows. H2: Broadband availability has no significant effect on change in county unemployment rates over a specific time period.
In the next section, we discuss the sources of the data used to test these hypotheses.

3. Data The latest available data, from December 2012, on broadband availability, connection speeds, and competition were obtained from the NBM database for all 3000-plus counties in the United States. For each location, broadband availability data was collected from the National Broadband Map, including data on percentage households with at least 6 Mbps download speed available, the percentage households with a combination of at least 3 mbps download and 768 kbps upload speeds available; percentage of households served by wireline broadband technologies; and the percentage of households with at least two wireline providers available. This data was cross-tallied with data on the total workforce and the persons employed at two points in time, in December 2008 and December 2012, from the Local Area Unemployment Statistics (LAUS) database of the Bureau of Labor Statistics (BLS). The county employment rate (and from it, the unemployment rate) was calculated as the percentage of persons employed in the total workforce. 44 The change in unemployment was then calculated over 2008-2012. To investigate whether variations between states in minimum wage standards or unemployment benefits had an effect on county unemployment rates, state minimum wages were

43 44

Ibid. According to the BLS, the total civilian labor force represents the subset of Americans who have jobs or are seeking a job, are at least 16 years old, are not serving in the military and are not institutionalized. In other words, the civilian labor force includes all Americans who are eligible to work in the everyday U.S. economy. See BLS Website, http://www.bls.gov/dolfaq/bls_ques23.htm.

TPRC, 2013, p. 13

collected from the database maintained by the Labor Law Center, 45 and state unemployment benefits from Fileunemployment.org. 46 Higher minimum wages are a disincentive for job creation leading to the expectation of a positive correlation with county unemployment, while higher unemployment benefits too are expected to be positively correlated with unemployment by reducing at the margin the incentives to seek work. Finally, it may be argued that construction activity will have a major impact on unemployment because this was the primary sector in which the 2008 economic crisis originated. However, Jayakar and Park tested the impact of changes in total construction spending at the county level, 2007-10 on the unemployment rate and found that the variable had no effect on the unemployment rate.47 Though the crisis originated in the construction sector, the effects on employment were far more widespread. Therefore, we did not include construction spending as a predictor in this paper. To control for the positive effect of education on employment found by some researchers, 48 we also collected census data on education (the percentage of the county workforce with a high school diploma). Other controls included the percentage of minorities in the county population, expected to be correlated positively with unemployment, and total area of the county as a proxy for the urbanization of the county. Finally, there is a potential positive correlation between county unemployment rates and unemployment rates in the region; to account for this, we included the average annual unemployment rate in the state as a control. The full set of independent and dependent variables is reported in Table 1, with summary statistics.

45 46

http://www.laborlawcenter.com/t-State-Minimum-Wage-Rates.aspx http://fileunemployment.org/unemployment-benefits-comparison-by-state 47 Jayakar and Park 48 Atasoy

TPRC, 2013, p. 14

Table 1: Variables and Descriptive Statistics

DVs IVs Variables County avg. annual unemployment rate, 2012 Increase in unemployment rate, 2008-2012 County avg. annual unemployment rate, 2008 Percent households with at least 3 mbps download speed and 768 kbps upload speed available Percent households with at least 6 mbps download speed available Percentage households with at least 1 Gbps download speed available Percentage households with any wireline technology available (DSL, ADSL, sDSL, other copper wireline, Cable, Cable DOCSIS 3.0, other cable, Optical Fiber, or BPL) Percentage households with fiber available Percentage of households with at least 2 wireline providers available Percentage of county population aged 20-59 Percentage of minorities Median Income Percentage with high school diplomas Population in the county State minimum wage, 2012 Unemployment benefits State annual unemployment rate, 2012 Unit % % % % % % Variable Name UNEMPLOY_2012 UNEMP_CHANGE UNEMPLOY_2008 SPEEDCOMBO SPEED_DL_6 DOWNLOAD_1GB N 3142 3143 3140 3140 3143 3143 Mean 7.71 1.93 5.78 89.72 81.12 2.83 Std. Dev. 2.77 1.36 2.10 16.95 23.11 11.94

% % % % % $ % No. $ $ %

TECHWIRE FIBER PROVIDER ADULT MINORITY MED_INCOME EDUC_HS POPULATION MIN_WAGE UNEMP_BENE STATE_UNEMP

3143 3143 3143 3143 3143 3143 3143 3143 3143 3143 3143

86.40 .12 59.09 50.69 16.16 46,640 77.30 100,087 7.37 454.64 6.89

15.13 .23 30.45 5.24 18.49 12,541 8.89 318045 .41 106.15 1.54

As a first step in the data analysis procedure, Pearsons correlation coefficients were calculated between all the independent and dependent variables. Table 2 reports partial results of the correlation analysis (the correlations between the broadband deployment variables are not included in the interest of space). The first dependent variable, county unemployment rate (UNEMPLOY_2012), was significantly positively correlated with the 2008 county unemployment rate, the percentage of minorities in the population, and the 2012 state unemployment rate. It was negatively correlated with broadband availability (SPEEDCOMBO), median income

(MED_INCOME), and education (the percentage of high school graduates, EDUC_HS).

TPRC, 2013, p. 15

Table 3: Bivariate Correlations

UNEM PLOY_ 2012 UNEMPLOY_ 2008 SPEED COMBO SPEED _DL_6 SPEED _DL_1G TECHWIRE UNEM PLOY_ 2008 SPEED COMBO SPEED _DL_6 SPEED_DL _1G TECH WIRE FIBER PROVI DER ADULT MINOR ITY MED_I NCOME EDUC_ HS POPUL ATION MIN _WAGE STATE _UNEMP

.887** -.056** -.032 -.104** -.084** -.158** .043* .302** .371** -.265** -.408** .053** .213** .626** -.142**

1 -.064** -.052** -.100** -.093** -.186** .018 .240** .262** -.315** -.393** .010 .152** .593** -.099** 1 .708** .073** .545** .115** .514** .218** -.012 .234** .178** .158** -.010 .070** .021 1 .085** .637** .185** .601** .258** -.052** .334** .273** .206** .046* .117** .067** 1 .101** .494** .166** -.078** -.051** .029 .081** .010 .125** -.169** -.036* 1 .254** .599** .166** -.052** .278** .241** .207** -.021 .060** .043* .183** -.067** -.017 .210** .179** .121** .041* -.159** .032 1 .372** .072** .383** .230** .293** .060** .193** .082** 1 .206** .133** -.033 .161** .056** .377** -.032 1 -.050** -.405** .212** .005 .174** -.177** 1 .584** .317** .107** -.049** .217** 1 .103** .126** -.261** .198** 1 .120** .115** .054** 1 .273** .138** 1 -.123**

FIBER

PROVIDER

ADULT

MINORITY MED _INCOME EDUC_HS POPULATION MIN _WAGE STATE _UNEMP UNEMP _BENE

Note: Shaded cells indicate high correlations, |r| > 0.5; ** = significant at p<0.01; * = significant at p<0.05.

In addition, it can be seen that SPEEDCOMBO was highly correlated with SPEED_DL_6, TECHWIRE, and PROVIDER. A rule of thumb is that a value of 0.7 or above for the Pearsons r statistic indicates a very strong positive relationship between two variables.49 Thus, for example, SPEEDCOMBO may be considered to be strongly associated with SPEED_DL_6, r = .708.

49

Dennis E. Hinkle, William Wiersma, and Stephen D. Jurs. Applied statistics for the behavioral sciences. (Boston, Mass: Houghton Mifflin, 2003).

TPRC, 2013, p. 16

Results Using data from the NBM, BLS, and the Census Bureaus Censtat database, the hypotheses identified in the previous Section 2 were tested using regression analysis. As a preparatory step to regression analysis, the assumptions of normality, absence of outliers, and absence of multicollinearity were checked. Normality checking indicated that MIN_WAGE was not normal, and it was transformed to convert the extremely positively skewed data to normality. But, since linear regression analysis is fairly robust against deviations from normality and the other variables presented less serious problems, it was decided to proceed with the analysis without modifying any of the other variables. To determine whether there are any outliers that were influential, the Cook's Distance statistic was checked and suggested no problem. 50 Multicollinearity, which results when two or more independent variables are highly correlated with each other, was examined by inspecting correlation coefficients. The high correlations between some broadband availability variables reported in Table 2 suggest that multicollinearity might result if these variables are simultaneously included in regression models. In the presence of multicollinearity, coefficient estimates are less precise and standard errors are inflated, but the results are still unbiased if the models are correctly specified.51 To avoid potential problems, it was decided to retain only one of the broadband deployment variables, SPEEDCOMBO in further analysis and other variables strongly correlated with SPEEDCOMBO, such as SPEED_DL_6, TECHWIRE, PROVIDER, were not included in the regression models. Once the Ordinary Least Squares (OLS) regressions were run, the results were further checked for independence of errors, homoscedasticity, and normality of residuals. A Durbin-

50

As a rule of thumb, if the Cook's Distance values are above 1, they should be investigated (Cook & Weisberg, 1999). 51 Ranjit Paul, Multicollinearity: causes, effects and remedies. http://www.iasri.res.in/seminar/AS-299/ebooks /2005-2006/Msc/trim2/3.%20Multicollinearity-%20Causes,Effects%20and%20Remedies-Ranjit.pdf

TPRC, 2013, p. 17

Watson statistic of 2.00 indicated independence of errors. Homoscedasticity was checked by plotting the studentized residuals against the unstandardized predicted values. Tests of multicollinearity, such as tolerance and variance influence factor (VIF), were run and indicated no significant problem, since all indicators of broadband availability other than SPEEDCOMBO had been deleted from the regression models.52 The first hypothesis states that after controlling for other factors, broadband deployment will have no significant impact on county unemployment rate in 2012. To test this hypothesis while controlling for other influential factors (percentage of working age population, percentage of minorities, education, the state minimum wage, total unemployment benefits, the county unemployment rate in 2008 and the state unemployment rate in 2012), multiple regression analysis was conducted. The dependent variable was the 2012 county unemployment rate, which was regressed on broadband availability, along with the aforementioned control variables. We began with a full model and proceeded by stepwise backward elimination, dropping the coefficient with the lowest t-statistic in each round (Table 3). Ideally, stepwise procedures are not the best method for model selection, but as Derksen and Keselmans widely cited study has argued, it might be acceptable in situations where theory does not offer firm guidance on variable selection, or when data are too difficult/expensive to collect, or when researchers seek to identify the most parsimonious explanation for the phenomenon under consideration. 53 Since both reasons one and three appear to fit the present circumstances, we rely on stepwise procedures. Greene however cautions against the use of forward selection approaches because they may be

52

If the Tolerance value is less than 0.1 - which is a VIF value greater than 10 - there is a collinearity problem. In this data, all the Tolerance values were greater than 0.1 (the lowest was 0.369), so it may be stated with reasonable confidence that collinearity was not a problem in this dataset. 53 . Shelley Derksen and H. J. Keselman, Backward, forward and stepwise automated subset selection algorithms: Frequency of obtaining authentic and noise variables, British Journal of Mathematical and Statistical Psychology 45 (1992): 265-282.

TPRC, 2013, p. 18

subject to biases caused by the incomplete specification in the early steps when there are only a few included variables.54 The more attractive strategy according to Greene is the general-tosimple approach in which one begins with a full model and progressively eliminates variables. Greene however cautions that this strategy too is subject to Type 1 errors when the number of variables in the full model is very largebut this is not the case in the present paper. It was therefore decided to use stepwise backward elimination beginning with the full model, as reported in Table 3. The final model has broadband availability negatively and significantly associated with the unemployment rate after controlling for other factors (= -.01, p=.06), leading to a rejection of the null hypothesis. However, the effect was not large: a 10% increase of broadband availability produced a 0.1% decrease in the unemployment rate, other factors being the same. The county unemployment rate in 2008 included as a control was a strong predictor, indicating that unemployment patterns tended to persist over time. The state unemployment rate in 2012, included to control for regional effects was also a positive predictor. But contrary to the findings of Jayakar and Park using a smaller sample drawn from 8 states, the effect of state unemployment benefits was significantly negative: this may be because economically better-off states, that presumably had higher unemployment benefits, had lower unemployment rates as well. The coefficients were very stable under various specifications, and the models accounted for about 83% of the variation in the DV.

54

. William H. Greene, Econometric analysis. (Boston, Mass.: Prentiss-Hall, 2011).

TPRC, 2013, p. 19

Table 3: Regression Analysis for DV: County unemployment rate, 2012 (%)
Model 1 CONSTANT Speed Combination (DL>3 UL>768) Pct. working age population Minority Median income Pct. with high school diploma Population State minimum wage 2012 (Transformed) Unemployment benefits County unemployment rate, 2008 State unemployment rate, 2012 F(df) p R2 Adj. R2 N Beta [t] 4.26 *** [7.37] -.01 * [-1.82] .05 *** [6.02] .12 ***[13.90] .00 [-.30] -.03 *** [-2.86] .00 [-.19] -.08 ***[-9.54] -.03 ***[-4.05] .75 *** [74.08] .11 ***[10.63] F(10, 3129)=1514.08 p < .000 .83 .83 3140 -.08 ***[-9.55] -.03 ***[-4.06] .75 *** [74.10] .11 ***[10.63] F(9, 3130)=1682.83 p < .000 .83 .83 3140 -.08 ***[-9.55] -.03 ***[-4.20] .75 *** [76.98] .11 ***[10.79] F(8, 3131)=1893.70 p < .000 .83 .83 3140 Model 2 Beta [t] 4.26 *** [7.39] -.01 * [-1.84] .05 *** [6.01] .12 ***[14.15] .00 [-.35] -.03 *** [-2.86] -.03 *** [-3.66] Model 3 Beta [t] 4.30 *** [7.58] -.01 * [-1.88] .05 *** [6.01] .12 ***[14.59]

Note: Betas refers to standardized regression coefficients. *** p <0.01, ** p<0.05, * p<0.1

Turning now to Hypothesis 2, this was intended to test the effects of broadband availability on increases in unemployment from 2008 to 2012. Over this period, the recession in the United States caused widespread layoffs and plant closures, with the result that the average annual unemployment rate for the counties in our sample increased from 5.8% in 2008 to 7.7% in 2012 (Table 1). The null hypothesis predicted that broadband availability had no impact on
TPRC, 2013, p. 20

increases in unemployment over time, after controlling for other factors. To test this hypothesis, a new set of regressions were run, with the increase in unemployment (county unemployment rate in 2012 county unemployment rate in 2008) as the dependent variable, and all the independent variables in the previous set of regressions (see Table 4). As before, we started with a full model of all the variables and progressively eliminated variables stepwise. In these regressions, the main variable of interest, broadband availability as measured by the percentage of households with at least 3Mbps in download and 768 kbps upload speed, was a significant negative predictor of increase in unemployment. Previous research using similar methods but a smaller dataset had found broadband availability was not a significant predictor;55 here, with more available information, broadband availability found to significantly reduce increases in unemployment. Turning now to the other variables, the 2008 county unemployment rate was not a significant predictor of the increase in unemployment from 2008 to 2012. Instead, a larger workforce (the percentage of working age population), the minority percentage and regional effects (as measured by the state unemployment rate) were correlated with higher increases in unemployment; while education, the state minimum wages, and unemployment benefits all reduced increases in unemployment. Overall, after controlling for these other predictor variables, the second null hypothesis that predicted that broadband availability will have no effect was rejected.

55

Jayakar and Park.

TPRC, 2013, p. 21

Table 4: Regression Analysis for DV: Increase in county unemployment rate, 2008-2012
Model 1 CONSTANT Speed combination (DL>3 UL>768) Pct. working age population Minority Median income Pct. With high school diploma Population State minimum wage 2012 (Transformed) Unemployment benefits County unemployment rate, 2008 State unemployment rate, 2012 Beta [t] 4.26 *** [7.37] -.03 * [-1.82] .11 *** [6.02] .26 ***[13.90] .00 [-.30] -.06 *** [-2.86] .00 [-.19] -.16 ***[-9.54] -.07 ***[-4.05] -.02 [-.80] .23 ***[10.63] F(10, 3129)=108.90 p < .000 .26 .26 3140 -.16 ***[-9.55] -.07 ***[-4.06] -.02 [-.80] .23 ***[10.63] F(9, 3130)=121.03 p < .000 .26 .26 3140 -.16 ***[-9.55] -.07 ***[-4.20] -.02 [-.80] .23 ***[10.79] F(8, 3131)=136.18 p < .000 .26 .26 3140 .22 ***[11.98] F(7, 3132)=155.58 p < .000 .26 .26 3140 -.16 ***[-9.55] -.07 ***[-4.22] Model 2 Beta [t] 4.27 *** [7.39] -.03 * [-1.84] .11 *** [6.01] .26 ***[14.15] .00 [-.35] -.06 *** [-2.86] -.07 *** [-3.66] -.07 *** [-3.59] Model 3 Beta [t] 4.30 *** [7.58] -.03 * [-1.88] .10 *** [6.01] .25 ***[14.59] Model 4 Beta [t] 4.26 *** [7.55] -.03 * [-1.83] .10 *** [5.98] .25 ***[14.59]

F(df) p R2 Adj. R2 N

Note: Betas refers to standardized regression coefficients. *** p <0.01, ** p<0.05, * p<0.1

However as in the case of the unemployment level (Table 3) the effect of broadband availability on increases in unemployment too were weak, though significant. A 10% increase in the availability of broadband would have reduced the increase in unemployment by 0.3%. Also, the model as a whole has only weak explanatory power, as evidenced by the low R2 (.26). There
TPRC, 2013, p. 22

might be an explanation in the literature for these weak effects. As several papers have argued, broadband enables firms to realize productivity gains that may permit them to replace labor.56 In a recessionary economic environment, regions with better broadband connectivity may be better placed to reduce their labor force while maintaining productivity through ICT-based outsourcing. Thus, counties with better broadband availability are likely to have some losses in employment rates due to better labor productivity. Coupled with the fact that research shows a correlation between ICT deployment and economic growth which may create some jobs, the effect on unemployment turned out to be low. These results present a more complicated picture of the impact of broadband availability on unemployment. Clearly, unemployment rates are affected by regional economies, at the level of the state and beyond. The affluence and urbanization levels of the state might also indicate the mix of economic activities performed in those states, implying that broadband availability will affect employment opportunities differently, as pointed out by several previous studies. 57 While broadband availability does have a beneficial impact on employment opportunities, it is needless to say only one of several factors that affect unemployment rates. 4. Conclusions and Discussion When Congress authorized the investment of a portion of ARRA funds on broadband deployment, the expectation was that these investments will contribute to the advancement of the Recovery Acts goals of stimulating employment and spurring investments in long-term growth. This expectation was supported by an extensive theoretical literature hypothesizing a positive relationship between broadband deployment and economic growth. However, the evidence in favor of a positive relationship between broadband and employment was much more tenuous and
56 57

Fornefeld, Delaunay and Elixmann; Shideler, Badasyan and Taylor. Crandall, Lehr and Litan; Shideler, Badasyan and Taylor.

TPRC, 2013, p. 23

contradictory. While some authors have found a significant positive effect, others found a nonsignificant or even negative effect, and still others argued that other variables such as education affects the relationship. Previous work done by the authors of this paper using a smaller sample of counties drawn from 8 states had provided inconclusive results: broadband availability was found to correlate negatively with unemployment, but has no significant impact on changes in employment over the course of the recession. 58 Therefore, the objective of this paper was to investigate the relationship between broadband investments and employment, utilizing a larger, national sample of counties drawn from the newly available National Broadband Map data. The results in this paper present a positive relationship between broadband availability and employment. Counties with better broadband availability had lower unemployment rates in 2012 and lower increases in unemployment from 2008 to 2012, even after controlling for other factors. However, the size of the effect is very small. Changes in broadband availability are correlated only with small changes in unemployment. Nevertheless, this is good news for advocates of universal broadband, because the impacts of broadband are not limited to employment generation, but extend to general economic growth and small business development. The evidence in favor of a positive impact of broadband in these areas is much stronger; coupled with the positive results on employment generation found in this paper, the case for universal broadband has been strengthened. Though the data show that counties with better broadband availability were better off in terms of employment, we are still not able to determine conclusively with the available data what effect broadband has on jobs during the recession. Panel data tracking changes in broadband deployment and unemployment over time and cross-sectionally would elucidate the relationships much better, but unfortunately, enough observations of broadband deployment over time are not
58

Jayakar and Park.

TPRC, 2013, p. 24

yet available. As future installments of the National Broadband Map data become available, opportunities for further research will be opened up. The literature cited previously had suggested that the effects of broadband deployment on employment are variable depending on the type of industries active in the local economy. For example, broadband deployment may have negative effects on employment in sectors which use less educated and less-skilled labor inputs, while it benefits better-educated and higher paid workers, and creates more jobs in more knowledge-intensive industries. The composition of the local economy, in terms of the types of industries active in it, is likely to be a mediating factor in determining the effects of broadband deployment. County unemployment rates are possibly influenced by other factors not considered in this paper. More work is required to clarify this complex relationship.

References Albery, Brooks. What level of dialtone penetration constitutes universal service? Telecommunications Policy 19 (1995): 365-380. Atasoy, Hilal. The effects of broadband internet expansion on labor market outcomes, SSRN, 2011. http://ssrn.com/abstract=1890709. Beard, T. Randolph, George Ford, and Richard Saba. Internet use and job search. Phoenix Center Policy Bulletin No. 39. (Phoenix Center: Washington, DC, 2010). Bureau of Labor Statistics [BLS] Household data: Annual averages. U.S. Department of Labor, BLS, http://www.bls.gov/cps/cpsaat01.pdf Cook, R. Dennis and Sanford Weisberg. Applied regression including computing and graphics. New York: John Wiley, 1999. Crandall, Robert, William Lehr, and Robert Litan. The effects of broadband deployment on output and employment: A cross-sectional analysis of U.S. data. The Brookings Institution: Issues in Economic Policy No.5, 2007. http://www.brookings.edu /papers/2007/06labor_crandall.aspx
TPRC, 2013, p. 25

Cronin, Francis, Edwin Parker, Elizabeth Colleran, and Mark Gold. Telecommunications infrastructure and economic growth: An analysis of causality. Telecommunications Policy 15 (1991): 529-535. Cronin, Francis, Edwin Parker, Elizabeth Colleran, and Mark Gold. Telecommunications infrastructure investments and economic development. Telecommunications Policy 17 (1993): 415-430. Czernich, Nina, Oliver Falck, Tobias Kretschmer, and Ludger Woessmann. Broadband infrastructure and economic growth. CESIFO Working Paper No. 2861, Category 6: Fiscal Policy, Macroeconomics and Growth, 2009. http://www.CESifo-group.org/wp Datta, Anasua, and Sumit Agarwal. Telecommunications and economic growth: A panel data approach. Applied Economics 36, no. 15 (2004): 16491654. Derksen, Shelley and H. J. Keselman, Backward, forward and stepwise automated subset selection algorithms: Frequency of obtaining authentic and noise variables, British Journal of Mathematical and Statistical Psychology 45 (1992): 265-282. Duch, Raymond. Privatizing the economy: Telecommunications policy in comparative perspective. Ann Arbor, MI: University of Michigan Press, 1991. Federal Communications Commission. Connecting America: The national broadband plan. Washington, D.C.: FCC, 2010.http://download.broadband.gov/plan/national-broadbandplan.pdf Ford, George. Challenges in using the National Broadband Maps data. Phoenix Center Policy Bulletin No. 27. (Phoenix Center: Washington, DC, 2011). Foreman-Peck, James. Competition and performance in the U.K. telecommunications industry. Telecommunications Policy 9 (1985): 215-228. Fornefeld, Martin, Gilles Delaunay, and Dieter Elixmann, The impact of broadband on growth and productivity. Brussels: European Commission, Directorate General of Information, Media and Society, 2008. Goetz, Stephan J, Yicheol Han, Jill L. Findeis, and Kathryn J. Brasier, U.S. commuting networks and economic growth: measure and implications for spatial policy, Growth and Change 41(2) (2010): 276-302. Greene, William H. Econometric analysis. Boston, MA: Prentiss-Hall, 2011. Grubesic, Tony H. Spatial data constraints: Implications for measuring broadband. Telecommunications Policy 32, no. 7 (2008): 490-502. Grubesic, Tony H. The U.S. National Broadband Map: Data limitations and implications. Telecommunications Policy 36, no. 2 (2012): 113-126.
TPRC, 2013, p. 26

Hardy, Andrew P. The role of the telephone in economic development. Telecommunications Policy 4 (1980): 278-286. Hinkle, Dennis E., William Wiersma, and Stephen D. Jurs. Applied statistics for the behavioral sciences. Boston, Mass: Houghton Mifflin, 2003. Howell, David. C. Statistical methods for psychology. Belmont, CA: Wadsworth, 1997. Jipp, A. Wealth of nations and telephone density. Telecommunications Journal, (1963): 199201. Jayakar, Krishina P., & Eun-A Park. Broadband availability and employment: an analysis of county-level data from the national broadband map. Journal of Information Policy, 3 (2013): 181-200. Kolko, Jed. Does broadband boost local economic development? San Francisco, CA: Public Policy Institute of California, 2010. http://www.ppic.org/main/publication.asp?i=866 Koutroumpis, Pantelis. The economic impact of broadband on growth: A simultaneous approach. Telecommunications Policy 33, (2009): 471-485. Madden, Gary and Scott C. Savage. CEE telecommunications investment and economic growth. Information Economics and Policy 10 (1998): 173-195. National Telecommunications and Information Administration. The Broadband Technology Opportunities Program: Expanding broadband access and adoption in communities across America: Overview of grant awards. Washington, DC: NTIA, 2010. http://www.ntia.doc.gov/report/2010/expanding-broadband-access-and-adoptioncommunities-across-america-overview-grant-awards Preiger, James E., and Wei-Min Hu. The broadband digital divide and the nexus of race, competition, and quality. Information Economics and Policy 20, no. 2 (2008): 150-167. Saunders, Robert J., Jeremy J. Warford, and Bjorn Wellenius. Telecommunications and economic development. Baltimore, MD: Johns Hopkins University Press, 1994. Shideler, David, Narine Badasyan, and Laura Taylor. The economic impact of broadband deployment in Kentucky. Federal Reserve Bank of St. Louis Regional Economic Development 3, no. 2 (2007): 88-118. Shuai, Xiaobing, Who benefits from job creation at county level? An analysis of leakage and spillover of new employment opportunities in Virginia, Business Economics 45 (2010): 38-48.

TPRC, 2013, p. 27

Singer, Hal J., and Jeffrey D. West. Economic effects of broadband infrastructure deployment and tax incentives for broadband deployment. Dallas, TX: Fiber To The Home Council, 2010. http://www.neofiber.net/Articles/Economic_Effects_of_FTTH.pdf Sweet, Dustin L., Forecasting county-level unemployment accounting for spatial correlation, Doctoral dissertation (2011), University of Missouri, Columbia. Wallsten, Scott J. An econometric analysis of telecom competition, privatization and regulation in Africa and Latin America. Journal of Industrial Economics 49, no. 1 (2001): 1-19.

TPRC, 2013, p. 28