Sie sind auf Seite 1von 41

Introduction to Path Analysis and

Structural Equation Modelling with


AMOS
Daniel Stahl
Biostatistics, IoP

April/May 2010

Outline
Course will cover:
• Path analysis and SEM
• Gain familiarity with some of common statistical procedures,
coupled with their application through the use of statistical
software AMOS:
• Mediation analysis
• Confirmatory factor analysis
• Simple SEM
• Multigroup comparison (“Factorial invariance”)

By the end of the class, you will be able to do:


• Model specification, identification, and estimation
• Evaluating model fit
• Interpreting parameter estimates 2

Daniel Stahl: SEM 2010 1


SEM Books
SEM
• Barbara M. Byrne (2009) Structural Equation Modeling with AMOS.
• Randall E. Schumacker and Richard G. Lomax (2004) A Beginner's Guide
to Structural Equation Modeling. (AMOS examples)
• Rex B. Kline (2004) Principles and Practice of Structural Equation
Modeling, new edition in Juli 2010
• Raykow, T. and Marcouldes, G. A. (2008) A first course in structural
equation modelling. (Lisrel, EQS, Mplus examples)
• Bill Shipley (2004) Cause and Correlation in Biology: A User's Guide to
Path Analysis, Structural Equations and Causal Inference.
• James L. Arbuckle (2007) Amos™ 7.0 User’s Guide.
• Bollen, K. A. (1989) Structural Equations with Latent Variables (classic,
advanced)
Specialised books:
• Brown, T. A. (2006) Confirmatory factor analysis for applied research
• MacKinnon, D. (2008) Introduction to Statistical Mediation Analysis
Writing up the results of a SEM analysis:
• Boomsma, A. (2000). Reporting analyses of covariance structures.
Structural Equation Modeling, 7, 461-483. 3

Some web pages

• G. David Garson: SEM (with AMOS)


– http://faculty.chass.ncsu.edu/garson/PA765/structur.htm
• David Kenny
– http://davidakenny.net/cm/causalm.htm
• Norman and Streiner (2003): PDQ Statistics: Chapter 17
Path Analysis and Structural Equation Modeling
– http://www.bcdecker.com/SampleOfChapter/1-55009-207-3.pdf
• University of Texas: “Structural Equation Modeling using
AMOS”:
– http://ssc.utexas.edu/consulting/tutorials/stat/amos/

Daniel Stahl: SEM 2010 2


Path analysis and SEM

“Path analysis and SEM are extensions of the general


linear model (GLM) that enables a researcher to test a
set of regression equations simultaneously.”
“A key feature of SEM is that observed variables are
understood to represent a small number of "latent
traits“ or “latent constructs” that cannot be directly
measured, only inferred from the observed measured
variables.”

Today:
What are path analyses and SEMs?
– Recap correlation and regression
– Basic concepts of path analysis and SEM
– Unobservable latent traits
– Relationship between correlation,
regression, factor analysis, path analysis
and SEM
Introduction to AMOS
– Simple analyses with AMOS

Daniel Stahl: SEM 2010 3


Example

Study of the effect of room temperature on water


consumption:
The following data were collected for 50 subjects:
• Room temperature (ºC)
• Water consumption (decilitre)
• Self reports of thirst on a 1-5 scale (not at all thirsty -very
thirsty)

What kind of analysis could we do?

(Hypothetical study from MacKinnon 2008)

Correations
Correlations

Temperature Water consumption

Thirst Water consumption

Pearson’s correlation describes the linear association


between two variables. 8

Daniel Stahl: SEM 2010 4


Relationship between room
temperature and water consumption
6.0

5.0
Pearson’s r=0.66
Water consumption (dl)

4.0

3.0

2.0

1.0

0.0

17 18 19 20 21 22 23
Temperature
9

Examples of correlations
correlation=0
4 correlation = 0.35
2

0
0

-2
-2
-2 0 2
-3 -2 -1 0 1 2 3

correlation=.73 correlation=.94
10
10
5

0 0

-5

-10 -10
-2 0 2 -2 0 2 10

Daniel Stahl: SEM 2010 5


Simple linear regressions

βtemp
Temperature Water consumption

βthirst
Thirst Water consumption

• A simple linear regression models the linear relationship


between a response variable Y and a predictor variable X.
• We assume that y (water consumption) depends on X
(Temperature) and use linear regression to describe the effect
11
of x on y.

Regression: Temp. on water consumption


6.0

5.0
Water consumption (dl)

4.0

3.0
b=0.8 dl

2.0
1C
1.0
R Sq Linear = 0.434

0.0

17 18 19 20 21 22 23
Temperature 12

Daniel Stahl: SEM 2010 6


Linear regression
Coefficients a

Unstandardized Standardized
Coefficients Coefficients
Model B Std. Error Beta t Sig.
1 (Constant) -14.320 2.886 -4.961 .000
temp Temperature .836 .138 .659 6.071 .000
a. Dependent Variable: water Water consumption (dl)

water consumption= -14.32 + 0.836 *Temperature


Question:
Temperature = 20 degree
Water consumption= ?
13

Answer to Question 1:
Temperature = 20 degree
Water consumption= ?
Water consumption = -14.32 + 0.836 *20
= -14.32+16.72
= 2.4 dl
Question 2:
water consumption= -14.32 + 0.836 *Temperature
Temperature = 0 degree
Water consumption= ?
14

Daniel Stahl: SEM 2010 7


Multiple linear regression

Temperature

Water consumption

Thirst

15

Multiple linear regression


• In multiple linear regression, the effects of several
independent variables on the dependent variable are
estimated simultaneously.
• The effect of each independent variable is adjusted for all
other independent variables (= other variables are kept
constant) used in the regression.
• The dependent variable y is modeled as

y = β0 + β1 x1 + β2 x2 + ... + βk xk + ε where

– β0 is the intercept or constant: the value of y when all xi are 0


– β1 is the increase in y when x1 increases by one unit and all other x
are fixed
– βk is the increase in y when xk increases by one unit and all other x
are fixed
– ε is the error term or residual 16

Daniel Stahl: SEM 2010 8


Regression
A linear regression describes the effect of one or
more independent (predictor) variables on a
dependent (explanatory) variable:

y = β0 +β1x1 + β2x2 + … βnxn+ ε(N,σ2)

Water consumption = β0 + β1*temperature + β2*thirst + ε(N,σ2)

Water consumption =
-5.068 + 0.2681*temperature + 0.779*thirst + ε(N,σ2)

17

Multiple linear regression

Temperature
β1

Water consumption

β2

Thirst

Water consumption = β0 + β1*temperature + β2*thirst + ε(N,σ2)

What is missing in our path diagram?


18

Daniel Stahl: SEM 2010 9


Regression

A linear regression describes the effect of one or more


independent (predictor) variables on a dependent variable:

y = constant +β1x1 + β2x2 + … βnxn+ ε(N,σ2)

Water consumption = constant + β1*temp. + β2*thirst + ε(N,σ2)

Error also influences the outcome variable y (water


consumption) and error terms need to be incorporated in
the model diagram.

19

Multiple regression

Error

Tem perature 1
β1

W ater consum ption

β2
Thirst

•AMOS requires you to “draw” a correlation between


your independent variables!
•Regression assumes that there is no measurement
error in the independent variables.
•Is this a reasonable assumption? 20

Daniel Stahl: SEM 2010 10


• Beyond simple regressions

21

Stimulus-Organism-Response Model
Theory suggests that the effect of a stimulus on a response is
mediated by the organism:
Higher temperature increases thirst, which then leads to water
consumption.
(Woodworth 1928) Stimulus

Organism

Response
22

Daniel Stahl: SEM 2010 11


Mediation (Path analysis)

Temperature

Error
1

Thirst

Error
1

Water consumption

Baron, R. M., & Kenny, D. A. (1986). The moderator-mediator variable distinction in social
psychological research: Conceptual, strategic and statistical considerations. Journal of 23
Personality and Social Psychology, 51, 1173-1182.

Path analysis

• Technique to examine the (assumed) causal


relationships between two or more variables.
• Path analysis assess the direct and indirect (mediating)
relationships among a set of variables!
 XY: direct effect of X on Y

 X MY: indirect effect of X on Y via M

 Total effect of X on Y = direct + indirect effect

• Regression is a subset of path analysis. It only studies


the direct effects of one or more independent variables
on (usually) one dependent variable

24

Daniel Stahl: SEM 2010 12


Path analysis-regression relationship

• Path analysis (and SEM) is a multivariate extension of


the multiple linear regression to a series of multiple
regressions, where all the equations are fitted
simultaneously.
• For example: the simple mediation model consists of two
linear regressions:
• XM Temperature  Thirst
• MY Thirst  Water consumption

25

Path analysis
Independent variable X

Error
1

Direct effect from X


Mediator M on Y (“partial
mediation”)

Error
Indirect 1
effect from
X on Y via
M Dependent variable Y

26

Daniel Stahl: SEM 2010 13


Measurement error and latent traits

Thirst distress: the degree to which a person is bothered by


thirst or its associated discomfort.
Self reports of thirst on a 1-5 scale (not at all thirsty -very
thirsty) is probably not a reliable measure of thirst.
How can we obtain a better measure of thirst?

27

Measurement error and latent traits

Thirst is a latent and subjective trait, which can’t be


observed and measured directly.
We develop/use a questionnaire with several items, which
should be an accurate (reliable and valid) measure of the
unobservable (latent) trait “thirst”.

28

Daniel Stahl: SEM 2010 14


Six Items Thirst Distress Scale

1. My thirst causes me discomfort


2. My thirst bothers me a lot.
3. I am very uncomfortable when I am thirsty.
4. My mouth feels like cotton when I am thirsty.
5. My saliva is very thick when I am thirsty.
6. When I drink less, my thirst gets worse.
The items were scored on 5-point rating scales
from 1 (strongly disagree) to 5 (strongly agree).

Welch, J. L. (2002) Development of the thirst distress


scale. Nephrology Nursing Journal
29

Latent trait “Thirst”


e1 e2 e3 e4 e5

1 1 1 1 1

Item 1 Item 2 Item 3 Item 4 Item 5

Latent trait "Thirst"

This model takes measurement error of our observed


variables into account. The latent trait is measured without
error. (But we have to assume that the latent trait is a valid
30
measure of “Thirst”)

Daniel Stahl: SEM 2010 15


Example of factor analysis: One Factor Model

e1 e2 e3 e4 e5

1 1 1 1 1

Item 1 Item 2 Item 3 Item 4 Item 5

Latent trait "Thirst"

31

Path analysis with latent trait “thirst”

Temperature 1
Item 1 e1
1 1
e7 Item 2 e2
1
Latent trait "Thirst" Item 3 e3
1
Item 4 e4
e6 1
Item 5 e5
Water consumption

32

Daniel Stahl: SEM 2010 16


Another example

We are interested about the influence of pain and


body functioning on depression.
We collected the following variables of 40 cancer
patients:
– Body function
– Pain
– Depression
The variables were measured with a small
questionnaire with three questions for each
variable
We assume that pain and body functioning
influence depression
33

Error 1 Error 2 Error 3

1 1 1
SEM
Item 1 Item 2 Item 3

Pain
error
1
1 Item 1 Error 1

1
Depression Item 2 Error 2

1
Item 3 Error 3

Function

Item 3 Item 2 Item 1

1 1 1

34
Error 3 Error 2 Error 1

Daniel Stahl: SEM 2010 17


Structural Equation Models

SEM consists of two parts: a measurement model and a


structural model.
• The structural model deals with the relationship between
the latent variables
• while the measurement model describes the relationship
between our measured variables and the latent variables
• For example: Relationship between the measurement
model and the structural model relating pain and function
to depression:

35

Error 1 Error 2 Error 3

1 1 1 Measurement model

Item 1 Item 2 Item 3

Pain
1
1 Item 1 Error 1

1
Depression Item 2 Error 2

1
Item 3 Error 3

Function

Item 3 Item 2 Item 1

1 1 1
Structural model
36
Error 3 Error 2 Error 1

Daniel Stahl: SEM 2010 18


Structural equation models

• Structural equation modelling allows to study the effect


of unmeasured latent variables.
• Latent variables cannot be observed and must be
inferred from measured variable.
• The use of latent variables allows us to take
measurement error of the observed variables into
account!

37

SEM can be extended: it allows to include more


latent and observable variables in the analysis:
e6 e7 e8 e9

1 1 1 1

1
Item 5
e5 contri- Item 1 Item 2 Item 3 Item 4
bution
1 Item 4 1
e4
Coworker
1
Item3 1
e3 other Self Esteem e10
Depression
people
1
Item 2
e2 super- 1
visor
1 Item 1
e1
"work"

Age

38
e

Daniel Stahl: SEM 2010 19


Family Tree of SEM

Structural
Multiple Path
Bivariate Regression
Equation
Analysis
Correlation Modeling

Confirmatory
Factor Factor
Analysis Analysis

Exploratory
Factor
Analysis

39

Path analysis and SEM


• In our path analysis model we assume that each latent
variable is almost perfectly measured with one observed
variable (perfect correlation = no measurement error).
•  The latent variable is inferred from only one measured
variable.
e.g. temperature change can be seen as a latent
variable measured as the change in a quicksilver
column.
• Path analysis can also be seen as SEM.
Error Error

1 1

Change in Temperature
quicksilver column How thirsty are you? Latent trait "Thirst"
40

Daniel Stahl: SEM 2010 20


• More about latent traits

41

Unobservable traits
• In psychology and health sciences we are often
concerned with questions which are more subjective
than questions in other fields of science.
• These includes measurements of:
abilities, knowledge, emotions, feelings, attitudes or
personality traits.
• All traits have got in common that they are unobservable
traits = latent traits.

42

Daniel Stahl: SEM 2010 21


Unobservable, latent traits
• The effect of a drug may prolong the life of a patient or
cure a symptom but it may also effects on the general
well-being.
• While life prolonging is rather easy to define, it is not
easy to define “well-being”
• And different people may have got different definitions.
• The field of psychometrics is concerned with the theory
and technique of measurement of such psychological
and mental phenomena.

43

Latent traits in psychology


• Intelligence
• Memory
• Extraversion
• Self-esteem
• Depression
• Anxiety
• Knowledge
• Beliefs
• Feelings and Emotions: Joy, sadness,
• Senses and Perception: smell of flower
• Attitude about something, e.g. foreigners, risk
• Motivation
• Ability to learn statistics or a new language
44

Daniel Stahl: SEM 2010 22


Latent traits in medical research
• Pain
• Mental disorders
– Depression
– Schizophrenia
– Autism
• Mobility/Function (gerontology)
• Arthritis
• Quality of life
• Patient satisfaction (e.g. in hospital)

45

Latent vs. observed variables

• An observed variable, like body height, is directly


observable and can be measured easily.
• A latent variable or trait or construct is not directly
observable. Instead, it is inferred from variables
(items) than can be observed.

• The main approach of psychometric measurements


involves applying interviews, questionnaires and tests (=
instruments).

46

Daniel Stahl: SEM 2010 23


Latent trait and items
“I feel good about my
work” 1 2 3 4 5 6 7

“I am proud of my relationship
with my supervisor” 1 2 3 4 5

“On the whole I get along with


others well” 1 2 3 4 5
Self Esteem
“I can feel that my co-worker
respect me” 1 2 3 4 5

“I feel that I am making a useful


contribution to work” 1 2 3 4 5

Here, the latent trait “Self esteem” elicits to each item a


response from 1 (strongly disagree) to 5 (strongly
agree). The sum of the observed responses allows a
conclusion about the person’s self esteem. 47

Item and latent traits

• All 5 items are measuring the latent trait “self esteem”.


• They should therefore correlate with the latent trait.
• A single item will never measure a construct perfectly
(and hence will never correlate perfectly),
• but the 5 items should be an accurate predictor of the
latent trait.
•  Basic idea of SEM
• SEM in form of factor analysis is an important tool to
develop such tests.

48

Daniel Stahl: SEM 2010 24


Confirmatory factor analysis model

Item 1 1
e1
"work"
1
Item 2 1
super- e2
visor
Item3 1
Self Esteem other e3
people
Item 4 1
e4
Coworker
Item 5 1
contri- e5
bution

49

• SEM allows the complex modelling of


latent traits:

50

Daniel Stahl: SEM 2010 25


SEM can be extended: it allows to include more
latent and observable variables in the analysis:
e6 e7 e8 e9

1 1 1 1

1
Item 5
e5 contri- Item 1 Item 2 Item 3 Item 4
bution
1 Item 4 1
e4
Coworker
1
Item3 1
e3 other Self Esteem e10
Depression
people
1
Item 2
e2 super- 1
visor
1 Item 1
e1
"work"

Age

51
e

Some more applications of SEM:

• Path analysis models


• Confirmatory Factor analysis models
• Longitudinal Factor analysis
• Structural regression models (CFA with
explanatory variables)
• Latent change or Latent growth curve models
(longitudinal data analysis)
• Latent class models (modelling latent class
which allows to categorize individuals into
distinctive groups or classes based on their
responses to a set of observed variables.
52

Daniel Stahl: SEM 2010 26


SEM: Longitudinal CFA

1
1 pain e1
e3 function
1

1 Well-being Well-being 1
e2 depress depress e2
Time 1 Time 2

1
1
1 function e3
e1 pain

53

Structural regression model:


Factor model of verbal memory
with age as covariate

error
1
1 1 paragrap err_p

1
Age verbal sentence err_s

1
wordmean err_w

54

Daniel Stahl: SEM 2010 27


Longitudinal data analysis using SEM
• A common approach to the analysis of longitudinal data is
multilevel modelling
• but we can also use the structural equation modelling
(SEM) framework to form what are known as “latent
curve” or “latent trajectory” models.
• Rovine & Molenaar (2001) demonstrated the
mathematical equivalence of MLM and SEM with
balanced data.
• SEM Latent trajectory analysis is is more flexible.

55

Amos Setup: Simple Growth Curve Model with


Random Slope and Intercept with 4 time points
E1 E2 E3 E4

1 1 1 1

X1 X2 X3 X4

ICEPT SLOPE

56

Daniel Stahl: SEM 2010 28


Growth Curve Model with Random Slope and
Intercept withg14correlated errors
g13 g24

g12 g23 g34

0, g1 0, g2 0, g3 0, g4
e1 e2 e3 e4
1 1 1 1
mg1 mg2 mg3 mg4
y1 y2 y3 y4

4
2 6
1 1
1 1 0

ICEPT Slope

57

Latent Class Analysis

• Latent variable analysis in which the observed predictor


variables are categorical or continuous and the latent
(unobserved) variable is also categorical.
• a statistical method for finding subtypes of related cases
(latent classes) from multivariate data.
– attempts to find groups or classes of observations
that are similar to one another.
– also called Model-based cluster analysis
(not possible in AMOS 7 but in new version)

58

Daniel Stahl: SEM 2010 29


Literature
Growth curve modelling:
• Terry E. Duncan, Susan C. Duncan, Lisa A. Strycker
(2006) An Introduction to Latent Variable Growth Curve
Modeling: Concepts, Issues, and Applications.
Latent Class modelling
• McCutheon, A.L. (1987). Latent Class Analysis.
Newbury Park, CA: Sage.
• Muthen, B.O. (2001). Latent variable mixture modeling.
In G.A. Marcoulides & R.E. Schumacker, New
Developments and techniques in structural equation
modeling. Mahway, NJ: Erlbaum.
• Everitt, B, Landau, S, Leese, M. and Stahl, D. (2010??)
Cluster analysis. 5th edition
59

AMOS

• We will start with simple drawing…

60

Daniel Stahl: SEM 2010 30


SEM: observed and unobserved variables

• Observed variables: rectangles


– Indicator variables
– Manifest variables
– Reference variables
• Unobserved variables: ovals
– Latent variables
– Latent constructs
– Latent factors

61

SEM diagram symbols

Observed variable
1
Latent variable with
disturbance or error

1 Observed variable
with measurement
error (“endogenous Unidirectional path
variable”) (“regression”)

Correlation between
1 1 variables
Latent variable with
items (= observed
1 variables) Reciprocal relation
between variables

62

Daniel Stahl: SEM 2010 31


1 2 3
1. Observed variables
4 5 6 2. Unobserved variables
7 3. Drawing latent variable (draws latent variable and items)
8 8 8 4. Drawing path (causal relationship regression)
9 10 5. Draw covariances (correlation, no direction)
6. Unique variable (error variable, add e.g. to each dependent var
7. List variables (open data file first, then drag and drop variables)
11 12 13
8. Select one object, select all, deselect
14 15 16
9. Move object
10. Delete
11. Select data file
12. Analysis properties (choose statistics)
13. Calculate estimates (starts the analysis)
14. Copy graph in clipboard
15. View test (see results) 63

16. Save

Exercise 1

• Draw and label the following 4 models with


AMOS
• Try to identify what kind of analysis you
are “doing”. (For example: Correlation,
regression, SEM, Path model…)
• Move some of your models around
• Delete part of your models

64

Daniel Stahl: SEM 2010 32


Model 1
Tem perature water consum ption

e1
1

Tem perature W ater consumption

e2

Tem perature 1

W ater consum ption

Thirst

65

Model 2
Temp

1 1
e1 Thirst activity e2

water consumption

e3
66

Daniel Stahl: SEM 2010 33


Model 3

e1 e2 e3 e4 e5

1 1 1 1 1

a b c d e

F1

67

Model 4
e1 e2 e3 e4 e5

1 1 1 1 1

a b c d e

F1

1
F2 e12

p q r s t u

1 1 1 1 1 1

68
e11 e10 e9 e8 e7 e6

Daniel Stahl: SEM 2010 34


Attaching the data
Amos supports input of several common database formats:
SPSS *.sav, Excel, Access, .txt and .csv files
SPSS files are the most convenient ones
File Data files
Tick box “select data file:
Click “file name”
and select your data file
View data: SPSS opens and you can view the data
Click okay

69

Naming the variables

• Click the “List Variables in Data Set” icon and


then drag each variable name to the appropriate
rectangle

70

Daniel Stahl: SEM 2010 35


Doing the analysis
• “Analysis properties”
Select statistics:

• “Calculate estimates”: Run the analysis

• “View text”: See the results – Output


.66


• See results on graph Temperature W ater consumption (dl)

71

Example

Data set:
• Water_consumption.sav

72

Daniel Stahl: SEM 2010 36


Correlation: Output
Standardised estimates:

.66

Temperature W ater consumption (dl)

73

Regression

Unstandardised estimates (Beta)


.83

e1

1
.92

.84 W ater consumption (dl)


Temperature

Standardised estimates (standardised beta)


e1

.43

.66 W ater consumption (dl)


Temperature

74

Daniel Stahl: SEM 2010 37


Multiple regression
Unstandardised estimates
.57

.92 e1
1
Temperature
.27

.67 Water consumption (dl)


.92
.78

Thirst

Standardised estimates
e1

Temperature .61
.21

.73 Water consumption (dl)


.62

Thirst
75

Exercise
Use SPSS and open the data file “mediation.sav”
1. Do the following analysis with SPSS:
• Correlation matrix of all three variables
• Simple linear regressions between:
– Function  Depression
– Pain  Depression
• Multiple regression:
– Pain +Function  Depression
2. Do the same analysis with AMOS
3. Optional: Delete a few observations of the independent
variables the data set and rerun the multiple regression
in SPSS and AMOS. Are the results still the same?
Compare the sample sizes of both analyses. 76

Daniel Stahl: SEM 2010 38


Data set

• Barsevick, Andrea M.; Dudley, William N.; Beck, Susan


L. (2006) Cancer-related Fatigue, Depressive
Symptoms, and Functional Status: A Mediation Model.
Nursing Research. 55(5):366-372,
• The effect of pain and body function on depressive
symptoms of cancer patient after the second
chemotherapy treatment or during the last week of
radiotherapy was assessed.

77

AMOS: Main steps

• Draw path diagram


• Move data into appropriate box
• Name variables
• Analysis properties: Select statistics
• Calculate estimates: Run analysis
• View text: View results

78

Daniel Stahl: SEM 2010 39


Correlation results
Correlation .34
coefficient r

Pain Depression

79

Regression results
0, .40
Unstandardised regression coefficient
e1 Error
Variance Variance
1
4.93, 4.08 1.00

.11
Pain Depression

Explained
e1 variance r2
Standardised regression coefficient
.11
.34
Pain Depression
80

Daniel Stahl: SEM 2010 40


Multiple regression results 0, .36

Unstandardised regression coefficient e1


4.93, 4.08

1
Pain .06 2.61

-.40 2.62, .19


Depression
-.52

Body Function

Explained
Standardised regression coefficient e1 variance r2

Pain .18 .20

-.46 Depression
Correlation -.34
between
independent Body Function
81
variables

Next week: Mediation analysis with AMOS


Model
Model 21 Mediation (Path
Model 2 analysis)

pain
Error 2
pain
1
Error 1
1
depress

function
function
Error 2
1
1

Error 1 depress

82

Daniel Stahl: SEM 2010 41

Das könnte Ihnen auch gefallen