5.4 In-Class Exercises
This week, we will wrap up our re-analysis of the Kestilä (2006) results. During this practical, you will conduct a CFA of the Attitudes toward Immigration items and compare the results to those obtained from your previous EFA- and PCA-based replications of Kestilä (2006).
5.4.1
Load the ESS data.
- The relevant data are contained in the ess_round1.rds file.
We are going to conduct a CFA to evaluate the measurement model implied by the five-factor representation of the Attitudes toward Immigration items that you should have found via the EFA you conducted in the Week 4 In-Class Exercises.
Caveat: Technically, the following CFA result have no confirmatory value because we’ll be estimating our CFA models from the same data that we used for our EFA. Practicing the techniques will still be useful, though.
5.4.2
Define the lavaan model syntax for the CFA implied by the five-factor solution
from 4.4.9.
- Enforce a simple structure; do not allow any cross-loadings.
- Covary the five latent factors.
- Do not specify any mean structure.
- Save this model syntax as an object in your environment.
Hints:
- You can algorithmically enforce a simple structure by assigning each item to the factor upon which it loads most strongly.
- You can download the fitted
psych::efa()object for the five-factor solution here. - The pattern matrix for the five-factor EFA solution in our Week 4 exercises is equivalent to the solution presented in Table 3 of Kestilä (2006).
Click to show code
mod_5f <- '
## Immigration Policy:
ip =~ imrcntr + eimrcnt + eimpcnt + imsmetn + impcntr + imdfetn
## Social Threat:
st =~ imbgeco + imbleco + imwbcnt + imwbcrm + imtcjob + imueclt
## Refugee Policy:
rp =~ gvrfgap + imrsprc + rfgbfml + rfggvfn + rfgawrk + rfgfrpc + shrrfg
## Cultural Threat:
ct =~ qfimchr + qfimwht + pplstrd + vrtrlg
## Economic Threat:
et =~ imwgdwn + imhecop
'Note: We don’t have to specify the latent covariances in the model syntax, we
can tell lavaan to estimate all latent covariances when we fit the model.
5.4.3
Estimate the CFA model you defined above, and summarize the results.
- Use the
lavaan::cfa()function to estimate the model. - Use the default settings for the
cfa()function. - Request the model fit statistics with the summary by supplying the
fit.measures = TRUEargument tosummary(). - Request the standardized parameter estimates with the summary by supplying the
standardized = TRUEargument tosummary().
Check the results, and answer the following questions:
- Does the model fit the data well?
- How are the latent variances and covariances specified when using the default settings?
- How is the model identified when using the default settings?
Click to show code
## Load the lavaan package:
library(lavaan)
## Estimate the CFA model:
fit_5f <- cfa(mod_5f, data = ess)
## Summarize the fitted model:
summary(fit_5f, fit.measures = TRUE, standardized = TRUE)## lavaan 0.6-18 ended normally after 72 iterations
##
## Estimator ML
## Optimization method NLMINB
## Number of model parameters 60
##
## Used Total
## Number of observations 14243 19690
##
## Model Test User Model:
##
## Test statistic 18631.556
## Degrees of freedom 265
## P-value (Chi-square) 0.000
##
## Model Test Baseline Model:
##
## Test statistic 159619.058
## Degrees of freedom 300
## P-value 0.000
##
## User Model versus Baseline Model:
##
## Comparative Fit Index (CFI) 0.885
## Tucker-Lewis Index (TLI) 0.869
##
## Loglikelihood and Information Criteria:
##
## Loglikelihood user model (H0) -520035.133
## Loglikelihood unrestricted model (H1) -510719.354
##
## Akaike (AIC) 1040190.265
## Bayesian (BIC) 1040644.106
## Sample-size adjusted Bayesian (SABIC) 1040453.432
##
## Root Mean Square Error of Approximation:
##
## RMSEA 0.070
## 90 Percent confidence interval - lower 0.069
## 90 Percent confidence interval - upper 0.071
## P-value H_0: RMSEA <= 0.050 0.000
## P-value H_0: RMSEA >= 0.080 0.000
##
## Standardized Root Mean Square Residual:
##
## SRMR 0.048
##
## Parameter Estimates:
##
## Standard errors Standard
## Information Expected
## Information saturated (h1) model Structured
##
## Latent Variables:
## Estimate Std.Err z-value P(>|z|) Std.lv Std.all
## ip =~
## imrcntr 1.000 0.617 0.748
## eimrcnt 0.942 0.011 84.943 0.000 0.582 0.696
## eimpcnt 1.127 0.010 113.413 0.000 0.695 0.898
## imsmetn 0.982 0.010 98.753 0.000 0.606 0.796
## impcntr 1.150 0.010 113.623 0.000 0.710 0.900
## imdfetn 1.132 0.010 111.802 0.000 0.698 0.887
## st =~
## imbgeco 1.000 1.608 0.728
## imbleco 0.826 0.012 69.222 0.000 1.327 0.619
## imwbcnt 1.046 0.012 88.056 0.000 1.682 0.792
## imwbcrm 0.713 0.011 63.102 0.000 1.146 0.564
## imtcjob 0.751 0.011 66.787 0.000 1.207 0.597
## imueclt 1.008 0.013 78.043 0.000 1.621 0.698
## rp =~
## gvrfgap 1.000 0.659 0.610
## imrsprc 0.855 0.016 51.881 0.000 0.563 0.535
## rfgbfml 1.047 0.019 56.174 0.000 0.690 0.593
## rfggvfn 0.849 0.016 51.714 0.000 0.559 0.533
## rfgawrk 0.653 0.016 41.044 0.000 0.430 0.405
## rfgfrpc -0.810 0.016 -51.095 0.000 -0.534 -0.525
## shrrfg -0.999 0.017 -58.381 0.000 -0.658 -0.625
## ct =~
## qfimchr 1.000 1.836 0.629
## qfimwht 0.941 0.017 54.250 0.000 1.728 0.659
## pplstrd -0.366 0.007 -51.585 0.000 -0.673 -0.600
## vrtrlg 0.252 0.006 41.294 0.000 0.462 0.443
## et =~
## imwgdwn 1.000 0.723 0.667
## imhecop 1.151 0.023 49.736 0.000 0.832 0.771
##
## Covariances:
## Estimate Std.Err z-value P(>|z|) Std.lv Std.all
## ip ~~
## st -0.605 0.012 -48.693 0.000 -0.610 -0.610
## rp 0.264 0.006 45.566 0.000 0.648 0.648
## ct 0.634 0.015 41.007 0.000 0.560 0.560
## et -0.206 0.006 -35.411 0.000 -0.462 -0.462
## st ~~
## rp -0.838 0.017 -48.329 0.000 -0.792 -0.792
## ct -1.622 0.041 -39.091 0.000 -0.550 -0.550
## et 0.675 0.017 39.083 0.000 0.580 0.580
## rp ~~
## ct 0.626 0.018 34.950 0.000 0.518 0.518
## et -0.233 0.007 -33.007 0.000 -0.490 -0.490
## ct ~~
## et -0.592 0.020 -30.127 0.000 -0.446 -0.446
##
## Variances:
## Estimate Std.Err z-value P(>|z|) Std.lv Std.all
## .imrcntr 0.299 0.004 77.941 0.000 0.299 0.440
## .eimrcnt 0.359 0.005 79.638 0.000 0.359 0.515
## .eimpcnt 0.116 0.002 62.821 0.000 0.116 0.193
## .imsmetn 0.212 0.003 75.580 0.000 0.212 0.366
## .impcntr 0.119 0.002 62.454 0.000 0.119 0.191
## .imdfetn 0.132 0.002 65.344 0.000 0.132 0.213
## .imbgeco 2.288 0.033 70.261 0.000 2.288 0.470
## .imbleco 2.837 0.037 76.688 0.000 2.837 0.617
## .imwbcnt 1.677 0.027 63.198 0.000 1.677 0.372
## .imwbcrm 2.810 0.036 78.612 0.000 2.810 0.682
## .imtcjob 2.630 0.034 77.524 0.000 2.630 0.643
## .imueclt 2.761 0.038 72.515 0.000 2.761 0.512
## .gvrfgap 0.733 0.010 73.584 0.000 0.733 0.628
## .imrsprc 0.791 0.010 77.119 0.000 0.791 0.714
## .rfgbfml 0.877 0.012 74.508 0.000 0.877 0.648
## .rfggvfn 0.788 0.010 77.203 0.000 0.788 0.716
## .rfgawrk 0.945 0.012 80.870 0.000 0.945 0.836
## .rfgfrpc 0.749 0.010 77.501 0.000 0.749 0.724
## .shrrfg 0.676 0.009 72.682 0.000 0.676 0.609
## .qfimchr 5.142 0.080 64.113 0.000 5.142 0.604
## .qfimwht 3.891 0.064 60.623 0.000 3.891 0.566
## .pplstrd 0.804 0.012 67.054 0.000 0.804 0.640
## .vrtrlg 0.872 0.011 76.990 0.000 0.872 0.804
## .imwgdwn 0.652 0.012 53.300 0.000 0.652 0.555
## .imhecop 0.472 0.014 34.353 0.000 0.472 0.405
## ip 0.381 0.007 51.578 0.000 1.000 1.000
## st 2.584 0.054 47.795 0.000 1.000 1.000
## rp 0.434 0.012 36.748 0.000 1.000 1.000
## ct 3.371 0.096 35.174 0.000 1.000 1.000
## et 0.523 0.015 34.944 0.000 1.000 1.000Click for explanation
No, the model does not seem to fit the data well.
- The SRMR looks good, but one good looking fit statistic is not enough.
- The TLI and CFI are in the “unacceptable” range.
- RMSEA is in the “questionable” range.
- The \(\chi^2\) is highly significant, but we don’t care.
The cfa() function is just a wrapper for the lavaan() function with several
options set at the defaults you would want for a standard CFA.
- By default:
- All latent variances and covariances are freely estimated (due to the
argument
auto.cov.lv.x = TRUE) - The model is identified by fixing the first factor loading of each factor
to 1 (due to the argument
auto.fix.first = TRUE)
- All latent variances and covariances are freely estimated (due to the
argument
To see a full list of the (many) options you can specify to tweak the behavior
of lavaan estimation functions run ?lavOptions.
Now, we will consider a couple of alternative factor structures for the Attitudes toward Immigration CFA. First, we will go extremely simple by estimating a one-factor model wherein all Attitude items are explained by a single latent variable.
5.4.4
Define the lavaan model syntax for a one-factor model of the Immigration items.
- Save this syntax as an object in your environment.
Click to show code
5.4.5
Estimate the one-factor model, and summarize the results.
- Compare the fit measures for the one-factor and five-factor models
- Which model better fits the data?
Note: Remember, you can use the lavaan::fitMeasures() function to extract
only the model fit information from a fitted lavaan object.
Click to show code
## Estimate the one factor model:
fit_1f <- cfa(mod_1f, data = ess)
## Summarize the results:
summary(fit_1f)## lavaan 0.6-18 ended normally after 47 iterations
##
## Estimator ML
## Optimization method NLMINB
## Number of model parameters 50
##
## Used Total
## Number of observations 14243 19690
##
## Model Test User Model:
##
## Test statistic 49510.917
## Degrees of freedom 275
## P-value (Chi-square) 0.000
##
## Parameter Estimates:
##
## Standard errors Standard
## Information Expected
## Information saturated (h1) model Structured
##
## Latent Variables:
## Estimate Std.Err z-value P(>|z|)
## ati =~
## imrcntr 1.000
## eimrcnt 0.937 0.012 78.324 0.000
## eimpcnt 1.114 0.011 101.263 0.000
## imsmetn 0.987 0.011 90.990 0.000
## impcntr 1.147 0.011 102.371 0.000
## imdfetn 1.153 0.011 103.148 0.000
## imbgeco -2.055 0.032 -64.749 0.000
## imbleco -1.625 0.031 -52.533 0.000
## imwbcnt -2.173 0.030 -71.324 0.000
## imwbcrm -1.432 0.029 -48.849 0.000
## imtcjob -1.532 0.029 -52.519 0.000
## imueclt -2.198 0.033 -65.876 0.000
## gvrfgap 0.807 0.016 51.746 0.000
## imrsprc 0.757 0.015 49.790 0.000
## rfgbfml 0.861 0.017 51.272 0.000
## rfggvfn 0.722 0.015 47.671 0.000
## rfgawrk 0.530 0.015 34.448 0.000
## rfgfrpc -0.755 0.015 -51.462 0.000
## shrrfg -0.931 0.015 -61.438 0.000
## qfimchr 1.597 0.042 37.835 0.000
## qfimwht 1.769 0.038 46.697 0.000
## pplstrd -0.873 0.016 -53.994 0.000
## vrtrlg 0.602 0.015 39.940 0.000
## imwgdwn -0.682 0.016 -43.576 0.000
## imhecop -0.773 0.016 -49.611 0.000
##
## Variances:
## Estimate Std.Err z-value P(>|z|)
## .imrcntr 0.327 0.004 79.021 0.000
## .eimrcnt 0.388 0.005 80.422 0.000
## .eimpcnt 0.161 0.002 70.832 0.000
## .imsmetn 0.235 0.003 77.101 0.000
## .impcntr 0.158 0.002 69.688 0.000
## .imdfetn 0.150 0.002 68.791 0.000
## .imbgeco 3.381 0.041 82.203 0.000
## .imbleco 3.666 0.044 83.130 0.000
## .imwbcnt 2.839 0.035 81.477 0.000
## .imwbcrm 3.399 0.041 83.334 0.000
## .imtcjob 3.260 0.039 83.130 0.000
## .imueclt 3.683 0.045 82.092 0.000
## .gvrfgap 0.938 0.011 83.176 0.000
## .imrsprc 0.906 0.011 83.285 0.000
## .rfgbfml 1.092 0.013 83.203 0.000
## .rfggvfn 0.917 0.011 83.394 0.000
## .rfgawrk 1.031 0.012 83.913 0.000
## .rfgfrpc 0.832 0.010 83.192 0.000
## .shrrfg 0.803 0.010 82.499 0.000
## .qfimchr 7.613 0.091 83.803 0.000
## .qfimwht 5.772 0.069 83.442 0.000
## .pplstrd 0.988 0.012 83.040 0.000
## .vrtrlg 0.958 0.011 83.728 0.000
## .imwgdwn 1.010 0.012 83.583 0.000
## .imhecop 0.954 0.011 83.294 0.000
## ati 0.353 0.007 48.941 0.000## Compare fit statistics:
fitMeasures(fit_5f,
fit.measures = c("npar", # Estimated parameters
"chisq", "df", "pvalue", # Model fit vs. saturated
"cfi", "tli", # Model fit vs. baseline
"rmsea", "srmr"), # Model fit vs. saturated
output = "text")##
## Model Test User Model:
##
## Test statistic 18631.556
## Degrees of freedom 265
## P-value 0.000
##
## User Model versus Baseline Model:
##
## Comparative Fit Index (CFI) 0.885
## Tucker-Lewis Index (TLI) 0.869
##
## Root Mean Square Error of Approximation:
##
## RMSEA 0.070
##
## Standardized Root Mean Square Residual:
##
## SRMR 0.048fitMeasures(fit_1f,
fit.measures = c("npar", # Estimated parameters
"chisq", "df", "pvalue", # Model fit vs. saturated
"cfi", "tli", # Model fit vs. baseline
"rmsea", "srmr"), # Model fit vs. saturated
output = "text")##
## Model Test User Model:
##
## Test statistic 49510.917
## Degrees of freedom 275
## P-value 0.000
##
## User Model versus Baseline Model:
##
## Comparative Fit Index (CFI) 0.691
## Tucker-Lewis Index (TLI) 0.663
##
## Root Mean Square Error of Approximation:
##
## RMSEA 0.112
##
## Standardized Root Mean Square Residual:
##
## SRMR 0.087Click for explanation
The one-factor model definitely seems to fit worse than the five-factor model.
5.4.6
Given the CFA results from the five factor model, would a second-order CFA be appropriate for the Attitudes towards Immigration data? Why or why not?
Click for explanation
Yes, a second-order CFA model is a theoretically appropriate representation of the Attitudes towards Immigration items.
- The first order latent variables in the five-factor model are all significantly correlated.
- The first order latent variables in the five-factor model seem to tap different aspects of some single underlying construct.
5.4.7
Define the lavaan model syntax for a second-order CFA model of the Attitudes
towards Immigration items, estimate it, and inspect the results.
- Use the five factors defined in 5.4.2 as the first order factors.
Click to show code
mod_2o <- paste(mod_5f,
'ati =~ ip + rp + st + ct + et',
sep = '\n')
fit_2o <- cfa(mod_2o, data = ess)
summary(fit_2o, fit.measures = TRUE)## lavaan 0.6-18 ended normally after 94 iterations
##
## Estimator ML
## Optimization method NLMINB
## Number of model parameters 55
##
## Used Total
## Number of observations 14243 19690
##
## Model Test User Model:
##
## Test statistic 19121.111
## Degrees of freedom 270
## P-value (Chi-square) 0.000
##
## Model Test Baseline Model:
##
## Test statistic 159619.058
## Degrees of freedom 300
## P-value 0.000
##
## User Model versus Baseline Model:
##
## Comparative Fit Index (CFI) 0.882
## Tucker-Lewis Index (TLI) 0.869
##
## Loglikelihood and Information Criteria:
##
## Loglikelihood user model (H0) -520279.910
## Loglikelihood unrestricted model (H1) -510719.354
##
## Akaike (AIC) 1040669.820
## Bayesian (BIC) 1041085.841
## Sample-size adjusted Bayesian (SABIC) 1040911.056
##
## Root Mean Square Error of Approximation:
##
## RMSEA 0.070
## 90 Percent confidence interval - lower 0.069
## 90 Percent confidence interval - upper 0.071
## P-value H_0: RMSEA <= 0.050 0.000
## P-value H_0: RMSEA >= 0.080 0.000
##
## Standardized Root Mean Square Residual:
##
## SRMR 0.048
##
## Parameter Estimates:
##
## Standard errors Standard
## Information Expected
## Information saturated (h1) model Structured
##
## Latent Variables:
## Estimate Std.Err z-value P(>|z|)
## ip =~
## imrcntr 1.000
## eimrcnt 0.943 0.011 85.095 0.000
## eimpcnt 1.126 0.010 113.523 0.000
## imsmetn 0.982 0.010 98.910 0.000
## impcntr 1.149 0.010 113.651 0.000
## imdfetn 1.130 0.010 111.789 0.000
## st =~
## imbgeco 1.000
## imbleco 0.822 0.012 68.916 0.000
## imwbcnt 1.047 0.012 88.172 0.000
## imwbcrm 0.709 0.011 62.846 0.000
## imtcjob 0.747 0.011 66.424 0.000
## imueclt 1.013 0.013 78.434 0.000
## rp =~
## gvrfgap 1.000
## imrsprc 0.854 0.017 51.127 0.000
## rfgbfml 1.048 0.019 55.377 0.000
## rfggvfn 0.853 0.017 51.170 0.000
## rfgawrk 0.657 0.016 40.785 0.000
## rfgfrpc -0.828 0.016 -51.249 0.000
## shrrfg -1.020 0.017 -58.369 0.000
## ct =~
## qfimchr 1.000
## qfimwht 0.939 0.018 51.902 0.000
## pplstrd -0.389 0.008 -51.072 0.000
## vrtrlg 0.271 0.006 41.908 0.000
## et =~
## imwgdwn 1.000
## imhecop 1.158 0.024 48.877 0.000
## ati =~
## ip 1.000
## rp 1.264 0.024 53.732 0.000
## st -3.123 0.051 -61.058 0.000
## ct 2.638 0.058 45.467 0.000
## et -1.000 0.024 -42.490 0.000
##
## Variances:
## Estimate Std.Err z-value P(>|z|)
## .imrcntr 0.299 0.004 77.900 0.000
## .eimrcnt 0.359 0.005 79.597 0.000
## .eimpcnt 0.116 0.002 62.698 0.000
## .imsmetn 0.211 0.003 75.502 0.000
## .impcntr 0.119 0.002 62.476 0.000
## .imdfetn 0.133 0.002 65.406 0.000
## .imbgeco 2.285 0.033 70.158 0.000
## .imbleco 2.852 0.037 76.762 0.000
## .imwbcnt 1.668 0.027 62.920 0.000
## .imwbcrm 2.821 0.036 78.653 0.000
## .imtcjob 2.646 0.034 77.607 0.000
## .imueclt 2.734 0.038 72.213 0.000
## .gvrfgap 0.740 0.010 73.738 0.000
## .imrsprc 0.797 0.010 77.211 0.000
## .rfgbfml 0.885 0.012 74.621 0.000
## .rfggvfn 0.791 0.010 77.189 0.000
## .rfgawrk 0.946 0.012 80.833 0.000
## .rfgfrpc 0.741 0.010 77.149 0.000
## .shrrfg 0.665 0.009 72.020 0.000
## .qfimchr 5.347 0.081 65.623 0.000
## .qfimwht 4.084 0.065 62.673 0.000
## .pplstrd 0.778 0.012 64.838 0.000
## .vrtrlg 0.854 0.011 75.931 0.000
## .imwgdwn 0.655 0.012 52.977 0.000
## .imhecop 0.468 0.014 33.353 0.000
## .ip 0.177 0.004 44.418 0.000
## .st 0.596 0.023 26.030 0.000
## .rp 0.101 0.005 21.784 0.000
## .ct 1.745 0.060 29.185 0.000
## .et 0.316 0.010 31.813 0.000
## ati 0.204 0.005 37.371 0.0005.4.8
Compare the model fit of the first- and second-order five-factor models using
the fitMeasures() function.
- Which model offers the better fit?
- Which model is more complex?
Click to show code
fitMeasures(fit_5f,
fit.measures = c("npar", # Estimated parameters
"chisq", "df", "pvalue", # Model fit vs. saturated
"cfi", "tli", # Model fit vs. baseline
"rmsea", "srmr"), # Model fit vs. saturated
output = "text")##
## Model Test User Model:
##
## Test statistic 18631.556
## Degrees of freedom 265
## P-value 0.000
##
## User Model versus Baseline Model:
##
## Comparative Fit Index (CFI) 0.885
## Tucker-Lewis Index (TLI) 0.869
##
## Root Mean Square Error of Approximation:
##
## RMSEA 0.070
##
## Standardized Root Mean Square Residual:
##
## SRMR 0.048fitMeasures(fit_2o,
fit.measures = c("npar", # Estimated parameters
"chisq", "df", "pvalue", # Model fit vs. saturated
"cfi", "tli", # Model fit vs. baseline
"rmsea", "srmr"), # Model fit vs. saturated
output = "text")##
## Model Test User Model:
##
## Test statistic 19121.111
## Degrees of freedom 270
## P-value 0.000
##
## User Model versus Baseline Model:
##
## Comparative Fit Index (CFI) 0.882
## Tucker-Lewis Index (TLI) 0.869
##
## Root Mean Square Error of Approximation:
##
## RMSEA 0.070
##
## Standardized Root Mean Square Residual:
##
## SRMR 0.048Click for explanation
The CFI and TLI are both slightly better in the original five factor model, but the RMSEA and SRMR of both models don’t differ out to three decimal places. As usual, both models have a significant \(\chi^2\), but that doesn’t tell us much.
Qualitative comparisons of model fit are fine, but we’d like to have an actual statistical test for these fit differences. As it happens, we have just such a test: a nested model \(\Delta \chi^2\) test (AKA, chi-squared difference test, change in chi-squared test, likelihood ratio test).
In the Week 7 lecture, we’ll cover nested models and tests thereof, but it will be useful to start thinking about these concepts now. Two models are said to be nested if you can define one model by placing constraints on the other model.
By way of example, consider the following two CFA models.