11  What the models can and cannot tell us

The models in this compendium were fitted to the same dataset, and most of them added one feature to the previous one.

11.1 The sequence of models and the questions they answer

The models follow the order of the workshop. Table 11.1 lists the question that each model answers together with its fit. Loneliness scores are modeled from the no growth model to the multigroup models. Anxiety and depression symptoms are used in the parallel and sequential models.

Code 
rng <- function(x, d = 3) {
  r <- round(range(x), d)
  if (r[1] == r[2]) nz(r[1]) else paste(nz(r[1]), "to", nz(r[2]))
}
fitrng <- diag %>%
  group_by(family) %>%
  summarize(N = paste(unique(range(n_listwise)), collapse = " "),
            CFI = rng(cfi), RMSEA = rng(rmsea), .groups = "drop")
qs <- data.frame(
  family = c("No growth", "Linear", "Quadratic", "Bilinear",
             "Time-varying covariate", "Time-invariant covariate", "Both covariates",
             "Distal outcome", "Multigroup", "Parallel", "Sequential"),
  Question = c(
    "Is one constant level enough, with differences between people but no change over time?",
    "Is average change a straight line, and do people differ in starting level and rate of change?",
    "Does the rate of change itself change, so that the trajectory curves?",
    "Does the rate of change differ between two phases of the observation period? (three codings of time)",
    "Is the score at each wave related to hostility at the same wave, with growth in the model?",
    "Do starting level and rate of change differ by the variable female?",
    "Do both relations hold when the two kinds of covariate are in the same model?",
    "Are starting level and rate of change related to substance use at wave 5?",
    "Do the growth parameters differ between the two groups of female, once equality constraints are released step by step?",
    "Do the growth factors of anxiety and depression, measured at the same waves, go together?",
    "How is a model for a later process built? A demonstration of the syntax only."))
tab <- merge(qs, fitrng, by = "family", sort = FALSE)
tab <- tab[match(qs$family, tab$family), ]
names(tab)[1] <- "Model"
knitr::kable(tab, row.names = FALSE, align = "llrrr")
Table 11.1: Models of the compendium, the question each answers, and the fit of the workshop fits
Model Question N CFI RMSEA
No growth Is one constant level enough, with differences between people but no change over time? 321 .607 .204
Linear Is average change a straight line, and do people differ in starting level and rate of change? 321 .852 .143
Quadratic Does the rate of change itself change, so that the trajectory curves? 321 .908 .145
Bilinear Does the rate of change differ between two phases of the observation period? (three codings of time) 321 .921 .134
Time-varying covariate Is the score at each wave related to hostility at the same wave, with growth in the model? 321 .975 .087
Time-invariant covariate Do starting level and rate of change differ by the variable female? 320 .840 .133
Both covariates Do both relations hold when the two kinds of covariate are in the same model? 320 .976 .077
Distal outcome Are starting level and rate of change related to substance use at wave 5? 290 .854 .128
Multigroup Do the growth parameters differ between the two groups of female, once equality constraints are released step by step? 320 .609 to .661 .179 to .184
Parallel Do the growth factors of anxiety and depression, measured at the same waves, go together? 321 .584 to .927 .101 to .242
Sequential How is a model for a later process built? A demonstration of the syntax only. 321 .581 .238

The columns CFI and RMSEA give the range over the fits within a family. Several models were fitted for the bilinear, multigroup and parallel questions. The column N is the number of participants used.

The models describe change in these scores. They describe the average trajectory, the differences between people around it, and how these differences are related to other variables. They cannot tell why people change. The data are observational, so a coefficient from a covariate on the slope factor describes an association in this dataset, and the models give no basis for causal statements.

11.2 How to report a growth model

  • Describe the data. State the number of participants, the number of waves, the number of participants used in the model and how missing data were handled (Chapter 1).
  • State the time scores. Give the values and say what time zero means. The coding of time changes the meaning of the intercept (Biesanz et al., 2004).
  • Name the software and the estimator. In this compendium the models were fitted with lavaan version 0.6.17 (Rosseel, 2012).
  • Report the fit. Give the chi-square statistic with its degrees of freedom, CFI, RMSEA with its confidence interval and SRMR. Cut-offs are guides and not universal rules (Hu & Bentler, 1999). If the fit is poor, say so. Remember that only models with acceptable fit should be compared and interpreted substantively.
  • Report the estimates. Give the means of the intercept and slope factors, their variances, the covariance between them and the residual variances, each with a standard error.
  • Report model comparisons in words. When a likelihood ratio test is significant, the reduced model is rejected and the more complex model fits better. Say which model is nested in which. A test of a variance fixed to zero is a test on the boundary of the parameter space, and the usual chi-square reference distribution does not apply (Self & Liang, 1987).
  • Report warnings. If lavaan warns about a latent covariance matrix that is not positive definite or about a negative variance, say so and say what was done about it.
  • Report checks. If you ran a sensitivity analysis, for example with a different missing data method, report it as such.

11.3 Limitations of this compendium

Listwise deletion. The workshop code calls growth() with its default settings, which is listwise deletion (Chapter 1). Of the 442 rows in the dataset, 321 were used for the linear model and 290 for the distal outcome model. The multigroup models lose one further row (N = 320) because of a missing value on female. As an addition to the workshop, the models were refitted with full information maximum likelihood (FIML) and used between 412 and 439 participants. For the linear model the mean of the intercept factor is 1.466 under listwise deletion and 1.490 under FIML, and the mean of the slope factor is -0.025 and -0.027. The variance of the intercept factor is larger under FIML (0.139 against 0.165). The chapters report the listwise results.

Skewed scores. Across the loneliness, anxiety and depression variables the skewness ranges from 1.5 to 3.8 and the kurtosis goes up to 19.7 (see Chapter 1 for the caution that follows from this). The models were also refitted with FIML and the robust ML estimator (Yuan & Bentler, 2000), again as an addition to the workshop. One fit is left out of the FIML and robust statistics, the time-invariant covariate model, because in that refit the mean of the covariate was not modeled (see Chapter 5) and its fit is not comparable. For the remaining 17 fits the robust CFI is higher than the listwise one (for the linear model .902 against .852), but 9 of them still have a robust CFI below .90. A scaled difference test is the tool for comparing nested models under a robust estimator (Satorra & Bentler, 2001). The model comparisons in the chapters use the ordinary likelihood ratio test. This test is unreliable when the models are misspecified, and as noted above when a variance is close to its boundary, which is the case for several slope factors.

Poor fit. By CFI, 11 of the 18 workshop fits are below .90, and only 2 are at or above .95. The two models with a time-varying covariate fit well (CFI .975 and .976). The multigroup models and the models with anxiety and depression fit poorly. The parallel model with cross-domain residual covariances has a higher CFI (.927), but its RMSEA is .101 and its SRMR is .141, so it is not well fitting either. The estimates of the poorly fitting models limit what can be said about this dataset.

Near-zero variances and warnings. Of the 18 fits, 8 are flagged as not admissible. For the quadratic, the three bilinear, the first two parallel and the sequential model, lavaan warns that the covariance matrix of the latent variables is not positive definite, and in these fits the variance of at least one growth factor is very small. In the parallel model with cross-domain residual covariances the warning is about negative latent variances, which are improper solutions of the kind described by Dillon et al. (1987). The options are listed in Chapter 9. We kept the workshop syntax and reported the warnings instead.

The sequential model. As explained in Chapter 10, the sequential model of the workshop uses the same five anxiety and five depression measurements under new labels.

11.4 Where to go next

This compendium does not cover the following topics.

  • Multilevel formulation of growth models. The same models can be written as multilevel models, as described by Singer and Willett (2003), Raudenbush and Bryk (2002) and Grimm et al. (2016). This is also the route for data in which participants are nested in higher-level units.
  • Individually varying times of observation. Models in which the time of measurement differs between participants, as described by Mehta and West (2000).
  • Nonlinear growth. Other functional forms for the trajectory, as described by Grimm et al. (2011).
  • Within-person and between-person effects. The separation of the two kinds of effect for time-varying covariates, as described by Curran and Bauer (2011).
  • Autoregressive and cross-lagged processes. Combining autoregressive paths with growth factors, as described by Bollen and Curran (2004), and the criticism of the cross-lagged panel model by Hamaker et al. (2015).
  • Growth mixture models. Models in which different groups of people have different trajectories, as described by Ram and Grimm (2009). The mixture modeling on which they build is described by Muthén and Shedden (1999).
  • Measurement invariance over time and groups. Statistical approaches are given by Millsap (2011) and reporting conventions by Putnick and Bornstein (2016).
  • Missing data. Applied missing data analysis is covered by Enders (2010).
  • Longitudinal structural equation modeling in general. Little (2013) and Bollen and Curran (2006) give book-length treatments.