This file contains final MLMs for reaction times on the flanker task for the PD Inhibition DDM project.

Note: I have adopted the common convention of only analyzing RTs for correct responses. This assumes that whatever process gives rise to the RT for an incorrect response is not commensurable. The alternative would be to allow for correct to enter into the model as a predict of the current trial’s RT. This could support moderation analyses – for example, whether error RTs are faster in the mostly congruent condition for incongruent trials. The downside is all of the ensuing multi-way interactions, many of which are not of central interest.

N.B. For the AIC comparisons to be valid, we need to use method=‘ML’ throughout. Once we have a winning model, or one where the parameters matter, re-estimate with REML. update(mm, method='REML')

To reduce the number of models, I don’t consider block diagonal matrices for L2 random effects. This is achieved with pdDiag() and basically estimates a unique variance component for each level of a factor. But the fits of these models are extremely close to the more conventional block within id id/block parameterization for random effects. Given that the latter is much more conventional, let’s stick with that when we want to check for heterogeneity in L2 by condition.

LME approach

These models permit heterogeneous L1 residual variability

m1: design effects, random intercept of subject

minfo[["m1"]] <- c(fixed="Congruency, Block", l1="Homogeneous", l2="Subject: Intercept")
m1 <- lme(rt_inv ~ cond*block, 
          random = ~ 1 | id,
          na.action = na.exclude, 
          data = flanker, method='ML')

m1_reml <- update(m1, method='REML')

flextable(car::Anova(m1, type=3)) %>% autofit()

Chisq	Df	Pr(>Chisq)
8196.32	1	0.00e+00
162.18	1	3.78e-37
7.38	1	6.58e-03
14.82	1	1.18e-04

#Get model-predicted means in original RT scale. Force back onto original RT using custom function
rg <- update(ref_grid(m1_reml), tran=list(linkinv=function(x) { -1000/x }, mu.eta=function(x) { -1000/x }), predict.type="response")
res_trans <- emmeans(rg, ~cond|block)

flextable(as.data.frame(res_trans)) %>% colformat_num(j=c("response", "SE", "lower.CL", "upper.CL"), digits=2) %>% autofit()

cond	block	response	SE	df	lower.CL	upper.CL
congruent	most_incon	370.13	11.09	106	362.17	378.46
incongruent	most_incon	386.56	11.38	106	378.02	395.48
congruent	most_con	373.50	11.00	106	365.54	381.83
incongruent	most_con	397.91	11.95	106	388.70	407.56

m2: design effects, allow for L2 random intercept to vary by block

Use nested RE specification.

minfo[["m2"]] <- c(fixed="Congruency, Block", l1="Homogeneous", l2="Block Within Subject: Intercept")
m2 <- lme(rt_inv ~ cond*block,
          random=~1|id/block,
          na.action = na.exclude,
          data = flanker, method='ML')

flextb_vif(m2)

Predictor	vif
cond	2.08
block	1.39
cond:block	2.09

flextable(car::Anova(m2, type=3)) %>% autofit()

Chisq	Df	Pr(>Chisq)
7997.73	1	0.00e+00
166.21	1	4.98e-38
3.62	1	5.71e-02
14.51	1	1.39e-04

m3: design effects, allow for L1 residual variance to vary by block

minfo[["m3"]] <- c(fixed="Congruency, Block", l1="Per Block", l2="Block within subject: Intercept")
m3 <- lme(rt_inv ~ cond*block,
          random=~1|id/block,
          weights=varIdent(form=~1|block),
          na.action = na.exclude,
          data = flanker, method='ML')
summary(m3)

## Linear mixed-effects model fit by maximum likelihood
##  Data: flanker 
##     AIC   BIC logLik
##   14536 14597  -7260
## 
## Random effects:
##  Formula: ~1 | id
##         (Intercept)
## StdDev:       0.295
## 
##  Formula: ~1 | block %in% id
##         (Intercept) Residual
## StdDev:      0.0666    0.358
## 
## Variance function:
##  Structure: Different standard deviations per stratum
##  Formula: ~1 | block 
##  Parameter estimates:
## most_incon   most_con 
##       1.00       1.07 
## Fixed effects: rt_inv ~ cond * block 
##                                Value Std.Error    DF t-value p-value
## (Intercept)                   -2.702   0.03015 16039   -89.6  0.0000
## condincongruent                0.115   0.00864 16039    13.4  0.0000
## blockmost_con                  0.024   0.01265   106     1.9  0.0581
## condincongruent:blockmost_con  0.049   0.01275 16039     3.8  0.0001
##  Correlation: 
##                               (Intr) cndncn blckm_
## condincongruent               -0.198              
## blockmost_con                 -0.244  0.471       
## condincongruent:blockmost_con  0.134 -0.677 -0.478
## 
## Standardized Within-Group Residuals:
##     Min      Q1     Med      Q3     Max 
## -4.4977 -0.5985  0.0157  0.5960  4.1934 
## 
## Number of Observations: 16255
## Number of Groups: 
##            id block %in% id 
##           107           214

#stargazer(m3, type="html")
flextable(car::Anova(m3, type=3)) %>% autofit()

Chisq	Df	Pr(>Chisq)
8031.10	1	0.00e+00
178.53	1	1.01e-40
3.67	1	5.54e-02
14.49	1	1.41e-04

flextable(render_aictab(minfo=minfo, m1, m2, m3)) %>% autofit()

model	fixed	l1	l2	logLik	AIC	dAIC	df	weight
m1	Congruency, Block	Homogeneous	Subject: Intercept	-7343	14698	162.6	6	4.98e-36
m2	Congruency, Block	Homogeneous	Block Within Subject: Intercept	-7279	14571	35.9	7	1.59e-08
m3	Congruency, Block	Per Block	Block within subject: Intercept	-7260	14536	0.0	8	1.00e+00

m4: Allow L1 residual variability to depend on both condition and block

minfo[["m4"]] <- c(fixed="Congruency, Block", l1="Per Block and Condition", l2="Block within subject: Intercept")
m4 <- lme(rt_inv ~ cond*block,
          random=~1|id/block,
          weights=varIdent(form=~1|block*cond),
          na.action = na.exclude,
          data = flanker, method='ML')
summary(m4)

## Linear mixed-effects model fit by maximum likelihood
##  Data: flanker 
##     AIC   BIC logLik
##   14530 14607  -7255
## 
## Random effects:
##  Formula: ~1 | id
##         (Intercept)
## StdDev:       0.296
## 
##  Formula: ~1 | block %in% id
##         (Intercept) Residual
## StdDev:      0.0667    0.361
## 
## Variance function:
##  Structure: Different standard deviations per stratum
##  Formula: ~1 | block * cond 
##  Parameter estimates:
## most_incon*incongruent   most_incon*congruent     most_con*congruent   most_con*incongruent 
##                  1.000                  0.982                  1.049                  1.104 
## Fixed effects: rt_inv ~ cond * block 
##                                Value Std.Error    DF t-value p-value
## (Intercept)                   -2.702   0.03014 16039   -89.6  0.0000
## condincongruent                0.115   0.00858 16039    13.5  0.0000
## blockmost_con                  0.024   0.01258   106     1.9  0.0569
## condincongruent:blockmost_con  0.048   0.01286 16039     3.8  0.0002
##  Correlation: 
##                               (Intr) cndncn blckm_
## condincongruent               -0.194              
## blockmost_con                 -0.242  0.465       
## condincongruent:blockmost_con  0.130 -0.667 -0.464
## 
## Standardized Within-Group Residuals:
##     Min      Q1     Med      Q3     Max 
## -4.4718 -0.6017  0.0136  0.5967  4.1736 
## 
## Number of Observations: 16255
## Number of Groups: 
##            id block %in% id 
##           107           214

flextable(render_aictab(minfo=minfo, m1, m2, m3, m4)) %>% autofit()

model	fixed	l1	l2	logLik	AIC	dAIC	df	weight
m1	Congruency, Block	Homogeneous	Subject: Intercept	-7343	14698	168.26	6	2.74e-37
m2	Congruency, Block	Homogeneous	Block Within Subject: Intercept	-7279	14571	41.59	7	8.78e-10
m3	Congruency, Block	Per Block	Block within subject: Intercept	-7260	14536	5.68	8	5.51e-02
m4	Congruency, Block	Per Block and Condition	Block within subject: Intercept	-7255	14530	0.00	10	9.45e-01

Interim conclusions about random effects structure:

Allow for per-block random intercept (nested random effect)
Allow for L1 residual variance to vary by block and trial condition

But!! Obtaining p-values and denominator degrees of freedom is a somewhat tortured process because neither Satterthwaite nor Kenward-Roger corrections for degrees of freedom are provided. Moreover, we don’t have any conceptual interest in L1 heterogeneity. Likewise, when we switch to glmer for accuracy, we no longer have the option for heterogeneous L1 variances since nlme doesn’t provide a binary logistic model as far as I know.

Thus, switch to lmer() since the best-fitting model (m30) depends primarily on nested L2 random effects. This makes it easier to switch to Mplus, too.

LMER-based modeling

m1: design effects, random intercept of subject

minfo <- list() #re-initialize now that we're eliminating l1 heterogeneity

minfo[["m1"]] <- c(fixed="Congruency, Block", l2="Subject: Intercept")
m1 <- lmer(rt_inv ~ cond*block + (1|id),
          na.action = na.exclude, 
          data = flanker, REML=FALSE)
m1_reml <- update(m1, REML=TRUE)

flextable(anova(m1) %>% rownames_to_column("Effect")) %>% autofit() #lmerTest reports Satterthwaite degrees of freedom

Effect	Sum Sq	Mean Sq	NumDF	DenDF	F value	Pr(>F)
cond	66.10	66.10	1	16149	471.4	4.70e-103
block	8.21	8.21	1	16148	58.5	2.12e-14
cond:block	2.08	2.08	1	16148	14.8	1.19e-04

#flextable(car::Anova(m1, type=3)) %>% autofit()

#Get model-predicted means in original RT scale. Force back onto original RT using custom function
rg <- update(ref_grid(m1_reml), tran=list(linkinv=function(x) { -1000/x }, mu.eta=function(x) { -1000/x }), predict.type="response")
res_trans <- emmeans(rg, ~cond|block, lmer.df = "satterthwaite")

# emmeans(m1_reml, ~cond|block, lmer.df = "satterthwaite") #in inverse units

flextable(as.data.frame(res_trans)) %>% 
  colformat_num(j=c("response", "SE", "lower.CL", "upper.CL"), digits=2) %>% 
  autofit()

cond	block	response	SE	df	lower.CL	upper.CL
congruent	most_incon	370.13	11.09	118	362.18	378.45
incongruent	most_incon	386.56	11.38	110	378.03	395.48
congruent	most_con	373.50	11.00	110	365.54	381.82
incongruent	most_con	397.91	11.95	119	388.71	407.55

m2: design effects, allow for L2 random intercept to vary by block

Use nested RE specification.

minfo[["m2"]] <- c(fixed="Congruency, Block", l2="Block within subject: Intercept")
m2 <- lmer(rt_inv ~ cond*block + (1|id/block),
          na.action = na.exclude,
          data = flanker, REML=FALSE)
m2_reml <- update(m2, REML=TRUE)

flextable(anova(m2) %>% rownames_to_column("Effect")) %>% autofit()

Effect	Sum Sq	Mean Sq	NumDF	DenDF	F value	Pr(>F)
cond	66.21	66.21	1	16052	479.9	7.75e-105
block	2.63	2.63	1	120	19.0	2.73e-05
cond:block	2.00	2.00	1	16073	14.5	1.40e-04

#car::Anova(m2, type=3)
flextable(render_aictab(minfo=minfo, m1, m2)) %>% autofit()

model	fixed	l2	logLik	AIC	dAIC	df	weight
m1	Congruency, Block	Subject: Intercept	-7343	14698	127	6	3.12e-28
m2	Congruency, Block	Block within subject: Intercept	-7279	14571	0	7	1.00e+00

rg <- update(ref_grid(m2_reml), 
             tran=list(linkinv=function(x) { -1000/x }, mu.eta=function(x) { -1000/x }), 
             predict.type="response")

res_trans <- emmeans(rg, ~cond|block, lmer.df = "satterthwaite")
pairs(res_trans)

## Note: Use 'contrast(regrid(object), ...)' to obtain contrasts of back-transformed estimates

## block = most_incon:
##  contrast                estimate      SE    df t.ratio p.value
##  congruent - incongruent   -0.115 0.00895 16069 -12.890 <.0001 
## 
## block = most_con:
##  contrast                estimate      SE    df t.ratio p.value
##  congruent - incongruent   -0.164 0.00908 16052 -18.060 <.0001 
## 
## Note: contrasts are still on the custom scale 
## Degrees-of-freedom method: satterthwaite

# emmeans(m1_reml, ~cond|block, lmer.df = "satterthwaite") #in inverse units

flextable(as.data.frame(res_trans)) %>% 
  colformat_num(j=c("response", "SE", "lower.CL", "upper.CL"), digits=2) %>% 
  autofit()

cond	block	response	SE	df	lower.CL	upper.CL
congruent	most_incon	370.13	11.23	123	362.08	378.55
incongruent	most_incon	386.64	11.54	115	378.00	395.68
congruent	most_con	373.48	11.14	115	365.42	381.91
incongruent	most_con	397.84	12.09	124	388.54	407.59

pairs(emmeans(m2_reml, ~cond|block))

## block = most_incon:
##  contrast                estimate      SE    df t.ratio p.value
##  congruent - incongruent   -0.115 0.00895 16069 -12.890 <.0001 
## 
## block = most_con:
##  contrast                estimate      SE    df t.ratio p.value
##  congruent - incongruent   -0.164 0.00908 16052 -18.060 <.0001 
## 
## Degrees-of-freedom method: satterthwaite

pairs(emmeans(m2_reml, ~ cond * block), simple="each")

## $`simple contrasts for cond`
## block = most_incon:
##  contrast                estimate      SE    df t.ratio p.value
##  congruent - incongruent   -0.115 0.00895 16069 -12.890 <.0001 
## 
## block = most_con:
##  contrast                estimate      SE    df t.ratio p.value
##  congruent - incongruent   -0.164 0.00908 16052 -18.060 <.0001 
## 
## Degrees-of-freedom method: satterthwaite 
## 
## $`simple contrasts for block`
## cond = congruent:
##  contrast              estimate     SE  df t.ratio p.value
##  most_incon - most_con  -0.0242 0.0128 204 -1.900  0.0594 
## 
## cond = incongruent:
##  contrast              estimate     SE  df t.ratio p.value
##  most_incon - most_con  -0.0728 0.0129 213 -5.630  <.0001 
## 
## Degrees-of-freedom method: satterthwaite

pairs(pairs(emmeans(m2_reml, ~cond|block)), by=NULL) #difference of differences

##  contrast                                                              estimate     SE    df t.ratio p.value
##  congruent - incongruent,most_incon - congruent - incongruent,most_con   0.0485 0.0127 16071 3.810   0.0001 
## 
## Degrees-of-freedom method: satterthwaite

m3: Add trial_z as fixed

minfo[["m3"]] <- c(fixed="Congruency, Block, Trial", l2="Block within subject: Intercept")
m3 <- lmer(rt_inv ~ cond*block + trial_z + (1|id/block),
          na.action = na.exclude,
          data = flanker, REML=FALSE)

m4: Add trial_z as random, too, but only at subject level, not block-specific

minfo[["m4"]] <- c(fixed="Congruency, Block, Trial", l2="Block within subject:Intercept\nSubject: Trial")
m4 <- lmer(rt_inv ~ cond*block + trial_z + (1 + trial_z | id) + (1 | id:block),
          na.action = na.exclude,
          data = flanker, REML=FALSE)

m5: Add trial_z as random both per subject and block

minfo[["m5"]] <- c(fixed="Congruency, Block, Trial", l2="Block within subject: Intercept, Trial")
m5 <- lmer(rt_inv ~ cond*block + trial_z + (1 + trial_z | id/block),
          na.action = na.exclude,
          data = flanker, REML=FALSE)

summary(m5)

## Linear mixed model fit by maximum likelihood . t-tests use Satterthwaite's method ['lmerModLmerTest']
## Formula: rt_inv ~ cond * block + trial_z + (1 + trial_z | id/block)
##    Data: flanker
## 
##      AIC      BIC   logLik deviance df.resid 
##    13910    14003    -6943    13886    16243 
## 
## Scaled residuals: 
##    Min     1Q Median     3Q    Max 
## -4.490 -0.599  0.014  0.592  4.297 
## 
## Random effects:
##  Groups   Name        Variance Std.Dev. Corr 
##  block:id (Intercept) 0.00462  0.0680        
##           trial_z     0.00687  0.0829   0.16 
##  id       (Intercept) 0.08809  0.2968        
##           trial_z     0.00271  0.0520   -0.12
##  Residual             0.12951  0.3599        
## Number of obs: 16255, groups:  block:id, 214; id, 107
## 
## Fixed effects:
##                                  Estimate  Std. Error          df t value Pr(>|t|)    
## (Intercept)                      -2.70173     0.03030   123.72588  -89.16  < 2e-16 ***
## condincongruent                   0.11571     0.00868 15870.60190   13.33  < 2e-16 ***
## blockmost_con                     0.02343     0.01263   200.01585    1.85   0.0651 .  
## trial_z                          -0.02571     0.00848   102.10784   -3.03   0.0031 ** 
## condincongruent:blockmost_con     0.04934     0.01236 15854.09419    3.99 0.000066 ***
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## Correlation of Fixed Effects:
##             (Intr) cndncn blckm_ tril_z
## condncngrnt -0.198                     
## blockmst_cn -0.247  0.474              
## trial_z     -0.058 -0.001  0.022       
## cndncngrn:_  0.139 -0.702 -0.477  0.000

flextable(render_aictab(minfo=minfo, m1, m2, m3, m4, m5)) %>% autofit()

model	fixed	l2	logLik	AIC	dAIC	df	weight
m1	Congruency, Block	Subject: Intercept	-7343	14698	787.8	6	8.38e-172
m2	Congruency, Block	Block within subject: Intercept	-7279	14571	661.2	7	2.68e-144
m3	Congruency, Block, Trial	Block within subject: Intercept	-7242	14500	589.9	8	8.11e-129
m4	Congruency, Block, Trial	Block within subject:Intercept Subject: Trial	-6991	14002	91.7	10	1.20e-20
m5	Congruency, Block, Trial	Block within subject: Intercept, Trial	-6943	13910	0.0	12	1.00e+00

m6: Add trial x block interaction as fixed

minfo[["m6"]] <- c(
  fixed="Congruency, Block, Congruency x Block, Trial, Block x Trial", 
  l2="Block within subject: Intercept, Trial")

m6 <- lmer(rt_inv ~ cond*block + trial_z*block + (1 + trial_z | id/block),
          na.action = na.exclude,
          data = flanker, REML=FALSE)

flextable(render_aictab(minfo=minfo, m1, m2, m3, m4, m5, m6)) %>% autofit()

model	fixed	l2	logLik	AIC	dAIC	df	weight
m1	Congruency, Block	Subject: Intercept	-7343	14698	787.84	6	6.09e-172
m2	Congruency, Block	Block within subject: Intercept	-7279	14571	661.17	7	1.95e-144
m3	Congruency, Block, Trial	Block within subject: Intercept	-7242	14500	589.88	8	5.89e-129
m4	Congruency, Block, Trial	Block within subject:Intercept Subject: Trial	-6991	14002	91.74	10	8.72e-21
m5	Congruency, Block, Trial	Block within subject: Intercept, Trial	-6943	13910	0.00	12	7.27e-01
m6	Congruency, Block, Congruency x Block, Trial, Block x Trial	Block within subject: Intercept, Trial	-6943	13912	1.96	13	2.73e-01

m7: Trial within run (run_trial) as fixed

trial x block is not significant, drop from consideration

minfo[["m7"]] <- c(
  fixed="Congruency, Block, Congruency x Block, Trial, Run Trial", 
  l2="Block within subject: Intercept, Trial")

m7 <- lmer(rt_inv ~ cond*block + trial_z + run_trial_z + (1 + trial_z | id/block),
          na.action = na.exclude,
          data = flanker, REML=FALSE)

flextable(render_aictab(minfo=minfo, m1, m2, m3, m4, m5, m6, m7)) %>% autofit()

model	fixed	l2	logLik	AIC	dAIC	df	weight
m1	Congruency, Block	Subject: Intercept	-7343	14698	811.9	6	4.89e-177
m2	Congruency, Block	Block within subject: Intercept	-7279	14571	685.3	7	1.57e-149
m3	Congruency, Block, Trial	Block within subject: Intercept	-7242	14500	614.0	8	4.73e-134
m4	Congruency, Block, Trial	Block within subject:Intercept Subject: Trial	-6991	14002	115.8	10	7.01e-26
m5	Congruency, Block, Trial	Block within subject: Intercept, Trial	-6943	13910	24.1	12	5.84e-06
m6	Congruency, Block, Congruency x Block, Trial, Block x Trial	Block within subject: Intercept, Trial	-6943	13912	26.1	13	2.20e-06
m7	Congruency, Block, Congruency x Block, Trial, Run Trial	Block within subject: Intercept, Trial	-6930	13886	0.0	13	1.00e+00

m8: Add run trial as random at subject level only

minfo[["m8"]] <- c(
  fixed="Congruency, Block, Trial, Run Trial", 
  l2="Block within subject: Intercept, Trial\nSubject: Run Trial")

m8 <- lmer(rt_inv ~ cond*block + trial_z + run_trial_z + 
             (1 + trial_z + run_trial_z | id) + 
             (1 + trial_z | id:block),
          na.action = na.exclude, control=lmerControl(optCtrl=list(maxfun=2e4)),
          data = flanker, REML=FALSE)

## Warning in checkConv(attr(opt, "derivs"), opt$par, ctrl = control$checkConv, : Model failed to converge with max|grad| = 0.00213811 (tol =
## 0.002, component 1)

flextb_vif(m8)

Predictor	vif
cond	2.07
block	1.36
trial_z	1.12
run_trial_z	1.12
cond:block	2.08

flextable(render_aictab(minfo=minfo, m1, m2, m3, m4, m5, m6, m7, m8)) %>% autofit()

model	fixed	l2	logLik	AIC	dAIC	df	weight
m1	Congruency, Block	Subject: Intercept	-7343	14698	915	6	2.37e-199
m2	Congruency, Block	Block within subject: Intercept	-7279	14571	788	7	7.57e-172
m3	Congruency, Block, Trial	Block within subject: Intercept	-7242	14500	717	8	2.29e-156
m4	Congruency, Block, Trial	Block within subject:Intercept Subject: Trial	-6991	14002	219	10	3.39e-48
m5	Congruency, Block, Trial	Block within subject: Intercept, Trial	-6943	13910	127	12	2.82e-28
m6	Congruency, Block, Congruency x Block, Trial, Block x Trial	Block within subject: Intercept, Trial	-6943	13912	129	13	1.06e-28
m7	Congruency, Block, Congruency x Block, Trial, Run Trial	Block within subject: Intercept, Trial	-6930	13886	103	13	4.83e-23
m8	Congruency, Block, Trial, Run Trial	Block within subject: Intercept, Trial Subject: Run Trial	-6876	13783	0	16	1.00e+00

m9: Add run trial as random at subject and block level

Indeed, we see a big boost allowing run_trial to be random at subject and block level

minfo[["m9"]] <- c(
  fixed="Congruency, Block, Congruency x Block, Trial, Run Trial", 
  l2="Block within subject:Intercept, Trial, Run Trial")

m9 <- lmer(rt_inv ~ cond*block + trial_z + run_trial_z + 
             (1 + trial_z + run_trial_z | id) + 
             (1 + trial_z + run_trial_z | id:block),
          na.action = na.exclude,
          data = flanker, REML=FALSE)

## boundary (singular) fit: see ?isSingular

if (isSingular(m9)) { print(check_singularity(m9)) }

## id.run_trial_z 
##              0

But! The singularity check reveals the run_trial_z RE at id level is zero. This suggests that the boost we’re getting in fit has to do with the correlation between run_trial_z and other REs at the id level.

If we drop run_trial_z at the id level, fit is definitely worse (m8 vs. m9). Thus, tolerate the singularity for now.

flextb_vif(m9)

Predictor	vif
cond	2.07
block	1.36
trial_z	1.13
run_trial_z	1.13
cond:block	2.08

flextable(render_aictab(minfo=minfo, m1, m2, m3, m4, m5, m6, m7, m8, m9)) %>% autofit()

model	fixed	l2	logLik	AIC	dAIC	df	weight
m1	Congruency, Block	Subject: Intercept	-7343	14698	1015	6	3.74e-221
m2	Congruency, Block	Block within subject: Intercept	-7279	14571	888	7	1.20e-193
m3	Congruency, Block, Trial	Block within subject: Intercept	-7242	14500	817	8	3.61e-178
m4	Congruency, Block, Trial	Block within subject:Intercept Subject: Trial	-6991	14002	319	10	5.35e-70
m5	Congruency, Block, Trial	Block within subject: Intercept, Trial	-6943	13910	227	12	4.46e-50
m6	Congruency, Block, Congruency x Block, Trial, Block x Trial	Block within subject: Intercept, Trial	-6943	13912	229	13	1.68e-50
m7	Congruency, Block, Congruency x Block, Trial, Run Trial	Block within subject: Intercept, Trial	-6930	13886	203	13	7.63e-45
m8	Congruency, Block, Trial, Run Trial	Block within subject: Intercept, Trial Subject: Run Trial	-6876	13783	100	16	1.58e-22
m9	Congruency, Block, Congruency x Block, Trial, Run Trial	Block within subject:Intercept, Trial, Run Trial	-6823	13683	0	19	1.00e+00

m10: Add prev_rt_z as fixed effect

Using the z-scored variant to get parameter estimates on a similar scale, which can speed up estimation and occasionally avoid convergence problems.

Gargantuan improvement for prev_rt_z. Also note that this knocks out the fixed effect of run_trial_z. Drop that moving forward.

minfo[["m10"]] <- c(
  fixed="Congruency, Block, Congruency x Block, Trial, Run Trial, Previous RT", 
  l2="Block within subject: Intercept, Trial, Run Trial")

m10 <- lmer(rt_inv ~ cond*block + trial_z + run_trial_z + prev_rt_z + 
              (1 + trial_z + run_trial_z | id/block),
          na.action = na.exclude,
          data = flanker, REML=FALSE)

## boundary (singular) fit: see ?isSingular

if (isSingular(m10)) { print(check_singularity(m10)) }

## id.run_trial_z 
##      0.0000655

Same problem as m9: the run_trial_z variance estimate is zero.

summary(m10)

## Linear mixed model fit by maximum likelihood . t-tests use Satterthwaite's method ['lmerModLmerTest']
## Formula: rt_inv ~ cond * block + trial_z + run_trial_z + prev_rt_z + (1 +      trial_z + run_trial_z | id/block)
##    Data: flanker
## 
##      AIC      BIC   logLik deviance df.resid 
##    11993    12147    -5977    11953    15757 
## 
## Scaled residuals: 
##    Min     1Q Median     3Q    Max 
## -4.598 -0.594  0.009  0.599  4.654 
## 
## Random effects:
##  Groups   Name        Variance Std.Dev. Corr       
##  block:id (Intercept) 0.002207 0.0470              
##           trial_z     0.003459 0.0588    0.11      
##           run_trial_z 0.001558 0.0395   -0.34 -0.51
##  id       (Intercept) 0.051824 0.2276              
##           trial_z     0.001400 0.0374   -0.15      
##           run_trial_z 0.000172 0.0131    0.74  0.55
##  Residual             0.117920 0.3434              
## Number of obs: 15777, groups:  block:id, 214; id, 107
## 
## Fixed effects:
##                                  Estimate  Std. Error          df t value Pr(>|t|)    
## (Intercept)                      -2.70722     0.02353   126.88543 -115.07  < 2e-16 ***
## condincongruent                   0.11658     0.00843 15269.86562   13.83  < 2e-16 ***
## blockmost_con                     0.01817     0.01045   266.94207    1.74   0.0832 .  
## trial_z                          -0.02017     0.00658    96.16376   -3.07   0.0028 ** 
## run_trial_z                      -0.00220     0.00423   161.87874   -0.52   0.6029    
## prev_rt_z                         0.11693     0.00355 15367.00287   32.97  < 2e-16 ***
## condincongruent:blockmost_con     0.04952     0.01199 15260.17171    4.13 0.000037 ***
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## Correlation of Fixed Effects:
##             (Intr) cndncn blckm_ tril_z rn_tr_ prv_r_
## condncngrnt -0.248                                   
## blockmst_cn -0.279  0.559                            
## trial_z     -0.070  0.000  0.017                     
## run_trial_z  0.177  0.001 -0.005 -0.235              
## prev_rt_z   -0.001 -0.004 -0.001  0.025  0.022       
## cndncngrn:_  0.174 -0.702 -0.563 -0.001  0.002  0.003
## convergence code: 0
## boundary (singular) fit: see ?isSingular

flextb_vif(m10)

Predictor	vif
cond	2.14
block	1.59
trial_z	1.06
run_trial_z	1.06
prev_rt_z	1.00
cond:block	2.15

flextable(render_aictab(minfo=minfo, m1, m2, m3, m4, m5, m6, m7, m8, m9, m10)) %>% autofit()

model	fixed	l2	logLik	AIC	dAIC	df	weight
m1	Congruency, Block	Subject: Intercept	-7343	14698	2705	6	0
m2	Congruency, Block	Block within subject: Intercept	-7279	14571	2578	7	0
m3	Congruency, Block, Trial	Block within subject: Intercept	-7242	14500	2507	8	0
m4	Congruency, Block, Trial	Block within subject:Intercept Subject: Trial	-6991	14002	2009	10	0
m5	Congruency, Block, Trial	Block within subject: Intercept, Trial	-6943	13910	1917	12	0
m6	Congruency, Block, Congruency x Block, Trial, Block x Trial	Block within subject: Intercept, Trial	-6943	13912	1919	13	0
m7	Congruency, Block, Congruency x Block, Trial, Run Trial	Block within subject: Intercept, Trial	-6930	13886	1893	13	0
m8	Congruency, Block, Trial, Run Trial	Block within subject: Intercept, Trial Subject: Run Trial	-6876	13783	1790	16	0
m9	Congruency, Block, Congruency x Block, Trial, Run Trial	Block within subject:Intercept, Trial, Run Trial	-6823	13683	1690	19	0
m10	Congruency, Block, Congruency x Block, Trial, Run Trial, Previous RT	Block within subject: Intercept, Trial, Run Trial	-5977	11993	0	20	1

m11: Allow prev_rt_z x run_trial interaction

Yes, there is an interaction (N.B. mean centering of prev_rt is essential to avoid collinearity).

minfo[["m11"]] <- c(
  fixed="Congruency, Block, Congruency x Block, Trial,\nRun Trial, Previous RT, Run Trial x Previous RT",
  l2="Block within subject: Intercept, Trial, Run Trial")

m11 <- lmer(rt_inv ~ cond*block + trial_z + run_trial_z * prev_rt_z +
              (1 + trial_z + run_trial_z | id/block),
          na.action = na.exclude,
          data = flanker, REML=FALSE)

## boundary (singular) fit: see ?isSingular

if (isSingular(m11)) { print(check_singularity(m11)) }

id.run_trial_z 0

Continued singularity due to the zero variance in run_trial_z at id level.

flextb_vif(m11)

Predictor	vif
cond	2.14
block	1.58
trial_z	1.06
run_trial_z	1.06
prev_rt_z	1.00
cond:block	2.15
run_trial_z:prev_rt_z	1.00

flextable(render_aictab(minfo=minfo, m1, m2, m3, m4, m5, m6, m7, m8, m9, m10, m11)) %>% autofit()

model	fixed	l2	logLik	AIC	dAIC	df	weight
m1	Congruency, Block	Subject: Intercept	-7343	14698	2717.2	6	0.00000
m2	Congruency, Block	Block within subject: Intercept	-7279	14571	2590.5	7	0.00000
m3	Congruency, Block, Trial	Block within subject: Intercept	-7242	14500	2519.2	8	0.00000
m4	Congruency, Block, Trial	Block within subject:Intercept Subject: Trial	-6991	14002	2021.1	10	0.00000
m5	Congruency, Block, Trial	Block within subject: Intercept, Trial	-6943	13910	1929.4	12	0.00000
m6	Congruency, Block, Congruency x Block, Trial, Block x Trial	Block within subject: Intercept, Trial	-6943	13912	1931.3	13	0.00000
m7	Congruency, Block, Congruency x Block, Trial, Run Trial	Block within subject: Intercept, Trial	-6930	13886	1905.3	13	0.00000
m8	Congruency, Block, Trial, Run Trial	Block within subject: Intercept, Trial Subject: Run Trial	-6876	13783	1802.5	16	0.00000
m9	Congruency, Block, Congruency x Block, Trial, Run Trial	Block within subject:Intercept, Trial, Run Trial	-6823	13683	1702.1	19	0.00000
m10	Congruency, Block, Congruency x Block, Trial, Run Trial, Previous RT	Block within subject: Intercept, Trial, Run Trial	-5977	11993	12.4	20	0.00202
m11	Congruency, Block, Congruency x Block, Trial, Run Trial, Previous RT, Run Trial x Previous RT	Block within subject: Intercept, Trial, Run Trial	-5969	11981	0.0	21	0.99798

m12: Add prev_rt_z as random

minfo[["m12"]] <- c(
  fixed="Congruency, Block, Congruency x Block, Trial, Previous RT", 
  l2="Block within subject: Intercept, Trial, Run Trial, Previous RT")

m12 <- lmer(rt_inv ~ cond*block + trial_z + prev_rt_z + 
              (1 + trial_z + run_trial_z + prev_rt_z | id/block),
          na.action = na.exclude, control=lmerControl(optCtrl=list(maxfun=2e4)), #bump up iterations
          data = flanker, REML=FALSE)

## boundary (singular) fit: see ?isSingular

if (isSingular(m12)) { print(check_singularity(m12)) }

## id.prev_rt_z 
##            0

flextb_vif(m12)

Predictor	vif
cond	2.15
block	1.65
trial_z	1.00
prev_rt_z	1.00
cond:block	2.17

flextable(render_aictab(minfo=minfo, m1, m2, m3, m4, m5, m6, m7, m8, m9, m10, m11, m12)) %>% autofit()

model	fixed	l2	logLik	AIC	dAIC	df	weight
m1	Congruency, Block	Subject: Intercept	-7343	14698	2814.0	6	0.00e+00
m2	Congruency, Block	Block within subject: Intercept	-7279	14571	2687.4	7	0.00e+00
m3	Congruency, Block, Trial	Block within subject: Intercept	-7242	14500	2616.1	8	0.00e+00
m4	Congruency, Block, Trial	Block within subject:Intercept Subject: Trial	-6991	14002	2117.9	10	0.00e+00
m5	Congruency, Block, Trial	Block within subject: Intercept, Trial	-6943	13910	2026.2	12	0.00e+00
m6	Congruency, Block, Congruency x Block, Trial, Block x Trial	Block within subject: Intercept, Trial	-6943	13912	2028.2	13	0.00e+00
m7	Congruency, Block, Congruency x Block, Trial, Run Trial	Block within subject: Intercept, Trial	-6930	13886	2002.1	13	0.00e+00
m8	Congruency, Block, Trial, Run Trial	Block within subject: Intercept, Trial Subject: Run Trial	-6876	13783	1899.3	16	0.00e+00
m9	Congruency, Block, Congruency x Block, Trial, Run Trial	Block within subject:Intercept, Trial, Run Trial	-6823	13683	1798.9	19	0.00e+00
m10	Congruency, Block, Congruency x Block, Trial, Run Trial, Previous RT	Block within subject: Intercept, Trial, Run Trial	-5977	11993	109.2	20	1.90e-24
m11	Congruency, Block, Congruency x Block, Trial, Run Trial, Previous RT, Run Trial x Previous RT	Block within subject: Intercept, Trial, Run Trial	-5969	11981	96.8	21	9.38e-22
m12	Congruency, Block, Congruency x Block, Trial, Previous RT	Block within subject: Intercept, Trial, Run Trial, Previous RT	-5915	11884	0.0	27	1.00e+00

m13: Add condition as random at subject level

minfo[["m13"]] <- c(
  fixed="Congruency, Block, Congruency x Block, Trial, Previous RT", 
  l2="Block within subject: Intercept, Trial, Run Trial, Previous RT\nSubject: Congruency")

m13 <- lmer(rt_inv ~ cond*block + trial_z + prev_rt_z + 
              (1 + trial_z + run_trial_z + prev_rt_z + cond | id) +
              (1 + trial_z + run_trial_z + prev_rt_z | id:block),
          na.action = na.exclude, control=lmerControl(optCtrl=list(maxfun=2e4)), #bump up iterations
          data = flanker, REML=FALSE)

## boundary (singular) fit: see ?isSingular

## Warning: Model failed to converge with 2 negative eigenvalues: -1.6e-01 -1.5e+02

if (isSingular(m13)) { print(check_singularity(m13)) }

id.run_trial_z 0

flextb_vif(m13)

Predictor	vif
cond	1.63
block	1.59
trial_z	1.01
prev_rt_z	1.00
cond:block	1.87

flextable(render_aictab(minfo=minfo, m1, m2, m3, m4, m5, m6, m7, m8, m9, m10, m11, m12, m13)) %>% autofit()

model	fixed	l2	logLik	AIC	dAIC	df	weight
m1	Congruency, Block	Subject: Intercept	-7343	14698	2836.7	6	0.00e+00
m2	Congruency, Block	Block within subject: Intercept	-7279	14571	2710.0	7	0.00e+00
m3	Congruency, Block, Trial	Block within subject: Intercept	-7242	14500	2638.7	8	0.00e+00
m4	Congruency, Block, Trial	Block within subject:Intercept Subject: Trial	-6991	14002	2140.6	10	0.00e+00
m5	Congruency, Block, Trial	Block within subject: Intercept, Trial	-6943	13910	2048.8	12	0.00e+00
m6	Congruency, Block, Congruency x Block, Trial, Block x Trial	Block within subject: Intercept, Trial	-6943	13912	2050.8	13	0.00e+00
m7	Congruency, Block, Congruency x Block, Trial, Run Trial	Block within subject: Intercept, Trial	-6930	13886	2024.7	13	0.00e+00
m8	Congruency, Block, Trial, Run Trial	Block within subject: Intercept, Trial Subject: Run Trial	-6876	13783	1921.9	16	0.00e+00
m9	Congruency, Block, Congruency x Block, Trial, Run Trial	Block within subject:Intercept, Trial, Run Trial	-6823	13683	1821.5	19	0.00e+00
m10	Congruency, Block, Congruency x Block, Trial, Run Trial, Previous RT	Block within subject: Intercept, Trial, Run Trial	-5977	11993	131.9	20	2.33e-29
m11	Congruency, Block, Congruency x Block, Trial, Run Trial, Previous RT, Run Trial x Previous RT	Block within subject: Intercept, Trial, Run Trial	-5969	11981	119.5	21	1.15e-26
m12	Congruency, Block, Congruency x Block, Trial, Previous RT	Block within subject: Intercept, Trial, Run Trial, Previous RT	-5915	11884	22.6	27	1.23e-05
m13	Congruency, Block, Congruency x Block, Trial, Previous RT	Block within subject: Intercept, Trial, Run Trial, Previous RT Subject: Congruency	-5899	11861	0.0	32	1.00e+00

m14: Add condition as random at subject and block level

No improvement compared to m13. Keep condition at id level only.

minfo[["m14"]] <- c(
  fixed="Congruency, Block, Congruency x Block, Trial, Previous RT", 
  l2="Block within subject: Intercept, Trial, Run Trial, Previous RT, Congruency")

m14 <- lmer(rt_inv ~ cond*block + trial_z + prev_rt_z + 
              (1 + trial_z + run_trial_z + prev_rt_z + cond | id) + 
              (1 + trial_z + run_trial_z + prev_rt_z + cond | id:block),
          na.action = na.exclude, control=lmerControl(optCtrl=list(maxfun=2e4)), #bump up iterations
          data = flanker, REML=FALSE)

## boundary (singular) fit: see ?isSingular

if (isSingular(m14)) { print(check_singularity(m14)) }

## id:block.condincongruent       id.condincongruent 
##                0.0000282                0.0000000

flextb_vif(m14)

Predictor	vif
cond	1.67
block	1.55
trial_z	1.02
prev_rt_z	1.01
cond:block	1.86

flextable(render_aictab(minfo=minfo, m1, m2, m3, m4, m5, m6, m7, m8, m9, m10, m11, m12, m13, m14)) %>% autofit()

model	fixed	l2	logLik	AIC	dAIC	df	weight
m1	Congruency, Block	Subject: Intercept	-7343	14698	2836.7	6	0.00e+00
m2	Congruency, Block	Block within subject: Intercept	-7279	14571	2710.0	7	0.00e+00
m3	Congruency, Block, Trial	Block within subject: Intercept	-7242	14500	2638.7	8	0.00e+00
m4	Congruency, Block, Trial	Block within subject:Intercept Subject: Trial	-6991	14002	2140.6	10	0.00e+00
m5	Congruency, Block, Trial	Block within subject: Intercept, Trial	-6943	13910	2048.8	12	0.00e+00
m6	Congruency, Block, Congruency x Block, Trial, Block x Trial	Block within subject: Intercept, Trial	-6943	13912	2050.8	13	0.00e+00
m7	Congruency, Block, Congruency x Block, Trial, Run Trial	Block within subject: Intercept, Trial	-6930	13886	2024.7	13	0.00e+00
m8	Congruency, Block, Trial, Run Trial	Block within subject: Intercept, Trial Subject: Run Trial	-6876	13783	1921.9	16	0.00e+00
m9	Congruency, Block, Congruency x Block, Trial, Run Trial	Block within subject:Intercept, Trial, Run Trial	-6823	13683	1821.5	19	0.00e+00
m10	Congruency, Block, Congruency x Block, Trial, Run Trial, Previous RT	Block within subject: Intercept, Trial, Run Trial	-5977	11993	131.9	20	1.99e-29
m11	Congruency, Block, Congruency x Block, Trial, Run Trial, Previous RT, Run Trial x Previous RT	Block within subject: Intercept, Trial, Run Trial	-5969	11981	119.5	21	9.82e-27
m12	Congruency, Block, Congruency x Block, Trial, Previous RT	Block within subject: Intercept, Trial, Run Trial, Previous RT	-5915	11884	22.6	27	1.05e-05
m13	Congruency, Block, Congruency x Block, Trial, Previous RT	Block within subject: Intercept, Trial, Run Trial, Previous RT Subject: Congruency	-5899	11861	0.0	32	8.52e-01
m14	Congruency, Block, Congruency x Block, Trial, Previous RT	Block within subject: Intercept, Trial, Run Trial, Previous RT, Congruency	-5895	11865	3.5	37	1.48e-01

m15: add prev_error as fixed

Post-error slowing? Indeed

minfo[["m15"]] <- c(
  fixed="Congruency, Block, Congruency x Block,\nTrial, Previous RT, Previous Error", 
  l2="Block within subject: Intercept, Trial, Run Trial, Previous RT\nSubject: Congruency")

m15 <- lmer(rt_inv ~ cond*block + trial_z + prev_rt_z + prev_error +
              (1 + trial_z + run_trial_z + prev_rt_z + cond | id) +
              (1 + trial_z + run_trial_z + prev_rt_z | id:block),
          na.action = na.exclude, control=lmerControl(optCtrl=list(maxfun=2e4)), #bump up iterations
          data = flanker, REML=FALSE)

## boundary (singular) fit: see ?isSingular

## Warning: Model failed to converge with 1 negative eigenvalue: -8.3e+00

if (isSingular(m15)) { print(check_singularity(m15)) }

## id.run_trial_z 
##              0

flextb_vif(m15)

Predictor	vif
cond	1.63
block	1.59
trial_z	1.02
prev_rt_z	1.01
prev_error	1.01
cond:block	1.86

flextable(render_aictab(minfo=minfo, m1, m2, m3, m4, m5, m6, m7, m8, m9, m10, m11, m12, m13, m14, m15)) %>% autofit()

model	fixed	l2	logLik	AIC	dAIC	df	weight
m1	Congruency, Block	Subject: Intercept	-7343	14698	2983	6	0.00e+00
m2	Congruency, Block	Block within subject: Intercept	-7279	14571	2856	7	0.00e+00
m3	Congruency, Block, Trial	Block within subject: Intercept	-7242	14500	2785	8	0.00e+00
m4	Congruency, Block, Trial	Block within subject:Intercept Subject: Trial	-6991	14002	2287	10	0.00e+00
m5	Congruency, Block, Trial	Block within subject: Intercept, Trial	-6943	13910	2195	12	0.00e+00
m6	Congruency, Block, Congruency x Block, Trial, Block x Trial	Block within subject: Intercept, Trial	-6943	13912	2197	13	0.00e+00
m7	Congruency, Block, Congruency x Block, Trial, Run Trial	Block within subject: Intercept, Trial	-6930	13886	2171	13	0.00e+00
m8	Congruency, Block, Trial, Run Trial	Block within subject: Intercept, Trial Subject: Run Trial	-6876	13783	2068	16	0.00e+00
m9	Congruency, Block, Congruency x Block, Trial, Run Trial	Block within subject:Intercept, Trial, Run Trial	-6823	13683	1968	19	0.00e+00
m10	Congruency, Block, Congruency x Block, Trial, Run Trial, Previous RT	Block within subject: Intercept, Trial, Run Trial	-5977	11993	278	20	4.40e-61
m11	Congruency, Block, Congruency x Block, Trial, Run Trial, Previous RT, Run Trial x Previous RT	Block within subject: Intercept, Trial, Run Trial	-5969	11981	266	21	2.18e-58
m12	Congruency, Block, Congruency x Block, Trial, Previous RT	Block within subject: Intercept, Trial, Run Trial, Previous RT	-5915	11884	169	27	2.32e-37
m13	Congruency, Block, Congruency x Block, Trial, Previous RT	Block within subject: Intercept, Trial, Run Trial, Previous RT Subject: Congruency	-5899	11861	146	32	1.89e-32
m14	Congruency, Block, Congruency x Block, Trial, Previous RT	Block within subject: Intercept, Trial, Run Trial, Previous RT, Congruency	-5895	11865	150	37	3.28e-33
m15	Congruency, Block, Congruency x Block, Trial, Previous RT, Previous Error	Block within subject: Intercept, Trial, Run Trial, Previous RT Subject: Congruency	-5825	11715	0	33	1.00e+00

m16: add prev_error as random at ID level

minfo[["m16"]] <- c(
  fixed="Congruency, Block, Congruency x Block,\nTrial, Previous RT, Previous Error", 
  l2="Block within subject: Intercept, Trial, Run Trial, Previous RT\nSubject: Congruency, Previous Error")

m16 <- lmer(rt_inv ~ cond*block + trial_z + prev_rt_z + prev_error +
              (1 + trial_z + run_trial_z + prev_rt_z + cond + prev_error | id) +
              (1 + trial_z + run_trial_z + prev_rt_z | id:block),
          na.action = na.exclude, control=lmerControl(optCtrl=list(maxfun=2e4)), #bump up iterations
          data = flanker, REML=FALSE) # control=lmerControl(optimizer = "nlminbwrap"))

## Warning in checkConv(attr(opt, "derivs"), opt$par, ctrl = control$checkConv, : unable to evaluate scaled gradient

## Warning in checkConv(attr(opt, "derivs"), opt$par, ctrl = control$checkConv, : Model failed to converge: degenerate Hessian with 1 negative
## eigenvalues

## Warning: Model failed to converge with 1 negative eigenvalue: -2.3e+00

if (isSingular(m16)) { print(check_singularity(m16)) }

flextb_vif(m16)

Predictor	vif
cond	1.66
block	1.59
trial_z	1.03
prev_rt_z	1.09
prev_error	1.11
cond:block	1.86

flextable(render_aictab(minfo=minfo, m1, m2, m3, m4, m5, m6, m7, m8, m9, m10, m11, m12, m13, m14, m15, m16)) %>% autofit()

model	fixed	l2	logLik	AIC	dAIC	df	weight
m1	Congruency, Block	Subject: Intercept	-7343	14698	3014.0	6	0.00e+00
m2	Congruency, Block	Block within subject: Intercept	-7279	14571	2887.4	7	0.00e+00
m3	Congruency, Block, Trial	Block within subject: Intercept	-7242	14500	2816.1	8	0.00e+00
m4	Congruency, Block, Trial	Block within subject:Intercept Subject: Trial	-6991	14002	2317.9	10	0.00e+00
m5	Congruency, Block, Trial	Block within subject: Intercept, Trial	-6943	13910	2226.2	12	0.00e+00
m6	Congruency, Block, Congruency x Block, Trial, Block x Trial	Block within subject: Intercept, Trial	-6943	13912	2228.2	13	0.00e+00
m7	Congruency, Block, Congruency x Block, Trial, Run Trial	Block within subject: Intercept, Trial	-6930	13886	2202.1	13	0.00e+00
m8	Congruency, Block, Trial, Run Trial	Block within subject: Intercept, Trial Subject: Run Trial	-6876	13783	2099.3	16	0.00e+00
m9	Congruency, Block, Congruency x Block, Trial, Run Trial	Block within subject:Intercept, Trial, Run Trial	-6823	13683	1998.9	19	0.00e+00
m10	Congruency, Block, Congruency x Block, Trial, Run Trial, Previous RT	Block within subject: Intercept, Trial, Run Trial	-5977	11993	309.2	20	7.04e-68
m11	Congruency, Block, Congruency x Block, Trial, Run Trial, Previous RT, Run Trial x Previous RT	Block within subject: Intercept, Trial, Run Trial	-5969	11981	296.8	21	3.48e-65
m12	Congruency, Block, Congruency x Block, Trial, Previous RT	Block within subject: Intercept, Trial, Run Trial, Previous RT	-5915	11884	200.0	27	3.71e-44
m13	Congruency, Block, Congruency x Block, Trial, Previous RT	Block within subject: Intercept, Trial, Run Trial, Previous RT Subject: Congruency	-5899	11861	177.4	32	3.02e-39
m14	Congruency, Block, Congruency x Block, Trial, Previous RT	Block within subject: Intercept, Trial, Run Trial, Previous RT, Congruency	-5895	11865	180.9	37	5.24e-40
m15	Congruency, Block, Congruency x Block, Trial, Previous RT, Previous Error	Block within subject: Intercept, Trial, Run Trial, Previous RT Subject: Congruency	-5825	11715	31.3	33	1.60e-07
m16	Congruency, Block, Congruency x Block, Trial, Previous RT, Previous Error	Block within subject: Intercept, Trial, Run Trial, Previous RT Subject: Congruency, Previous Error	-5803	11684	0.0	39	1.00e+00

m17: remove run_trial_z from id level

We see a modest boost for eliminating run_trial_z from id level, which is more parsimonious.

My notation for the table doesn’t really accommodate this…

minfo[["m17"]] <- c(
  fixed="Congruency, Block, Congruency x Block,\nTrial, Previous RT, Previous Error", 
  l2="Block within subject: Intercept, Trial, Run Trial, Previous RT\nSubject: Congruency, Previous Error")

m17 <- lmer(rt_inv ~ cond*block + trial_z + prev_rt_z + prev_error +
              (1 + trial_z + prev_rt_z + cond + prev_error | id) +
              (1 + trial_z + run_trial_z + prev_rt_z | id:block),
          na.action = na.exclude, control=lmerControl(optCtrl=list(maxfun=2e4)), #bump up iterations
          data = flanker, REML=FALSE) # control=lmerControl(optimizer = "nlminbwrap"))

## Warning in checkConv(attr(opt, "derivs"), opt$par, ctrl = control$checkConv, : Model failed to converge with max|grad| = 0.00367693 (tol =
## 0.002, component 1)

if (isSingular(m17)) { print(check_singularity(m17)) }

flextb_vif(m17)

Predictor	vif
cond	1.66
block	1.59
trial_z	1.03
prev_rt_z	1.09
prev_error	1.12
cond:block	1.86

flextable(render_aictab(minfo=minfo, m1, m2, m3, m4, m5, m6, m7, m8, m9, m10, m11, m12, m13, m14, m15, m16, m17)) %>%
  autofit()

model	fixed	l2	logLik	AIC	dAIC	df	weight
m1	Congruency, Block	Subject: Intercept	-7343	14698	3015.51	6	0.00e+00
m2	Congruency, Block	Block within subject: Intercept	-7279	14571	2888.84	7	0.00e+00
m3	Congruency, Block, Trial	Block within subject: Intercept	-7242	14500	2817.55	8	0.00e+00
m4	Congruency, Block, Trial	Block within subject:Intercept Subject: Trial	-6991	14002	2319.41	10	0.00e+00
m5	Congruency, Block, Trial	Block within subject: Intercept, Trial	-6943	13910	2227.67	12	0.00e+00
m6	Congruency, Block, Congruency x Block, Trial, Block x Trial	Block within subject: Intercept, Trial	-6943	13912	2229.62	13	0.00e+00
m7	Congruency, Block, Congruency x Block, Trial, Run Trial	Block within subject: Intercept, Trial	-6930	13886	2203.57	13	0.00e+00
m8	Congruency, Block, Trial, Run Trial	Block within subject: Intercept, Trial Subject: Run Trial	-6876	13783	2100.80	16	0.00e+00
m9	Congruency, Block, Congruency x Block, Trial, Run Trial	Block within subject:Intercept, Trial, Run Trial	-6823	13683	2000.40	19	0.00e+00
m10	Congruency, Block, Congruency x Block, Trial, Run Trial, Previous RT	Block within subject: Intercept, Trial, Run Trial	-5977	11993	310.71	20	2.29e-68
m11	Congruency, Block, Congruency x Block, Trial, Run Trial, Previous RT, Run Trial x Previous RT	Block within subject: Intercept, Trial, Run Trial	-5969	11981	298.30	21	1.13e-65
m12	Congruency, Block, Congruency x Block, Trial, Previous RT	Block within subject: Intercept, Trial, Run Trial, Previous RT	-5915	11884	201.46	27	1.21e-44
m13	Congruency, Block, Congruency x Block, Trial, Previous RT	Block within subject: Intercept, Trial, Run Trial, Previous RT Subject: Congruency	-5899	11861	178.85	32	9.83e-40
m14	Congruency, Block, Congruency x Block, Trial, Previous RT	Block within subject: Intercept, Trial, Run Trial, Previous RT, Congruency	-5895	11865	182.35	37	1.70e-40
m15	Congruency, Block, Congruency x Block, Trial, Previous RT, Previous Error	Block within subject: Intercept, Trial, Run Trial, Previous RT Subject: Congruency	-5825	11715	32.76	33	5.20e-08
m16	Congruency, Block, Congruency x Block, Trial, Previous RT, Previous Error	Block within subject: Intercept, Trial, Run Trial, Previous RT Subject: Congruency, Previous Error	-5803	11684	1.46	39	3.25e-01
m17	Congruency, Block, Congruency x Block, Trial, Previous RT, Previous Error	Block within subject: Intercept, Trial, Run Trial, Previous RT Subject: Congruency, Previous Error	-5808	11683	0.00	33	6.75e-01

library(sjPlot)
gg <- plot_model(m17, type="std", show.p=TRUE, ci.lvl=.95, dot.size=5, line.size=2) + 
  ylim(-0.1, 0.3) + theme_bw(base_size=18) +
  geom_hline(yintercept = 0, size=1.2) + 
  theme(panel.grid.major.y=element_blank()) +
  ggtitle("") + ylab("Standardized regression coefficient")

## Variable `prev_error` contains only two different values. Consider converting it to a factor.

## boundary (singular) fit: see ?isSingular

## Warning: Model failed to converge with 2 negative eigenvalues: -3.2e-01 -4.8e+02

## Scale for 'y' is already present. Adding another scale for 'y', which will replace the existing scale.

plot(gg)

ggsave(gg, filename="Flanker_RT_Params.pdf", width=9, height=9, useDingbats=FALSE)

m18: Allow for prev_rt_z x prev_error interaction

Not a slam dunk, but a small interaction. Omit for simplicity.

minfo[["m18"]] <- c(
  fixed="Congruency, Block, Trial, Congruency x Trial, Previous RT,\nPrevious Error, Previous RT x Previous Error", 
  l2="Block within subject: Intercept, Trial, Run Trial, Previous RT\nSubject: Congruency, Previous Error")

m18 <- lmer(rt_inv ~ cond*block + trial_z + prev_rt_z * prev_error +
              (1 + trial_z + prev_rt_z + cond + prev_error | id) +
              (1 + trial_z + run_trial_z + prev_rt_z | id:block),
          na.action = na.exclude, control=lmerControl(optCtrl=list(maxfun=2e4)), #bump up iterations
          data = flanker, REML=FALSE) # control=lmerControl(optimizer = "nlminbwrap"))

## Warning in checkConv(attr(opt, "derivs"), opt$par, ctrl = control$checkConv, : Model failed to converge with max|grad| = 0.00656541 (tol =
## 0.002, component 1)

if (isSingular(m18)) { print(check_singularity(m18)) }

flextb_vif(m18)

Predictor	vif
cond	1.67
block	1.59
trial_z	1.03
prev_rt_z	1.09
prev_error	1.21
cond:block	1.87
prev_rt_z:prev_error	1.13

car::Anova(m18, type=3)

## Analysis of Deviance Table (Type III Wald chisquare tests)
## 
## Response: rt_inv
##                         Chisq Df Pr(>Chisq)    
## (Intercept)          12489.75  1    < 2e-16 ***
## cond                   143.60  1    < 2e-16 ***
## block                    4.65  1    0.03098 *  
## trial_z                 11.83  1    0.00058 ***
## prev_rt_z              426.43  1    < 2e-16 ***
## prev_error              76.40  1    < 2e-16 ***
## cond:block              14.56  1    0.00014 ***
## prev_rt_z:prev_error     4.21  1    0.04029 *  
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

flextable(render_aictab(minfo=minfo, m1, m2, m3, m4, m5, m6, m7, m8, m9, m10, m11, m12, m13, m14, m15, m16, m17, m18)) %>%
  autofit()

model	fixed	l2	logLik	AIC	dAIC	df	weight
m1	Congruency, Block	Subject: Intercept	-7343	14698	3016.91	6	0.00e+00
m2	Congruency, Block	Block within subject: Intercept	-7279	14571	2890.24	7	0.00e+00
m3	Congruency, Block, Trial	Block within subject: Intercept	-7242	14500	2818.95	8	0.00e+00
m4	Congruency, Block, Trial	Block within subject:Intercept Subject: Trial	-6991	14002	2320.81	10	0.00e+00
m5	Congruency, Block, Trial	Block within subject: Intercept, Trial	-6943	13910	2229.07	12	0.00e+00
m6	Congruency, Block, Congruency x Block, Trial, Block x Trial	Block within subject: Intercept, Trial	-6943	13912	2231.02	13	0.00e+00
m7	Congruency, Block, Congruency x Block, Trial, Run Trial	Block within subject: Intercept, Trial	-6930	13886	2204.97	13	0.00e+00
m8	Congruency, Block, Trial, Run Trial	Block within subject: Intercept, Trial Subject: Run Trial	-6876	13783	2102.20	16	0.00e+00
m9	Congruency, Block, Congruency x Block, Trial, Run Trial	Block within subject:Intercept, Trial, Run Trial	-6823	13683	2001.80	19	0.00e+00
m10	Congruency, Block, Congruency x Block, Trial, Run Trial, Previous RT	Block within subject: Intercept, Trial, Run Trial	-5977	11993	312.11	20	9.71e-69
m11	Congruency, Block, Congruency x Block, Trial, Run Trial, Previous RT, Run Trial x Previous RT	Block within subject: Intercept, Trial, Run Trial	-5969	11981	299.70	21	4.80e-66
m12	Congruency, Block, Congruency x Block, Trial, Previous RT	Block within subject: Intercept, Trial, Run Trial, Previous RT	-5915	11884	202.86	27	5.12e-45
m13	Congruency, Block, Congruency x Block, Trial, Previous RT	Block within subject: Intercept, Trial, Run Trial, Previous RT Subject: Congruency	-5899	11861	180.25	32	4.17e-40
m14	Congruency, Block, Congruency x Block, Trial, Previous RT	Block within subject: Intercept, Trial, Run Trial, Previous RT, Congruency	-5895	11865	183.75	37	7.23e-41
m15	Congruency, Block, Congruency x Block, Trial, Previous RT, Previous Error	Block within subject: Intercept, Trial, Run Trial, Previous RT Subject: Congruency	-5825	11715	34.16	33	2.21e-08
m16	Congruency, Block, Congruency x Block, Trial, Previous RT, Previous Error	Block within subject: Intercept, Trial, Run Trial, Previous RT Subject: Congruency, Previous Error	-5803	11684	2.86	39	1.38e-01
m17	Congruency, Block, Congruency x Block, Trial, Previous RT, Previous Error	Block within subject: Intercept, Trial, Run Trial, Previous RT Subject: Congruency, Previous Error	-5808	11683	1.40	33	2.86e-01
m18	Congruency, Block, Trial, Congruency x Trial, Previous RT, Previous Error, Previous RT x Previous Error	Block within subject: Intercept, Trial, Run Trial, Previous RT Subject: Congruency, Previous Error	-5807	11681	0.00	34	5.76e-01

m19: try to simplify random effect structure further: Drop trial_z within block

Eliminate small variance components for the trial_z id:block piece

minfo[["m19"]] <- c(
  fixed="Congruency, Block, Congruency x Block,\nTrial, Previous RT, Previous Error", 
  l2="Block within subject:Intercept, Run Trial, Previous RT\nSubject: Congruency, Trial, Previous Error")

m19 <- lmer(rt_inv ~ cond*block + trial_z + prev_rt_z + prev_error +
              (1 + trial_z + prev_rt_z + cond + prev_error | id) +
              (1 + run_trial_z + prev_rt_z | id:block),
          na.action = na.exclude, control=lmerControl(optCtrl=list(maxfun=2e4)), #bump up iterations
          data = flanker, REML=FALSE) # control=lmerControl(optimizer = "nlminbwrap"))

## Warning in checkConv(attr(opt, "derivs"), opt$par, ctrl = control$checkConv, : Model failed to converge with max|grad| = 0.0141655 (tol =
## 0.002, component 1)

if (isSingular(m19)) { print(check_singularity(m19)) }

flextb_vif(m19)

Predictor	vif
cond	1.66
block	1.60
trial_z	1.03
prev_rt_z	1.10
prev_error	1.11
cond:block	1.87

#car::Anova(m19, type=3)

flextable(render_aictab(minfo=minfo, m1, m2, m3, m4, m5, m6, m7, m8, m9, m10, m11, m12, 
                        m13, m14, m15, m16, m17, m18, m19)) %>% autofit()

model	fixed	l2	logLik	AIC	dAIC	df	weight
m1	Congruency, Block	Subject: Intercept	-7343	14698	3016.91	6	0.00e+00
m2	Congruency, Block	Block within subject: Intercept	-7279	14571	2890.24	7	0.00e+00
m3	Congruency, Block, Trial	Block within subject: Intercept	-7242	14500	2818.95	8	0.00e+00
m4	Congruency, Block, Trial	Block within subject:Intercept Subject: Trial	-6991	14002	2320.81	10	0.00e+00
m5	Congruency, Block, Trial	Block within subject: Intercept, Trial	-6943	13910	2229.07	12	0.00e+00
m6	Congruency, Block, Congruency x Block, Trial, Block x Trial	Block within subject: Intercept, Trial	-6943	13912	2231.02	13	0.00e+00
m7	Congruency, Block, Congruency x Block, Trial, Run Trial	Block within subject: Intercept, Trial	-6930	13886	2204.97	13	0.00e+00
m8	Congruency, Block, Trial, Run Trial	Block within subject: Intercept, Trial Subject: Run Trial	-6876	13783	2102.20	16	0.00e+00
m9	Congruency, Block, Congruency x Block, Trial, Run Trial	Block within subject:Intercept, Trial, Run Trial	-6823	13683	2001.80	19	0.00e+00
m10	Congruency, Block, Congruency x Block, Trial, Run Trial, Previous RT	Block within subject: Intercept, Trial, Run Trial	-5977	11993	312.11	20	9.71e-69
m11	Congruency, Block, Congruency x Block, Trial, Run Trial, Previous RT, Run Trial x Previous RT	Block within subject: Intercept, Trial, Run Trial	-5969	11981	299.70	21	4.80e-66
m12	Congruency, Block, Congruency x Block, Trial, Previous RT	Block within subject: Intercept, Trial, Run Trial, Previous RT	-5915	11884	202.86	27	5.12e-45
m13	Congruency, Block, Congruency x Block, Trial, Previous RT	Block within subject: Intercept, Trial, Run Trial, Previous RT Subject: Congruency	-5899	11861	180.25	32	4.17e-40
m14	Congruency, Block, Congruency x Block, Trial, Previous RT	Block within subject: Intercept, Trial, Run Trial, Previous RT, Congruency	-5895	11865	183.75	37	7.23e-41
m15	Congruency, Block, Congruency x Block, Trial, Previous RT, Previous Error	Block within subject: Intercept, Trial, Run Trial, Previous RT Subject: Congruency	-5825	11715	34.16	33	2.21e-08
m16	Congruency, Block, Congruency x Block, Trial, Previous RT, Previous Error	Block within subject: Intercept, Trial, Run Trial, Previous RT Subject: Congruency, Previous Error	-5803	11684	2.86	39	1.38e-01
m17	Congruency, Block, Congruency x Block, Trial, Previous RT, Previous Error	Block within subject: Intercept, Trial, Run Trial, Previous RT Subject: Congruency, Previous Error	-5808	11683	1.40	33	2.86e-01
m18	Congruency, Block, Trial, Congruency x Trial, Previous RT, Previous Error, Previous RT x Previous Error	Block within subject: Intercept, Trial, Run Trial, Previous RT Subject: Congruency, Previous Error	-5807	11681	0.00	34	5.76e-01
m19	Congruency, Block, Congruency x Block, Trial, Previous RT, Previous Error	Block within subject:Intercept, Run Trial, Previous RT Subject: Congruency, Trial, Previous Error	-5821	11701	19.27	29	3.76e-05

m20: drop run_trial_z random effect

minfo[["m20"]] <- c(
  fixed="Congruency, Block, Congruency x Block,\nTrial, Previous RT, Previous Error", 
  l2="Block within subject: Intercept, Trial, Previous RT\nSubject: Congruency, Previous Error")

m20 <- lmer(rt_inv ~ cond*block + trial_z + prev_rt_z + prev_error +
              (1 + trial_z + prev_rt_z + cond + prev_error | id) +
              (1 + trial_z + prev_rt_z | id:block),
          na.action = na.exclude, control=lmerControl(optCtrl=list(maxfun=2e4)), #bump up iterations
          data = flanker, REML=FALSE) # control=lmerControl(optimizer = "nlminbwrap"))

## Warning in checkConv(attr(opt, "derivs"), opt$par, ctrl = control$checkConv, : Model failed to converge with max|grad| = 0.00474248 (tol =
## 0.002, component 1)

if (isSingular(m20)) { print(check_singularity(m20)) }

flextb_vif(m20)

Predictor	vif
cond	1.67
block	1.61
trial_z	1.03
prev_rt_z	1.09
prev_error	1.12
cond:block	1.88

#car::Anova(m20, type=3)

flextable(render_aictab(minfo=minfo, m1, m2, m3, m4, m5, m6, m7, m8, m9, m10, m11, 
                        m12, m13, m14, m15, m16, m17, m18, m19, m20)) %>% autofit()

model	fixed	l2	logLik	AIC	dAIC	df	weight
m1	Congruency, Block	Subject: Intercept	-7343	14698	3016.91	6	0.00e+00
m2	Congruency, Block	Block within subject: Intercept	-7279	14571	2890.24	7	0.00e+00
m3	Congruency, Block, Trial	Block within subject: Intercept	-7242	14500	2818.95	8	0.00e+00
m4	Congruency, Block, Trial	Block within subject:Intercept Subject: Trial	-6991	14002	2320.81	10	0.00e+00
m5	Congruency, Block, Trial	Block within subject: Intercept, Trial	-6943	13910	2229.07	12	0.00e+00
m6	Congruency, Block, Congruency x Block, Trial, Block x Trial	Block within subject: Intercept, Trial	-6943	13912	2231.02	13	0.00e+00
m7	Congruency, Block, Congruency x Block, Trial, Run Trial	Block within subject: Intercept, Trial	-6930	13886	2204.97	13	0.00e+00
m8	Congruency, Block, Trial, Run Trial	Block within subject: Intercept, Trial Subject: Run Trial	-6876	13783	2102.20	16	0.00e+00
m9	Congruency, Block, Congruency x Block, Trial, Run Trial	Block within subject:Intercept, Trial, Run Trial	-6823	13683	2001.80	19	0.00e+00
m10	Congruency, Block, Congruency x Block, Trial, Run Trial, Previous RT	Block within subject: Intercept, Trial, Run Trial	-5977	11993	312.11	20	9.71e-69
m11	Congruency, Block, Congruency x Block, Trial, Run Trial, Previous RT, Run Trial x Previous RT	Block within subject: Intercept, Trial, Run Trial	-5969	11981	299.70	21	4.80e-66
m12	Congruency, Block, Congruency x Block, Trial, Previous RT	Block within subject: Intercept, Trial, Run Trial, Previous RT	-5915	11884	202.86	27	5.12e-45
m13	Congruency, Block, Congruency x Block, Trial, Previous RT	Block within subject: Intercept, Trial, Run Trial, Previous RT Subject: Congruency	-5899	11861	180.25	32	4.17e-40
m14	Congruency, Block, Congruency x Block, Trial, Previous RT	Block within subject: Intercept, Trial, Run Trial, Previous RT, Congruency	-5895	11865	183.75	37	7.23e-41
m15	Congruency, Block, Congruency x Block, Trial, Previous RT, Previous Error	Block within subject: Intercept, Trial, Run Trial, Previous RT Subject: Congruency	-5825	11715	34.16	33	2.21e-08
m16	Congruency, Block, Congruency x Block, Trial, Previous RT, Previous Error	Block within subject: Intercept, Trial, Run Trial, Previous RT Subject: Congruency, Previous Error	-5803	11684	2.86	39	1.38e-01
m17	Congruency, Block, Congruency x Block, Trial, Previous RT, Previous Error	Block within subject: Intercept, Trial, Run Trial, Previous RT Subject: Congruency, Previous Error	-5808	11683	1.40	33	2.86e-01
m18	Congruency, Block, Trial, Congruency x Trial, Previous RT, Previous Error, Previous RT x Previous Error	Block within subject: Intercept, Trial, Run Trial, Previous RT Subject: Congruency, Previous Error	-5807	11681	0.00	34	5.76e-01
m19	Congruency, Block, Congruency x Block, Trial, Previous RT, Previous Error	Block within subject:Intercept, Run Trial, Previous RT Subject: Congruency, Trial, Previous Error	-5821	11701	19.27	29	3.76e-05
m20	Congruency, Block, Congruency x Block, Trial, Previous RT, Previous Error	Block within subject: Intercept, Trial, Previous RT Subject: Congruency, Previous Error	-5827	11712	30.89	29	1.13e-07

m21: drop intercept at id/block

minfo[["m21"]] <- c(
  fixed="Congruency, Block, Congruency x Block,\nTrial, Previous RT, Previous Error", 
  l2="Block within subject:Previous RT, Previous Error\nSubject: Congruency")

m21 <- lmer(rt_inv ~ cond*block + trial_z + prev_rt_z + prev_error +
              (1 + trial_z + prev_rt_z + cond + prev_error | id) +
              (0 + trial_z + prev_rt_z | id:block),
          na.action = na.exclude, control=lmerControl(optCtrl=list(maxfun=2e4)), #bump up iterations
          data = flanker, REML=FALSE) # control=lmerControl(optimizer = "nlminbwrap"))

## boundary (singular) fit: see ?isSingular

## Warning: Model failed to converge with 2 negative eigenvalues: -4.4e-02 -7.1e+01

flextb_vif(m21)

Predictor	vif
cond	1.69
block	2.01
trial_z	1.01
prev_rt_z	1.05
prev_error	1.08
cond:block	2.10

#car::Anova(m21, type=3)

flextable(render_aictab(minfo=minfo, m1, m2, m3, m4, m5, m6, m7, m8, m9, m10, m11, m12,
                        m13, m14, m15, m16, m17, m18, m19, m20, m21)) %>% autofit()

model	fixed	l2	logLik	AIC	dAIC	df	weight
m1	Congruency, Block	Subject: Intercept	-7343	14698	3016.91	6	0.00e+00
m2	Congruency, Block	Block within subject: Intercept	-7279	14571	2890.24	7	0.00e+00
m3	Congruency, Block, Trial	Block within subject: Intercept	-7242	14500	2818.95	8	0.00e+00
m4	Congruency, Block, Trial	Block within subject:Intercept Subject: Trial	-6991	14002	2320.81	10	0.00e+00
m5	Congruency, Block, Trial	Block within subject: Intercept, Trial	-6943	13910	2229.07	12	0.00e+00
m6	Congruency, Block, Congruency x Block, Trial, Block x Trial	Block within subject: Intercept, Trial	-6943	13912	2231.02	13	0.00e+00
m7	Congruency, Block, Congruency x Block, Trial, Run Trial	Block within subject: Intercept, Trial	-6930	13886	2204.97	13	0.00e+00
m8	Congruency, Block, Trial, Run Trial	Block within subject: Intercept, Trial Subject: Run Trial	-6876	13783	2102.20	16	0.00e+00
m9	Congruency, Block, Congruency x Block, Trial, Run Trial	Block within subject:Intercept, Trial, Run Trial	-6823	13683	2001.80	19	0.00e+00
m10	Congruency, Block, Congruency x Block, Trial, Run Trial, Previous RT	Block within subject: Intercept, Trial, Run Trial	-5977	11993	312.11	20	9.71e-69
m11	Congruency, Block, Congruency x Block, Trial, Run Trial, Previous RT, Run Trial x Previous RT	Block within subject: Intercept, Trial, Run Trial	-5969	11981	299.70	21	4.80e-66
m12	Congruency, Block, Congruency x Block, Trial, Previous RT	Block within subject: Intercept, Trial, Run Trial, Previous RT	-5915	11884	202.86	27	5.12e-45
m13	Congruency, Block, Congruency x Block, Trial, Previous RT	Block within subject: Intercept, Trial, Run Trial, Previous RT Subject: Congruency	-5899	11861	180.25	32	4.17e-40
m14	Congruency, Block, Congruency x Block, Trial, Previous RT	Block within subject: Intercept, Trial, Run Trial, Previous RT, Congruency	-5895	11865	183.75	37	7.23e-41
m15	Congruency, Block, Congruency x Block, Trial, Previous RT, Previous Error	Block within subject: Intercept, Trial, Run Trial, Previous RT Subject: Congruency	-5825	11715	34.16	33	2.21e-08
m16	Congruency, Block, Congruency x Block, Trial, Previous RT, Previous Error	Block within subject: Intercept, Trial, Run Trial, Previous RT Subject: Congruency, Previous Error	-5803	11684	2.86	39	1.38e-01
m17	Congruency, Block, Congruency x Block, Trial, Previous RT, Previous Error	Block within subject: Intercept, Trial, Run Trial, Previous RT Subject: Congruency, Previous Error	-5808	11683	1.40	33	2.86e-01
m18	Congruency, Block, Trial, Congruency x Trial, Previous RT, Previous Error, Previous RT x Previous Error	Block within subject: Intercept, Trial, Run Trial, Previous RT Subject: Congruency, Previous Error	-5807	11681	0.00	34	5.76e-01
m19	Congruency, Block, Congruency x Block, Trial, Previous RT, Previous Error	Block within subject:Intercept, Run Trial, Previous RT Subject: Congruency, Trial, Previous Error	-5821	11701	19.27	29	3.76e-05
m20	Congruency, Block, Congruency x Block, Trial, Previous RT, Previous Error	Block within subject: Intercept, Trial, Previous RT Subject: Congruency, Previous Error	-5827	11712	30.89	29	1.13e-07
m21	Congruency, Block, Congruency x Block, Trial, Previous RT, Previous Error	Block within subject:Previous RT, Previous Error Subject: Congruency	-5843	11738	56.67	26	2.84e-13

In short, dropping any of the id:block random effects (m19-m21) substantially worsens fit.

final model table

m17 is the winner – keep 1-17 for supp

flextable(render_aictab(minfo=minfo[1:17], m1, m2, m3, m4, m5, m6, m7, m8, m9, m10, m11, m12, 
                        m13, m14, m15, m16, m17)) %>% autofit()

model	fixed	l2	logLik	AIC	dAIC	df	weight
m1	Congruency, Block	Subject: Intercept	-7343	14698	3015.51	6	0.00e+00
m2	Congruency, Block	Block within subject: Intercept	-7279	14571	2888.84	7	0.00e+00
m3	Congruency, Block, Trial	Block within subject: Intercept	-7242	14500	2817.55	8	0.00e+00
m4	Congruency, Block, Trial	Block within subject:Intercept Subject: Trial	-6991	14002	2319.41	10	0.00e+00
m5	Congruency, Block, Trial	Block within subject: Intercept, Trial	-6943	13910	2227.67	12	0.00e+00
m6	Congruency, Block, Congruency x Block, Trial, Block x Trial	Block within subject: Intercept, Trial	-6943	13912	2229.62	13	0.00e+00
m7	Congruency, Block, Congruency x Block, Trial, Run Trial	Block within subject: Intercept, Trial	-6930	13886	2203.57	13	0.00e+00
m8	Congruency, Block, Trial, Run Trial	Block within subject: Intercept, Trial Subject: Run Trial	-6876	13783	2100.80	16	0.00e+00
m9	Congruency, Block, Congruency x Block, Trial, Run Trial	Block within subject:Intercept, Trial, Run Trial	-6823	13683	2000.40	19	0.00e+00
m10	Congruency, Block, Congruency x Block, Trial, Run Trial, Previous RT	Block within subject: Intercept, Trial, Run Trial	-5977	11993	310.71	20	2.29e-68
m11	Congruency, Block, Congruency x Block, Trial, Run Trial, Previous RT, Run Trial x Previous RT	Block within subject: Intercept, Trial, Run Trial	-5969	11981	298.30	21	1.13e-65
m12	Congruency, Block, Congruency x Block, Trial, Previous RT	Block within subject: Intercept, Trial, Run Trial, Previous RT	-5915	11884	201.46	27	1.21e-44
m13	Congruency, Block, Congruency x Block, Trial, Previous RT	Block within subject: Intercept, Trial, Run Trial, Previous RT Subject: Congruency	-5899	11861	178.85	32	9.83e-40
m14	Congruency, Block, Congruency x Block, Trial, Previous RT	Block within subject: Intercept, Trial, Run Trial, Previous RT, Congruency	-5895	11865	182.35	37	1.70e-40
m15	Congruency, Block, Congruency x Block, Trial, Previous RT, Previous Error	Block within subject: Intercept, Trial, Run Trial, Previous RT Subject: Congruency	-5825	11715	32.76	33	5.20e-08
m16	Congruency, Block, Congruency x Block, Trial, Previous RT, Previous Error	Block within subject: Intercept, Trial, Run Trial, Previous RT Subject: Congruency, Previous Error	-5803	11684	1.46	39	3.25e-01
m17	Congruency, Block, Congruency x Block, Trial, Previous RT, Previous Error	Block within subject: Intercept, Trial, Run Trial, Previous RT Subject: Congruency, Previous Error	-5808	11683	0.00	33	6.75e-01

summary(m17)

## Linear mixed model fit by maximum likelihood . t-tests use Satterthwaite's method ['lmerModLmerTest']
## Formula: rt_inv ~ cond * block + trial_z + prev_rt_z + prev_error + (1 +  
##     trial_z + prev_rt_z + cond + prev_error | id) + (1 + trial_z +      run_trial_z + prev_rt_z | id:block)
##    Data: flanker
## Control: lmerControl(optCtrl = list(maxfun = 20000))
## 
##      AIC      BIC   logLik deviance df.resid 
##    11683    11936    -5808    11617    15744 
## 
## Scaled residuals: 
##    Min     1Q Median     3Q    Max 
## -4.709 -0.592  0.005  0.593  4.646 
## 
## Random effects:
##  Groups   Name            Variance Std.Dev. Corr                   
##  id:block (Intercept)     0.001488 0.0386                          
##           trial_z         0.002911 0.0540   -0.05                  
##           run_trial_z     0.001344 0.0367   -0.08 -0.34            
##           prev_rt_z       0.000877 0.0296   -0.44  0.23  0.49      
##  id       (Intercept)     0.056884 0.2385                          
##           trial_z         0.001785 0.0423   -0.05                  
##           prev_rt_z       0.001688 0.0411   -0.04 -0.11            
##           condincongruent 0.003409 0.0584   -0.36 -0.33 -0.13      
##           prev_error      0.023561 0.1535    0.16 -0.05  0.54 -0.39
##  Residual                 0.113909 0.3375                          
## Number of obs: 15777, groups:  id:block, 214; id, 107
## 
## Fixed effects:
##                                  Estimate  Std. Error          df t value Pr(>|t|)    
## (Intercept)                      -2.71651     0.02444   119.04772 -111.17  < 2e-16 ***
## condincongruent                   0.12045     0.01006   260.54362   11.98  < 2e-16 ***
## blockmost_con                     0.02119     0.00995   253.48965    2.13  0.03412 *  
## trial_z                          -0.02207     0.00640    91.17503   -3.45  0.00085 ***
## prev_rt_z                         0.12047     0.00591   117.18438   20.40  < 2e-16 ***
## prev_error                        0.23397     0.02404    78.80894    9.73  3.7e-15 ***
## condincongruent:blockmost_con     0.04545     0.01184 15114.19541    3.84  0.00012 ***
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## Correlation of Fixed Effects:
##             (Intr) cndncn blckm_ tril_z prv_r_ prv_rr
## condncngrnt -0.388                                   
## blockmst_cn -0.259  0.478                            
## trial_z     -0.037 -0.119  0.002                     
## prev_rt_z   -0.026 -0.052 -0.002  0.053              
## prev_error   0.076 -0.125  0.005 -0.018  0.284       
## cndncngrn:_  0.166 -0.581 -0.583 -0.001  0.001 -0.006
## convergence code: 0
## Model failed to converge with max|grad| = 0.00367693 (tol = 0.002, component 1)

#checking diagnostics: does the lower bound on the random effect encompass zero (singularity)
#https://rstudio-pubs-static.s3.amazonaws.com/33653_57fc7b8e5d484c909b615d8633c01d51.html
tt <- getME(m1_sc,"theta")
ll <- getME(m1_sc,"lower")
min(tt[ll==0])

Final Flanker RT Analyses

PD Inhibition Project

Michael Hallquist

28 May 2020

LME approach

m1: design effects, random intercept of subject

m2: design effects, allow for L2 random intercept to vary by block

m3: design effects, allow for L1 residual variance to vary by block

m4: Allow L1 residual variability to depend on both condition and block

LMER-based modeling

m1: design effects, random intercept of subject

m2: design effects, allow for L2 random intercept to vary by block

m3: Add trial_z as fixed

m4: Add trial_z as random, too, but only at subject level, not block-specific

m5: Add trial_z as random both per subject and block

m6: Add trial x block interaction as fixed

m7: Trial within run (run_trial) as fixed

m8: Add run trial as random at subject level only

m9: Add run trial as random at subject and block level

m10: Add prev_rt_z as fixed effect

m11: Allow prev_rt_z x run_trial interaction

m12: Add prev_rt_z as random

m13: Add condition as random at subject level

m14: Add condition as random at subject and block level

m15: add prev_error as fixed

m16: add prev_error as random at ID level

m17: remove run_trial_z from id level

m18: Allow for prev_rt_z x prev_error interaction

m19: try to simplify random effect structure further: Drop trial_z within block

m20: drop run_trial_z random effect

m21: drop intercept at id/block

final model table