样本量规划

Abstract:

With the development of data-collection technics and increasing complexity of study designs, nested data widely exists in psychological research. Linear mixed-effects models, unfortunately with an unreasonable hypothesis that the residual variances are homogenous, are generally used in nested data analysis. Meanwhile, Mixed-Effects Location-Scale Models (MELSM) has become more and more popular, because they can handle heterogenous residual variances and are able to add predictors for the two substructures (i.e., mean structure denoted as location model and variance structure denoted as scale model) in different levels. MELSM can avoid estimation bias due to inappropriate assumptions of homogenous variance and explore the relationship among traits and simultaneously investigate the inter- and intra-individual variability, as well as their explanatory variables. This study, aims at developing the methods of model construction and sample size planning for MELSM, using simulated studies and empirical studies. In detail, the main contents of this project are as follows. Study 1 focuses on comparing and selecting candidate models based on Bayesian fit indices to construct MELSM, taking into consideration the estimated method for complicated models. We propose that model selection for location model and scale model can be completed sequentially. Study 2 explores the method of sample size planning for MELSM, according to both power analysis (based on Monte Carlo simulation) and the accuracy in parameter estimation analysis (based on the credible interval of the posterior distribution). Adequate sample size is required for both the power and the accuracy in parameter estimation. Study 3 extends the sample size planning method for MELSM to better frame the considerations of uncertainty. By specifying the prior distribution of effect sizes, repeating sampling and selecting model based on the robust Bayesian fit index suggested by Study 1, three main sources of uncertainty can be well controlled: the uncertainty due to unknown population effect size, sampling variability and model approximation. With the simulated study results, we are able to provide reliable Bayesian fit indices for MELSM construction, and summary the process of sample size planning for MELSM in both determinate and uncertain situations. Moreover, Study 4 illustrates the application of MELSM in two empirical psychological studies and verifies the operability of the conclusions of the simulated studies in practice. The unique contribution of this paper is to further promote the methods of model construction and sample size planning for MELSM, as well as provide methodological foundation for researchers. In addition, we plan to integrate the functions above to develop a user-friendly R package for MELSM and provide a basis for promotion and application of MELSM, which help researchers make sample size planning, model construction and parameter estimation for MELSM easily, according to their specification. If these statistical models are widely implemented, the reproducibility and replicability of psychological studies will be enhanced finally.

Key words: nested data, mixed-effects location-scale models, model construction, sample size planning

步骤	数据收集前：样本量规划(研究2, 研究3)
	模型确定		模型不确定
	效应量确定	效应量不确定	效应量确定	效应量不确定
第1步	根据先验信息确定1个效应量的值;	定义效应量参数先验分布, 并从中抽取 S 个效应量的值;	根据先验信息确定1个效应量的值;	定义效应量参数先验分布, 并从中抽取 S 个效应量的值;
第2步	基于待拟合模型生成 R 个样本量为 N 的样本, 共可得到 R 个样本;	基于待拟合模型生成 R 个样本量为 N 的样本, 共可得到 R×S 个样本;	基于备选模型中最复杂模型生成 R 个样本量为 N 的样本, 共可得到 R 个样本;	基于备选模型中最复杂模型生成 R 个样本量为 N 的样本, 共可得到 R×S 个样本;
第3步	基于待拟合模型拟合数据, 计算检验力和效应量准确性;	基于待拟合模型拟合数据, 计算检验力和效应量准确性;	基于各备选模型拟合数据, 并根据贝叶斯拟合指标(研究1)选择最佳模型的结果用于计算检验力和效应量准确性;	基于各备选模型拟合数据, 并根据贝叶斯拟合指标(研究1)选择最佳模型的结果用于计算检验力和效应量准确性;
第4步	整合结果, 得到样本量为 N 时的检验力和效应量准确性指标的值。	整合结果, 得到样本量为 N 时的检验力和效应量准确性指标的分布。	整合结果, 得到样本量为 N 时的检验力和效应量准确性指标的值。	整合结果, 得到样本量为 N 时的检验力和效应量准确性指标的分布。
步骤	数据收集后：模型建构(研究1)
	模型确定		模型不确定
第1步	直接拟合模型。		根据拟合指标确定最佳的均值模型;
第2步			根据拟合指标确定最佳的方差模型;
第3步			拟合模型选择得到的最佳MELSM。

表2 MELSM的样本量规划和模型建构的理论范式

步骤	数据收集前：样本量规划(研究2, 研究3)
	模型确定		模型不确定
	效应量确定	效应量不确定	效应量确定	效应量不确定
第1步	根据先验信息确定1个效应量的值;	定义效应量参数先验分布, 并从中抽取 S 个效应量的值;	根据先验信息确定1个效应量的值;	定义效应量参数先验分布, 并从中抽取 S 个效应量的值;
第2步	基于待拟合模型生成 R 个样本量为 N 的样本, 共可得到 R 个样本;	基于待拟合模型生成 R 个样本量为 N 的样本, 共可得到 R×S 个样本;	基于备选模型中最复杂模型生成 R 个样本量为 N 的样本, 共可得到 R 个样本;	基于备选模型中最复杂模型生成 R 个样本量为 N 的样本, 共可得到 R×S 个样本;
第3步	基于待拟合模型拟合数据, 计算检验力和效应量准确性;	基于待拟合模型拟合数据, 计算检验力和效应量准确性;	基于各备选模型拟合数据, 并根据贝叶斯拟合指标(研究1)选择最佳模型的结果用于计算检验力和效应量准确性;	基于各备选模型拟合数据, 并根据贝叶斯拟合指标(研究1)选择最佳模型的结果用于计算检验力和效应量准确性;
第4步	整合结果, 得到样本量为 N 时的检验力和效应量准确性指标的值。	整合结果, 得到样本量为 N 时的检验力和效应量准确性指标的分布。	整合结果, 得到样本量为 N 时的检验力和效应量准确性指标的值。	整合结果, 得到样本量为 N 时的检验力和效应量准确性指标的分布。
步骤	数据收集后：模型建构(研究1)
	模型确定		模型不确定
第1步	直接拟合模型。		根据拟合指标确定最佳的均值模型;
第2步			根据拟合指标确定最佳的方差模型;
第3步			拟合模型选择得到的最佳MELSM。

胡传鹏, 王非, 过继成思, 宋梦迪, 隋洁, 彭凯平. ( 2016 ). 心理学研究中的可重复性问题:从危机到契机. 心理科学进展 , 24 (9), 1504-1518. doi: 10.3724/SP.J.1042.2016.01504 温忠麟, 范息涛, 叶宝娟, 陈宇帅. ( 2016 ). 从效应量应有的性质看中介效应量的合理性. 心理学报 , 48 (4), 435-443. Anderson, S. F., Kelley, K., & Maxwell, S. E. ( 2017 ). Sample-size planning for more accurate statistical power: A method adjusting sample effect sizes for publication bias and uncertainty. Psychological Science , 28 (11), 1547-1562. doi: 10.1177/0956797617723724 pmid: 28902575 Arend, M. G., & Schäfer, T. ( 2019 ). Statistical power in two-level models: A tutorial based on Monte Carlo simulation. Psychological Methods , 24 (1), 1-19. doi: 10.1037/met0000195 pmid: 30265048 Asparouhov, T., & Muthén, B. ( 2021 ). Advances in Bayesian model fit evaluation for structural equation models. Structural Equation Modeling: A Multidisciplinary Journal , 28 (1), 1-14. doi: 10.1080/10705511.2020.1764360 Baird, R., & Maxwell, S. E. ( 2016 ). Performance of time-varying predictors in multilevel models under an assumption of fixed or random effects. Psychological Methods , 21 (2), 175-188. doi: 10.1037/met0000070 pmid: 26950731 Barr, D. J., Levy, R., Scheepers, C., & Tily, H. J. ( 2013 ). Random effects structure for confirmatory hypothesis testing: Keep it maximal. Journal of Memory and Language , 68 , 255-278. doi: 10.1016/j.jml.2012.11.001 Blozis, S. A., McTernan, M., Harring, J. R., & Zheng, Q. ( 2020 ). Two-part mixed-effects location scale models. Behavior Research Methods , 52 (5), 1836-1847. doi: 10.3758/s13428-020-01359-7 Brauer, M., & Curtin, J. J. ( 2018 ). Linear mixed-effects models and the analysis of nonindependent data: A unified framework to analyze categorical and continuous independent variables that vary within-subjects and/or within-items. Psychological Methods , 23 (3), 389-411. doi: 10.1037/met0000159 pmid: 29172609 Brunton-Smith, I., Sturgis, P., & Leckie, G. ( 2017 ). Detecting and understanding interviewer effects on survey data by using a cross-classified mixed effects location-scale model. Journal of the Royal Statistical Society: Series A (Statistics in Society) , 180 (2), 551-568. Brysbaert, M., & Stevens, M. ( 2018 ). Power analysis and effect size in mixed effects models: A tutorial. Journal of Cognition , 1 (1), 9. https://doi.org/10.5334/joc.10 doi: 10.5334/joc.10 pmid: 31517183 Bürkner, P. C. ( 2017 ). brms: An R package for Bayesian multilevel models using Stan. Journal of Statistical Software , 80 (1), 1-28. Courvoisier, D., Walls, T. A., Cheval, B., & Hedeker, D. ( 2019 ). A mixed-effects location scale model for time-to- event data: A smoking behavior application. Addictive Behaviors , 94 , 42-49. doi: S0306-4603(18)30961-4 pmid: 30181016 Depaoli, S., & Clifton, J. P. ( 2015 ). A Bayesian approach to multilevel structural equation modeling with continuous and dichotomous outcomes. Structural Equation Modeling: A Multidisciplinary Journal , 22 (3), 327-351. doi: 10.1080/10705511.2014.937849 Faul, F., Erdfelder, E., Lang, A. -G., & Buchner, A. ( 2007 ). G* Power 3: A flexible statistical power analysis program for the social, behavioral, and biomedical sciences. Behavior Research Methods , 39 (2), 175-191. doi: 10.3758/BF03193146 Gelman, A., Meng, X. L., & Stern, H. ( 1996 ). Posterior predictive assessment of model fitness via realized discrepancies. Statistica Sinica , 6 (4), 733-760. González, B. J., de Boeck, P., & Tuerlinckx, F. ( 2014 ). Linear mixed modelling for data from a double mixed factorial design with covariates: A case-study on semantic categorization response times. Journal of the Royal Statistical Society: Series C , 63 (2), 289-302. doi: 10.1111/rssc.12031 Halsey, L. G., Curran-Everett, D., Vowler, S. L., & Drummond, G. B. ( 2015 ). The fickle p value generates irreproducible results. Nature Methods , 12 (3), 179-185. doi: 10.1038/nmeth.3288 pmid: 25719825 Hedeker, D., Mermelstein, R. J., & Demirtas, H. ( 2008 ). An application of a mixed-effects location scale model for analysis of ecological momentary assessment (EMA) data. Biometrics , 64 (2), 627-634. doi: 10.1111/j.1541-0420.2007.00924.x pmid: 17970819 Hedeker, D., Mermelstein, R. J., Demirtas, H., & Berbaum, M. L. ( 2016 ). A mixed-effects location-scale model for ordinal questionnaire data. Health Services and Outcomes Research Methodology , 16 (3), 117-131. pmid: 27570476 Hoijtink, H., Mulder, J., van Lissa, C., & Gu, X. ( 2019 ). A tutorial on testing hypotheses using the Bayes factor. Psychological Methods , 24 (5), 539-556. doi: 10.1037/met0000201 pmid: 30742472 Hox, J. J., Moerbeek, M., & van de Schoot, R. ( 2017 ). Multilevel analysis: Techniques and applications (3rd ed.). New York, Routledge. Judd, C. M., Westfall, J., & Kenny, D. A. ( 2012 ). Treating stimuli as a random factor in social psychology: A new and comprehensive solution to a pervasive but largely ignored problem. Journal of Personality & Social Psychology , 103 (1), 54-69. Judd, C. M., Westfall, J., & Kenny, D. A. ( 2017 ). Experiments with more than one random factor: designs, analytic models, and statistical power. Annual Review of Psychology , 68 (1), 601-625. doi: 10.1146/psych.2017.68.issue-1 Lee, W. Y. ( 2018 ). Generalized linear mixed effect models with crossed random effects for experimental designs having non-repeated items: Model specification and selection (Unpublished doctorial dissertation). Vanderbilt University. Lin, X., Mermelstein, R. J., & Hedeker, D. ( 2018 ). A 3-level Bayesian mixed effects location scale model with an application to ecological momentary assessment data. Statistics in Medicine , 37 (13), 2108-2119. doi: 10.1002/sim.7627 pmid: 29484693 Liu, X., & Wang, L. ( 2019 ). Sample size planning for detecting mediation effects: A power analysis procedure considering uncertainty in effect size estimates. Multivariate Behavioral Research , 54 (6), 822-839. doi: 10.1080/00273171.2019.1593814 pmid: 30983425 Martínez-Huertas, J. Á., Olmos, R., & Ferrer, E. ( 2021 ). Model selection and model averaging for mixed-effects models with crossed random effects for subjects and items. Multivariate Behavioral Research , 59 (4), 390-412. Matuschek, H., Kliegl, R., Vasishth, S., Baayen, H., & Bates, D. ( 2017 ). Balancing Type I error and power in linear mixed models. Journal of Memory and Language , 94 , 305-315. doi: 10.1016/j.jml.2017.01.001 Maxwell, S. E. ( 2004 ). The persistence of underpowered studies in psychological research: Causes, consequences, and remedies. Psychological Methods , 9 (2), 147-163. doi: 10.1037/1082-989X.9.2.147 pmid: 15137886 Nosek, B. A., Hardwicke, T. E., Moshontz, H., Allard, A., Corker, K. S., Almenberg, A. D.,... Vazire, S. ( 2022 ). Replicability, robustness, and reproducibility in psychological science. Annual Review of Psychology , 73 , 719-748. doi: 10.1146/psych.2022.73.issue-1 Park, J., & Pek, J. ( 2022 ). Conducting Bayesian-classical hybrid power analysis with R package hybridpower. Multivariate Behavioral Research . https://doi.org/10.1080/00273171.2022.2038056 Pek, J., & Park, J. ( 2019 ). Complexities in power analysis: Quantifying uncertainties with a Bayesian-classical hybrid approach. Psychological Methods , 24 (5), 590-605. doi: 10.1037/met0000208 pmid: 30816728 Rast, P., & Ferrer, E. ( 2018 ). A mixed-effects location scale model for dyadic interactions. Multivariate Behavioral Research , 53 (5), 756-775. doi: 10.1080/00273171.2018.1477577 pmid: 30395725 Schultzberg, M., & Muthén, B. ( 2018 ). Number of subjects and time points needed for multilevel time-series analysis: A simulation study of dynamic structural equation modeling. Structural Equation Modeling: A Multidisciplinary Journal , 25 (4), 495-515. doi: 10.1080/10705511.2017.1392862 Spiegelhalter, D. J., Best, N. G., Carlin, B. P., & van der Linde, A. ( 2002 ). Bayesian measures of model complexity and fit. Journal of the Royal Statistical Society. Series B (Statistical Methodology) , 64 (4), 583-639. doi: 10.1111/1467-9868.00353 Thoemmes, F., MacKinnon, D. P., & Reiser, M. R. ( 2010 ). Power analysis for complex mediational designs using Monte Carlo methods. Structural Equation Modeling , 17 (3), 510-534. doi: 10.1080/10705511.2010.489379 van Erp, S., & Browne, W. J. ( 2021 ). Bayesian multilevel structural equation modeling: An investigation into robust prior distributions for the doubly latent categorical model. Structural Equation Modeling: A Multidisciplinary Journal , 28 (6), 875-893. doi: 10.1080/10705511.2021.1915146 Walters, R. W., Hoffman, L., & Templin, J. ( 2018 ). The power to detect and predict individual differences in intra-individual variability using the mixed-effects location-scale model. Multivariate Behavioral Research , 53 (3), 360-374. doi: 10.1080/00273171.2018.1449628 pmid: 29565691 Wasserstein, R. L., & Lazar, N. A. ( 2016 ). The ASA statement on p -values: Context, process, and purpose. The American Statistician , 70 (2), 129-133. doi: 10.1080/00031305.2016.1154108 Williams, D. R., Martin, S. R., Liu, S., & Rast, P. ( 2020 ). Bayesian multivariate mixed-effects location scale modeling of longitudinal relations among affective traits, states, and physical activity. European Journal of Psychological Assessment , 36 (6), 981-997. doi: 10.1027/1015-5759/a000624 pmid: 34764628 Williams, D. R., Martin, S. R., & Rast, P. ( 2022 ). Putting the individual into reliability: Bayesian testing of homogeneous within-person variance in hierarchical models. Behavior Research Methods , 54 (3), 1272-1290. doi: 10.3758/s13428-021-01646-x Williams, D. R., Mulder, J., Rouder, J. N., & Rast, P. ( 2021 ). Beneath the surface: Unearthing within-person variability and mean relations with Bayesian mixed models. Psychological Methods , 26 (1), 74-89. doi: 10.1037/met0000270 Williams, D. R., Zimprich, D. R., & Rast, P. ( 2019 ). A Bayesian nonlinear mixed-effects location scale model for learning. Behavior Research Methods , 51 (5), 1968-1986. doi: 10.3758/s13428-019-01255-9 pmid: 31069713 Zhang, Z. ( 2014 ). Monte Carlo based statistical power analysis for mediation models: Methods and software. Behavior Research Methods , 46 (4), 1184-1198. doi: 10.3758/s13428-013-0424-0 pmid: 24338601