Differential item functioning; Health behaviour in school‑aged children; Psychosomatic health complaints; Measurement invariance; Self‑reported health complaints,; HBSC symptom checklist; Subjective health complaints; Cross-national; Adolescents
Résumé :
[en] Background
The Symptom Checklist (SCL) developed by the Health Behaviour in School-aged Children (HBSC) study is a non-clinical measure of psychosomatic complaints (e.g., headache and feeling low) that has been used in numerous studies. Several studies have investigated the psychometric characteristics of this scale; however, some psychometric properties remain unclear, among them especially a) dimensionality, b) adequacy of the Graded Response Model (GRM), and c) measurement invariance across countries.
Methods
Data from 229,906 adolescents aged 11, 13 and 15 from 46 countries that participated in the 2018 HBSC survey were analyzed. Adolescents were selected using representative sampling and surveyed by questionnaire in the classroom. Dimensionality was investigated using exploratory graph analysis. In addition, we investigated whether the GRM provided an adequate description of the data. Reliability over the latent variable continuum and differential test functioning across countries were also examined.
Results
Exploratory graph analyses showed that SCL can be considered as one-dimensional in 16 countries. However, a comparison of the unidimensional with a post-hoc bifactor GRM showed that deviation from a hypothesized one-dimensional structure was negligible in most countries. Multigroup invariance analyses supported configural and metric invariance, but not scalar invariance across 32 countries. Alignment analysis showed non-invariance especially for the items irritability, feeling nervous/bad temper and feeling low.
Conclusion
HBSC-SCL appears to represent a consistent and reliable unidimensional instrument across most countries. This bodes well for population health analyses that rely on this scale as an early indicator of mental health status.
Disciplines :
Santé publique, services médicaux & soins de santé
Auteur, co-auteur :
HEINZ, Andreas ; University of Luxembourg > Faculty of Humanities, Education and Social Sciences (FHSE) > Department of Social Sciences (DSOC)
SISCHKA, Philipp ; University of Luxembourg > Faculty of Humanities, Education and Social Sciences (FHSE) > Department of Behavioural and Cognitive Sciences (DBCS)
CATUNDA, Carolina ; University of Luxembourg > Faculty of Humanities, Education and Social Sciences (FHSE) > Department of Social Sciences (DSOC)
Cosma, Alina
García-Moya, Irene
Lyyra, Nelli
Kaman, Anne
Ravens-Sieberer, Ulrike
Pickett, William
Co-auteurs externes :
yes
Langue du document :
Anglais
Titre :
Item response theory and differential test functioning analysis of the HBSC-Symptom-Checklist across 46 countries
Vaičiūnas T, Šmigelskas K. The Role of School-Related Well-Being for Adolescent Subjective Health Complaints. IJERPH. 2019. 10.3390/ijerph16091577. DOI: 10.3390/ijerph16091577
Lyyra N, Välimaa R, Tynjälä J. Loneliness and subjective health complaints among school-aged children. Scand J Public Health. 2018;46:87–93. 10.1177/1403494817743901. DOI: 10.1177/1403494817743901
Cosma A, Stevens G, Martin G, Duinhof EL, Walsh SD, Garcia-Moya I, et al. Cross-National Time Trends in Adolescent Mental Well-Being From 2002 to 2018 and the Explanatory Role of Schoolwork Pressure. J Adolesc Health. 2020;66:S50–8. 10.1016/j.jadohealth.2020.02.010. DOI: 10.1016/j.jadohealth.2020.02.010
Norell-Clarke A, Hagquist C. Child and adolescent sleep duration recommendations in relation to psychological and somatic complaints based on data between 1985 and 2013 from 11 to 15 year-olds. J Adolesc. 2018;68:12–21. 10.1016/j.adolescence.2018.07.006. DOI: 10.1016/j.adolescence.2018.07.006
Låftman SB, Magnusson C. Do health complaints in adolescence negatively predict the chance of entering tertiary education in young adulthood? Scand J Public Health. 2017;45:878–85. 10.1177/1403494817713649. DOI: 10.1177/1403494817713649
Bohman H, Jonsson U, Päären A, von Knorring L, Olsson G, von Knorring A-L. Prognostic significance of functional somatic symptoms in adolescence: a 15-year community-based follow-up study of adolescents with depression compared with healthy peers. BMC Psychiatry. 2012;12:90. 10.1186/1471-244X-12-90. DOI: 10.1186/1471-244X-12-90
Kinnunen P, Laukkanen E, Kylmä J. Associations between psychosomatic symptoms in adolescence and mental health symptoms in early adulthood. Int J Nurs Pract. 2010;16:43–50. 10.1111/j.1440-172X.2009.01782.x. DOI: 10.1111/j.1440-172X.2009.01782.x
Haugland S, Wold B. Subjective health complaints in adolescence - reliability and validity of survey methods. J Adolesc. 2001;24:611–24. 10.1006/jado.2000.0393. DOI: 10.1006/jado.2000.0393
Torsheim T, Wold B. School-related stress, support, and subjective health complaints among early adolescents: A multilevel approach. J Adolesc. 2001;24:701–13. 10.1006/jado.2001.0440. DOI: 10.1006/jado.2001.0440
Currie C, Inchley J, Molcho M, Lenzi M, Veselska Z, Wild F. Health Behaviour in School-aged Children (HBSC) Study Protocol: Background, Methodology and Mandatory items for the 2013/14 Survey. St. Andrews; 2014.
Heinz A, Catunda C, van Duin C, Willems H. Suicide Prevention: Using the Number of Health Complaints as an Indirect Alternative for Screening Suicidal Adolescents. J Affect Disord. 2020;260:61–6. 10.1016/j.jad.2019.08.025. DOI: 10.1016/j.jad.2019.08.025
Ravens-Sieberer U, Torsheim T, Hetland J, Vollebergh W, Cavallo F, Jericek H, et al. Subjective health, symptom load and quality of life of children and adolescents in Europe. Int J Public Health. 2009;54(Suppl 2):151–9. 10.1007/s00038-009-5406-8. DOI: 10.1007/s00038-009-5406-8
Haugland S, Wold B, Stevenson J, Aaroe LE, Woynarowska B. Subjective health complaints in adolescence A cross-national comparison of prevalence and dimensionality. Eur J Public Health. 2001;11:4–10. 10.1093/eurpub/11.1.4. DOI: 10.1093/eurpub/11.1.4.
Gariepy G, McKinnon B, Sentenac M, Elgar FJ. Validity and Reliability of a Brief Symptom Checklist to Measure Psychological Health in School-Aged Children. Child Ind Res. 2016;9:471–84. 10.1007/s12187-015-9326-2. DOI: 10.1007/s12187-015-9326-2
Dey M, Jorm AF, Mackinnon AJ. Cross-sectional time trends in psychological and somatic health complaints among adolescents: a structural equation modelling analysis of “Health Behaviour in School-aged Children” data from Switzerland. Soc Psychiatry Psychiatr Epidemiol. 2015;50:1189–98. 10.1007/s00127-015-1040-3. DOI: 10.1007/s00127-015-1040-3
Potrebny T, Wiium N, Haugstvedt A, Sollesnes R, Torsheim T, Wold B, Thuen F. Health complaints among adolescents in Norway: A twenty-year perspective on trends. PLoS One. 2019;14:e0210509. 10.1371/journal.pone.0210509. DOI: 10.1371/journal.pone.0210509
Reise SP, Cook KF, Moore TM. Evaluating the Impact of Multidimensionality on Unidimensional Item Response Theory Model Parameters. In: Reise SP, Revicki DA, editors. Handbook of Item Response Theory Modeling: Applications to Typical Performance Assessment. New York: Routledge, Taylor & Francis Group; 2015. p. 13–40.
Marsman M, Borsboom D, Kruis J, Epskamp S, van Bork R, Waldorp LJ, et al. An Introduction to Network Psychometrics: Relating Ising Network Models to Item Response Theory Models. Multivar Behav Res. 2018;53:15–35. 10.1080/00273171.2017.1379379. DOI: 10.1080/00273171.2017.1379379
Clark LA, Watson D. Constructing validity: New developments in creating objective measuring instruments. Psychol Assess. 2019;31:1412–27. 10.1037/pas0000626. DOI: 10.1037/pas0000626
Argyriou AA, Mitsikostas D-D, Mantovani E, Litsardopoulos P, Panagiotopoulos V, Tamburin S. An updated brief overview on post-traumatic headache and a systematic review of the non-pharmacological interventions for its management. Expert Rev Neurother. 2021;21:475–90. 10.1080/14737175.2021.1900734. DOI: 10.1080/14737175.2021.1900734
Escobar JI, Gureje O. Influence of cultural and social factors on the epidemiology of idiopathic somatic complaints and syndromes. Psychosom Med. 2007;69:841–5. 10.1097/psy.0b013e31815b007e. DOI: 10.1097/psy.0b013e31815b007e
Martinková P, Drabinová A, Liaw Y-L, Sanders EA, McFarland JL, Price RM. Checking Equity: Why Differential Item Functioning Analysis Should Be a Routine Part of Developing Conceptual Assessments. CBE Life Sci Educ. 2017. 10.1187/cbe.16-10-0307. DOI: 10.1187/cbe.16-10-0307
Hagquist C, Andrich D. Recent advances in analysis of differential item functioning in health research using the Rasch model. Health Qual Life Outcomes. 2017;15:181. 10.1186/s12955-017-0755-0. DOI: 10.1186/s12955-017-0755-0
Hagquist C, Due P, Torsheim T, Välimaa R. Cross-country comparisons of trends in adolescent psychosomatic symptoms - a Rasch analysis of HBSC data from four Nordic countries. Health Qual Life Outcomes. 2019. 10.1186/s12955-019-1097-x. DOI: 10.1186/s12955-019-1097-x
Ravens-Sieberer U, Erhart M, Torsheim T, Hetland J, Freeman J, Danielson M, Thomas C. An international scoring system for self-reported health complaints in adolescents. Eur J Public Health. 2008;18:294–9. 10.1093/eurpub/ckn001. DOI: 10.1093/eurpub/ckn001
Depaoli S, Tiemensma J, Felt JM. Assessment of health surveys: fitting a multidimensional graded response model. Psychol Health Med. 2018;23:13–31. 10.1080/13548506.2018.1447136. DOI: 10.1080/13548506.2018.1447136
Gershon RC, Hays RD, Kallen MA. Health Measurement. In: van der Linden WJ, editor. Handbook of Item Response Theory. Boca Raton, FL: CRC Press; 2017. p. 349–63.
Hagquist C. Discrepant trends in mental health complaints among younger and older adolescents in Sweden: an analysis of WHO data 1985–2005. J Adolesc Health. 2010;46:258–64. 10.1016/j.jadohealth.2009.07.003. DOI: 10.1016/j.jadohealth.2009.07.003
Hagquist C. Explaining differential item functioning focusing on the crucial role of external information - an example from the measurement of adolescent mental health. BMC Med Res Methodol. 2019;19:185. 10.1186/s12874-019-0828-3. DOI: 10.1186/s12874-019-0828-3
Torsheim T, Ravens-Sieberer U, Hetland J, Välimaa R, Danielson M, Overpeck M. Cross-national variation of gender differences in adolescent subjective health in Europe and North America. Soc Sci Med. 2006;62:815–27. 10.1016/j.socscimed.2005.06.047. DOI: 10.1016/j.socscimed.2005.06.047
Wang D, Wang C, Chen S, Zuo C, Dong D, Wang Y. Psychometric properties of the subjective health complaints for Chinese children: parent- and self-reports. Curr Psychol. 2020;39:2357–65. 10.1007/s12144-018-9943-2. DOI: 10.1007/s12144-018-9943-2
Petanidou D, Giannakopoulos G, Tzavara C, Dimitrakaki C, Kolaitis G, Tountas Y. Adolescents’ multiple, recurrent subjective health complaints: investigating associations with emotional/behavioural difficulties in a cross-sectional, school-based study. Child Adolesc Psychiatry Ment Health. 2014;8:3. 10.1186/1753-2000-8-3. DOI: 10.1186/1753-2000-8-3
Sijtsma K. On the Use, the Misuse, and the Very Limited Usefulness of Cronbach’s Alpha. Psychometrika. 2009;74:107–20. 10.1007/s11336-008-9101-0. DOI: 10.1007/s11336-008-9101-0
Inchley J, Currie D, Cosma A, Samdal O, editors. Health Behaviour in School-aged Children (HBSC) Study Protocol: background, methodology and mandatory items for the 2017/18 survey. St. Andrews; 2018.
Golino H, Shi D, Christensen AP, Garrido LE, Nieto MD, Sadana R, et al. Investigating the Performance of Exploratory Graph Analysis and Traditional Techniques to Identify the Number of Latent Factors: A Simulation and Tutorial. Psychol Methods. 2020;25:292–320. 10.1037/met0000255. DOI: 10.1037/met0000255
Golino HF, Epskamp S, Voracek M. Exploratory Graph Analysis: A New Approach for Estimating the Number of Dimensions in Psychological Research. PLoS One. 2017;12:e0174035. 10.1371/journal.pone.0174035. DOI: 10.1371/journal.pone.0174035
Cai L, Monroe S. A New Statistic for Evaluating Item Response Theory Models for Ordinal Data. 2014. https://files.eric.ed.gov/fulltext/ED555726.pdf. Accessed 17 Aug 2022.
Monroe S, Cai L. Evaluating Structural Equation Models for Categorical Outcomes: A New Test Statistic and a Practical Challenge of Interpretation. Multivar Behav Res. 2015;50:569–83. 10.1080/00273171.2015.1032398. DOI: 10.1080/00273171.2015.1032398
Cai L, Hansen M. Limited-Information Goodness-of-Fit Testing of Hierarchical Item Factor Models: Testing Hierarchical Item Factor Models. Br J Math Stat Psychol. 2013;66:245–76. 10.1111/j.2044-8317.2012.02050.x. DOI: 10.1111/j.2044-8317.2012.02050.x
Cai L, Monroe S. IRT Model Fit Evaluation from Theory to Practice: Progress and Some Unanswered Questions. Measurement: Interdiscip Res Perspective. 2013;11:102–6. 10.1080/15366367.2013.835172. DOI: 10.1080/15366367.2013.835172
Maydeu-Olivares A, Joe H. Assessing Approximate Fit in Categorical Data Analysis. Multivar Behav Res. 2014;49:305–28. 10.1080/00273171.2014.911075. DOI: 10.1080/00273171.2014.911075
Kang T, Chen TT. Performance of the Generalized S-X2 Item Fit Index for the Graded Response Model. Asia Pac Educ Rev. 2011;12:89–96. 10.1007/s12564-010-9082-4. DOI: 10.1007/s12564-010-9082-4
Wells CS, Hambleton RK. Model Fit with Residual Analyses. In: van der Linden WJ, editor. Handbook of Item Response Theory, Volume Two Statistical Tools. New Yorck: CRC Press; 2016. p. 395–413.
Brown A. Item Response Theory Approaches to Test Scoring and Evaluating the Score Accuracy. In: Irwing FP, Booth T, Hughes DJ, editors. The Wiley Handbook of Psychometric Testing: A Multidisciplinary Reference on Survey, Scale, and Test Development. Hoboken: Wiley Blackwell; 2018. p. 607–638.
Wilson M, Draney K. A technique for setting standards and maintaining them over time. In: Nishisato S, Baba Y, Bozdogan H, Kanefugi K, editors. Measurement and multivariate analysis (Proceedings of the International Conference on Measurement and Multivariate Analysis, Banff, Canada, May 12-14, 2000). Tokyo: Springer-Verlag, 2000; 2002. p. 325-332.
Toland MD, Sulis I, Giambona F, Porcu M, Campbell JM. Introduction to Bifactor Polytomous Item Response Theory Analysis. J Sch Psychol. 2017;60:41–63. 10.1016/j.jsp.2016.11.001. DOI: 10.1016/j.jsp.2016.11.001
Stucky BD, Edelen MO. Using Hierarchical IRT Models to Create Unidimensional Measures From Multidimensional Data. In: Paul Reise SP, Revicki DA, editors. Handbook of Item Response Theory Modeling: Applications to Typical Performance Assessment. New York: Routledge, Taylor & Francis Group; 2015. p. 183–206.
Bonifay W, Lane SP, Reise SP. Three Concerns With Applying a Bifactor Model as a Structure of Psychopathology. Clin Psychol Sci. 2017;5:184–6. 10.1177/2167702616657069. DOI: 10.1177/2167702616657069
Stucky BD, Thissen D, Orlando EM. Using Logistic Approximations of Marginal Trace Lines to Develop Short Assessments. Appl Psychol Meas. 2013;37:41–57. 10.1177/0146621612462759. DOI: 10.1177/0146621612462759
Rodriguez A, Reise SP, Haviland MG. Evaluating Bifactor Models: Calculating and Interpreting Statistical Indices. Psychol Methods. 2016;21:137–50. 10.1037/met0000045. DOI: 10.1037/met0000045
ten Berge JMF, Sočan G. The Greatest Lower Bound to the Reliability of a Test and the Hypothesis of Unidimensionality. Psychometrika. 2004;69:613–25. 10.1007/BF02289858. DOI: 10.1007/BF02289858
DeMars CE. Alignment as an Alternative to Anchor Purification in DIF Analyses. Struct Equ Modeling. 2020;27:56–72. 10.1080/10705511.2019.1617151. DOI: 10.1080/10705511.2019.1617151
Marsh HW, Guo J, Parker PD, Nagengast B, Asparouhov T, Muthén B, Dicke T. What to Do When Scalar Invariance Fails: The Extended Alignment Method for Multi-Group Factor Analysis Comparison of Latent Means across Many Groups. Psychol Methods. 2018;23:524–45. 10.1037/met0000113. DOI: 10.1037/met0000113
Muthén B, Asparouhov T. IRT Studies of Many Groups: The Alignment Method. Front Psychol. 2014. 10.3389/fpsyg.2014.00978. DOI: 10.3389/fpsyg.2014.00978
Kim ES, Cao C, Wang Y, Nguyen DT. Measurement Invariance Testing with Many Groups: A Comparison of Five Approaches. Struct Equ Modeling. 2017;24:524–44. 10.1080/10705511.2017.1304822. DOI: 10.1080/10705511.2017.1304822
Chalmers RP. Model-Based Measures for Detecting and Quantifying Response Bias. Psychometrika. 2018;83:696–732. 10.1007/s11336-018-9626-9. DOI: 10.1007/s11336-018-9626-9
Kline RB. Principles and Practice of Structural Equation Modeling. New York: Guilford Press; 2016.
Little TD. Longitudinal Structural Equation Modeling. New York: Guilford Press; 2013.
Lai K, Green SB. The Problem with Having Two Watches: Assessment of Fit When RMSEA and CFI Disagree. Multivar Behav Res. 2016;51:220–39. 10.1080/00273171.2015.1134306. DOI: 10.1080/00273171.2015.1134306
Tuerlinckx F, de Boeck P. Modeling Local Item Dependencies in Item Response Theory. Psychologica Belgica. 1998;38:61. 10.5334/pb.925. DOI: 10.5334/pb.925
Prisciandaro JJ, Tolliver BK. An item response theory evaluation of the young mania rating scale and the montgomery-asberg depression rating scale in the systematic treatment enhancement program for bipolar disorder (STEP-BD). J Affect Disord. 2016;205:73–80. 10.1016/j.jad.2016.06.062. DOI: 10.1016/j.jad.2016.06.062
Inchley J, Currie D, Budisavljevic S, Torsheim T, Jåstad A, Cosma A, et al, editors. Spotlight on adolescent health and well-being. Findings from the 2017/2018 Health Behaviour in School-aged Children (HBSC) survey in Europe and Canada. International report. Volume 1. Key findings. Copenhagen: WHO Regional Office for Europe; 2020.
Walsh SD, Gaspar T. Adolescents at Risk: Psychosomatic health complaints, low life satisfaction, excessive sugar consumption and their relationship with cumulative risks, Innocenti Working Papers no. 2016_13. 2016. https://www.unicef-irc.org/publications/pdf/IWP_2016_13.pdf. Accessed 17 Aug 2022.
OECD Child Well-being Dashboard. Organisation for Economic Co-operation and Development, Paris. 2022. https://www.oecd.org/els/family/child-well-being/data/dashboard/. Accessed 17 Aug 2022.
R Core Team. R: A Language and Environment for Statistical Computing. Vienna: R Foundation for Statistical Computing; 2021.
Wickham H, Averick M, Bryan J, Chang W, McGowan LDA, François R, et al. Welcome to the tidyverse. J Open Source Softw. 2019;4:1686. 10.21105/joss.01686. DOI: 10.21105/joss.01686
Fox J, Weisberg S. An R Companion to Applied Regression. Thousand Oaks: Sage; 2019.
Larmarange J. Labelled: Manipulating Labelled Data. R Package Version 2.8.0 2021.
Lüdecke D. Sjlabelled: Labelled Data Utility Functions. R Package Version 1.1.8 2021. 10.5281/zenodo.1249215.
Pasek J, Tahk A, Culter G, Schwemmle M. Weights: Weighting and Weighted Statistics. R Package Version 1.0.2 2021.
Parchami A. Weighted.Desc.Stat: Weighted Descriptive Statistics. R Package Version 1.0 2016.
O’Connor BP. EFA.Dimensions: Exploratory Factor Analysis Functions for Assessing Dimensionality. R Package Version 0.1.7.2 2021.
Revelle W. Psych: Procedures for Psychological, Psychometric, and Personality Research. R Package Version 2.1.6 2021. Evanston.
Lubbe D. Parallel Analysis with Categorical Variables: Impact of Category Probability Proportions on Dimensionality Assessment Accuracy. Psychol Methods. 2019;24:339–51. 10.1037/met0000171. DOI: 10.1037/met0000171
Golino H, Christensen AP. EGAnet: Exploratory Graph Analysis A Framework for Estimating the Number of Dimensions in Multivariate Data Using Network Psychometrics. R Package Version 0.9.8 2021.
Chalmers RP. Mirt: A Multidimensional Item Response Theory Package for the R Environment. J Stat Softw. 2012. 10.18637/jss.v048.i06. DOI: 10.18637/jss.v048.i06
Lim H. Irtplay: Unidimensional Item Response Theory Modeling. R Package Version 1.6.2 2020.
Wickham H. Ggplot2: Elegant Graphics for Data Analysis: Springer-Verlag New York; 2016.
Kassambara A. Ggpubr: ’ggplot2’ Based Publication Ready Plots. R Package Version 0.4.0 2020.
Hallquist MN, Wiley JF. MplusAutomation: An R Package for Facilitating Large-Scale Latent Variable Analyses in Mplus. Struct Equ Model. 2018;25:621–38. 10.1080/10705511.2017.1402334. DOI: 10.1080/10705511.2017.1402334
