Search Advanced

AIMS Public Health

2018, Volume 5, Issue 3: 235-259. doi: 10.3934/publichealth.2018.3.235
Previous Article Next Article
Research article Topical Sections

Criterion scores, construct validity and reliability of a web-based instrument to assess physiotherapists’ clinical reasoning focused on behaviour change: ‘Reasoning 4 Change’

  • 1.
    Division of Physiotherapy, School of Health, Care and Social Welfare, Mälardalen University, Västerås, Sweden
  • 2.
    Division of Psychology, School of Health, Care and Social Welfare, Mälardalen University, Eskilstuna, Sweden
  • 3.
    Department of Physical Therapy, Faculty of Medicine, University of British Columbia, Vancouver, BC, Canada
  • 4.
    Department of Information Technology, Uppsala University, Uppsala, Sweden
  • Received: 28 April 2018 Accepted: 29 June 2018 Published: 06 July 2018

  • Background and aim: ‘Reasoning 4 Change’ (R4C) is a newly developed instrument, including four domains (D1–D4), to assess clinical practitioners’ and students’ clinical reasoning with a focus on clients’ behaviour change in a physiotherapy context. To establish its use in education and research, its psychometric properties needed to be evaluated. The aim of the study was to generate criterion scores and evaluate the reliability and construct validity of a web-based version of the R4C instrument. Methods: Fourteen physiotherapy experts and 39 final-year physiotherapy students completed the R4C instrument and the Pain Attitudes and Beliefs Scale for Physiotherapists (PABS-PT). Twelve experts and 17 students completed the R4C instrument on a second occasion. The R4C instrument was evaluated with regard to: internal consistency (five subscales of D1); test-retest reliability (D1–D4); inter-rater reliability (D2–D4); and construct validity in terms of convergent validity (D1.4, D2, D4). Criterion scores were generated based on the experts’ responses to identify the scores of qualified practitioners’ clinical reasoning abilities. Results: For the expert and student samples, the analyses demonstrated satisfactory internal consistency (a range: 0.67–0.91), satisfactory test-retest reliability (ICC range: 0.46–0.94) except for D3 for the experts and D4 for the students. The inter-rater reliability demonstrated excellent agreement within the expert group (ICC range: 0.94–1.0). The correlations between the R4C instrument and PABS-PT (r range: 0.06–0.76) supported acceptable construct validity. Conclusions: The web-based R4C instrument shows satisfactory psychometric properties and could be useful in education and research. The use of the instrument may contribute to a deeper understanding of physiotherapists’ and students’ clinical reasoning, valuable for curriculum development and improvements of competencies in clinical reasoning related to clients’ behavioural change.

    Citation: Maria Elvén, Jacek Hochwälder, Elizabeth Dean, Olle Hällman, Anne Söderlund. Criterion scores, construct validity and reliability of a web-based instrument to assess physiotherapists’ clinical reasoning focused on behaviour change: ‘Reasoning 4 Change’[J]. AIMS Public Health, 2018, 5(3): 235-259. doi: 10.3934/publichealth.2018.3.235

    Related Papers:

  • Background and aim: ‘Reasoning 4 Change’ (R4C) is a newly developed instrument, including four domains (D1–D4), to assess clinical practitioners’ and students’ clinical reasoning with a focus on clients’ behaviour change in a physiotherapy context. To establish its use in education and research, its psychometric properties needed to be evaluated. The aim of the study was to generate criterion scores and evaluate the reliability and construct validity of a web-based version of the R4C instrument. Methods: Fourteen physiotherapy experts and 39 final-year physiotherapy students completed the R4C instrument and the Pain Attitudes and Beliefs Scale for Physiotherapists (PABS-PT). Twelve experts and 17 students completed the R4C instrument on a second occasion. The R4C instrument was evaluated with regard to: internal consistency (five subscales of D1); test-retest reliability (D1–D4); inter-rater reliability (D2–D4); and construct validity in terms of convergent validity (D1.4, D2, D4). Criterion scores were generated based on the experts’ responses to identify the scores of qualified practitioners’ clinical reasoning abilities. Results: For the expert and student samples, the analyses demonstrated satisfactory internal consistency (a range: 0.67–0.91), satisfactory test-retest reliability (ICC range: 0.46–0.94) except for D3 for the experts and D4 for the students. The inter-rater reliability demonstrated excellent agreement within the expert group (ICC range: 0.94–1.0). The correlations between the R4C instrument and PABS-PT (r range: 0.06–0.76) supported acceptable construct validity. Conclusions: The web-based R4C instrument shows satisfactory psychometric properties and could be useful in education and research. The use of the instrument may contribute to a deeper understanding of physiotherapists’ and students’ clinical reasoning, valuable for curriculum development and improvements of competencies in clinical reasoning related to clients’ behavioural change.


    加载中
    [1] WHO (2013) World Health Organization. Global action plan for the prevention and control of noncommunicable diseases 2013–2020. Available from: http://apps.who.int/iris/bitstream/10665/94384/1/9789241506236_eng.pdf?ua=1.
    [2] Åsenlöf P, Denison E, Lindberg P (2005) Individually tailored treatment targeting activity, motor behavior, and cognition reduces pain-related disability: a randomized controlled trial in patients with musculoskeletal pain. J Pain 6: 588–603. doi: 10.1016/j.jpain.2005.03.008
    [3] Friedrich M, Gittler G, Arendasy M, et al. (2005) Long-term effect of a combined exercise and motivational program on the level of disability of patients with chronic low back pain. Spine 30: 995–1000. doi: 10.1097/01.brs.0000160844.71551.af
    [4] Dean E, de Andrade AD, O'Donoghue G, et al. (2014) The second physical therapy summit on global health: developing an action plan to promote health in daily practice and reduce the burden of non-communicable diseases. Physiother Theory Pract 30: 261–275. doi: 10.3109/09593985.2013.856977
    [5] Higgs J, Jones MA (2008) Clinical decision making and multiple problem spaces In: Higgs J, Jones MA, Loftus S et al. Editors, Clinical reasoning in the health professions, 3 Eds., Amsterdam: Butterworth-Heinemann, 3–14.
    [6] WCPT (2015) World Confederation for Physical Therapy. Policy statement: Non-communicable diseases. Available from: http://www.wcpt.org/policy/ps-ncd.
    [7] Christensen N, Black L, Furze J, et al. (2017) Clinical reasoning: survey of teaching methods, integration, and assessment in entry-level physical therapist academic education. Phys Ther 97: 175–186. doi: 10.2522/ptj.20150320
    [8] Yeung E, Kulasagarem K, Woods N, et al. (2016) Validity of a new assessment rubric for a short-answer test of clinical reasoning. BMC Med Educ 16: 192. doi: 10.1186/s12909-016-0714-1
    [9] APTA (2017) American Physical Therapy Association. Physical Therapist Clinical Performance Instrument (PT CPI). Version 2006 Update. Available from: http://www.apta.org/PTCPI/.
    [10] Dalton M, Davidson M, Keating JL (2012) The Assessment of Physiotherapy Practice (APP) is a reliable measure of professional competence of physiotherapy students: a reliability study. J Physiother 58: 49–56. doi: 10.1016/S1836-9553(12)70072-3
    [11] Lewis LK, Stiller K, Hardy F (2008) A clinical assessment tool used for physiotherapy students - is it reliable? Physiother Theory Pract 24: 121–134. doi: 10.1080/09593980701508894
    [12] Meldrum D, Lydon A-M, Loughnane M, et al. (2008) Assessment of undergraduate physiotherapist clinical performance: investigation of educator inter-rater reliability. Physiother 94: 212–219. doi: 10.1016/j.physio.2008.03.003
    [13] Elvén M, Hochwalder J, Dean E, et al. (2018) Development and initial evaluation of an instrument to assess physiotherapists' clinical reasoning focused on clients' behavior change. Physiother Theory Pract 34: 367–383. doi: 10.1080/09593985.2017.1419521
    [14] Elvén M, Hochwälder J, Dean E, et al. (2015) A clinical reasoning model focused on clients' behaviour change with reference to physiotherapists: Its multiphase development and validation Physiother Theory Pract 31: 231–243.
    [15] Elstein AS, Shulman LS, Sprafka SA (1978) Medical Problem Solving: An analysis of clinical reasoning, 1 Eds. Cambridge, Massachusetts: Harvard University Press.
    [16] Kreiter CD, Bergus G (2009) The validity of performance-based measures of clinical reasoning and alternative approaches. Med Educ 43: 320–325. doi: 10.1111/j.1365-2923.2008.03281.x
    [17] Durning SJ, Artino JAR, Schuwirth L, et al. (2013) Clarifying assumptions to enhance our understanding and assessment of clinical reasoning. Acad Med 88: 442–448. doi: 10.1097/ACM.0b013e3182851b5b
    [18] Fischer MR, Kopp V, Holzer M, et al. (2005) A modified electronic key feature examination for undergraduate medical students: validation threats and opportunities. Med Teach 27: 450–455. doi: 10.1080/01421590500078471
    [19] Fournier J, Demeester A, Charlin B (2008) Script concordance tests: Guidelines for construction. BMC Med Inform Decis: 8:18. doi: 10.1186/1472-6947-8-18
    [20] Cook DA, Triola MM (2009) Virtual patients: a critical literature review and proposed next steps. Med Educ 43: 303–311. doi: 10.1111/j.1365-2923.2008.03286.x
    [21] Dory V, Gagnon R, Vanpee D, et al. (2012) How to construct and implement script concordance tests: insights from a systematic review. Med Educ 46: 552–563. doi: 10.1111/j.1365-2923.2011.04211.x
    [22] Charlin B, Boshuizen HPA, Custers EJ, et al. (2007) Scripts and clinical reasoning. Med Educ 41: 1178–1184. doi: 10.1111/j.1365-2923.2007.02924.x
    [23] Norman GR, Tugwell P, Feightner JW, et al. (1985) Knowledge and clinical problem-solving. Med Educ 19: 344–356. doi: 10.1111/j.1365-2923.1985.tb01336.x
    [24] Charlin B, Roy L, Brailovsky C, et al. (2000) The Script Concordance test: a tool to assess the reflective clinician. Teach Learn Med 12: 189–195. doi: 10.1207/S15328015TLM1204_5
    [25] Streiner DL, Norman GR (2008) Health measurement scales. A practical guide to their development and use., 4 Eds. Oxford: University Press.
    [26] Houben RM, Ostelo RW, Vlaeyen JW, et al. (2005) Health care providers' orientations towards common low back pain predict perceived harmfulness of physical activities and recommendations regarding return to normal activity. Eur J Pain 9: 173–183. doi: 10.1016/j.ejpain.2004.05.002
    [27] Ostelo RWJG, Stomp-van den Berg SGM, Vlaeyen JWS, et al. (2003) Health care provider's attitudes and beliefs towards chronic low back pain: the development of a questionnaire. Man Ther 8: 214–222. doi: 10.1016/S1356-689X(03)00013-4
    [28] World Medical Association (2013) Declaration of Helsinki: ethical principles for medical research involving human subjects. JAMA 310: 2191–2194. doi: 10.1001/jama.2013.281053
    [29] Gagnon R, Charlin B, Coletti M, et al. (2005) Assessment in the context of uncertainty: how many members are needed on the panel of reference of a script concordance test? Med Educ 39: 284–291. doi: 10.1111/j.1365-2929.2005.02092.x
    [30] Baker J, Lovell K, Harris N (2006) How expert are the experts? An exploration of the concept of 'expert' within Delphi panel techniques. Nurse Researcher 14: 59–70.
    [31] Polit DF, Beck CT (2010) Essentials of nursing research. Appraising evidence for nursing practice, 7 Eds. Philadelphia: Lippincott Williams & Wilkins.
    [32] Farmer EA, Page G (2005) A practical guide to assessing clinical decision-making skills using the key features approach. Med Educ 39: 1188–1194. doi: 10.1111/j.1365-2929.2005.02339.x
    [33] Johnson J (2014) Designing with the mind in mind., 2 Eds. Amsterdam: Morgan Kaufmann, Elsevier Inc.
    [34] Tidwell J (2011) Designing interfaces: Patterns for effective interaction design., 2 Eds. Sebastopol: O'Reilly Media, Inc.
    [35] Charlin B, Desaulniers M, Gagnon R, et al. (2002) Comparison of an aggregate scoring method with a consensus scoring method in a measure of clinical reasoning capacity. Teach Learn Med 14: 150–156. doi: 10.1207/S15328015TLM1403_3
    [36] Overmeer T, Boersma K, Main CJ, et al. (2009) Do physical therapists change their beliefs, attitudes, knowledge, skills and behaviour after a biopsychosocially orientated university course? J Eval Clin Pract 15: 724–732. doi: 10.1111/j.1365-2753.2008.01089.x
    [37] Mutsaers JHAM, Peters R, Pool-Goudzwaard AL, et al. (2012) Systematic review: Psychometric properties of the Pain Attitudes and Beliefs Scale for Physiotherapists: A systematic review. Man Ther 17: 213–218. doi: 10.1016/j.math.2011.12.010
    [38] Eland ND, Kvale A, Ostelo R, et al. (2017) The Pain Attitudes and Beliefs Scale for Physiotherapists: Dimensionality and Internal Consistency of the Norwegian Version. Physiother Res Int 22: e1670. doi: 10.1002/pri.1670
    [39] Field A (2013) Discovering statistics using IBM SPSS statistics, 4 Eds. London: Sage.
    [40] Cronbach L (1951) Coefficient alpha and the internal structure of tests. Psychometrika 16: 297–334. doi: 10.1007/BF02310555
    [41] Nunnally JM, Bernstein IH (1994) Psychometric theory, 3 Eds. New York: McGraw-Hill.
    [42] Streiner DL (2003) Starting at the beginning: an introduction to coefficient alpha and internal consistency. J Pers Assess 80: 99–103. doi: 10.1207/S15327752JPA8001_18
    [43] Hallgren KA (2012) Computing inter-rater reliability for observational data: An overview and tutorial. Tutor Quant Methods Psychol 8: 12–34.
    [44] Schuck P (2004) Assessing reproducibility for interval data in health-related quality of life questionnaires: which coefficient should be used? Qual Life Res 13: 571–586. doi: 10.1023/B:QURE.0000021318.92272.2a
    [45] Cicchetti DV (2001) The precision of reliability and validity estimates re-visited: distinguishing between clinical and statistical significance of sample size requirements. J Clin Exp Neuropsychol 23: 695–700. doi: 10.1076/jcen.23.5.695.1249
    [46] DeVon HA, Block ME, Moyle-Wright P, et al. (2007) A psychometric toolbox for testing validity and reliability. J Nurs Scholarsh 39: 155–164. doi: 10.1111/j.1547-5069.2007.00161.x
    [47] Terwee CB, Bot SD, de Boer MR, et al. (2007) Quality criteria were proposed for measurement properties of health status questionnaires. J Clin Epidemiol 60: 34–42. doi: 10.1016/j.jclinepi.2006.03.012
    [48] Dawson T, Comer L, Kossick MA, et al. (2014) Can script concordance testing be used in nursing education to accurately assess clinical reasoning skills? J Nurs Educ 53: 281–286. doi: 10.3928/01484834-20140321-03
    [49] Humbert AJ, Johnson MT, Miech E, et al. (2011) Assessment of clinical reasoning: A Script Concordance test designed for pre-clinical medical students. Med Teach 33: 472–477. doi: 10.3109/0142159X.2010.531157
    [50] Nouh T, Boutros M, Gagnon R, et al. (2012) The script concordance test as a measure of clinical reasoning: a national validation study. Am J Surg Pathol 203: 530–534. doi: 10.1016/j.amjsurg.2011.11.006
    [51] Bland AC, Kreiter CD, Gordon JA (2005) The psychometric properties of five scoring methods applied to the script concordance test. Acad Med 80: 395–399. doi: 10.1097/00001888-200504000-00019
    [52] Lineberry M, Kreiter CD, Bordage G (2013) Threats to validity in the use and interpretation of script concordance test scores. Med Educ 47: 1175–1183. doi: 10.1111/medu.12283
    [53] Lubarsky S, Dory V, Duggan P, et al. (2013) Script concordance testing: From theory to practice: AMEE Guide No. 75. Med Teach 35: 184–193. doi: 10.3109/0142159X.2013.760036
    [54] Lubarsky S, Charlin B, Cook DA, et al. (2011) Script concordance testing: a review of published validity evidence. Med Educ 45: 329–338. doi: 10.1111/j.1365-2923.2010.03863.x
    [55] Elvén M, Dean E (2017) Factors influencing physical therapists' clinical reasoning: qualitative systematic review and meta-synthesis. Phys Ther Rev 22: 60–75. doi: 10.1080/10833196.2017.1289647
    [56] Wainwright SF, Shepard KF, Harman LB, et al. (2011) Factors that influence the clinical decision making of novice and experienced physical therapists. Phys Ther 91: 87–101. doi: 10.2522/ptj.20100161
    [57] Gatchel RJ, Peng YB, Peters ML, et al. (2007) The biopsychosocial approach to chronic pain: scientific advances and future directions. Psychol Bull 133: 581–624. doi: 10.1037/0033-2909.133.4.581
    [58] Soderlund A (2011) The role of educational and learning approaches in rehabilitation of whiplash-associated disorders in lessening the transition to chronicity. Spine 36: 280–285. doi: 10.1097/BRS.0b013e3182388220
    [59] Gray H, Howe T (2013) Physiotherapists' assessment and management of psychosocial factors (Yellow and Blue Flags) in individuals with back pain. Phys Ther Rev 18: 379–394. doi: 10.1179/1743288X13Y.0000000096
    [60] Gilliland S, Wainwright SF (2017) Patterns of clinical reasoning in physical therapist students. Phys Ther 97: 499–511. doi: 10.1093/ptj/pzx028
    [61] Solvang PK, Fougner M (2016) Professional roles in physiotherapy practice: Educating for self-management, relational matching, and coaching for everyday life. Physiother Theory Pract 32: 591–602. doi: 10.1080/09593985.2016.1228018
    [62] Foster NE, Delitto A (2011) Embedding psychosocial perspectives within clinical management of low back pain: integration of psychosocially informed management principles into physical therapist practice-challenges and opportunities. Phys Ther 91: 790–803. doi: 10.2522/ptj.20100326
    [63] DeVellis RF (2012) Scale Development. Theory and Applications, 3 Eds. Thousands Oaks: SAGE Publications
    [64] Cortina JM (1993) What is coefficient alpha? J Appl Psychol 78: 98–104. doi: 10.1037/0021-9010.78.1.98
    [65] Netemeyer RG, Bearden WO, Sharma S (2003) Scaling Procedures. Issues and Applications, 1 Eds. Thousands Oaks: Sage Publications.
    [66] Darlow B, Fullen BM, Dean S, et al. (2012) The association between health care professional attitudes and beliefs and the attitudes and beliefs, clinical management, and outcomes of patients with low back pain: A systematic review. Eur J Pain 16: 3–17. doi: 10.1016/j.ejpain.2011.06.006
    [67] Simmonds MJ, Derghazarian T, Vlaeyen JW (2012) Physiotherapists' knowledge, attitudes, and intolerance of uncertainty influence decision making in low back pain. Clin J Pain 28: 467–474. doi: 10.1097/AJP.0b013e31825bfe65
    [68] Cook DA, Beckman TJ (2006) Current concepts in validity and reliability for psychometric instruments: theory and application. Am J Med 119: 166.e7–16.
    [69] Koo TK, Li MY (2016) A guideline of selecting and reporting intraclass correlation coefficients for reliability research. J Chiropr Med 15: 155–163. doi: 10.1016/j.jcm.2016.02.012
    [70] Kottner J, Audige L, Brorson S, et al. (2011) Guidelines for reporting reliability and agreement studies (GRRAS) were proposed. Int J Nurs Stud 48: 661–671. doi: 10.1016/j.ijnurstu.2011.01.016
    [71] Bennett RE (2011) Formative assessment: a critical review. Assess Educ Princ Pol Pract 18: 5–25.
  • publichealth-05-03-235-s001.pdf
  • Reader Comments
  • © 2018 the Author(s), licensee AIMS Press. This is an open access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/4.0)
通讯作者: 陈斌, bchen63@163.com
  • 1. 

    沈阳化工大学材料科学与工程学院 沈阳 110142

  1. 本站搜索
  2. 百度学术搜索
  3. 万方数据库搜索
  4. CNKI搜索
AIMS Public Health
3.3 3.4

Metrics

Article views(4092) PDF downloads(940) Cited by(8)

Preview PDF
Download XML
Export Citation
Article outline

Figures and Tables

Tables(6)

Other Articles By Authors

Related pages

Tools

/

DownLoad:  Full-Size Img  PowerPoint
Return
Return

Catalog