Journal Screenshot

International Journal of Academic Research in Business and Social Sciences

Open Access Journal

ISSN: 2222-6990

Validating an Instrument for Competency Measurement: The Art of Using Rasch Measurement Model

Muhammad Aiman Arifin

http://dx.doi.org/10.6007/IJARBSS/v11-i6/10105

Open access

Assessment of competency including the knowledge, skills, and abilities of an employee is vital to provide good information for an employer to enhance employees’ future performances as well as maintaining organization performances. To measure the current level of employee competencies, tools such as an instrument for competency measurement that is valid and reliable should be used. To provide the validity and reliability evidence, instead of using the classical test theory (CTT), another measurement model that can be utilized to conduct a more precise measurement is the Rasch Measurement Model (RMM). This paper describes the application of the Rasch Measurement Model toward a study on competency measurement instrument development and validation. A discussion of several RMM diagnoses such as Person and Item Separation Index, Item Polarity, Fit Statistics, Item Dimensionality, Standardized Residual Correlation, and Differential Item Functioning (DIF) can be used to guide social science researchers in developing an instrument to measure employee competencies.

Arifin, M. A., Rasdi, R. M., Mohd, M. A., & Omar, M. K. (2017). Competencies of Vocational Teacher?: A Personnel Measurement Framework. 7, 147–164. https://doi.org/10.6007/IJARBSS/v7-i14/3659
Awopeju, O. A., & Afolabi, E. R. I. (2016). Comparative Analysis of Classical Test Theory and Item Response Theory Based Item Parameter Estimates of Senior School Certificate Mathematics Examination. European Scientific Journal, ESJ.
Aziz, A. A., Khatimin, N., Mastor, K. A., Zaharim, A., & Yasin, S. H. M. (2013). A rasch analysis on total learning experience of UKM engineering students. International Education Studies, 6(6), 202–212. https://doi.org/10.5539/ies.v6n6p8
Bond, T. G., & Fox, C. M. (2007). Applying the Rasch Model: Fundamental Measurement in the Human Sciences, Second Edition. Journal of Educational Measurement, 2(2nd), 360. https://doi.org/10.1111/j.1745-3984.2003.tb01103.x
Bond, T. G., & Fox, C. M. (2015). Applying the Rasch Model: Fundamental Measurement in the Human Sciences. In International Journal of Testing 1(3). https://doi.org/10.1207/S15327574IJT013&4_10
Boxall, P., & Purcell, J. (2011). Strategy and the process of strategic management. In Strategy and human resource management.
Boyatzis, R. E. (1982). Competence and job performance. Competence and Performance, 10–39. https://doi.org/Samsung/Academico/Material Didatico MKZ/GC
Boyatzis, Richard E, & Boyatzis, R. E. (2008). Competencies in the 21st century. https://doi.org/10.1108/02621710810840730
Cernusca, L., & Dima, C. (2007). Competency and Human Resource Management. Revista Da Faculdade de Ciências Humanas e Sociais, 162–171.
Cronbach, L. J. (1951). Coefficient alpha and the internal structure of tests. Psychometrika. https://doi.org/10.1007/BF02310555
Dubois, D., & Rothwell, W. (2004). Competency-Based or a Traditional Approach to Training? T & D, 58(4), 46.
Engelhard Jr., G. (2013). Invariant measurement: Using Rasch models in the social, behavioral, and health sciences. In Invariant Measurement: Using Rasch Models in the Social, Behavioral, and Health Sciences LA - English.
Evarts. (1987). Enhancing managerial competence: the american management association competency programme. Journal of Management Development, 6(4), 7–18. https://doi.org/10.1108/09574090910954864
Greenstein, L. (2012). Assessing 21st century skills: A guide to evaluating mastery and authentic learning. In Assessing 21st century skills: A guide to evaluating mastery and authentic learning.
Hager, P., Gonczi, A., & Athanasou, J. (1994). General Issues about Assessment of Competence. Assessment & Evaluation in Higher Education, 19(1), 3–16. https://doi.org/10.1080/0260293940190101
Hambleton, R. K., Swaminathan, H., & Rogers, H. J. (1991). Fundamentals of item response theory. Contemporary Sociology, 21(2), 289. https://doi.org/10.2307/2075521
Hambleton, Ronald K., & Jones, R. W. (1993). Comparison of classical test theory and item response theory and their applications to test development. In Educational Measurement Issues and Practice, 12(3), 39–47. https://doi.org/10.1097/01.mlr.0000245426.10853.30
Hoffmann, T. (1999). The meanings of competency. Journal of European Industrial Training, 23(6), 275–286. https://doi.org/10.1108/03090599910284650
Huang, Y. H., Zohar, D., Robertson, M. M., Garabet, A., Lee, J., & Murphy, L. A. (2013). Development and validation of safety climate scales for lone workers using truck drivers as exemplar. Transportation Research Part F: Traffic Psychology and Behaviour. https://doi.org/10.1016/j.trf.2012.08.011
Internacional Project Management Association. (2006). ICB - IPMA Competence Baseline, Version 3.0. In Internacional Project Management Association.
Linacre, J. M. (2004). Rasch Model Estimation: Further Topics. Journal of Applied Measurement, 5(1), 95–110.
McClelland, D. (1973). Testing for competence rather than for “intelligence”. The American Psychologist, 28(1), 1–14. https://doi.org/10.1037/h0038240
Moore, D. R., Cheng, M. I., Dainty, A. R. J., Moore, D. R., & Cheng, M. (2006). Competence , competency and competencies?: performance assessment in organisations. https://doi.org/10.1108/00438020210441876
Muñiz, J., van der Linden, W. J., & Hambleton, R. K. (2008). Handbook of Modern Item Response Theory. In European Journal of Psychological Assessment, 14(1). https://doi.org/10.1027/1015-5759.14.1.91
Nornazira, S., Aede H. M., Nor, F. M. A., & Noor, K. A. J. (2015). A preliminary study of the needs for the development of competency measurement instrument for Malaysian chefs. Pertanika Journal of Social Sciences and Humanities, 23(August), 159–172.
Novick, M. R. (1966). The axioms and principal results of classical test theory. Journal of Mathematical Psychology. https://doi.org/10.1016/0022-2496(66)90002-2
Paalman, C. H., Terwee, C. B., Jansma, E. P., & Jansen, L. M. C. (2013). Instruments Measuring Externalizing Mental Health Problems in Immigrant Ethnic Minority Youths: A Systematic Review of Measurement Properties. PLoS ONE.
https://doi.org/10.1371/journal.pone.0063109
Petrillo, J., Cano, S. J., McLeod, L. D., & Coon, C. D. (2015). Using classical test theory, item response theory, and rasch measurement theory to evaluate patient-reported outcome measures: A comparison of worked examples. Value in Health, 18(1), 25–34. https://doi.org/10.1016/j.jval.2014.10.005
Reckase, M. D. (1979). Unifactor Latent Trait Models Applied to Multifactor Tests: Results and Implications. Journal of Educational Statistics.
https://doi.org/10.3102/10769986004003207
Sanchez, J. I. (2000). Anntoinette D. Lucia and Richard Lepsinger. The Art and Science of Competency Models: Pinpointing Critical Success Factors. Personnel Psychology, 53(2), 509–511.
Sanghi, S. (2007). The Handbook of Competency Mapping. In SAGE Publication. https://doi.org/10.4135/9788132108481
Schley, D. G. (2003). The Art and Science of Competency Models: Pinpointing Critical Success Factors in Organizations (Book). Academy of Management Learning & Education, 2, 210–212. https://doi.org/10.5465/AMLE.2003.9901707
Spencer, L. M., & Spencer, S. M. (1993). Competence at Work?: Models for Superior Performance. In John Wiley & Sons (pp. 1–372).
Sumintono, B. (2018). Rasch Model Measurements as Tools in Assesment for Learning. https://doi.org/10.2991/icei-17.2018.11
Wright, P. M., & Snell, S. A. (2008). Human resources, organizational resources, and capabilities. In The Routledge Companion to Strategic Human Resource Management. https://doi.org/10.4324/9780203889015-32

In-Text Citation: (Arifin, 2021)
To Cite this Article: Arifin, M. A. (2021). Validating an Instrument for Competency Measurement: The Art of Using Rasch Measurement Model. International Journal of Academic Research in Business and Social Sciences, 11(6), 159–172.