Journal Screenshot

International Journal of Academic Research in Business and Social Sciences

Open Access Journal

ISSN: 2222-6990

A Content Validity Study for Vocational Teachers’ Assessment Literacy Instrument (VoTAL)

Jimmy Chong, Siti Eshah Mokshein, Ramlee Mustapha

http://dx.doi.org/10.6007/IJARBSS/v11-i4/9547

Open access

The importance of content validity has made it a crucial step in the development of a measurement instrument. This study aimed to establish content validity for Vocational Teachers’ Assessment Literacy (VoTAL) instrument using Lawshe’s Content Validity Ratio (CVR) analysis. In this study, fourteen professional experts and seven lay experts were selected to assess the VoTAL instrument. A purposive sampling technique was employed to choose the expert panels based on their expertise that appropriate to the domain constructs of the instrument. The VoTAL instrument consists of three primary constructs represented by 125 items. Overall, the CVR analysis suggested that 100 items met the minimum requirements of the overall CVR value (CVR ? 0.428) and mean of judgments (x ? ? 1.5), whereby 25 items were excluded. In conclusion, the VoTAL instrument shows good content validity for measuring vocational teachers’ assessment literacy. Hence, further study should be carried out to examine additional psychometric properties of this instrument.

Ahmad, N. A., Drus, S. M., Kasim, H., & Othman, M. M. (2019). Assessing Content Validity of Enterprise Architecture Adoption Questionnaire (EAAQ) Among Content Experts. 2019 IEEE 9th Symposium on Computer Applications & Industrial Electronics (ISCAIE), 160–165. https://doi.org/10.1109/ISCAIE.2019.8743918
Allahyari, T., Rangi, N. H., Khosravi, Y., & Zayeri, F. (2009). Development and evaluation of a new questionnaire for rating of cognitive failures at work. International Journal of Occupational Hygiene, 3(1) 6–11.
Alonzo, A. C. (2020). Defining Trustworthiness for Teachers’ Multiple Uses of Classroom Assessment Results. In Susan M. Brookhart & J. H. Mcmillan (Eds.), Classroom Assessment and Educational Measurement (pp. 120–145). Taylor & Francis.
American Federation of Teachers [AFT], National Council on Measurement in Education [NCME], & National Educational Association [NEA]. (1990). Standards for Teacher Competence in Educational Assessment of Students. National Council on Measurement in Education.
Bennett, R. E. (2011). Formative Assessment: A Critical Review. Assessment in Education: Principles, Policy & Practice, 18(1), 5–25.
https://doi.org/10.1080/0969594X.2010.513678
Birenbaum, M., DeLuca, C., Earl, L., Heritage, M., Klenowski, V., Looney, A., Smith, K., Timperley, H., Volante, L., & Wyatt-Smith, C. (2015). International trends in the implementation of assessment for learning: Implications for policy and practice. Policy Futures in Education, 13(1), 117–140. https://doi.org/10.1177/1478210314566733
Brookhart, S. M. (2011). Educational Assessment Knowledge and Skills for Teachers. Educational Measurement: Issues and Practice, 30(1), 3–12.
https://doi.org/10.1111/j.1745-3992.2010.00195.x
Campbell, C., Murphy, J. A., & Holt, J. K. (2002). Psychometric analysis of an assessment literacy instrument: Applicability to preservice teachers. Annual Meeting of the Mid-Western Educational Research Association, Columbus, OH.
Chalavi, L. H., Alamdari, S., & Asadi, K. (2015). Improving Indices in Evaluation of Medical Research Centers in Iran. International Journal of Advanced Research in Biological Sciences, 2(2), 41–52.
DeLuca, C. (2012). Preparing Teachers for the Age of Accountability: Toward a Framework for Assessment Education. Action in Teacher Education, 34(5–6), 576–591. https://doi.org/10.1080/01626620.2012.730347
DeLuca, C., & Klinger, D. A. (2010). Assessment Literacy Development: Identifying Gaps in Teacher Candidates’ Learning. Assessment in Education: Principles, Policy & Practice, 17(4), 419–438. https://doi.org/10.1080/0969594X.2010.516643
DeLuca, C., LaPointe-McEwan, D., & Luhanga, U. (2016a). Approaches to Classroom Assessment Inventory: A New Instrument to Support Teacher Assessment Literacy. Educational Assessment, 21(4), 248–266.
https://doi.org/10.1080/10627197.2016.1236677
DeLuca, C., Valiquette, A., Coombs, A., LaPointe-McEwan, D., & Luhanga, U. (2016b). Teachers’ approaches to classroom assessment: a large-scale survey. Assessment in Education: Principles, Policy & Practice, 25(4), 355–375.
https://doi.org/10.1080/0969594X.2016.1244514
DeVellis, R. F. (2003). Scale Development: Theory and Applications. SAGE Publications.
Economic Planning Unit. (2015). 11th Malaysia Plan (2016-2020). In Prime Minister Department. http://www.epu.gov.my
Furr, M. (2011). Scale Construction and Psychometrics for Social and Personality Psychology. SAGE Publications.
Gable, R. K., & Wolf, M. B. (1993). Instrument Development in the Affective Domain (2nd ed.). Springer Netherlands.
Gotch, C. M., & French, B. F. (2014). A Systematic Review of Assessment Literacy Measures. Educational Measurement: Issues and Practice, 33(2), 14–18. https://doi.org/10.1111/emip.12030
Grant, J. S., & Davis, L. L. (1997). Selection and use of content experts for instrument development. Research in Nursing & Health, 20(3), 269–274. https://doi.org/10.1163/15718182-02502011
Harlen, W. (2012). On the Relationship between Assessment for Formative and Summative Purposes. In J. Gardner (Ed.), Assessment and Learning (2nd ed.). SAGE Publications. https://doi.org/10.4135/9781446250808
Kelly, M. P., Feistman, R., Dodge, E., St. Rose, A., & Littenberg-Tobias, J. (2020). Exploring the dimensionality of self-perceived performance assessment literacy (PAL). Educational Assessment, Evaluation and Accountability, 32(4), 499–517. https://doi.org/10.1007/s11092-020-09343-7
Kershaw, I. (1993). Ohio vocational education teachers’ perceived use of student assessment information in educational decision-making. The Ohio State University.
Klinger, D., J. McDivitt, P., B. Howard, B., A. Munoz, M., Roger, T. W., & Wylie, E. (2015). Classroom Assessment Standards for PreK-12 Teachers. Joint Committee on Standards for Educational Evaluation.
Lawshe, C. H. (1975). A Quantitative Approach to Content Validity. Personnel Psychology, 28(4), 563–575. https://doi.org/10.1111/j.1744-6570.1975.tb01393.x
Leedy, P. D., & Ormrod, J. E. (2016). Practical Research: Planning and Design. Pearson Education.
Lynn, M. R. (1986). Determination and Quantification of Content Validity. Nursing Research, 35(6), 382–386.
Maclellan, E. (2004). Initial knowledge states about assessment: novice teachers’ conceptualizations. Teaching and Teacher Education, 20(5), 523–535. https://doi.org/10.1016/j.tate.2004.04.008
Mertler, C. A. (2003). Preservice versus inservices teachers’ assessment literacy: Does classroom experience make a difference? Paper presented at the Annual Meeting of the Mid-Western Educational Research Association, Columbus, OH.
Mohamed, S., Kamis, A., & Ali, N. (2016). Gauging the assessment literacy of Malaysia’s home economics teachers: An empirical study. Geografia-Malaysian Journal of Society and Space, 3(3), 130–138.
Nunnally, J. C., & Bernstein, I. H. (1994). Psychometric theory. McGraw-Hill.
Plake, B. S. (1993). Teacher Assessment Literacy: Teachers’ Competencies in the Educational Assessment of Students. Mid-Western Educational Researcher, 6(1), 21-27.
Popham, W. J. (2014). Classroom Assessment: What Teachers Need to Know. Pearson Education.
Popham, W. J. (2016). Classroom Assessment: What Teachers Need to Know. Pearson Education.
Powell, C. (2003). The Delphi technique: myths and realities. Journal of Advanced Nursing, 41(4), 376–382. https://doi.org/10.1046/j.1365-2648.2003.02537.x
Rahman, A. A., Hanafi, N. M., Mukhtar, M. I., & Ahmad, J. (2014). Assessment Practices for Competency based Education and Training in Vocational College, Malaysia. Procedia - Social and Behavioral Sciences, 112, 1070–1076.
https://doi.org/10.1016/j.sbspro.2014.01.1271
Retnawati, H., Hadi, S., & Nugraha, A. C. (2016). Vocational High School Teachers’ Difficulties in Implementing the Assessment in Curriculum 2013 in Yogyakarta Province of Indonesia. International Journal of Instruction, 9(1), 33–48. https://doi.org/10.12973/iji.2016.914a
Rubio, D. M. G., Berg-Weger, M., Tebb, S. S., Lee, E. S., & Rauch, S. (2003). Objectifying content validity: Conducting a content validity study in social work research. Social Work Research, 27(2), 94–104. https://doi.org/10.1093/swr/27.2.94
Shepard, L. A. (2020). Discussion of Part II: Should “Measurement” Have a Role in Teacher Learning about Classroom Assessment? In Susan M. Brookhart & J. H. Mcmillan (Eds.), Classroom Assessment and Educational Measurement (pp. 192–206). Taylor & Francis.
Tilden, V. P., Nelson, C. A., & May, B. A. (1990). Use of Qualitative Methods to Enhance Content Validity. Nursing Research, 39(3), 172–175. https://doi.org/10.1097/00006199-199005000-00015
Willis, J., Adie, L., & Klenowski, V. (2013). Conceptualizing teachers’ assessment literacies in an era of curriculum and assessment reform. The Australian Educational Researcher, 40(2), 241–256. https://doi.org/10.1007/s13384-013-0089-9
Wilson, F. R., Pan, W., & Schumsky, D. A. (2012). Recalculation of the Critical Values for Lawshe’s Content Validity Ratio. Measurement and Evaluation in Counseling and Development, 45(3), 197–210. https://doi.org/10.1177/0748175612440286
Zamanzadeh, V., Ghahramanian, A., Rassouli, M., Abbaszadeh, A., Alavi-Majd, H., & Nikanfar, A.-R. (2015). Design and Implementation Content Validity Study: Development of an instrument for measuring Patient-Centered Communication. Journal of Caring Sciences, 4(2), 165–178. https://doi.org/10.15171/jcs.2015.017

In-Text Citation: (Chong et al., 2021)
To Cite this Article: Chong, J., Mokshein, S. E., & Mustapha, R. (2021). A Content Validity Study for Vocational Teachers’ Assessment Literacy Instrument (VoTAL). International Journal of Academic Research in Business and Social Sciences, 11(4), 868–883.