Validity and Reliability of Self-Initiated Professional Development (SI-PD) Instrument: A Rasch Measurement Model approach for Pilot Study

This pilot study was conducted to generate empirical evidence on the validity and reliability of elements of Self-Initiated Professional Development (SI- PD) for TVET teachers’ in vocational colleges. Construct validity was used by analyzing point-measure correlation index (PTMEA CORR), infit and outfit MNSQ values, standard residual correlations, item reliability and person reliability. The instrument was distributed to 40 TVET teachers in Malaysian vocational colleges. This instrument was developed to measure ten SI-PD elements, namely, Understanding, Rewards, School Administrators’ Role, Professional Development System, Attitude, Readiness, Collaborative Environment, Industrial Attachment Opportunity, Autonomy and Strategy. The reliability value based on Alpha Cronbach value was 0.98 which indicated that the instrument was very good and effective with high degree of consistency. The person reliability value was 0.97 while the item reliability value was 0.86. This indicated that the respondents selected were highly reliable because the value were more than 0.80. The separation index value for respondents was 5.68 which indicated that there were six groups levels in giving agreement on SI-PD elements, while, item separation index of 2.46 indicated there were three levels of item difficulty. Removing and refining the items was done by taking account supervisors’ and experts’ views and evaluation. This pilot study showed that only five items did not meet the criteria and 76 items remained to be used for actual study. In conclusion, findings from the study clearly showed that the developed instrument have a high level of validity and reliability and capable to be used in measuring SI-PD elements.


Introduction
TVET in Malaysia has experienced a major transformation which can be seen through the upgrading of vocational schools to vocational colleges. Vocational colleges focus on developing human capital with relevant skills and possessed diploma qualifications besides overcoming dropout problems among students in academic field. The transformation aimed to elevate the standard of TVET to be at par with mainstream education. This is because society often regard TVET as second-class education (OECD, 2020;Ab Rahim, 2017).
The Shared Prosperity Vision (SPV) 2030, TVET teachers in vocational colleges are challenged to adapt with the transformation which required them to maintain teaching and learning quality at the same time. The main factor to improve the quality of TVET graduates are to rely on the competencies of TVET teachers. Engaging in meaningful professional development activities will keep TVET teachers updated with the current knowledge and skills because demand for TVET graduates in Malaysia is expected to increase to 7.98 million by the year 2030 (Devi, 2019). Professional development for TVET teachers is important because they need to ensure the students they produce are relevant to the current market (Syuhada, 2015). However, the competencies of TVET teachers depend on how well they continue to learn and update their knowledge and skills.

Literature Review
Professional development plays an essential role in ensuring that TVET teachers are able to keep abreast of developments in their own field on teaching as well as changes in teaching methods and technologies. According to Andersson and Köpsén (2015), TVET teachers must have current vocational identity that includes all of the knowledge and skills necessary to proficiently practice their vocational in a way that meets current expectations. While, according to Sirk et al. (2016), the work of vocational teachers has become increasingly more complex, coupled with growing societal demands and expectation upon them. This was aligned to Zuhaili and Ramlee (2017) study on TVET teachers in vocational colleges who found that TVET teachers' workloads were increased since the upgrading of vocational schools to vocational colleges.
Recently, MOE focused on improving the value proposition and attractiveness of TVET because often TVET seems to be the last resort for less qualified students in academic area. This common stigma of TVET has also created an impression that TVET is to cater the school dropouts, rather than a key strategy to train competent future workforce (Affero & Razali, 2013). TVET teachers need ongoing opportunities to build their understanding and ability specially to cater students with diverse interests and backgrounds (Axmann et al., 2015). TVET teachers must be trained with relevant knowledge and skills linked with industry needs because they are responsible to address the economic growth issue that required skilled TVET graduates to fulfill the human resource needs in the industry (Khaizer et al., 2020). Their professional development should enable better links between theory and practice since majority courses offered are practical-laden courses (Kanwar et al., 2019;Ramli, 2018).
Self-Initiated Professional Development (SI-PD) is a professional development that arises from teacher's own initiative which to develop their intellect, experience as well as attitude and internally motivated (Mushayikwa, 2013). It is a new approach of professional development in Malaysia and most of the teachers still unfamiliar with it, thus they still need supports because sometimes teachers do not always recognize what they do not know (Riddle, 2018;Louws et al., 2017;Noorriati et al., 2016). SI-PD is an application of Self-Directed Learning (SDL) which is learner-driven learning that determined by the needs, interests and learning conditions of the individual (Tri et al., 2017). Some of the teachers were not familiar with professional development opportunities available within their working environment such as peer classroom observation, coaching, discussions, networking, as well as classroombased action research (Komba & Mwakabenga, 2019). Therefore, there are several elements that need to be given attention to successfully improve teachers' fullest potential (Kan'an & Osman, 2015).
Tight budgets are forcing school administrators to shift towards lower cost and higher impact professional development approach but it does not mean there is less learning and development going on, just it is being delivered differently (Tripon, 2019). As a whole, SI-PD offers inexpensive, versatile, personalized approach to simultaneously address teacher needs, organizational goals and student achievement (Jonathan, 2019;Mushayikwa, 2013). Besides that, SI-PD is crucial to the adult development because it accommodates individuals' learning styles and goals (Rana et al., 2016;Simegn, 2014). When designing professional development that comprised adults, it must reach the participants based on their level. They must be willing to keep learning, so that the knowledge and skills possessed according to current conditions (Marfu'ah et al., 2017).
The researcher intended to reduce the existing gaps between perceptions and expectations especially to TVET teachers in implementing SI-PD. Moreover, there were lots of studies regarding TVET teachers' professional development that not focusing on specific needs of the teachers, with most of the studies focusing on the benefits that can be gained from the professional development. There is a need to explore the elements for SI-PD framework which could contribute to TVET teachers' professionalism and strengthen their teaching practice (Junita et al., 2019). The purpose of this research is to develop a framework for implementation of SI-PD among TVET teachers in vocational colleges.

Methodology
Validity and reliability are important issues when developing research instruments. They are closely related but indicate different things. A measurement can be reliable without being valid. However, if a measurement is valid, it is usually also reliable (Cohen et al., 2018). Therefore, a pilot test was conducted to generate empirical evidence on the validity and reliability of SI-PD elements for TVET teachers' in vocational colleges. This process was important in order to detect any shortcomings or flaws with regards to the instrument used in the actual study. The pilot test was also implemented to improve the content, language use and the meaning of the items for easy comprehension.
In order to ensure instrument validity, face and content validation by experts was conducted to ensure that the questionnaire aligned with research objectives (Newman & Covrig, 2013;Wiersma & Jurs, 2009). Face validation is the extent to which a measurement method appears 'on its face' to measure the construct of interest. It is to check the language representation used and the questionnaire layout (Creswell & Clark, 2011). Meanwhile for content validity, it is the extent which a measure covers the construct of interest. The items in questionnaire should have the capability to measure what should be measured (Creswell & Clark, 2011). In this regard, fours experts involved and their selection were based on their expertise in TVET teachers' professional development, psychometric and language field. Through this validation stage, any misleading or do not meet the use of language items has been modified accordance to the suitability of the sample to answer the questionnaire. While, the reliability of the instrument is to measure the consistency of each item in the questionnaire.
The questionnaire consists of 81 items of five-point Likert scale that measured ten elements of SI-PD. Questionnaires were distributed to 40 TVET teachers in vocational colleges as recommended by Chua (2011) to determine the suitability of the elements for SI-PD framework. This also aligned to Cooper and Schindler (2011), the recommended number of respondents in the pilot study was between 25 to 100. The results of the pilot test were analysed using Winsteps software based on Rasch measurement model approach. The questionnaire consisted ten elements which were Understanding, Rewards, School Administrators' Role, Professional Development System, Attitude, Readiness, Industrial Attachment Opportunity, Autonomy and Strategy. The data gathered from the pilot test was analyzed using Winsteps software based on Rasch Measurement Model because it can produce a quality and high reliability instrument (Boone, 2018). The Rasch measurement model incorporates a method for ordering person according to their ability and ordering items according to their difficulty (Bond & Fox, 2015). There were four diagnoses performed, namely, item reliability; person reliability; item polarity based on PTMEA CORR value; and item fit. The Table 1 below shows the benchmarks that were used in this pilot study for determining the validity and reliability of the instrument.

Results and Discussion
Rasch Measurement model approach was used by the researcher to perform examination on the item functional in terms of (i) item reliability and separation of the respondents; (ii) detecting polarity items that measure the constructs based on the PTMEA CORR; (iii) items fit measuring elements and (iv) determine the item depends on the correlation of the standardized residuals.

Items and Person Reliability
The reliability of the instrument was determined by the reliability and separation index values as referred to the Rasch measurement model approach. Based on Rasch measurement model approach, the Alpha Cronbach (α) coefficient value that been accepted was between 0.71 until 0.99 as shows in Table 2 below (Bond & Fox, 2015). Table 2 Alpha Cronbach (α) coefficient value (Bond & Fox, 2015) Value Reliability 0.8 until 1.0 Very good and effective with a high degree of consistency 0.7 until 0.8 Good and acceptable 0.6 until 0.7 Acceptable < 0.6 Items need to be repaired < 0.5 Items need to be dropped Based on the pilot test analysis, the reliability value obtained based on Alpha Cronbach (α) value was 0.98. Therefore, the value represents that the instrument was very good and effective with high degree of consistency. The person reliability value was 0.97 while the item reliability value was also 0.86. These indicated that the respondents selected were reliable respondents and the items produced have high reliability because the value was more than 0.80 (Bond & Fox, 2015). The separation index value for respondent was 5.68 which indicated there were six groups levels in giving agreement on the SI-PD elements by TVET teachers. While, the item separation index was 2.46. It indicated there were three levels of item difficulty produced in measuring TVET teachers' agreement on the elements of SI-PD. This separation index value was following the criteria required by Rasch measurement model that was above the value of 2.0 (Linacre, 2005). Table 3 below showed the reliability and separation index value for the instrument. Item Polarity according to the Element The item polarity is to test the extent to which the development of the construct achieved its goal and to see the relationship between the developed items with the respondents of the research (Nurulhuda et al., 2018). The items measure the construct that want to be measured, should be positive (+) based on PT-Measure Corr value (Bond & Fox, 2015). If the value obtained is negative (-), it means that the developed items do not measure the construct and it should be dropped because it is difficult or do not address the question (not focus). Based on analysis, there were no negative values detected in the PT-Measure Correlation. The lowest value was 0.26. This indicated the developed items able to measure anything to be measured and the items moved in parallel with other items to measure the construct.

Item Fit
Items fit is measuring the elements that can be seen through the infit and outfit Mean Square (MNSQ). Items are considered to be accepted when the outfit MNSQ value in the range of 0.5 to 1.5 (Bond & Fox, 2015;Linacre, 2005). According to Boone et al. (2014), if the outfit MNSQ value is more than 1.5, it shows that the item is confusing and difficult to be answered by respondent. They added, if the outfit MNSQ value is less than 0.5, it shows that the item is too easy to predict. Therefore, the items that do not meet the MNSQ outfit requirement were being considered to drop or modify for actual study. Based on the analysis of pilot test found that there were three items that are not in the set of range and need to be reviewed or dropped. The items were A007, A008 and I072. Table 4 below shows the item fitness. Table 4 Item   The measurement of standard residual correlation values aims to determine whether there are overlapping items. High residual correlation for two items indicates that the items are not independent, either the items have similar characteristics of each other or because the two items shared several other dimensions. According to Linacre (2005), if the correlation value of the two items exceeds 0.7, it indicates that the correlation is high and only one item is required for measurement.
Based on Table 5 below, there were six pairs of items with high correlation value. It means that these items have the same measurement meaning or combine several other dimensions together. Therefore, these items were being given attention and one item need to be dropped for each pair of items involved.

Discussion
A review was conducted according to the requirements for validity and reliability of instrument based on Rasch measurement model approach. Based on analysis of pilot study, three items need to be removed because the outfit MNSQ value were larger than 1.5. This shows that the three items were difficult and confusing for respondents to respond. Removing and refining the items was done by taking account supervisors' and experts' views and evaluation. Therefore, item A008 was modified to make the purpose of the question clearer while item A007 and I072 were removed.
Based on standardized residual correlations value, six pairs of items were removed because the correlations value was more than 0.7. It shows that respondents viewed these pairs of items were the same thing or confusing. Therefore, two available options were whether the items needed to be retained and modified so the purpose of the items were clearer or removed the items. After referring to the MNSQ outfit value that less than 1.5 in Table 4, the items that been refined were H059, H060 and H062. This was because after referring to the experts, the items were relevant under industrial attachment element. While for items A003, A006, A008 and H058 were also retained because the MNSQ outfit value close to 1.00. Therefore, 76 items were determined to be used for actual study. The summary of the items for questionnaire was shown in Table 6 below.

Conclusion
As conclusion, the analysis using Rasch measurement model was used to determine the validity and reliability of SI-PD elements for TVET teachers' in vocational colleges. The reliability coefficient values obtained from this study were high indicating that the items were stable. Separation index for the level of difficulty exceeded the value of 2.0 indicating that the items were strongly accepted. It also shown that the instrument has the right quality to measure the suitability of SI-PD elements. Although most of the items were moving in parallel direction, there were some items that not measure the desired elements.
Based on research, the validity and reliability of an instrument is a very important aspect to carry out when developing a new instrument for a study. Rasch measurement model is a powerful tool to identify item and person fitness of a research instrument. Based on findings of validity and reliability, it indicates that the instrument was suitable to be used among TVET teachers' in vocational colleges context. Thus, the elements of SI-PD which consisted of Understanding, Rewards, School Administrators' Role, Professional Development System, Attitude, Readiness, Industrial Attachment Opportunity, Autonomy and Strategy were useful to improve the implementation of SI-PD.