Building Soft-Employability Skills (SES-KIT): Reliability, Face Validity and Content Validity Testing

The aim of this paper is to design and validate a survey instrument known as Soft-Employability Skills Kit (SES-KIT) and to assess its reliability among Technical and Vocation Education Training (TVET) students. Through literature review, soft skills are found to be essential factors in employability among TVET graduates. The development of a contemporary instrument to measure students’ s oft skills is necessary to promote awareness of the latest employability skills frameworks. SES-KIT was derived based on a mapping of eight employability skills frameworks and the top ten skills in the 21st century skills (World Economic Forum, 2020). Thirty-nine respondents from a TVET institution were included in this pilot study to measure reliability value and five experts were appointed to validate face and content validity. SES-KIT obtained a high reliability score of Cronbach alpha 0.961 and a good scale content validity index (S-CVI/Ave) value of 0.91. The next step of the research is to test this instrument to a larger scale of respondents.


Introduction
The issue of unemployment is a continuous topic as it revolves around dynamic changes from time to time. Intended to reduce unemployment rate, TVET was introduced in the Malaysian Eleventh Plan. However, the TVET industry faces some challenges. Industry Revolution 4.0 and Covid-19 endemic have driven some changes in the structure of employability skills. Even for highly technical-based programs like TVET, soft skills have become essential credentials in landing jobs (Abdul Karim & Maat, 2019;Sheh et al., 2020). Students must be aware and informed of the revolution of soft-employability skills to be competitive in the job market.
The term employability skills are used differently in different countries. Some known terms are core skills, essential skills, generic skills, transferable skills, key qualifications (NCVER, 2003). Despite the variations, this paper will adapt the term employability skills as it is the preferred term by the industry (Allen Consulting Group Report, 2006) and most of the studies in Malaysian context. The researcher coined the term 'soft-employability skills' to distinctly highlight the focus of the instrument, which is soft skills.
The building of SES-KIT as an instrument is timely as it aims at providing students with the recent soft skills requirement based on relevant employability skills frameworks. According to Henry et al (2005), fast growth of economy and technology leads to dynamic and flexible career opportunities. The goal in creating SES-KIT is not only to cater TVET students for the discipline-specific industry, but to help them be aware of the skills needed should they venture into other career prospects. Hence, this paper aims to: • design and validate a survey instrument known as Soft-Employability Skills Kit (SES-KIT) and to assess its reliability among Technical and Vocation Education Training (TVET) students In addition, the instrument will provide an opportunity for students to assess and make visible their soft skills. Longley and Kensington (2019) stressed that students should be able to 'see' their soft skills. An instrument like SES-KIT will enable them to assess and articulate their employability skills. As Rust (2016) mentioned, assessing oneself is crucial to employability as it allows students to describe their strengths and weaknesses. In a recent study, Scott and Willison (2021) echoes the sentiment by reporting that graduates who are more reflective are likely to be more sought after.

Development of the Instrument
There are three stages involved in building the SES-KIT instrument. The first stage was to analyse four employability skills frameworks from four countries that were formed as their national policies. The second stage was to examine four engineering employability skills frameworks established by accreditation bodies in four countries. These eight employability skills frameworks were selected from the United States, United Kingdom, Australia and Malaysia because these countries were registered under the Sydney Accord, an international agreement body for accrediting TVET programmes. Table 1 lists the eight employability skills frameworks used in developing SES-KIT instrument.
The summary of comparison between the eight frameworks is presented in Table 2. The table reveals six common skills across all the employability frameworks in the selected countries. The most mentioned skill is 'teamwork', followed by 'communication', 'lifelong learning' and 'professional ethics' with equal frequency. Although not cited in the engineering employability frameworks, the researcher decided to include 'critical thinking and problemsolving' into the SES-KIT instrument as this skill was mentioned in all national employability skills frameworks. Since the focus of this instrument is to create awareness and thus encourage the students to reflect on their employability skills for future flexible career opportunities, 'entrepreneurship' was included in the list of soft-employability skills as it applies to Australian and Malaysian contexts. 'Leadership' on the other hand was included under the construct of 'teamwork' as the characteristics of leadership were parallel to 'teamwork'.
The third stage was to map the common employability skills frameworks found in the first two stages to 21st century skills as reported in (The Future of Jobs, World Economic Forum, 2020). Table 3 presents the mapping of the six employability skills to the top ten 21 st century skills. It is found that most of the skills listed in the World Economic Forum (2020) completed the six employability skills reviewed in Stage 1 and Stage 2.

Methodology
To design and validate SES-KIT as a survey instrument, this research measured its reliability and validity. After extensive, systematic literature review, 88 items were generated into a questionnaire to represent six employability skills. Initially composed in English language, these items were then translated into Malay language to ensure absolute understanding among the respondents. Harkness (2006) stressed the importance of translation to accommodate a population that practice multiple languages.

Face Validity and Content Validity
Validation of survey instrument is a crucial step in research (Elangovan & Sundaravel, 2021). This paper adapted a systematic approach to measure content validity as introduced by Yusoff (2019). The approach includes 6 steps: a) preparing content validation form, b) selecting a review of panel of experts, c) conducting content validation, d) reviewing domain and items, e) providing score on each item and f) calculating Content Validity Index (CVI). The researcher prepared the content validation form by explaining in detail what is expected from the experts. Figure 1 shows example of instructions and the rating scale used to allow experts to judge the relevance of the items while Figure 2 shows an example of layout for content validation form consisting of definition of domain and items that represent the domain. A special column on the far right was created to enable the experts to give their personal feedback for face validity. For content validation, Yusoff (2019) reviewed that the minimum acceptable expert number is two and the maximum is ten. To validate the face validity and content validity of this survey instrument, five experts were selected by the researcher based on their experience. Three experts consist of former Deans and Deputy Dean of Student Development and Campus Lifestyle from two TVET campuses were appointed to assess the content validity of the instrument. Their expertise in the TVET industry and students' development programmes are essential in validating the items in the instrument. Another two experts (senior lecturers) were selected from the Faculty of Education, UiTM Puncak Alam for their in-depth knowledge in educational management and human development. Their decades of experience and knowledge teaching and developing modules for human development and classroom management will be beneficial to validate both face validity and content validity of the instrument.
Martinez (2017) defined face validity as a measure of items 'on its face', whether they are visibly relevant to the concept involved. Face validity requires experts to provide personal feedback of the presentation, relevance and clarity of the intended instrument. Hence, the researcher had requested the experts to validate the face validity of SES-KIT by providing subjective assessments towards criteria as suggested by (Oluwatayo, 2012) • the clarity and unambiguity of items • appropriateness of difficulty level for the respondents • correct spelling of difficult words • adequacy of instructions in the instrument • the structure of the instrument in terms of construction and well-thought-out format Oluwatayo (2012) defined content validity as a concept that stresses on the extent to which the instrument of measurement shows evidence of fairly and comprehensive coverage of the domain of items that it intends to cover. The content validity is represented by the Content Validity Index (CVI). CVI can be calculated using the Item-level content validity index (I-CVI) and Scale-level content validity index based on the average (S-CVI/Ave) (Yusuff, 2019; Rodrigues et al., 2016). The acceptable value for I-CVI is >0.79 while S-CVI/Ave value is ≥ 0.9 (Zamanzadeh et al., 2015).

Reliability
For reliability, this paper conducted a pilot study towards 39 respondents consisting of final year students in a local TVET university. The researcher conducted the pilot study through several GoogleMeet sessions since the nation was in lockdown mode due to Covid-19. Respondents were guided throughout the session. Bowling (2009) suggested that the testing of reliability through internal consistency measure to which extent do the items relate to a specific dimension. The pilot study attempts to measure the Cronbach Alpha value to determine the internal consistency. As reported by Oluwatayo (2012), Cronbach Alpha is one of the most widely used statistical tools to measure reliability in educational research. The acceptable Cronbach Alpha value to reflect good reliability should be ≥ 0.70 (Hair et al., 2010)

Results
The summary of face validity is shown in Table 4. It is vital for the researcher to investigate the following comments made by experts by comparing the result to content validity values before finalising which items will be rephrased, rearranged, or removed. Hence, Table 5 shows the I-CVI and S-CVI value for content validity of the instrument. Since there were 88 items, Table 5 only presents items with I-CVI value below 1. There were 10 items with I-CVI value below >0.79 and 14 items with I-CVI value within the range of 1>x >0.79. The S-CVI value of the instrument is 0.92. The structure of the instrument in terms of construction and wellthought-out format Change scale format from Strongly agree-Strongly disagree to Very true of me-Not true of me at all Items will be modified after comparing thew results of face validity and content validity to reliability. As seen in Figure 3, the Cronbach Alpha value for all 88 items is 0.972. In the discussion section, the researcher will compare this Cronbach Alpha value to a new Cronbach Alpha value after removing certain items.

Discussion
After analysing the I-CVI value and comments given by experts, 20 items were removed from the initial instrument. Since the acceptable value of I-CVI is >0.79 (Zamanzadeh et al., 2015), ten items with value lower than that were removed (B18, B32, B35, B58, B60, B61, B64, B65, B79, B82). Items with score between the range of 1>x>0.79 were reviewed. As a result, eight more items with I-CVI value 0.8 were removed (B10, B14, B23, B50, B53, B62, B63, B67). The research considered the comments by the experts and decided to remove Q8 and Q38 although it has a score of 1 because two of the experts criticized that there was a redundancy for its face validity. However, no changes were made to B1 and B3 although one expert suggested for the item to be split into two items. The first reason is because 4 other experts did not see the need to do so, and second, the items consist of adjoining verbs like 'listen and understand' and 'read and interpret'. Table 6 provides the example of B1 and B3. A total of 20 items were removed from the initial items generated for the SES-KIT instrument. A second reliability test was conducted to measure the Cronbach Alpha after the deletion to ensure that its internal consistency score is still good, if not acceptable. Figure 4 shows that the reliability value is still high, with a score of 0.961.

Cronbach's Alpha
Cronbach's Alpha Based on Standardized Items N of Items .961 .963 68 Fig. 4 Cronbach Alpha value with 68 items Table 7 describes the comparison of Cronbach Alpha values for six types of employability skills, before and after items deletion. As presented, the values of each construct are still good even after the deletion of 20 items. Hence, the researcher decided to retain the items from 88 to 68 items.

Conclusion
This aim of this paper is to design and validate a survey instrument known as Soft-Employability Skills Kit (SES-KIT). Six types of employability skills identified including communication skills, teamwork, critical thinking and problem-solving, lifelong learning, professional ethics, and entrepreneurship. Initially, 88 items were created as a result from extensive literature review on multiple employability frameworks. The items were then included into a content validation form before being distributed to five experts for face and content validity. Calculation of CVI and S-CVI/Ave were computed after recording the I-CVI scores. Once the items were deliberated based on its face validity and content validity, 20 items were removed. It can be summarised that the values of I-CVI and S-CVI/Ave were strong. Concurrently, the instrument was given to 39 students for a pilot study. The reliabity test was conducted by evaluating the Cronbach Alpha value. A highly reliable value of 0.961 was recorded for the internal consistency. This proved that SES-KIT was able to measure the employability skills of students. The researcher intends to examine the instrument to a larger sample to test its exploratory factor analysis.