Instrument Validity and Reliability UTAU-ZOOM2 Version

Zoom has emerged as an alternative application used by most communities in the world of education. The purpose of this study was to produce empirical evidence on the validity and reliability of the UTAU-ZOOM2 instrument for students who had attended Risale-i Nur learning sessions through ZOOM. This Risale-i Nur learning session is a non-formal education referring to a planned and structured personal and social education program and process designed to improve various skills and competencies outside the formal education curriculum. The objective of this study was to measure the instrument using item polarity test, standardized variance test, study instrument reliability test and individual-item distribution map in Rasch analysis. The findings of the study showed that only 16 items were received and met the requirements of the Rasch Model out of the total 24 items. However, all items can be improved in terms of sentence structure and language with content and language experts consultation to further improve the validity and reliability of the UTAU-ZOOM2 questionnaire. This study shows that Rasch analysis helps novice researchers improve systematically in term of constructing questionnaires to produce quality research data.


Introduction
The technology acceptance model for understanding academic intentions, and behaviours, suggests that experience using technology serves as an important factor in determining elearning policy (Alharbi & Drew, 2014). The use of educational technology indeed requires information on how consumers react to technology. In general, education is divided into three categories (i) Formal Education, (ii) Informal Education and (iii) Non-Formal Education (Grajcevci & Shala, 2016). However, some researchers classify into two forms, namely (i) Formal Education and (ii) Non -Formal Education (Sarju, Hamzah And Udin, 2010). Formal Education refers to the education system in schools starting from kindergarten, primary school, secondary school, vocational school, matriculation and institutions of higher learning. This formal education usually has recognition and certification. Non-formal education is the opposite, where non-formal education refers to planned and structured personal and social education programs and processes for young people designed to enhance various skills and competencies, outside the formal education curriculum (Othman & Din, 2021). Informal education is a lifelong learning process in which each individual acquires attitudes, values, skills and knowledge from the influence and resources of education in his environment and daily experience (Othman & Din, 2021). Examples are learning from family and neighbours, at the market, library, art exhibitions, work and through play, reading and sports activities. Learning in this way is often unplanned and unstructured. Formal, non-formal and informal education are complementary elements and mutually reinforce the lifelong learning process.
Various educational technologies in Malaysia require a complex and challenging application. The Zoom application is the most popular online meeting application globally and has the number one place in the hearts of users around the world, especially during the COVID19 pandemic. Zoom was developed in 2011 by Eric Yuan. Prior to Zoom, Eric worked at Webex. Of late, the Zoom app has emerged as an alternative app used by most communities in education. The official website for the Zoom app is https://zoom.us/about.
Zoom is a frontrunner in modern enterprise video communications, with a simple and reliable cloud platform for video and audio conferencing, collaboration, chat and webinars across mobile devices, desktops, phones and room systems (Zulherman et al., 2021). Zoom Room is an original software-based conference room solution used worldwide on ships, conferences, huddles and training rooms, as well as executive offices and classrooms. Zoom helps businesses and organizations unite their teams in friction-free environment to do more. During the COVID-19 outbreak, the Malaysian government issued an online learning policy in schools and universities. Which problem would occur if the management system supported the implementation of online learning? We can check from the aspect of individual ability. The aspects of internet networks and application devices are used to be considered so that online learning is carried out well (Ergun & Kiyici, 2019). Thus, what is the online learning solution for this work from home program?
The originality of this study is the intention to use Zoom during the COVID-19 outbreak in the context of non-formal education has never been investigated. Previous studies show that UTAUT2 instruments among ZOOM users developed in a local context are very limited. The existing UTAUT2 instrument only focuses on specific contexts and countries. This article differs from previous studies in the literature due to the effort to produce the UTAUT2 instrument among ZOOM users named Instrument UTAU-ZOOM2. Given that the UTAUT2 instrument is used in Malaysia, the issue of translation adaptation from other national instruments makes the application limited. Past researchers have built the UTAUT2 instrument; however, it is limited to the challenge in its context only (Zulherman, 2021). Therefore, testing a new UTAUT2 item in the context of Zoom users is a necessity to ensure that the item has good validity and reliability. According to Adnan and Mohd Matore (2019), the validity and reliability of using Rasch measurement analysis help in producing a good and high-quality instrument. The next section of this article will provide strong empirical evidence on UTAU-ZOOM2, especially from the perspective of the Rasch model.

Purpose
The purpose of this study was to produce empirical evidence on the validity and reliability of the UTAU-ZOOM2 instrument for students who had attended the Risale-i Nur study session through ZOOM.

Methodology
This study is a survey study. According to Khalid (2003), survey research aims to predict an impending phenomenon. In addition, survey research is also used to examine the validity and reliability of research instruments based on the data items obtained and ensure the function of the items is accurate and does not lead to giving multiple answers that are difficult to analyze (Abdullah & Wei 2017). The sample of this study consisted of 31 respondents. The selection of the study sample is purposive sampling. The respondents consisted of female students who had attended the Risale-i Nur learning session through ZOOM. This study uses a questionnaire as a research instrument. This questionnaire uses a four-point Likert scale with 1 representing 'strongly disagree', 2 = 'disagree', 3 = 'agree', 4 = 'strongly agree'. The use of midpoints is not involved in this instrument because it considers a biased response (Tsang 2012). The instrument of this study is divided into two parts. Section A contains demographic items, while Section B measures the ZOOM acceptance construct and factors that measure the constructs, as in table 1. The number of items shown in Table 1 below is 24 original items before the study was conducted.

Result and Finding
This study will focus on the main findings or outputs only such as (i) Item Fit Statistics, (ii) Item Polarity (iii) Unidimensionality, (iv) Local Independence, (v) Reability and Separation Index, and (vi) Item-Person Map (Wright Map). These six items are the main basis for determining the quality of items for measuring an instrument with acceptable validity and reliability of the UTAU-ZOOM2 .

Item Fit
The conditions for the model that is said to be suitable (fit) can also be deduced from Table 2 below according to the type of data used in the study instrument. According to Linacre (2005), if items fall below that range, they need to be segregated for modification or improved before being discarded. The suitability of this item is very important as it will affect the value of the reliability and validity of an individual instrument. As explained in Item fit, the cut-off point for item fit is between 0.6-1.4 (Bond & Fox 2007). If there is an item with a value <0.6, this means that construct overlap occurs and if the item has a value> 1.4, there are unexpected variables in the item and the existence of sub-groups of individuals who give biased and similar responses. Table 3 shows the fit items. The findings show only 16 items remain out of the total of 24 items regardless of the ZSTD value. Therefore, the 8 items that were dropped were items FC2, AC1,PE3, EE3, HM1, EE4, PE2 and SS2. The items were dropped because they did not meet the standards and the item suitability range was between 0.6 to 1.4 (Bond & Fox 2007). These items were also dropped to improve the quality of the instrument. Such items can also be refined by looking at research needs and expert views.

Item Polarity
A positive PTMEA value indicates the item measuring the construct (Bond & Fox 2012). This shows the items measuring the domains you want to measure in UTAU-ZOOM2. It also shows the extent in which the construction of this domain achieves its goals, and how well the relationship is between the item and the respondent. This analysis is a fundamental step for measuring domain validity. Based on Table 3, all PTMEA values are more than 0.30, i.e. from 0.64 to 0.84. Thus, it can be concluded that the items can contribute to the measurement in UTAU-ZOOM2.

Unidimensionality
Siti Rahayah (2008) states that one of the guidelines in determining quality items is to ensure that the items together measure the construct to be measured (unidimensional). In Table 4 below, it can be seen that the Raw Variance explained by measures is as much as 60.6 % while, the Raw Unexplained variance-total is as much as 39.4 %. This means that there is no presence of other dominant factors influencing the test. According to Linacre (2005), the value of variance explained should preferably exceed 60%, to allow us to safely say that there will be no other factors in the test conducted. The value of 60.6% shows that the item is reflected in the item that wants to be measured. Theoretically, the UTAUT model is established and maybe something wrong that sometimes the instrument is not good, but the fact that the content is there so that the value is 60.6 which is good. The Eigen value is 3.3 which is good that is less than 5. According to Zainal and Mohd Matore (2021), if the Eigen value is less than 5 and the noise of the item less than 15%, it shows that the unidimensionality of the items exist. The noise value shown in Table 4 is 7.5% which shows the unidimensionality of the items.  Table 5 shows ten-item matching constructs with residual correlation standard values between 0.54 and 0.41. To identify this, residual correlation values were examined, and pairs of items with a correlation greater than 0.3 were taken to indicate dependencies.

Reability and Separation Indexs
The individual separation index estimates the isolation or differences of groups of individuals that exist in a study according to the level of ability in the variables measured (Wreight & Master 1982). Table 5 below shows Cronbach's alpha values and item reliability. The individual reliability index is 0.89 and it is considered good and adequate in the range of 0.81 to 0.90, as suggested by Fisher (2007). The item reliability index is 0.58 and it is considered weak although may be acceptable because it is only a little above the range greater than or equal to 0.5, as suggested by George & Mallery (2003). According to George & Mallery (2003), rules of thumb for Alpha values less than or equal to 0.50 are unacceptable.
According to Linacre (2005) the individual separation index value of more than 2.00 is good. As for the individual separation index, the value is 2.81. Values above 2.0 indicate a good and acceptable index (Bond & Fox 2015). However, the value of the item separation index is 1.17, which is less than 2.00 as suggested by Bond & Fox (2015). Thus, the value of the item separation index is poor because the value is 1.17; less than 2.00. According to Green (2002), the value of the separation index must exceed the value of 1 so that the instrument can still be usable. For this study, it is suggested that the number of respondents be increased and the data be further analysed to obtain values for better reliability and separation index for the instrument items.   The difficulty of the item and the respondent's ability can be seen from Figure 1 above, where the item or respondent has a logit score that is farther from 0 and a positive value, then the item has a high level of difficulty. Meanwhile, the respondent is said to have a high level of ability in answering the items for the respondent's ability. The findings show that item EE1 is the easiest item and item AC2 is the most difficult item. In conclusion, the wright map above indicates that the whole item is simple while most respondents have a high ability level. Therefore, it is suggested that for further study, more difficult items are developed to meet the ability level of the students.

Disscussion
In this study, by using the Rasch Measurement Model as well, the researcher has obtained a high reliability value for a reliability test. The item and respondent reliability tests also indicated the questionnaires were tested and reliable. One of the advantages of modern psychometric methods including the Rasch modelling method is the ability of its formulas to identify misfit items and respondents (Din et al. 2009). For example, a very smart student should be able to answer a very easy question. This method can identify the level of difficulty of the item and the ability of the respondent (Bond & Fox, 2007). Thus the findings obtained related to the reliability and validity of the constructs for this research questionnaire can be accepted to answer both research questions.
A total of 8 items to note in this instrument. This does not mean that all of these items should be dropped. Improvements should be made to the sentence structure so as not to confuse the respondents. Items that are on the same difficulty level and have the same goal can be combined or dropped any less suitable items. Finally, this UTAU-ZOOM2 instrument can be used to achieve the objective of identifying the acceptance and use of ZOOM for students Risale-i Nur.
To obtain more accurate and consistent findings, future studies are proposed to use the same questionnaire and data to test the validity of constructs using structural equation modelling method or better known as the SEM method, the abbreviation of it. It is therefore suggested that future studies use the same questionnaire but collect data on a larger quantity of respondents.

Conclusion
Researchers, educators and item developers need to be concerned about the reliability and construct validity of an item so that the instrument can be measured fairly and have different difficulty levels. The reliability of items and individuals should be taken seriously to ensure that the instruments constructed and the samples have a high level of consistency. The validity and reliability of using Rasch measurement analysis help in producing a good and highquality instrument. To obtain consistent study findings, further studies are recommended using Structural Equation Modelling (SEM) techniques to test the validity of the constructs. Further research is also recommended by using more respondents and refining items to more complex questions to be in line with the level of ability of high ability respondents. In addition, further studies are also recommended by adding new constructs or incorporating existing constructs that other researchers have developed.