Testing the Validity and Reliability of a Writing Skill Assessment

Lower primary students in an international school have always been associated with writing difficulties. This pilot test is intended to assist in following the thread and specifically to assess the effectiveness of the instruments chosen and designed for the actual research, "Writing Skill Challenges Faced by Students in a Lower Primary, International School." Its aim is to assess the instrument's validity and reliability. Through the process of data analysis, two data collection tools, a questionnaire and face-to-face interview questions were statistically and thematically analysed. This pilot study included 15 lower primary students with similar characteristics to the actual samples. The data was then collected and analysed by using the Statistical Package Social Science (SPSS). After the data from the pilot study was entered into SPSS, the data's reliability was calculated and recorded using Cronbach's alpha formula. The findings revealed that all the items were positive, and the content was validated. It showed a very high degree of internal consistency between the given items based on reliability. As a result, no further changes were needed, and these instruments can be used in the actual study without hesitation.


Introduction
In preparation for a larger study, a pilot study can be described as a small study to test research procedures, data collection tools, sample recruitment strategies, and other research techniques (Junyong, 2017). The main aim of a pilot study is not to address specific research questions, but to discourage researchers from launching a large-scale study without sufficient knowledge of the methods proposed; in essence, a pilot study is carried out in a study that is expensive in time and resources to prevent the occurrence of a fatal flaw (Polit & Beck, 2017). A well-planned and conducted pilot study may also help researchers identify possible confounding variables not previously identified and determine the intensity of relationships between key variables to assist in sample size estimation (Polit & Beck, 2017). In short, researchers typically use pilot tests to test the adequacy of their proposed methods and procedures. The present research, which is also a pilot study, serves as a help following the thread and explicitly aims to evaluate the efficacy of the instrument selected as well as planned for the actual study entitled "Writing Skill Challenges Faced by Students in a Lower Primary, International School".

Background of the Pilot Study
In performing the pilot analysis, a smaller group of samples similar characteristics to the actual study was selected. The instrument that was chosen for this study was adapted from Shahrohan (2015) research about Factors Affecting the Writing Skills of Primary Schools. In line with that, to supplement the main research, an international school was selected in which to conduct the pilot study. In compliance with this, the pilot study was initiated at an international school in Selangor, Malaysia. It is not the site of the actual study, but it has a similar context to the actual study school. Just as the actual study's school, this school subscribes to the Cambridge curriculum from Year 1 up to A-Levels when it comes to curriculum. In communicating and conducting lessons, English is used as the primary language. Most of them are from China, Nigeria, Yemen, Libya, Japan, Korea and the Middle East countries, etc. The selected 15 lower primary students for the pilot studies, who are from 5 to 9 years old, have poor English language proficiency. The students migrated to Malaysia with either both parents or single parents, because of immigration, work prospects or finding refugee status. The students are mainly here to learn the English language. Some of them had no prior knowledge of English at school, where they could not speak or converse a word in English. This resulted in the students being unable to write in English and discouraged the students from gaining new knowledge of writing skills. The students did not use English previously when looking back at their use of the English language. Achieving competent levels of education is the globalization aim for all the elementary school students.

Purpose of the Pilot Study
The purpose to undergo this study is to test the content validity, face validity and internal reliability of the instrument to be used in the actual study after changes has been done. In order to ensure the validity and reliability of the instrument in calculating the expected outcome of a study, checking the validity and reliability of a data collection instrument is fundamental (Hazzi & Maldaon, 2015;Kinchin, Ismail, & Edwards, 2018). The instrument from Shahrohan (2015) research about Factors Affecting the Writing Skills of Primary Schools was adapted for this study. In order to demonstrate the efficacy of the instruments for the actual study, this modification of the instrument needs to be piloted. In line with that, the validity and reliability of an instrument was used to verify its effectiveness. Validity explains how well the information gathered covers the real field of inquiry (Ghauri and Gronhaug, 2005). Basically, validity means measuring what needs to be measured (Field, 2005). On the other side, reliability is defined as the degree to which test scores are free of error in measurement (Muijs, 2011). It is an indicator of an instrument's stability or internal accuracy in testing those concepts (Jackson, 2003). Hence, pilot study tested the content validity, face validity, triangulation and internal reliability of the instrument to be used in the actual study.

Pilot Study Objectives
This study aims to achieve the following goals: • To test the content and face validity of the instrument • To measure the reliability of the instrument

Pilot Study Questions
• What is the content and face validity of the instrument?
• What is the reliability of the instrument?

Development of the Instrument
The instruments used for the major was adapted from Shahrohan (2015) research on Factors Influencing the Writing Skills of Students in Primary Schools. Shahrohan has prepared and used his instrument for his research. The instrument was consisting of 9 items. The instrument has 9 "circle the correct answer" questions. The questions were about writing ability in the teaching and learning process of Year 4 students. The instrument was prepared in two pages.

Development of Adapted Instruments
As stated earlier, however, the instrument needs to undergo certain modifications to suit the objective and goals of the actual study. Hence, the questionnaire has been updated to Likert type. There are 19 items in the questionnaire. The instrument consists of a 3-points Likert type scale responses which are agree (3), neutral (2) and disagree (1). This practice based for individual item analysis to determine the collective degree of agreement of the participants on that topic mentioned by Joshi et al, (2020). As one of the tools, the purpose of the questionnaire is to identify the causes of the problems of the lower primary school students in their writing skill which is the first objective of the actual study. Next, title was given for the questionnaire which is 'Causes of Facing Problem in Writing Skill'.
Lastly, face-to-face interview questions will be the second part. This part will answer the second objective of the actual study. The questionnaire will be carried through face-to-face sessions in actual study by considering the lower level of primary students. For this face-toface interview, there are 6 items. 5 lower primary students were interviewed. Title was given for the questionnaire which is 'To explore the challenges faced by lower primary students in their writing skill'.

Testing Validity of the Adapted Instrument
In 2019, a pilot study was conducted over several months. In order to discover the validity of the instruments used, the current pilot study was carried out. The exact procedure of the major study was therefore followed by the pilot study process. The validation method was performed first. Validity represents the extent to which a calculation tests what it claims to measure said Bolarinwa (2015). There are several types, including face validity, construct validity, content validity, and criterion validity, have been defined (which could be concurrent and predictive validity). According to Wong et al, (2012) the validity tests are divided into two large components: internal and external validity. In this pilot study, content validation and face validity were used to validate the instruments. This will be followed by triangulation in conducting the pilot study.

Content Validity of the Adapted Instrument
The validity of the content refers to the degree to which the instrument completely tests the construct of interest or calculates it mentioned Bolarinwa (2015). The application of content validity is strongly recommended while the new instrument is being developed. Content validity usually involves reviewing a new survey instrument to ensure that it contains all the necessary items and excludes undesirable items from a specific construction domain (Lewis et al., 1995, Boudreau et al., 2001. The researchers need to submit questionnaires on content validity to experts working in different locations, where distance is not a constraint. For this pilot study, three expert panels from a private institute were selected to perform the validation process. The experts were Head of Key Stage 1, Head of English Department and senior English teacher. All the three experts are from the same field. The Lawshe (1975) model was used as a guide to the completion of the method by which the Content Validity Ratio (CVR) is determined according to the items provided in the validity determination instrument.

Face Validity of the Adopted Instrument
Face validity is developed when the questionnaire (instrument) is checked by a person (and or researcher) who is an expertise on the research topic who believes that it measures the feature or trait of interest said Bolarinwa (2015). If its content simply looks important to the individual taking the test, a test has facial validity. It tests the questionnaire's appearance in terms of viability, readability, style and formatting accuracy, and the clarity of the language used (Taherdoost, 2016). In general face validity refers to the subjective evaluation of the appearance and significance of the measurement instrument by researchers as to whether the items appear to be appropriate, fair, unambiguous and transparent in the instrument (Oluwatayo, 2012). For this study, face validity was done by referring to three experts. The three experts have gone through the items one by one.

Triangulations
Triangulation is the method of collecting data from various perspectives consisting of different sources of the issue being investigated in order to gain full understanding of the problem mentioned Marwa et al., (2019). This will in turn contribute towards a high degree of detailed comprehension of the problem. During this phase of the pilot testing, the researcher conducted a thorough interview of the expert teachers in order to gage their trustworthiness. The three expert teachers were very helpful.

Testing the Reliability of The Adapted Instrument
The pilot study also was done to find the reliability of the instruments used. Reliability is identified as the accuracy and consistency of an instrument's results (Braun, Clarke, Hayfield, & Terry, 2019;Sharifah, Jamal, & Hamidah, 2017). This implies that accurate instruments can be used many times in multiple timelines and show explicit and consistent results. There are some reliability testing measures which are, test-retest, equivalent form, internal consistency and reliability statistics mentioned by Wong & Yamat (2020). In this pilot study reliability statistics was utilized. Cronbach's Alpha was used to calculate the consistency of the instrument. Bonett and Wright (2014) said that Cronbach's alpha formula is one of the most frequently used reliability tests in social and organizational sciences. The instrument was then distributed and completed by different groups of Year 1 and Year 2 students as mentioned earlier. To avoid the students facing any issues with answering the questionnaire, the students were assisted during the questionnaire. The explanation why the students were assisted is that they are too young to respond to the questionnaire independently. 5 lower primary students were interviewed face to face after they finished with the questionnaire. One by one, the questions were posed. When the students were struggling to answer the questions, explanation and hints were provided. For each student, the face-to-face interview took place for approximately 20 minutes. During the interview session, the lower primary learners were encouraged to express their thoughts and ideas freely in a relaxed manner. Finally, the data was then gathered and analyzed using the Statistical Package Social Science (SPSS). The reliability of the data was measured and reported using Cronbach's alpha formula after the data gathered from the pilot study was keyed in the SPSS.

Findings of Validity Test
Validity is a benchmark for which methods are used to acquire data sources through checking or measuring. To validate the instrument for this analysis, content validity, face validity and triangulation were used. This instrument has been validated by three experts from a private institute. They concluded that this instrument can be used for the actual study. For the content validity, the result is marked as essential. Based on The Lawshe (1975) model, Content Validity Ratio (CVR) was used, and the result was 1.0. This is very clear that all the items are needed. For the face validity, the three experts said that the items were very clear and straight forward. Finally, the results of the triangulation verified the experts' trustworthiness levels which in turn validated the findings. All the items were positive and the content are validated. Hence, there is no changes need and this instrument can be used for the actual study. Cronbach's alpha formula was used to validate the reliability of this study. Based on Table 1, for all the 19 items, the cumulative alpha is 0.878, which indicates a very high degree of internal consistency between the items given. This indicates that the instrument is extremely accurate.

Conclusion
The results of a brief, well planned pilot study is published in this paper. Pilot studies often disseminate limited or statistically insignificant findings without adding comparative research benefits to the body's information. There were no any difficulty found throughout the pilot study. All the items were positive and the content are validated. Based on the reliability, it showed a very high degree of internal consistency between the provided items. No further changes were therefore, required. Hence, this instrument can be used in actual study without any doubt.