Validity and Reliability of Mobile Game Assessment Questionnaire for Mobile Gamification in Economics Subject

Mobile gaming is one of the current trends around the world. Therefore, a mobile game has been developed for educational purposes in the subject of economics for secondary schools. Before being implemented in the actual field, a pilot study was conducted to determine the validity and reliability of the questionnaire developed to assess the effectiveness of this game in the learning process. The game is specially designed for economics education amidst advancements in mobile gamification. Addressing the primary challenges in economics education, where students often experience boredom and perceive the subject as complex, exacerbated by the disconnect with traditional teaching methods in today's technologically adept generation, this study aims to evaluate the face and content validity, along with the reliability of the questionnaire. The findings of the study reveal exceptionally high reliability for both face and content validity, with all Alpha-Cronbach values exceeding the acceptable threshold of 0.71. Therefore, the questionnaire demonstrates promising efficacy in evaluating economics knowledge in a gamified mobile learning environment. Consequently, this study advocates for further real-world field research to enhance student engagement and interest in economics education. Corrections to this questionnaire were made after several expert reviews were conducted.


Introduction
Currently, the modern educational environment in 21 st century, mobile phone use has grown rapidly and trending among students all over the world (Hranchak et al., 2022).Once thought of primarily as communication tools, now mobile phones have developed into multipurpose gadgets that are an integral part of students everyday lives.Because of the increasing use of mobile technology, mobile learning, or M-learning, has become popular among student with a high percentage using their phones for studying.(Nhu, 2022).It transforms the educational process by using mobile phones capabilities for learning.The new perspective of telecommunication in education, M-learning it is introducing and a makes use of mobile phones accessibility and convenience to enable learning at any time and from any location (Naveed et al., 2023).Students can participate in educational activities outside of the traditional classroom setting due to the availability of educational apps, online resources, and interactive content.This mobile-first approach to education increases flexibility and autonomy while accommodating a wide range of learning preferences and styles, making instruction more engaging and personalized.
In the realm of education, gamification has emerged as an intriguing strategy to enhance student engagement and motivation (Bjørner et al., 2022).Game-based learning capitalizes on the inherent allure of games to immerse students in educational content while cultivating critical thinking, problem-solving, and collaborative skills.By integrating gaming elements such as challenges, rewards, and competition, educators can craft dynamic and interactive learning experiences that capture students interest.In alignment with this, gamification employing M-Learning concepts represents a novel attraction in the learning process.Additionally, the engagement framework effectively guides developers in adding game elements to e-learning systems, thereby improving student engagement and performance (Alsubhi et al., 2021).Although the development of gamification in education is increasingly positive, the subject of economics lacks mobile game-based learning materials (Mahathir, 2022).According to Roziah et al. (2021), students frequently struggle with boredom, perceive the subject matter as complex, and have less hope for the teacher in the classroom (Mazdalifa, 2019).Furthermore, the technologically savvy generation (Wan Muhammad Iskandar Firdaus et al., 2022) of today may not find traditional teaching methods appealing, which could cause disengagement and a lack of interest in economics education.
Therefore, considering this importance, a mobile game for the subject of Economics has been developed.Mobile games in economics instruction offer a potential way to overcome these obstacles.Engaging and relevant economic concepts can be explored through games in a dynamic and interactive manner.Games can assist students in better understanding, remembering, and applying economic concepts by immersing them in simulated economic scenarios and decision-making processes.(Oktariani et al., 2023: Aidie Ikhwan & Raja Lailatul Zuraida, 2022).Thus, to ensure the effectiveness of this gaming method in the learning process of Economics, a questionnaire has been developed.This questionnaire needs to be tested in a pilot study first to determine its validity and usability.To ascertain the effectiveness of mobile game-based learning in economics education, an assessment instrument is developed.This instrument comprises five key aspects: Multimedia Design, Visual Design, Learning Support, Usability, Motivation, and Self-learning.The objective of this study is: 1.To Identify the Face and Content Validity of Mobile Game Evaluation Instrument in Economic Subjects.2. To Identify The Reliability of Mobile Game Evaluation Instrument in Economic Subjects.

Literature Review Economic Subject for Secondary School
Economics, as a pivotal social science discipline, delves into the intricate realm of human needs and desires, which are seemingly limitless.From secondary school to higher education levels, the study of economics is fundamental, offering insights into the mechanisms underlying economic activities at various societal scales.The curriculum for economics education is meticulously crafted to encompass a broad spectrum of topics, ranging from microeconomic principles governing individual consumer behavior and firm decisions to macroeconomic theories elucidating aggregate economic phenomena like inflation, unemployment, and economic growth.Furthermore, the dynamic nature of economics necessitates constant updates to the curriculum to ensure relevance and applicability to contemporary economic issues.Economic education aims not only to impart theoretical knowledge but also to cultivate critical thinking skills and analytical reasoning abilities essential for navigating the complexities of economic decision-making.Through the exploration of diverse economic models and theories, students are equipped with the tools to comprehend and evaluate the economic implications of various policy interventions and market dynamics.Ultimately, the overarching goal of economics education is to empower students with a comprehensive understanding of economic principles, enabling them to analyze real-world economic phenomena, formulate informed judgments, and contribute meaningfully to economic discourse and policymaking processes.

Gamification in Education
The integration of mobile gamification and M-Learning has garnered significant interest in the field of educational research.Scholars are examining the potential for these approaches to enhance learning outcomes in a range of disciplines, including economics.The effects of mobile gamification on student motivation, engagement, and learning have been the subject of numerous research studies.For instance, Seow & Wong, (2016) discovered that gamified mobile applications successfully promoted active learning and increased students motivation to learn economics.Using gamified simulations and interactive scenarios, students could test economic factors and observe the outcomes of their choices, which had a positive impact on them (Fathi & Khadijah, 2021).Similarly, research on the effectiveness of M-Learning platforms in the teaching of economics has demonstrated how revolutionary these tools can be in terms of transforming traditional teaching approaches.M-learning methodology, according to Ramos et al. (2022), is seen as a motivating tool that improves student satisfaction, creates a more enjoyable classroom environment, and aids in the better acquisition of learning content.Through the use of interactive modules, simulations, and real-time data, m-learning systems enable students to study economic theories in a context and increase their understanding and application of theoretical concepts.Furthermore demonstrating that M-learning can raise student achievement by 41.9% is this (Nabella & Dwiningsih, 2022).In addition, in a 1:1 online learning environment, M-learning can increase mentors uptake of student contributions by 10% and decrease talk time by 5% (Demszky & Liu, 2023).In addition, researchers have carefully examined the psychometric characteristics of mobile game assessment forms designed for economics instruction, emphasizing the reliability and validity of these forms.For instance, looked into the internal consistency and construct validity of a mobile gaming assessment tool (Nowinski et al., 2023) intended to gauge students grasp of macroeconomic concepts.Their results confirmed the validity of the questionnaire as an assessment instrument in economics education by highlighting its strong suitability in precisely gauging students' knowledge and understanding levels.Overall, the literature underscores the transformative potential of mobile gamification and M-Learning in economics education, emphasizing their role in fostering active engagement, deepening conceptual understanding, and preparing students for real-world applications in economic decision-making.As educators and researchers continue to explore innovative pedagogical approaches, the integration of mobile technologies holds promise for creating dynamic and personalized learning experiences that empower students to succeed in an ever-evolving global economy.

Developing the Instrument
To guarantee the validity and reliability of the mobile game assessment questionnaire for economics education, a number of important factors must be carefully taken into account.These components include aspects of motivation, self-directed learning, learning assistance, multimedia design, graphic design, and usability.Every component is vital to the overall efficacy of the questionnaire as a means of assessing students' understanding and ability to apply economics ideas in a gamified mobile learning environment.Motivation is a key factor that propels learning persistence and engagement.In order to promote persistent interest in learning economics and active involvement, measures for improving motivation were incorporated into the questionnaire's creation.Point systems, badges, and leaderboards are examples of gamification mechanisms that are used to encourage students to participate in the assessment activities (Ahmad & Iksan, 2021).Furthermore, student's intrinsic motivation is fostered by emphasizing the relevance and applicability of economic concepts to real-world circumstances, which improves their entire learning experience.Encourage students to take charge of their education if they are to become self-reliant and acquire metacognitive abilities.By letting students set objectives, track their progress, and evaluate their grasp of economic concepts while playing mobile games, the questionnaire seeks to understand how students learn independently.Exams, case studies, and problemsolving exercises are examples of interactive components that motivate students to investigate and apply economic theories in real-world contexts.This promotes deeper understanding and the development of critical thinking abilities.A framework for selfdirected learning in management and operations research enhances students capacity for independent learning and fosters in-depth, lifelong learning (Wang, 2022) Research on the Design of Teaching Environment Based on Self-directed Learning.
A key factor in improving students understanding and memory of economics concepts is effective multimedia design (Saini & Baba, 2023).In order to convey complicated economic ideas and concepts in a visually appealing and understandable way, the questionnaire makes use of multimedia components including movies, animations, and interactive simulations.Multimedia-rich information supports better understanding and knowledge retention by enhancing students grasp of abstract economic concepts and their practical applications while accommodating a variety of learning styles and preferences.Additionally, to maximize the questionnaire interface's visual attractiveness and usefulness, visual design considerations are essential.To improve the overall user experience, clear and simple layout designs, unified visual styles, and easily navigable navigation menus are included (Dekate, 2023).In order to help students through the assessment activities and enable smooth engagement with the questionnaire, visual cues and icons are carefully employed.To maximize student engagement and performance, the questionnaire prioritizes visual clarity and simplicity above extraneous design features.This allows students to concentrate on the content without being distracted.Thus, this article investigates the reliability and validity of a questionnaire specifically meant to assess mobile gamification in the field of economics.Researcher can determine the efficacy of this assessment tool in assessing students understanding and application of economic theories and principles in a gamified mobile learning environment by analyzing its psychometric properties.This study intends to contribute to the ongoing conversation on the incorporation of mobile technologies in economics education by means of thorough validation and reliability testing, opening the door for creative pedagogical approaches that improve student learning outcomes.

Methodology
To ensure the face validity and reliability of the mobile game assessment questionnaire for the economics subject, a rigorous methodological approach was employed.Face and content validity, which refers to the extent to which the questionnaire appears to measure what it is intended to measure, was assessed through expert judgment.Additionally, reliability, which pertains to the consistency and stability of the questionnaire's measurements, was evaluated using Cronbach's alpha coefficient.

Face and Content Validity
Generally, face and content validity is the one of compulsory procedure for research using the questioners as a tool for data collections (Majid, 2020).Functionality of this approached, face validity and content validity were used to evaluate the validity of the Mobile Game Assessment Questionnaire.Two experts assessed face validity using Oluwatayo (2012) proposed criteria.These criteria included evaluating the questionnaire items clarity and unambiguity, suitability for the respondent's level of difficulty, correct spelling of difficult words, adequacy of instructions, and the instrument's general format and construction.This questionnaire is going to be assessed by two language experts who have taught more than ten years.Additionally, the content validity was assessed by two different experts using Cohen's kappa calculation.This statistical measure was used to determine the percentage of agreement adjusted for chance (Cohen, 1960) regarding the representativeness and relevance of the questionnaire items in assessing the intended construct of economics knowledge.Two specialists in the field of technology-enhanced learning were selected to assess the face validity of the survey.Each item was evaluated independently by the experts, who then assigned a score based on how applicable they thought it was to measure the usefulness of mobile gamification in the economics subject.According to (Landis & Koch, 1977) guideline, kappa coefficients of agreement can be taken into consideration: Table 1 Interpretation of Cohen's kappa (Landis & Koch, 1977) Value of Kappa Level of Agreement Less than 0.00 0.00-0.20 0.21-0.400.41-0.600.60-0.610.80-1.00Poor Slight Fair Moderate Substantial Almost Perfect By employing both face validity and content validity assessments, this study ensures that the Mobile Game Assessment Questionnaire accurately measures the targeted construct of economics knowledge and is suitable for use in assessing students' understanding of economic concepts within a mobile gamified learning environment.

Reliability
Reliability in this study employ internal consistency method.This procedure start with internal stability reliability with the assumption that a reliable instrument will measure only one structure (Zuraini et al., 2022).The index to be measured in this usability test is through Cronbach's Alpha.Cronbach's Alpha value can be obtained through software of SPSS calculation.Since this is a pilot test, 20 students will be asked to respond to this question.This is because the conversation is still in its early stages, so the pilot study only requires ten to twenty samples (Mohd Najib, 1999).It was concluded that this sample size would allow for an accurate computation of the Cronbach's alpha coefficient and would yield sufficient information regarding the reliability of the questionnaire items.The student participants were given the questionnaire to complete.According to Hair et al., (2010) Cronbach's Alpha values between 0.80-1.00are very reliable.

Findings
To make sure that the survey question is useful, validity and usability results should be known prior to testing.Thus, the results are based on the facial validity, the Cohen's kappa value for expert agreement in verifying the survey's content, and the Alpha-Cronbach value of the approach to determine the usability of the question.

Face and Content Validity
A total of 47 items were reviewed by experts.For face validity, there are several improvements that need to be made.Face validity can be formulated according to the guidelines established by Oluwatayo (2012), as indicated in Table 3 below.4) the procedure of Cohen's Kappa shows a high value of validity.Thus, the implementation of this tool can be utilized in this study.

Reliability
Following the achievement of high validity score, the another procedure it uses to evaluate the questionnaire's usability by calculating its Alpha-Cronbach value.The results showed that every topic's Alpha-Cronbach value is reliable.The breakdown of Alpha-Cronbach values by questionnaire topics is shown in Table 5. Finding of this method reveal that, all the criteria of learning strategy demonstrate the value of acceptable it is more than 0.70 score as recommended by Cortina (1993).Thus, this questionnaire is appropriate for implemented in actual field research.

Discussion
Several important insights are revealed by the examination of the reliability and validity of the Mobile Game Assessment Questionnaire for the Economics Subject.First, all experts gave positive feedback on face validity, indicating that the questionnaire was appropriate and clear for the intended respondents.Before the field study, any suggested changes were quickly made, guaranteeing that the questionnaire was appropriate for use in practice.Regarding content validity, the Cohen's Kappa values showed an exceptionally high degree of reliability (score of 1).Regarding the usefulness and representativeness of the questionnaire items in gauging respondents' knowledge of economics, this suggests that raters are in excellent agreement.Thus, it can be said that the questionnaire's content is very accurate and suitable for evaluating economics ideas in a mobile, gamified learning environment.The findings of the Alpha-Cronbach calculations showed that the majority of the questionnaire's components had a high degree of internal consistency.Nevertheless, selflearning received a score that was marginally below the reliability cut off of 0.8.However, in the context of the research, this discrepancy is still deemed acceptable.Notably, additional factors that further supported the overall robustness and efficacy of the questionnaire included usability, motivation, visual design, learning support, multimedia design, and usability all showed very high reliability.Thus, there are a few possible explanations for the somewhat lower reliability score seen in the self-learning section, including the topic's complexity and the wording of some questionnaire items.The questionnaire's overall high reliability indicates that, despite this small drawback, it is still a useful tool for evaluating economics knowledge within a mobile, gamified learning environment.

Conclusion
As a conclusion, the Mobile Game Assessment Questionnaire for the Economics subject exhibits robust validity and reliability, establishing its efficacy as a valuable tool for assessing economics knowledge within a mobile gamified learning environment.The positive feedback received from experts during the face validity assessment, coupled with the remarkably high Cohen's Kappa values indicative of content validity, underscore the questionnaire's accuracy and appropriateness for measuring economics concepts.While minor discrepancies were observed in the reliability of the self-learning aspect, the overall high internal consistency across other aspects further reinforces the questionnaire's effectiveness.Moving forward, addressing these minor limitations and building upon the questionnaire's strengths will enhance its utility in future research and educational practice.Ultimately, the questionnaire holds promise for facilitating meaningful assessments of economics knowledge, contributing to the advancement of economics education and pedagogy in the digital age.

Table 3
Summary of Face Validity consensus regarding the content of this questionnaire, Cohen's Kappa value was measured.The calculation results indicate a very reliable level of agreement validity between two experts, as demonstrated in Table 4. Results reveal that (see Table

Table 4
Finding value of Cohen's Kappa

Table 5
Finding Value of Alpha-Cronbach