Exploring Malaysian Tertiary Students’ Behavioural, Cognitive, Emotional and Social Engagement and Disengagement in ODL

Learning engagement is a determinant for success across any learning mode, putting emphasis on the quality of the students’ participation in their own academic journey. However, this is an especially daunting challenge for open and distance learning (ODL). This is further exacerbated when the onset of the COVID-19 pandemic forced the sudden shift to ODL mode, putting students in academic situations that they are unaccustomed to. To examine whether the engagement of tertiary-level students in Malaysia is affected by the change in learning mode, this research aims to identify indicators that make them engage as well as disengage during ODL lessons. A survey instrument known as Learner-EngagementTechnology (LET), was used to measure the level of engagement and disengagement in four categories each, namely behavioural, cognitive, emotional and social aspects. Responses from 129 undergraduates in a public higher learning institution in Malaysia were analysed. The descriptive analysis revealed that the students do engage more than disengage, albeit at a moderate level. The highest indicators of engagement and disengagement however were different. This research is part of an ongoing study and the results can help inform various stakeholders (e.g. lecturers, administrators, policy makers) of the possible causes of engagement and disengagement during ODL classes, so that a more comprehensive and decisive approach can be taken to improve the overall ODL experience for the students.


Introduction
The COVID-19 pandemic had, without a doubt, thrown a curveball to educational institutions especially in testing their readiness to embrace technology for not only supporting, but also possibly replacing the mode of instructions. In Malaysia, while many higher learning institutions have jumped on the Education 4.0 bandwagon and use technology-mediated learning environment in various forms (e.g. blended learning, flipped learning), a significant percentage of classes were still conducted offline or face-to-face. The pandemic that caused a worldwide stir forced the Malaysian government to announce the Movement Control Order back in March 2020, when many of these institutions were just warming up to the start of a new semester. Due to social and physical restrictions, technologymediated learning was the only viable option as the institutions raced to resume the academic sessions via open and distance learning (ODL).
ODL courses have been a 'staple' in public higher learning institutions but these courses were more commonly offered to postgraduate students or working adult learners who seek to pursue their studies part-time. Most courses for full-time undergraduate students were offered in conventional face-to-face or blended mode. The unfamiliarity of the best practices in implementing ODL for full-time undergraduates would jeopardize the teaching and learning experience.
One of the concerns in implementing ODL was how to engage the students in online classes, be it in the same as they would offline, or better. Technology is a double-edged sword where engagement is concerned. While digital technologies are touted to enhance students' learning engagement, challenges have also been identified which include students with limited technical ability, the difficulty in bonding with other students, feeling isolated during online learning, distraction with other applications and establishing limitations between class and personal life (Sun et al., 2016). As such, students need to possess a high level of motivation and self-determination since research has shown that online students need to rely on self-motivation in order to progress (Gómez-Parra & Huertas-Abril, 2019).
If these issues are not addressed, this could potentially lead to attrition (Wang et al., 2017), as online students have been linked to higher attrition rates compared to traditional students (Farrell & Brunton, 2020). The added pressure of COVID-19 related challenges could compound this problem, and it could cause a great loss to tertiary institutions as well as to the country if many of these students choose to withdraw. With these concerns in mind, this study therefore seeks to investigate how Malaysian undergraduates engage with digital technologies in an ODL environment, by identifying indicators that can help to improve the overall ODL experience. This study is guided by the following research questions: 1. What is the Malaysian undergraduates' level of engagement and disengagement in the ODL environment? 2. What is/are the indicator(s) of engagement and disengagement in the ODL environment?

Literature Review Student Engagement and Disengagement in Open and Distance Learning
Research on student engagement is becoming increasingly popular  as it has been linked to an array of traditional success factors including improved academic performance (Bowden et al., 2019;Finn & Zimmer, 2012) and is acknowledged as a valuable measure of 'quality' for teaching and learning (Ashwin & McVitty, 2015;Henrie et al., 2015). However, measuring engagement, especially in technology-mediated learning environment is especially a challenge since it is not directly observable compared to face-toface learning. It is even more trivial when instructors and students operate remotely such as the case of ODL.
Therefore, in what way do we measure engagement in such environment? We first begin with defining student engagement. We conceptualise student engagement following the widely accepted motivational framework by Fredricks et al. (2004). In this framework, student engagement is viewed as the commitment or investment shown by students in learning that is categorised into behavioural, emotional, and cognitive dimensions. These include identifying indicators where students show initiative actions to learn e.g. asking questions, taking notes or observable reactions towards learning tasks e.g. leaning forward while listening (behavioural), paying attention towards the object of learning to learn and master the material (cognitive) and projecting emotional responses such as interest, happiness, sadness, boredom and anxiety (emotional).
Although these dimensions were framed in the context of face-to-face learning, the key indicators can help inform engagement research in the technology-mediated setting. In the review by Henrie et al. (2015), it was found that existing research in technology-mediated setting examined either one aspect or combining two or all three dimensions using various measures. Later studies such as Halverson (2016) and Ma et al. (2018) have also used the same framework to measure engagement in the same context. Fredricks et al. (2016) and Wang et al. (2016) added the fourth aspect; the social dimension. This is to acknowledge that learning opportunities are embedded within a social context (Wang & Hofkens, 2019) and this can be observed from the students' participation in social interaction or collaboration in the learning process. Analysing the social dimension is arguably more crucial in the ODL setting, to see how this learning mode influence the way students and instructors engage virtually and remotely. Therefore, it is crucial for this dimension to be included in this study.
To add to the operational definition, we also recognise the other spectrum of engagement, namely disengagement, which is considered as not merely the bipolar opposite, but rather a unipolar construct in itself (Skinner et al., 2009). Previous studies have also used the terms disruptive engagement (Brown & Fletcher, 2002) or disaffection (e.g. Skinner et al., 2008). These terms, however, were only used to examine the emotional and/or behavioural engagement and did not address the cognitive and emotional dimensions. Additionally, disengagement may range from indicators such as withdrawals, absenteeism, and may go as far as to causing student attrition (Wang et al., 2017). This warrants the need for examining disengagement as technology may cause students to disengage from learning (Bergdahl, 2020b).
Therefore, as a way to 'observe' the Malaysian undergraduates' first experience in engaging with the ODL instructions, this study chooses to focus on the multidimensional indicators of student engagement (e.g. interest in learning, student interactions between/with instructors and peers, processing of information) while incorporating the indicators of disengagement. This study follows Bergdahl et al.'s (2020a) descriptors, that are contextualised in the technology-mediated environment and focusing on the use of digital technologies. As seen in Table 1, the descriptors of engagement and engagement are presented in a parallel but distinctly separate multifaceted constructs consisting of the four dimensions.

Methodology
In this study, a quantitative descriptive research design was employed to describe the indicators of engagement and disengagement among Malaysian tertiary students in the ODL environment. An online questionnaire, using Microsoft Forms, was sent out to 132 undergraduates enrolled in the English language courses in two campuses of a public higher learning institution in Malaysia with 129 returned responses. A purposive sampling method was used to select participants who underwent the ODL learning environment described below and had no prior experience with ODL instructions.

The ODL Environment
Bond and Bedenlier (2019) proposed a bioecological system on student engagement that exist on four levels: macro, exo, meso and microsystem. Due to the small scale of this study, only the description of the microsystem is provided (Figure 1), which places the student in the middle of the environment as they interact with the instructors, the curriculum, the technology and their peers. In this study, all the students were connected within the environment, via personal access to the Internet and their own digital devices (i.e. smartphones and/or laptops). The ODL curriculum was delivered using a combination of digital technologies, including learning management systems (e.g. Google Classroom, uFuture), social media applications (e.g. WhatsApp, YouTube) and web-conferencing sites (i.e. Google Meet) to share learning materials, to conduct lectures, oral and written practices as well as to implement the formative and summative assessments for various English language courses. The choices of these digital tools were made based on the considerations of available institutional support as well as students' varying level of access to the Internet.
The students interacted with their instructors and peers via these channels through the learning activities conducted, both in synchronous and asynchronous forms. The synchronous sessions include web conferencing for practices and presentations, while the asynchronous sessions were conducted by sharing pre-recorded or online materials as well as online discussions. Both individual and collaborative learning tasks were incorporated in this environment, as according to Farrell and Brunton (2020), online courses that promote interactions and social presence help to support online student engagement.

Learner-Engagement Technology (LET) Questionnaire
To collect the data, the study used a survey instrument known as the Learner-Engagement-Technology (LET) Questionnaire (Bergdahl et al., 2020b). Although there are other existing surveys such as the National Survey of Student Engagement (NSSE), this annual survey created and administered by the Indiana University Center for Postsecondary Research in the US consists of items that are known to contribute to favourable learning outcomes at the institution level instead of the course level, and focuses on only the three aspects of behavioural, cognitive and emotional engagement. Also, this study aims to address the issue of disengagement among the students which is not addressed by NSSE. Therefore, the LET Questionnaire was deemed to be more suitable due to its relevance to the scope and context of this study.
The survey consists of three sections: the first section comprises open and closed responses to obtain the participants background information, the second section requires the respondents to self-evaluate their digital skills for academic, and the final section comprises 31 items on engagement and disengagement indicators using the 5-point Likert scale (1=Strongly Agree to 5=Strongly Disagree). Some changes were made in phrasing the statements for indicators to match the ODL context of this study. To help ensure comprehensibility, the survey was presented bilingually, in English and Bahasa Malaysia, and all items were checked for linguistic difficulty and ambiguity. The data from the questionnaire was analysed descriptively using Version 22 of IBM SPSS in which the mean score (M) and standard deviation (SD) were tabulated for each item. A reliability analysis was also conducted which yielded the Cronbach's coefficient alpha value of .768. Table 2 presents an overview of the engagement and disengagement indicators as adapted from Bergdahl et al. (2020b) for behavioural (Beh/Dbeh), cognitive (Cog/Dcog), emotional (Emo/Demo) and social (Soc/Dsoc) dimensions.

Findings Demographic Profile of the Respondents
Based on the responses submitted by 129 participants who took part in this study, the following analysis is presented. The characteristics of the respondents are depicted in Table  3. Majority of the respondents (86.8%) were of the age 18 to 19 years old and the sample consists of 94 females (72.9%) and 35 males (27.1%). They were studying at either the foundation or diploma level programme and majority of them (46.5%) were in their first semester of study. The survey also requires the students to self-report their digital competency as digital skills is a necessity for operating in online learning. As can be seen in Table 3, the reported levels were from poor to high with majority of them perceiving their skills to be average (54.9%) and above average (36.3%).

Level and Indicators of Engagement and Disengagement in ODL
The objectives of this study were to identify the level and indicators of engagement and disengagement among the students when learning using digital technologies in the ODL classes. A descriptive analysis was conducted on the questionnaire items to identify the patterns of engagement and disengagement. The mean scores were first calculated for each item under the categories of engagement and disengagement indicators. The average mean scores were then calculated to identify the level of engagement and disengagement among the respondents. As reported in Table 4, the engagement score was moderately high (M=3.79, SD= 0.797) while the disengagement score was at the moderate level (M=3.31, SD=.903). Aside from the overall scores, the individual mean scores for each engagement and disengagement indicators are also presented in Table 4, from the highest to the lowest. Behavioural engagement was found to be the highest indicator, with a mean score of 3.92, followed by social, emotional, and cognitive engagement. The differences in mean scores between these four indicators were small from 3.65 to 3.92, suggesting that the respondents were engaged in all aspects in an almost similar way. In contrast, the differences in mean scores for disengagement indicators were much larger between the highest indicator (social disengagement at 3.79) and the remaining three aspects (between 3.09 to 3.27). This means that although the respondents sometimes disengage cognitively, emotionally, and behaviourally, they felt the most disengaged socially. It should be noted that for the social dimension, the mean scores for both engagement and disengagement were almost similar. This suggests that the students experienced both social engagement and disengagement when learning in the ODL mode.
To further understand what causes students to engage and disengage, Table 5 present the top ten main descriptors for engagement and disengagement from highest to lowest. This was reported in five categories: behavioural engagement (Beh), emotional engagement (Emo), social engagement (Soc), cognitive engagement (Cog) and social disengagement (Dsoc) with two indicators from each respective category.
From the list, it can be summarised that students engaged behaviourally by using digital technologies to support learning specifically for content mastery and research purposes. Emotionally, they perceived the value of using digital technologies in optimizing and supporting their learning. Socially, the students felt engaged when instructors used technology for monitoring and evaluating performance as well as providing a sense of belonging. However, they also became socially disengaged when they were left managing digital tools on their own as well as when they were repeatedly instructed to search online. Lastly, the students showed cognitive engagement by autonomously deciding and using technology for learning and when they feel that technology is needed to maximise their learning.

Discussion and Pedagogical Implications
From the results, two observations can be made that can potentially impact teaching and learning practices in technology-mediated learning, particularly in the ODL environment. First, despite having no experience in ODL, the students were able to engage across all dimensions as they recognised the needs, value, and effort to use technology for supporting, optimizing, and maximising learning. This is evident from the behavioural, cognitive and especially emotional indicators (Beh1, Emo2, Emo1, Cog3) as reported in Table 5. As learning in the ODL environment was delivered using various tools and using both synchronous and asynchronous media, it may have helped accommodate the students according to their needs, hence increasing engagement. Socially, the effective use of digital tools in providing feedback (Soc2) and promote inclusion (Soc3) were found to be the key to engaging the students. This may have been enabled by the ODL environment through the use of social media platforms such as WhatsApp for communicating and the use of collaborative learning activities. According to Bowden et al. (2019), when students are able to engage holistically, it provides a strong drive for their self-efficacy and self-esteem. This is important to ensure that students remain motivated in learning even in the ODL mode and reduce the risk of dropping out.
Another key finding from this study is that the students were as socially engaged as they were disengaged. This reinforces the need to measure engagement in technologymediated learning by incorporating the social aspect Wang et al., 2016) to the widely used behavioural, cognitive and emotional dimensions. Based on the indicators, though the students were able to engage using digital tools in a way that make them feel included (Soc3), they became disengaged when they were instructed to work on their own (Dsoc1) and were repeatedly instructed to do online search (Dsoc2) instead of receiving guidance. This reiterates the findings that online students often feel a sense of disconnection and isolation in their learning experience (Phirangee & Malec, 2017). These may have been caused by their lack of competency in using and managing digital tools effectively as most students gauged themselves as average digital users. Digital skills and knowledge are undeniably crucial for students to navigate the online learning environment (Hiar, 2020). Although they are dubbed the 'digital natives', some critical skills are necessitated when there is an integration of technology in the classroom (Alodail, 2020) and could be missing since they have not experienced ODL previously.
To ensure that digital technologies are used in a meaningful way to engage students, educators should plan the learning activities using mindful pedagogical approaches (D'Angelo, 2018) as only it will be more impactful on the students' learning (Alodail, 2020). With this in mind, we make the following recommendations for ODL instructors: 1. Students should be taught digital literacy skills. The social (dis)engagement viewpoint has brought to our attention that technologies may facilitate and impede the way students interact online. This is because, students need to interact with the curriculum, instructor, other students and technology within the ODL environment as mediated by digital tools. As found in other studies (Gilbert, 2017;Sain, et al., 2017), the current generation of students do not possess the adequate level of competence when it comes to using digital technologies for academic purposes. Therefore, they may need to be taught online learning skills such online research and comprehension (Sain et al., 2017) as well as strategies to utilise digital tools for online communication more effectively. The students' lack of confidence in their digital competency may be due to the lack of preparedness among Malaysian undergraduates in engaging in ODL as found by Chung et al. (2020). Therefore, if ODL instructions continue to be adopted regardless of the pandemic, higher learning institutions should consider giving students online learning orientations to increase their readiness and improve their academic digital skills (Abdous, 2019).
2. Use digital tools and asynchronous sessions for content rehearsal and mastery. The top indicators for engagement suggested that engagement, particularly behavioural and emotional, is very closely related to the use of digital technologies in the classroom. Educators should therefore integrate digital tools as a way for students to learn, rehearse, and master the content on their own, further promoting active learning. A research by Al-Bogami and Elyas (2020) found that the use of iPad applications like Quizlet for reading activities made students enjoy the lessons more and has positive impacts on the students' motivation, focus and engagement. Although the features in Quizlet (e.g. tests/quiz, flashcards, matching activity) are not much different from activities conducted in a traditional classroom, moving the content online and using digital tools for access to the materials have the potential to increase engagement, especially behaviourally.
For asynchronous learning sessions, there are a few activities that educators can use to boost engagement. Students can be asked to study related materials such as pre-recorded videos, slides or assigned readings to help reinforce learning and deepen understanding. They could then be asked to complete tasks relevant to the material. For example, Prince et al. (2020) suggested the thinking aloud paired problem solving (TAPPS) method. This method involves students working in pairs to solve the task given, then taking turns as the problem solver (vocalizing their thought process) and listener (asking questions and providing prompts if necessary). Another suggested activity is for students to develop online content and create guiding questions. Asynchronous activities like this helps students to prepare and master the content of the lessons on their own.
3. Incorporate elements of social interaction in learning activities. Farrell and Brunton (2020) posit that online courses that promote interactions and social presence help to support online student engagement. In the same vein, Aguilar et al. (2020) believe that designing lessons that promotes discussion is a way to "utilize students' natural inclination for conversation" that will in turn make them further engage in learning materials (p. 25). Collaborative discussion could be a way to boost cognitive engagement, as reported in studies by Coiro et al. (2014) and Kiili and Leu (2019). Through their work, collaborative discussion was shown to help students demonstrate better online search results and navigational practices as well as greater evidence-based reasoning acquired from online resources when completing academic tasks. Instructors could use online forums or chat groups to monitor and facilitate students' interaction by allowing them to reflect, integrate and explain concepts using their own words. Facilitating interaction is especially crucial to prevent negative student-student interaction (Phirangee, 2016).
The integration of technology that promotes interaction, like social media applications, also should be considered as it can create a more relaxing learning atmosphere as well allowing students to feel more confident and increasing their participation in the lessons (Mansor & Rahim, 2017). This does not necessarily mean that educators must incorporate social media in their classroom in order to create meaningful interaction among students. The same objective can be achieved by using the discussion and forum features present in most Learning Management Systems. For example, one type of classwork available in Google Classroom is Question. The use of Question allows educators to post various relevant materials (e.g. articles, webpage, YouTube videos) which students can react to by posting comments or answers. The collaborative learning experience can further be tweaked by enabling students to not just respond to the material, but to each other as well. One of the indicators for social engagement is the sense of inclusivity and belonging (Bergdahl et al., 2020b). In higher institutions, this may not be easily achieved as students are less likely to stay with the same group of classmates for each course. Thus, Poort et al. (2020) suggested establishing trust among group members, even before the group work began, for the purpose of improving engagement. Activities that allow comfortable and casual interaction between the group members, such as simple quizzes on Kahoot!, would be helpful for increasing social engagement.
4. Be mindful of potential pitfalls leading to disengagement. Although improved social engagement is no doubt beneficial for students, the findings of this study also indicated that students are more prone to be socially disengaged. Providing consistent and timely constructive feedback is one way to reduce social disengagement (Hiar, 2020). Nonetheless, Bergdahl et al. (2020b) caution that this solution does not address the potential disengagement among high achieving students, who may be more concerned with the active participation and self-sufficiency of their group members during a collaborative task, rather than receiving support from the lecturers.
While grouping students of mixed ability allows for peer learning, an aspect of social engagement according to Fredricks et al. (2016), it could also possibly create a situation for unequal distribution of responsibility and contribution while completing a particular task. This could cause higher achievers to feel irritated over having to put in more effort compared to the other members (Bergdahl et al., 2020b). Therefore, simply assigning group work is no guarantee that social engagement is achieved and could cause disengagement instead. More care and thought should go into assigning group tasks or activities to prevent disengagement. For example, students should be provided with a clear guideline on their responsibilities to the group. Most students may be unfamiliar with the skills required for effective teamwork, so they would need to be equipped with relevant information such as strategies to handle disagreements and managing group expectations (Hansen, 2006) Disengagement could also occur due to the students feeling frustrated with having to constantly navigate their ODL learning experience on their own and facilitate learning through finding information online (Bergdahl et al., 2020b). This could be prevented by using activities that encourage independent learning while also providing necessary online learning skills. Online scavenger hunt for example, could be useful in getting students to develop research skills in a more casual setting. This would enable them to practice the skills needed to make their ODL experience more enriching and not left with feeling burdened by having to work independently.

Conclusion
This study presented a description of engagement and disengagement levels and indicators among Malaysian tertiary students in undergoing ODL for the first time and contributes to the growing body of literature on learning engagement and disengagement for technology-mediated learning environment. The findings point towards the need for more studies on 'social engagement' as well as the exploration of the disengagement spectrum in further understanding online learning experience. Several pedagogical recommendations were made on how to maintain and improve engagement holistically, as well as on how to reduce social disengagement.
However, there are several limitations in this study that could potentially be addressed by future research. As this study only focuses on quantifying indicators of engagement and disengagement, future research may also include two other areas suggested by Schindler et al. (2017), namely factors influencing student engagement and outcomes of student engagement (p.5). Qualitative data collection through student interviews could help shed more light on the possible challenges that they may face in trying to engage in ODL and possibly add more input to the existing survey instrument. This study also focused on the ODL environment in a broad context. Therefore, focusing on synchronous or asynchronous mode or the use of specific digital tools or activities could potentially yield a better understanding of what shapes the students' experience in engaging or disengaging in ODL instructions. The possibility is endless as this study only addresses a small part of this important issue. Should the ODL instructions continue in the future and possibly replace the current delivery method in tertiary institutions, it is hoped that studies on engagement and disengagement would contribute to a solid and comprehensive implementation.