Teaching, Learning, and Assessment (TLA) Implementation to Address the Psychomotor Domain of Engineering Students During Open and Distance Learning (ODL): A Pilot Study

During the COVID-19 pandemic, most higher learning institutions (HLIs) in Malaysia had to change their face-to-face teaching and learning to Open and Distance Learning (ODL). Previous studies indicate difficulties associated with assessing psychomotor skills in engineering education via ODL. This study aims to observe the Teaching, Learning, and Assessment (TLA) implementation in carrying out laboratory work during ODL. This paper presents a pilot study involving 30 undergraduate engineering students. A descriptive research design applied using a document review and an online questionnaire survey to gather data on the effectiveness of the TLA related to psychomotor skills. The Cronbach's alpha values of more than 0.7, indicates that the instrument has obtained good internal reliability and consistency. The top-ranked challenge faced during ODL was a lack of equipment. Furthermore, the students recommended the provision of online simulations to improve online laboratory-based courses. The findings provide a basis and reference for improvements to carry out the main study on a larger scale that involves more engineering students in public and private universities in Malaysia. This study may facilitate the development of engineering programs in the HLIs to improve the practice-based learning curriculum during ODL.


Background of Study
The COVID-19 pandemic is an unprecedented event that occurred at the end of 2019 and caused a lockdown starting in March 2020 in Malaysia. The Movement Control Order (MCO) was introduced in various sectors to minimise the spread of disease. It causes significant disruption to various social and economic sectors, including education. It has posed inevitable challenges to students and educators in implementing Teaching, Learning and Assessment (TLA) related to the psychomotor domain in laboratory works where students are not allowed to be on campus, and the TLA was conducted fully online. It is very challenging for lecturers and students to adapt to the new approach of using remote teaching.
However, the pandemic has opened venues for Open and Distance Learning (ODL) with a completely new outlook for educators. In Malaysia, all higher learning institutions (HLIs) have changed the face-to-face TLA methods to ODL. Various virtual mediums are introduced and used by the lecturer to ensure that all students can participate in class by considering limited facilities, internet accessibility, and so on. Due to the abruptness of the worldwide situation, the HLIs were forced to design delivery methods around emergency remote learning systems. It seems to be accepted for lecture-based subjects. However, there are disadvantages to programs that offer subjects involving studios and laboratories. This is because both subjects require an assessment of psychomotor skills that cannot be carried out comprehensively online. Various ways and efforts have been made by the university and lecturers to ensure that the measurement of motor skills can be carried out according to guidelines issued by the accreditation bodies. The objectives of the study are (1) to study the effectiveness of the Teaching, Learning and Assessment (TLA) in addressing the psychomotor domain in laboratory courses during Open and Distance Learning (ODL) and (2) to determine the challenges and ways to improve the TLA for laboratory courses during ODL. This paper presents the findings from the pilot study focusing on the overall implementation of in laboratory courses for the civil engineering programme at a local university in Malaysia. It is hoped that this study facilitates the development of engineering programmes and helps the HLIs to improve and enhance the practice-based learning curriculum during ODL.

Literature Review Challenges faced in Open and Distance Learning (ODL)
The pandemic has changed the education system from physical learning to online learning. Teaching, Learning and Assessment (TLA) at the tertiary level have been reviewed and redesigned to suit remote learning. However, some studies suggest Open and Distance Learning (ODL) imposes various challenges, issues and effectiveness compared to traditional methods. Zainol et al (2021) have identified three major challenges during ODL, which are the usage barrier (insufficient devices and internet quota), value barrier (correct device to support TLA during ODL) and risk barrier (the acceptance of ICT knowledge and skills) for lowincome earners (B40) in Malaysia. This study, supported by other researchers, found that the major issues and challenges during ODL are competency in information and communication technology (ICT) among stakeholders, internet accessibility, and monetary issues among students (Musingafi et al., 2015;Ag-Ahmad, 2020;Adnan and Anwar, 2020) Furthermore, Aileen (2021) found that workload during ODL is increasing compared to physical learning and the home environment is not conducive to the learning process.
Lecturers need to find the best platform for online learning to accommodate students' locality, internet bandwidth, and quota. Hence, Google Meet and WhatsApp are the most preferred online media during ODL (Ag-Ahmad, 2020). In addition, Adam et al (2021) realised that time management among students plays an important role during ODL. Adnan and Anwar (2020) found that ODL in Pakistan is less effective than traditional methods due to infrastructure and financial issues. Meanwhile, Ag-Ahmad (2020) perceived that ODL is partially adopted by Malaysian students during the early stage of Movement Control Order (MCO), and Amir et al (2020) concluded that dental students in Indonesia well accepted the ODL and performed better during ODL. Later Chiew et. al (2022) conclude that ODL for engineering students only meet the minimum level of assessing psychomotor domain for laboratories courses. Hence, a study should be conducted on teaching, learning, and assessment (TLA) to address the psychomotor domain among undergraduate students in the Civil Engineering programme.

Psychomotor Domain
The psychomotor domain of learning outcomes is required to equip the engineering graduate with the necessary attributes and skills. In contrast with the cognitive learning domain, which generally uses exams and tests to assess the student'sperformance, the psychomotor domain learning outcome is not straightforwardly implemented. In addition, recent studies have found the difficulties associated with assessing psychomotor skills in engineering education via ODL (Chiew et al., 2022;Garcia-Alberti et al., 2021;Mustaffa and Shah, 2021). Novak-Pintarič and Kravanja (2020) viewed that programmes attached to Science, Technology, Engineering and Mathematics (STEM) seem to be less favourable for online learning before COVID-19 due to motor skill assessment related to laboratory work. Laboratories exercise is executed using a pre-recorded video on laboratory demonstration, which involves effort and hard work from the lecturer and laboratory assistant. Kapilan et al (2021) realised that laboratory work is difficult to complete during ODL. However, improvements have been made to enhance student attainment and skills development using virtual laboratories for mechanical engineering during 2020-2021. However, the virtual laboratory required high technology and extensive infrastructure to deliver remote learning for experimental work. Virtual Learning Environments (VLEs) were introduced by the University of Sheffield to accommodate a large number of students at the same time. Canvas, Blackboard, and Moodle are used for Teaching, Learning, and Assessment (TLA) to capture student engagement and knowledge during laboratory sessions (Bangert et al., 2022).
At the same time, Debacq et al (2021) studied the Learning Management System (LMS) for engineering food laboratories using remote learning in delivering the learning outcomes during COVID-19. However, VLEs and LMS seem to work for knowledge development but are not comprehensive in psychomotor domain profile assessment. Hoque (2016) reviewed three popular versions of psychomotor domain profiles that were started by Simpson and Harrow in 1972. Three years later, Dave reduced the difficulty level from 7 to 6. The list of the domain profiles is summarised in Table 1.  (2021) perceived engineers should possess well-developed motor skills during enrolment in engineering courses to ensure high competency of technical aspects in planning, handling, executing and evaluation of output. Since COVID-19, the mode of TLA has changed from physical learning to ODL. Therefore, psychomotor domains are a major concern among HLI that offer technical programmes and the Engineering Accreditation Council, Malaysia (EAC) in monitoring and maintaining the quality of graduates during ODL according to EAC Standards 2020. ODL disadvantages in laboratory courses, especially in assessing the psychomotor domain, with the limitation of tools, equipment, resources, and software in executing remote learning. Lecturers need to adjust based on limitations to accommodate all students in delivering the learning outcomes. Hence, an online questionnaire survey was conducted at the undergraduate level for Civil Engineering students who took laboratory courses during ODL, focusing on the psychomotor learning domain.

Methodology
A descriptive research design was adopted in this study. The document review was conducted on the documents related to laboratory courses, and an online survey questionnaire was disseminated to undergraduate engineering students. This paper presents the results of a pilot study focusing on the psychomotor learning domain acquired by undergraduate civil engineering students who took laboratory courses during ODL.

Sampling Design
The targeted respondents to answer the online survey questionnaire for the pilot study are civil engineering undergraduate programme students from an engineering programme who took laboratory courses in ODL mode between March 2020 and October 2021. Convenience sampling is adopted, and the number of respondents for the pilot study is set at 30 (Browne, 1995).

Instrument Design -Questionnaire Survey (Pilot Study)
An online survey questionnaire was designed based on previous studies related to the psychomotor domain. The questionnaire consists of six (6) sections: Section A assesses the demographics of the respondents, Section B assesses students' perception of ODL; Section C assesses students' psychomotor skills development during ODL, and Section D assesses the effectiveness of TLA in addressing the psychomotor domain during ODL, Section E assesses the challenges in implementing laboratory-based courses during ODL, and lastly, Section F captures students' recommendation for further improvement in conducting laboratory courses in ODL. Table 2 shows the questions included in each section of the questionnaire. Section C C1. Kindly indicate your level of agreement on how psychomotor skills contribute to students' abilities during ODL. Choose a scale from 1 (strongly disagree) to 5 (strongly agree). a) Students' ability to perceive nonverbal communication clues improves. b) Student's readiness for actions classified as physical, mental, or emotional is enhanced. c) The student's ability to learn through trial and error has improved. d) Students become more skilled and more confident when they learn more complex skills. e) When students acquire habitual reactions, they gain confidence and create skilled movement. f) Students can adapt their movement patterns to higher skills to tackle an issue. g) Through these psychomotor skills, students can develop new and unique movement patterns.
Section D D1. Please indicate the level of effectiveness of TLA in addressing the psychomotor domain during ODL. Choose a scale from 1 (not effective) to 5 (very effective). a) Do you think that studying engineering laboratories using ODL is an effective method to measure the psychomotor domain? b) Do you agree that online meetings, such as Google Meet, with the provision of notes, and recordings contribute to the effective delivery of laboratory courses during ODL? c) Do you think the designated assessment effectiveness during the pandemic is equivalent to the assessment conducted before ODL? d) Do you agree that time flexibility effectively improves your psychomotor domain during ODL?
D2. What kind of assessment is used to measure your psychomotor domain for the laboratory-based course during ODL?
D3. Kindly rate the appropriate online distance learning programme outcome (PO) attainment for your psychomotor domain development based on the statement below. Choose a scale from 1 (strongly disagree) to 5 (strongly agree). a) I am able to conduct investigations of complex engineering problems using research-based knowledge for laboratory courses b) I am able to determine the correct methods and procedures, including the design of experiment c) I am able to create, select, and apply appropriate techniques, resources, and modern engineering and IT tools to carry out the relevant experiment d) I am able to collect data from experiments accurately e) I am able to understand, analyse and interpret experimental results to discuss significant findings f) I am able to synthesise information towards providing relevant solutions to overcome problematic conditions g) I am able to propose a valid conclusion for the given problem Section E E1. Please indicate the level of challenge to undertaking laboratory activities during ODL based on the following statement. Choose a scale from 1 (not challenging) to 5 (very challenging). a) A non-conducive working space for experimentation. b) Poor internet connectivity c) Communication barriers between students and lecturer d) Lack of equipment e) Lack of resources to carry out experiments f) Insufficient support for mental health g) Lack of suitability of experiments to be adopted into virtual form h) Insufficient teaching delivery of learning outcomes related to psychomotor domain i) Ineffective assessments of learning outcomes related to psychomotor domain j) Ineffective student learning experience related to psychomotor domain Section F F1. Your comments and suggestions are essential to improving the online laboratory courses. Choose from scales of 1 (not significant) to 5 (very significant). a) Reinforce learning via e-lab or simulation-based laboratory experiments. b) Create an engaging live or recorded laboratory. c) Encourage students to conduct self-guided laboratories using home-based resources. d) Use high performance software accessible to students. e) Provide consultations with small groups of students. f) Provide financial support. g) Offer assistance with mental health issues. h) Improve assessment tools to suit the psychomotor domain. i) Revise the learning outcomes to align with the psychomotor domain. j) Provide special arrangements to post/transport components, materials, and/or tools required by students to carry out the experiments at their own premises. F2. In your opinion, what is/are other (s) appropriate improvements that can be made for teaching, learning and assessment for the laboratory courses during ODL?

Data Collection and Analysis
Data collection was done using an online Google form to obtain feedback from respondents. A Cronbach alpha reliability test is conducted on data collected from Sections B, C, D and E of the questionnaire. Results from the Cronbach alpha reliability test are reported in Table 3. The Cronbach alpha values for all sections are greater than 0.7 (ranging from 0.749 to 0.916), indicating that the questionnaire used is a reliable instrument (Tavakol & Dennick, 2011). 0.838 For the analysis of data, descriptive analysis is conducted, and the respective histogram, bar chart, and relative importance index are discussed in the following section.

Analysis and Discussion of the Pilot Study
Demographic Profile About 80% of undergraduate students are female, most aged between 22 and 25 years old. They were from a different year of study, between semesters 1 and 8, and about 56.7% of students were from semester 7. Most of the obtained CGPA ranged from 3.0 to 3.49. The rest gained a CGPA below 2.5, consisting of 3.3% of the overall respondents.
Students' perceptions of Open and Distance Learning (ODL) Figure 1 show various type of assessment used to measure the psychomotor domain chosen by the students during ODL. Figure 1: Type of assessment used to measure the psychomotor domain during ODL It is shown that the most preferred and used assessment tool to measure the psychomotor domain was by using videos prepared by the students to show the activities carried out at home (67%), followed by a real case study (56.7%) and a home-based experiment (53.3%). The assessment that has been given to measuring psychomotor needs to be adjusted according to certain criteria such as leading to the achievement of learning outcomes; being done by students; containing material that is following the scope of the curriculum; fair (unbiased by gender and socio-economic background); and including the time for the task (Hermawan et al., 2021). Through this method, it is much easier to assess the student's performance, which is related to the psychomotor domain directly. Most of the previous studies agreed that during COVID-19, the implementation of recording lab experiments and home-based experiments could partially help to assess the psychomotor domain (Asgari et al., 2021;Garcia-Alberti et al., 2021;Vielma & Brey, 2021). Figure 2 depicts the effectiveness of teaching and learning activities in addressing the psychomotor domain during ODL. Findings indicated that around 44% of the students opined that they understand and interpret experimental results to discuss significant findings, determine the correct procedure to conduct the experiment, and able to propose relevant solutions to overcome problematic conditions, while about 34% agreed that they were able to collect proper data based on the experiments and were to recognise the problems during online laboratory courses. However, only 27% said that they were able to demonstrate the equipment utilised appropriately during online laboratory courses. Thus, it can be concluded that most students can develop their psychomotor skills very well during the ODL. This was aligned with the finding by Mustaffa and Shah (2021), which showed that TLA during ODL with the combination of weekly assessments given to students could sustain students' interest and increase their performance. More than 70% of students strongly agreed that the online platforms are important for teaching and learning in laboratory-based courses during ODL, whilst 90% of students perceived that engineering laboratory learning through ODL is moderately effective (37%), less effective (37%) to not effective (10%) as can be seen in Figure 3. A recent study shows that a higher number of first-year students preferred distance learning compared to their seniors. Still, they preferred classroom learning for group discussion since distance learning resulted in more difficult communication and less learning satisfaction. However, they agreed that distance learning provides a more efficient learning method, which gives them more time to study and more flexibility to review study materials (Amir et al., 2020). Rowe et al. (2018) found that the student's responses to the laboratory experiences in their online courses were as good or better than those in their traditional laboratory courses. This was contributed by the fact that they had a stress-free environment and the ability to focus exclusively on content instead of logistical details.  The top rank challenge is the non-conducive working space experimentation which was agreed by 93% of the respondents as very challenging (37%), followed by challenging (53%) and moderately challenging (3%). In comparison, poor internet connectivity was agreed by 93% of the respondents as very challenging (30%), followed by challenging (40%) and moderately challenging (23%). The third rank challenge is the communication barriers between students and lecturers, which were regarded by about 93% of the respondents as very challenging (40%), followed by challenging (40%), and moderately challenging (13%). Finally, this study indicates that insufficient support for mental health was evaluated by 93% of the respondents as very challenging (53%), followed by challenging (17%), and moderately challenging (23%). However, a recent study found that the ODL poses unique challenges to adapting to new technology in a short period and developing new and innovative ways to teach hands-on psychomotor skills virtually while maintaining the quality of instruction (Plummer et al., 2021). Figure 5 shows that the top-ranked recommendations for improvement are to (1) provide online simulation, (2) improve assessment tools, (3) revise the content of the syllabus, (4) provide consultations with small groups and create live or recorded laboratory experiments, (5) have students conduct self-guided experiments and offer assistance with mental health issues, (6) use high-performance software that is accessible by students, and finally, (7) provide financial support. A previous study by Chernikova et al (2020) shows that simulationbased learning offers a wide range of opportunities to practice complex skills in higher education and to implement different types of scaffolding to facilitate effective learning.

Conclusion and Recommendation
This paper presented significant findings from a pilot study to observe the teaching, learning, and assessment (TLA) implementation in laboratory courses conducted during Open and Distance Learning (ODL). Overall, it can be concluded that the TLA activities related to psychomotor domain assessment via ODL were considered not as effective as the physical sessions for laboratory and design courses. However, ODL can be considered a good alternative to replace conventional TLA activities, provided that the activities are improved to ensure true and direct assessment of suitable learning domains designed for the laboratory courses.
Overall, the findings of this study provide some theoretical suggestions and insights for future research and practical instructional design addressing the psychomotor domain during ODL. First, this study reveals that assessment tools such as videos, real case studies and homebased experiments are the top preferred assessment tools for assessing the psychomotor domain during ODL. However, these findings come from the students' perspective; future research from the perspective of lecturers is needed. Second, the study also discovers the top challenges faced by the students during laboratory courses during ODL especially the lack of equipment accessible to them. Recommendations for improving the TLA that addresses the psychomotor domain during ODL include more online simulation. There is also a need for further exploration into a change in students' behaviour or attainment to a higher level through different TLA strategies.
The findings from the pilot study can be further used as a basis and reference for improvements to carry out the main study on a larger scale that involves more engineering students in various public and private universities in Malaysia. It is hoped that this study facilitates the development of engineering programmes and helps the HLIs improve and enhance the practice-based learning curriculum during ODL. The findings of this study support future TLA planning and design of engineering courses addressing psychomotor, specifically during ODL. For future research, a study on the effectiveness of incorporating blended learning in the curriculum related to laboratory courses can be carried out to improve the overall learning environment for engineering students.