Quality Assessments on Different Mode of Teaching and Learning Delivery: Analysis of the Outcome-Based Education Activities in Communication Engineering Course

Vol. 11(3) 2022, Pg. Abstract The Outcome-based Education (OBE) implementations in the higher learning institution have been significantly affected by the Covid-19 outbreak, but its progression from pandemic to endemic signals better health regulations and controls. This paper aims to examine the relationship between course Program Outcome (PO) attainment and students' perception of the basic communication engineering (ECM241) course. The utilized methods are PO measure ment through assessments and final examination as well as a student’s satisfaction survey. The analysis is done for three different delivery modes: face-to-face, hybrid, and online delivery. Through the PO measurement, the results show that students mostly preferred the hybrid delivery mode due to the flexibility in its assessment and activities conducted, reduced facility usage, and time constraints for the course completion. The survey method also showed that most of the students are satisfied with the course content and all factors contributing to the course delivery. In conclusion, this subject is suitable to be conducted using the hybrid mode as compared to the conventional method. In the future, this research could be expanded by considering all subjects in the semester to access overall OBE achievement and students' satisfaction with the Electrical Engineering program.


Introduction
The use of online learning has taken over educational institutions during the Covid-19 outbreak. The viral outspread of the pandemic forces traditional face-to-face educational structures to move swiftly to online classes. The United Nations Educational, Scientific and Cultural Organization (UNESCO) reported that 73.8% of students acknowledged the swift change from classroom sessions to online sessions impacted their studies due to the closure of the educational institutions. Consequently, once the pandemic experiences a sharp increase in reported Covid-19 cases, the Ministry of Higher Education (MoHE) in Malaysia reacted in time to release a circular note to temporarily close all public universities (Selvanathan et al., 2020). Hence, higher education students faced multiple challenges in terms of university closure, online classes, campus closures, border closures, quarantine, and isolation procedures (Marinoni & Land, 2020).
Despite the temporary closure of educational institutions, teaching and learning activities should continue since the academic calendar requires that the syllabus be completed by the given date. Hence, the pandemic urges both lecturers and students to look for new ways to ensure syllabus completion. The use of multiple online platforms as ways to ensure teaching and learning continues became a lifeline for most lecturers. The platform chosen usually can be accessed either synchronously or asynchronously, with relatively free or almost free access. This is to ensure learning activities can be continued anytime and anywhere. Hence providing the students the freedom to conduct their learning time (Dhawan, 2020).
Internet technologies have greatly influenced the learning idea, particularly in higher education, where the conventional classroom has been transformed into integrated blended learning, online learning, or Open-Distance Learning (ODL). The trends lead students to numerous potentials while also teaching them to be self-directed, autonomous, and lifelong learners (Singh et al., 2005). A bibliometric analysis conducted by (Wahid et al., 2020), resulted that the development of online MOOC courses is dominated in the fields of computer sciences, social sciences, and Engineering which top three countries where the study originated were the United States (23.03%), followed by China (14.69%) and Spain (11.61%). The most important factors during the integration from traditional to online learning programs are the students' characteristics, course content, and the learning context (Smart & Cappel, 2006). In online learning courses, the different students' cohorts give different expectations which are most influenced by their mode of study, and their perceptions of staff engagements, and proactive management gives a positive impact on the success of online learning (Jerry et al., 2008). A study by (Fedynich et al., 2015), suggested that the instructor's responsibilities were the most crucial in gaining student satisfaction since strong engagements were necessary to providing adequate facilities and appealing instructional design.
Nowadays, the curriculum system and teaching method in higher education use the OBE paradigm to achieve the desired output, skill, knowledge, and behavior of its designed program (Spady & Marshall, 1991). The curriculum design in individual courses focuses on course alignment, teaching technique, instructional process, and evaluation process. The OBE concept uses student-centered learning, which necessitates educators regularly measuring the Course Learning Outcome (CLO) and Program Outcome (PO) (Mohamed et al., 2010). The Universiti Teknologi MARA (UiTM) comprised OBE in its curriculum design, particularly in all engineering disciplines due to the demand for high-quality outputs and requirements set by the Engineering Technology Accreditation Council (ETAC).
UiTM offers outstanding platforms and tools to help in the OBE implementation. The Future platform is used by all UiTM users which this platform synchronizes the database of all registered students and instructors which makes the ODL classes more manageable. In addition, the Future platform can conduct an online assessment (quizzes and tests), as well as perform the Entrance-Exit Survey (EES) and Student Feedback Online (SUFO) at the end of each course. Besides, particularly in Engineering courses, various tools have been developed and upgraded from a simple spreadsheet to an advanced system to measure and analyze the CLO and PO achievement, such as OBE-Anas produced by (Idris et al., 2017) and CO-PO versus FKE used by (Mansor et al., 2008).
In OBE, the survey method is most suited for gathering student's perceptions of their perceived skills rather than real skills that may be tested through the assessment procedure. However, the online survey tool is a direct evaluation since multiple choice, rank order, grouping, and peer review questions may assist in determining student success (Wright et al., 2016). The study (Brennan & Hugo, 2016) uses a survey method to obtain the engineering graduate's self-efficacy based on the twelve graduate attributes provided by the Canadian Engineering Accreditation Board (CEAB) (Brennan & Hugo, 2016). Many institutions recognized the survey strategy as one of the efficient methods to access the program outcome to get insight into the curriculum's strengths and deficiencies in terms of student performance and satisfaction (Othman et al., 2011;Milne et al., 2014;Abedin et al., 2014). Indeed, the survey method is also the best practice to obtain student attainment and satisfaction as studied has been studied by (Mohamad et al., 2021;Ng et al., 2016).
The pandemic has opened a significant window of analyzing the effectiveness of online education in UiTM and if this method of teaching is suitable to be implemented via hybrid of the traditional classroom, hence enhancing the flexibility of the syllabus. Thus, this paper aims to observe and compare the quality of teaching and learning in different teaching modes i.e., ODL, face-to-face (F2F) and hybrid. This study intends to show different perspectives of teaching and learning from learning outcome of the course design and student's perception of the course during lockdown, restricted movement, and normal face-to-face post-pandemic classroom sessions. The learning outcome to be compared is the results of specific program outcomes and course outcomes set by the outcome-based education (OBE) framework as well as student's perception and feedback online system (SUFO) being implemented about Basic Communication Engineering (ECM241). The next section first elaborates on the mechanism of OBE execution in UiTM and then continues with its application in ECM241 subject particularly.

OBE Implementation in UiTM
OBE is one of the most famous ways in assessing the performance of students especially in higher educational level. OBE depends on a shift in focus from inputs to outcomes and on greater accountability for results (Bakar & Rosbi, 2019). The emphasis of outcome of the teaching instead of the material to teach is the key to OBE. In hindsight, OBE provides a platform to evaluate whether the student has acquired the knowledge they need to against new problems that are far more challenging and become competent worker once they leave the institution (Nakkeeran et al., 2018).
UiTM has lined out a clear OBE plan for an educational process that focuses on what students can do or the qualities they should develop after they are taught. This involves the restructuring of curriculum, assessment, and reporting practices in education to reflect the achievement of high order learning and mastery rather than accumulation of course credits. The engineering faculty requires both structures and curriculum to be designed to achieve those capabilities or qualities. It is also compulsory to report to the students and the lecturer of the students' performance after they have learnt the required skills and content. This is done via twice on each semester with a meeting between students and their academic advisor.
The OBE implementation process encompasses the establishment of Programme Educational Objectives (PEOs), followed by Programme Outcomes (POs), designing curriculum, teaching and learning (T&L) methods, assessment, continual quality improvement (CQI) and monitoring. PEOs are formulated in line with institutional mission statement and stakeholders' interests. It also addresses the graduate attainment within 3 to 5 years after their graduation. Programme Outcomes (POs), which consist of abilities to be attained by students before they graduate, are formulated based on the PEOs. POs address knowledge, skills, and attributes to be attained by students. Figure 1 below shows the relationship between lesson topics, CO, PO and PEO. The topics lead to lesson outcomes. Next, the group/individual lesson outcomes lead to course outcome (CO). After that, course outcomes must relate to programme outcomes (PO). Lastly programme outcomes must address the programme objectives (PEO) in answering a question on "What kind of "engineer" are produced?''.

A. Basic Communication Engineering (ECM241)
ECM241 is a subject made compulsories for all students undertaking a Diploma of Electrical Engineering in UiTM. This subject is offered in the third semester and consists of several COs and POs attainment. Justifiably in third semester students have attained strong mathematics and physics skills that enable them to conquer the COs and POs adjust to this subject. The COs mapped to ECM241 are: • CO1: Apply the basic knowledge of communication practices and transmission processes using relevant sketches and practical methods.
• CO2: Construct the proper waveform and spectrum of analog and digital transmission techniques based on the applied modulation or multiplexing.
• CO3: Explain in written form the basic elements, methods, and practical applications of communication system with appropriate diagrams There are 12 POs tested based on OBE for the Diploma of Electrical Engineering set by the UiTM. However not all PO will be tested on each subject. Hence the PO is mapped to certain subjects but at the end of 6 semesters the students have achieved all 12 POs. Here are the POs mapped to ECM241: • PO1: Apply knowledge of mathematics, natural science, engineering fundamentals and an engineering specialization to wide practical procedures and practices. • PO2: Identify and analyze well-defined engineering problems reaching substantiated conclusions using codified methods of analysis specific to their field of activity. • PO5: Apply appropriate techniques, resources, and modern engineering and IT tools to well-defined engineering problems, with an awareness of the limitations.
Organization of syllabus content is divided by weeks and all assessment to the subject is as follows.

B. Method of assessment on the mapped POs and COs.
The course introduces the basic concept of communication systems. It describes the basic implementation of communication systems technology. Among the subtopics on the course are basic concepts of digital transmission, modulation and multiplexing in communication system. The emphasis will be on Pulse Code Modulation (PCM), information theory and coding. The subject also deals with basic concepts of optical fiber, optical waveguides, optical cables, optical sources, couplers, and photodetectors. It also describes the power link budget, multiplexing, networking, and fiber loops. Optical test equipment and different measurements in optical fiber link are described. The learning process in outcome-basededucation is based on taxonomy level. The taxonomy level in functioning to make sure testing and assessment methods are appropriate to measure students understanding in achieving the outcomes. The assessment tools follow the domains of learning which are the cognitive, affective, and psychomotor. The domains are different components of learning outcome that are intangible of human capabilities to learn new knowledge. Since the subject is offered at the lower part of the program, it is known that the strength of complex analysis or synthesizing among the students are less developed. Hence the assessment is restricted to cognitive questions and some psychomotor skills which to be embarked during laboratory sessions. The POs are chosen to be mapped to the subject which are PO1, PO2 and PO5. PO1 and PO2 test the cognitive ability of the student while PO5 assess the psychomotor skills of multiple experiments run on the lab.

C. The COs Assessment Mapped to PO
PO1 is the ability for the student to apply knowledge of mathematics, natural science, English fundamentals and any key specialization to the practical procedure and practice. Hence the CO intendent the student to apply the basic knowledge of communication practices and transmission processes using relevant sketches and practical methods. For example, during test 1 the student is asked to explain the process of analogue and digital transmission of data transfer. Therefore, it is recognized as the assessment of CO1 since the student applied their knowledge on engineering fundamentals. The following table describes examples of assessment: To design appropriate questions with suitable verbs to increase clarity; a bloom's taxonomy is used. The bloom's taxonomy differentiates and classifies verbs into observable knowledge, abilities, attitudes, and abilities. Hence the hypothesis is predicated on the notion that there are levels of visible activities that signal brain activity (cognitive activity). The first level of taxonomy is remembering which includes student are examines to recall facts and basic concept. Example on table 1.1 depicts the type of questions normally asked on cognitive assessment. Based on the example stated, to answer question 1, the student must apply the knowledge of electronic communication which includes the knowledge of engineering aspect of FM communication. The same can be applied to question 2, whereby student needs to illustrate the shape of AM signal. The last example required students to apply theoretical knowledge of engineering aspects of both satellite and microwave communication. 1. An AM signal with a modulation index of 0.8 and carrier voltage of 80V is transmitted for an AM radio broadcasting. The carrier frequency is 600 kHz and modulating frequency is 50 kHz. If the carrier power is 2kW, determine the following: (Marks 12) i) modulating voltage ii) minimum and maximum voltage iii) lower sideband frequency iv) upper sideband frequency v) total transmitted power vi) Sketch and label all values of the AM spectrum.

vii) Differentiate between the Double Sideband Suppressed Carrier (DSBSC) and Single Sideband (SSB) in terms of frequency domain display and application
2. An audio signal with 10 kHz frequency and 20 Vpp amplitude is to be transmitted using PCM system. The signal is sampled at 15% higher than the minimum sampling frequency and the number of bits per sample is 8 bits. Compute the: (Marks 10) i) sampling frequency ii) quantization level iii) resolution iv) maximum quantization error v) transmission bit rate

FM signal is given in trigonometric form as below: (Marks 14) VFM(t)=20cos(200π×106t +2sin6π×103t
i) Predict the number of sets of significant sidebands. ii) Compute the carrier swing. iii) Compute the maximum and minimum frequencies of FM signal. iv) Compute the bandwidth by using Bessel Function Table and Carson's rule. v) Compute the average power of FM if the resistor load is 50 Q. vi) Compute all sidebands frequency. vii) Next sketch the frequency spectrum for the FM signal complete with its relative amplitude and frequency of sidebands Method: Quiz, test, and final examination. Table 1.2 shows the mapping of the second CO and PO. The CO2 basically examine the ability of the student to understands and analyses the AM, FM and modulation technique which specifically falls on the field of digital communication engineering. Here the student is expected to apply mathematic skills parallel to their engineering theories knowledge. The marks allocated for each question are also significantly increased compared to previous CO, PO mapping. The reasons are due to the complex nature of the theories applied to mathematical skills. The CO levels require the student to understand and test their understanding of complex engineering problems occurred naturally in the fields. Conclusion: Based on your simulation experiment, write the findings in your own words.
Method: Practical Test Table 1.3 shows the last CO, PO mapping on the subject which are the CO3 and PO5. The CO attainment required the student to perform practical test and proceed to write the scientific reports pertaining to the findings during the practical test. Each student is assessed twice, i.e., once during the practical test to measure the student's practical competencies and the second assessment is during report writing. PO5 required students to evaluate, justify, and reason any value that they get during lab sessions to produce good technical reports. Whenever their hypothesis is not concluded by their results, the students are required to provide suitable judgement on materials and method used.

Methodology A. Sample
Due to the different modes of Teaching and Learning (T&L) delivery, this paper is assessing the quality of the OBE implementation in the basic communication engineering course in the Department of Electrical Engineering (EE), UiTM, Johor, Malaysia by using a comparative analysis between the Face-to-Face (F2F), Online-Distance-Learning (ODL) and Hybrid modes. Data from these three (3) mainstream semesters were recorded and analyzed. Table 2.1 tabulates and characterizes the sample distributions for this analysis work.

B. Instruments
As presented in the previous section, the OBE implementation in UiTM is done using various instruments. Despite the different mode, the same OBE procedures are applied for all semesters: F2F, ODL and Hybrid. The first and most important instruments are related to the course assessments in measuring the COs and POs. During the first week of lectures, students are introduced to the course lesson plan, materials, and platform to be conducted. Throughout each semester, all assessments: quiz/assignment, test, practical, presentation are handed out to students in particular week according to the course lesson plan that prepared by the course coordinator. Students are assessed for their final examination after finishing the fourteen (14) weeks of lectures. Test, practical and final examination questions are standardized to all classes following the same rubrics criteria. However, there are some flexibilities in the quiz/assignment and presentation assessments. All the assessment marks achieved for each student reflect the course COs and POs attainment. As one of the EE core courses, the subject ECM241 which consist of five (5) chapters and dominated by analog and digital modulation techniques, has been chosen for this case study. Table 2.2 summarizes the assessments distributions for this course. The second instrument is by means of survey. Regarding the course content, two (2) stages of surveys i.e., 1) entry survey and 2) exit survey, have been performed. On the first meeting for that semester, the lecturers distributed the entry survey to obtain an early expectation of the students' readiness and limitations on the course content. At the end of the semester, exit survey are conducted to measure the students' knowledge level on the course content after undergoing the T & L activities. The point differences between these entrance-exit surveys show the improvement of the students' knowledge on this course.
Another survey named Student Feedback Online (SuFO) is also performed with standardized questions for all UiTM courses to evaluate the whole factors contributing to the course T&L effectiveness. The questions asked in this survey reflect the students' impression about course, feedback on the lecturer professionalism, effective T&L activities, and conducive classroom facilities. All the mentioned surveys have been conducted using the established UiTM uFuture online application which needs a login from student to access the course materials and perform required assessments. The survey data are saved in the uFuture database which can be accessed by all course's instructors. The Microsoft Excel spreadsheet has been used to analyze the collected results. Even though the results from these surveys are not counted into the COs and POs attainment of the course, the data are used to give feedback to the UiTM management and stakeholders for future improvement actions. Accordingly, the next section presents the discussion of results from all methodologies performed in achieving the paper's objectives.

Results and Analysis
PO attainment for three (3) mainstream semesters are tabulated in Table 3 and graphically presented in Figure 1, comparing the achievement between F2F, ODL and Hybrid modes. F2F semester was just before the Covid-19 hit. ODL was conducted during the pandemic with still high cases. Hybrid mode started when most of the students and staff have completed their vaccinations. From these data, it is clearly seen that for PO1 which focusing on students' cognitive capabilities, Hybrid mode has the highest attainment. During this semester, the PO2: Problem Analysis was assessed for the first time in this course. PO5 which focuses on the psychomotor through the 30% of practical laboratory also shown a significant attainment. For F2F mode, the 60 % Final Examination assessment was one of the biggest factors to produce the lowest PO1 attainment. However, for PO5 and PO10 (Communicationpresentation assessment) they are all above the 65 % of target. ODL mode also produced a good PO attainment but with lower PO1, which is most probably due to lessen T&L activities conducted during ODL to reduce the burden of students' screen time with the computer. In overall, students were more preferred to have a mixture on the T&L activities as highest average attainment shown by Hybrid delivery mode.  The quality of the OBE implementations in this course were also evaluated from the entrance and exit survey results. These surveys are performed at the beginning and end of each semester respectively with same survey questions given to students. The point difference between these entrance-exit questions shows the students' knowledge level on the course content and directly reflects the effectiveness of the T&L activities. There are nine (9) questions in this survey with a minimum of 1 and maximum 5 mark allocated for each question. As for example, Figure 2 shows the point difference obtained for question 1 which "I can explain the concept of modulation". It is significantly noticed that most of the students obtained 3 points of improvement mark. The Hybrid mode also shows significant results as compared to ODL and F2F. During ODL and F2F, there are students who did not achieve the objective of the course content by obtaining the 0 (9.86 %) and -1 (3.52 %) point for this entrance-exit survey. Figure 3 illustrates the overall perception of the course content from the entrance-exit survey. In overall, the questions obtained 3-point of improvement mark. Again, the Hybrid mode has the highest perception on the course content which the frequency of 3 and 4 points are 56.27 % and 29.13 % respectively. While entrance and exit survey focused on the course content, SuFO survey is conducted to get an overall evaluation on factors which contributing to the quality of T&L activities in the UiTM OBE system. The results from this survey are summarized in Figure 4 and 5. From Figure  4, majority of students were 'highly agreed' or 'agreed' that the course content is impressive, high professionalism shown by lecturers, effective T&L activities conducted and adequate and conducive T&L facilities available. Only a noticeable number of students (1.88 %) 'did not agree' with the SuFO statements. Almost similar performance was recorded for all F2F, ODL and Hybrid mode of delivery. In overall, students gave good feedback during the ODL mode since the highest percentage recorded for all criteria as shown in Figure 5, followed by Hybrid and the least, F2F. However, the least percentage (85.84 %) recorded for the Hybrid mode in feedback from students on the Facilities. Students were expecting more conducive facilities prepared by UiTM. Actually, this batch of students are entering the campus for the first time after 2 semesters having the ODL lessons. Thus, their expectations on the campus facilities (classroom, laboratory, hostel, internet etc.) are reasonably high. Thus, to have a mixture on the T&L activities delivery, the facilities factor is one of the biggest rooms for improvement.

Conclusion
This paper aims to study the relation of student's achievement with respect to method of T&L in UiTM for subject of ECM241. The achievement scores are set to be the CO-PO, the entrance and exit survey, and the SuFO survey. The paper first discussed the OBE implementation used in UiTM and explained in great details the CO and PO used to assess the student. Next, the same CO and PO are used in F2F, hybrid, and online T&L to measure the performance of the students. Based on the results, the hybrid classroom scored significantly higher compared to F2F and online classes. The flexibility of having a hybrid online class with a face-to-face class indeed increased the overall score of CO and PO measured. Besides, any unanswered question experienced due to technical difficulties during an online class is solved during a face-to-face session. Based on the entrance and exit surveys conducted, students also acknowledge that the hybrid classroom did give them the most improvement in tackling the subject when compared to other types of T&L. Lastly, the SuFO is a survey assessing the overall evaluation of factors contributing to the quality of T&L activities that also shows similar trends that favour the hybrid mode. However, the student rates the improvement to facilities as the most area to be improved. Hence, the improvement of technical facilities is crucially needed to cater to the increased volume of online classes and educational activities.