Challenges in Cognitive Domain Performance for Laboratory-Embedded Electrical Engineering Courses

In Malaysia, the Ministry of Higher Education (MoHE) and Engineering Technology Council (ETAC) have established a standard requirement for all engineering diploma programmes that include evaluating the effectiveness of the Programme Outcome (PO) to ensure the quality education of the programme. This paper aims to investigate the performance of the PO attainment for fundamental laboratory-embedded courses of the Diploma in Electrical Engineering (Electronic) programme based on the new assessment as stated in the ETAC Standard 2022. The samples of PO achievement were taken from three batches or cohorts before and after the implementation of the ETAC Standard: Oct 2019, Oct 2020, and Oct 2021. The analysis data involves focusing on the performance of two domains which are cognitive and psychomotor. The two-way ANOVA (Analysis of Variance) test was carried out using SPSS software for analyzing the data. The results reveal that while students perform well in psychomotor attainment, cognitive attainment gradually deteriorates over time. This finding suggests the need to revise the curriculum, syllabus, and assessment design to enhance the PO attainment to achieve the desired PO that is set by the council. Thus, the institutions can produce competent and proficient graduates with conceptual thinking and practical skills.


Introduction
Universities have a vital role in engineering disciplines, providing the necessary knowledge, skills, and training to produce highly skilled engineers.Nowadays, the growth and development of a nation or country depend on the technological and knowledge advancements offered by the engineering field (Al-Sheeb et al., 2019;Wall, 2010).To promote high-quality engineering education, the engineering programmes are assessed according to strict requirement standards by an accreditation body comprising the Engineering Accreditation Council (EAC), Engineering Technology Accreditation Council (ETAC), and Vol 13, Issue 3, (2024) E- ISSN: 2226-6348 To Link this Article: http://dx.doi.org/10.6007/IJARPED/v13-i3/21478DOI:10.6007/IJARPED/v13-i3/21478 Published Online: 28 May 2024 Malaysian Qualification Register (MQR), to ensure that universities meet the established benchmarks (Engineering Programme Accreditation Standard 2020, 2020; Engineering Technician Education Programme Accreditation Standard 2020, 2020; Malaysian Qualifications Register: Accreditation, 2010).All organisations play an important role in encouraging both public and private institutions to produce outstanding engineering graduates who possess the knowledge and skills required to thrive in their fields ETAC implemented a new evaluation system intended for aligning assessments to programme outcomes (PO).However, it is still unclear factor how the system could impact students' PO attainments.According to the qualifying requirements outlined by ETAC in the new assessment system, the curriculum must have a minimum of 90 Student Learning Time (SLT) credit units.At least 60 SLT credit units shall be engineering or engineering technology courses, and a minimum of 30 SLT credit units shall be allocated for practice-oriented components in the technical and specialist areas (Engineering Technician Education Programme Accreditation Standard 2020, 2020).Consequently, psychomotor skills have been incorporated into most engineering courses.Meaning, 30% of SLT must be practice-oriented in an engineering programme.Therefore, this paper aims to analyse a comparative study of the non-ETAC (with 10% practice-oriented) and ETAC syllabuses (with 25~30% practiceoriented) based on the achievement of the PO on cognitive and psychomotor attainments of fundamental laboratory-embedded courses in the Diploma in Electrical Engineering (Electronic).
All engineering programmes employ Bloom's Taxonomy as one of the primary assessment components in outcome-based education (OBE) implementation, which aims to determine the appropriate level and ensure the effectiveness of the course outcome.Bloom's taxonomy is a taxonomy of learning.It provides a framework for categorizing the complexity and specificity of a course outcome, course assessment, and course evaluation (Newton et al., 2020).The framework consists of a set of three hierarchical models: cognitive, psychomotor, and affective learning domains, as illustrated in Figure 1.The assessment given to students should align with their understanding and knowledge, ensuring that student achievements are measurable, verifiable, and capable of being effectively improved.

Cognitive and Psychomotor domains in Electrical Engineering
The connection that exists between cognitive and psychomotor domains specifically in the electrical engineering discipline focuses on the development of intellectual thinking and practical skills.The development of cognitive skills contained the conceptual theory for instance, the understanding of fundamental principles is learned before being applied in practical settings such as workshops, trainings, or laboratories (Mukhtar et al., 2019).Interrelation between both domains is crucial, particularly for electrical engineering students, as it allows them to effectively implement their learning in real-world working scenarios.
There is a positive correlation between student performance in cognitive assessments and their performances in psychomotor assignments as reported by Noor et al (2020) and Zanal et al. (2023).There is an obvious connection between students' comprehension of the theory and their proficiency in practical skills as their understanding improved, so did their practical skills (Noor et al. 2020).A study conducted by Mertayasa et al (2021) also found similar findings: cognitive styles can impact students' performance on tasks.The way students process information (cognitive skills) directly impacts their ability to perform physical (psychomotor) tasks.
In the following sections, the course versus PO based on fundamental engineering courses and POs comparison between Non-ETAC and ETAC is explained.The fundamental courses are the main constraint in PO achievement.Then, the POs and course attainment were analyzed.Finally, the factors that challenge the achievements will be discussed to illustrate the importance of syllabus design for sustainable development in the higher education sector.

Methodology A. Programme Background
The data used in this study were obtained from PO attainments of the Diploma in Electrical Engineering (Electronic) programme at Universiti Teknologi MARA Cawangan Pulau Pinang, Malaysia.The programme mode is full-time with the duration of six (6) semesters in three (3) years.
Ten (10) out of twenty (20) fundamental electrical engineering courses were selected for comparison analysis.It is important to note that these are the fundamental laboratoryembedded engineering courses that significantly contributed to the cognitive and psychomotor domains for the ETAC cohorts.The other ten (10) courses consist of laboratory courses, programming courses, industrial training, final-year project and elective courses that may vary depending on the student's choice.
Figure 2 illustrates the courses to PO mapping for the ETAC and non-ETAC cohorts.Only four (4) out of (10) selected courses have both cognitive and psychomotor assessments before ETAC implementation with a maximum 10% contribution for the psychomotor domain.Afterwards, all courses for the ETAC cohorts have a psychomotor assessment in the range of 25% to 30%, where the practice-oriented component has become the primary requirement.

Programme Outcome (PO) Comparison
The POs that relate to cognitive and psychomotor achievement for the non-ETAC and ETAC syllabuses are illustrated in Figure 3.In the ETAC syllabus, the PO statements were more focused and specific.

B. Statistical Data
Data used in the analysis are from PO attainments for cognitive and psychomotor learning domains.Three (3) cohorts of students were compared.The October 2019 cohort implemented the non-ETAC syllabus (non-ETAC), while October 2020 was the first cohort that fulfilled the ETAC syllabus (ETAC#1).In addition, a comparison of PO attainment was also made for the second cohort that applied ETAC syllabus which is October 2021 (ETAC#2).The details of the data samples are shown in Figure 4

Statistical Analysis
The statistical analysis of students' marks was carried out using the two-way ANOVA and Tukey HSD (Honestly Significant Difference) post hoc tests.Hypothesis testing for a twoway ANOVA was performed; null hypothesis testing will be rejected if the p-value or alpha is less than 0.05.

Research Hypothesis
The three null hypotheses of the two-way ANOVA are defined as follows: H0: There is no difference in the level of student achievement for any learning domain (cognitive and psychomotor) H0: There is no difference in the level of student achievement for any type of syllabus (non-ETAC and ETAC) H0: The effect of the independent variable (learning domains) does not depend on the effect of the other independent variable (syllabus)

Result, Analysis and Discussion
Programme Outcomes Attainment There are ten (10) fundamental courses, as mentioned in Figure 2, were used as data to analyse the PO attainments for cognitive and psychomotor domains.For the non-ETAC syllabus, there are five (5) courses consisting of both cognitive and psychomotor domains, while the other five (5) courses consist of only the cognitive domain.Meanwhile, for the ETAC syllabus, all ten (10) fundamental courses are laboratory-embedded courses, which consist of both cognitive and psychomotor domains.The motivation behind transitioning from the non-ETAC syllabus to the ETAC syllabus is to ensure students achieve a balance between theoretical/knowledge skills and practical skills.
Results for the PO attainments in both domains, which are cognitive and psychomotor for ten (10) fundamental courses were analyzed and shown in Figure 5.The Key Performance Indicator (KPI) of 50% was set by the faculty as an element for monitoring and improvement purposes.The psychomotor scores for all cohorts exceeded the KPI, while the cognitive scores did not meet expectations.For the non-ETAC syllabus, the achievement for course attainments in cognitive was slightly higher than KPI, and only one (1) course (10%) was below KPI.Meanwhile, for ETAC#1 syllabus, four (4) courses (40%) were below KPI in the cognitive domain.The cognitive domain was worsening whereby five (5) courses (50%) below KPI for ETAC#2 syllabus.
Figure 5. Fundamental courses attainment in cognitive and psychomotor domains.
The average attainment in cognitive and psychomotor domains is summarized in Figure 6.Referring to the cognitive achievement for course attainment for non-ETAC syllabus, it shows that students' understanding of the fundamental courses was good with an average of 68.25%, which is 18.25% higher than the KPI value.When the ETAC#1 syllabus was introduced, the percentage of psychomotor domain increased significantly from an average of 78.2% to 83.68%.The improvement in the psychomotor domain is due to the individual assessment that was carried out during the laboratory session.From the cognitive domain perspective, a drastic reduction was observed from an average of 68.25% to 54.73%.A similar trend was observed for ETAC#2, where the percentage of cognitive domains was reduced (≈1.85%),from 57.73% to 52.9%.However, cognitive performance was still maintained at the range of  2020), which found the students performed better in psychomotor and affective skills than in cognitive skills.
The error bars are used as the graphical representation of data variability, which can show the statistical significance by visual comparison.The overall performance on cognitive and psychomotor domains from Figure 5 is then translated into error bars as shown in Figure 6.As summaries, the standard deviation error bars in Figure 6 shows that both cohorts of the ETAC syllabus (ETAC#1 and ETAC#2) did not overlap between cognitive and psychomotor attainments, which means there were significant differences between the attainments.Meanwhile, for non-ETAC syllabus, the error bars were crossed or overlapped, meaning that there is no significant difference between cognitive and psychomotor.Figure 6.Average programme attainment in the cognitive and psychomotor domains between cohorts (using non-ETAC and ETAC syllabus).

Statistical Analysis Results
The statistical analysis was performed to test the factors that influence the students' achievement which includes learning domains (cognitive, and psychomotor) and syllabus (non-ETAC, and ETAC).
The significance values obtained from Levene's test are greater than 0.05, for both cognitive and psychomotor marks, indicating variances were homogeneous for marks.
Next, a two-way ANOVA was conducted on marks.The learning domain and syllabus were predictors.The result of the two-way ANOVA investigates whether either of the two independent variables (learning domain, syllabus) or their interaction (learning domain and In SPSS, only variables with more than three groups can perform an additional post-hoc test.Learning domain is a variable that has two groups, cognitive and psychomotor.Thus, this variable cannot undergo a post hoc test for further investigation.On the other hand, the General Linear Model (GLM) outcome can demonstrate how the variables are different and dominating.
The result of the Estimate Marginal Means (EMM) between the learning domain and syllabus is displayed in Table 2.The EMM results show that psychomotor marks were not statistically significant (p>0.05),while cognitive marks for both ETAC and non-ETAC syllabuses were significant (p<0.05).Thus, it can be concluded that only cognitive marks are affected by syllabus differences.Zanal et al ( 2023) reported a similar conclusion, stating that students' cognitive domain achievement is influenced by the implementation of psychomotor elements and students have better psychomotor skills.Next, a Post Hoc comparison using the Tukey HSD test was conducted on cognitive marks to find out the difference in achievement between cohorts.The test results are shown in Table 3 below.Based on the results, the achievement of cognitive marks for all ETAC cohorts, ETAC#1 and ETAC#2 were in the same group, while non-ETAC was in a different group and had relatively higher marks than the ETAC cohort.This means that there is strong evidence to conclude that non-ETAC and ETAC syllabuses have produced a different level of cognitive achievement levels among engineering students (Rohaya et al., 2020).The main contributing factors that lead to poor performance in cognitive attainment in the ETAC syllabus compared to non-ETAC might be due to the unbalanced syllabus design for assessment distribution between cognitive and psychomotor.In the ETAC syllabus (ETAC#1 and ETAC#2) the distribution of assessments in the most fundamental laboratory-embedded courses consists of 70% evaluating the cognitive domain (Test 20% and Final Examination 50%) and 30% evaluating the psychomotor domain (Laboratory 20% and Practical Test 10%).The first factor, the high percentage of assessing the cognitive domain ~70% is not consistent with the main objective of developing the ETAC syllabus.The second factor contributing to poor performance in the cognitive domain in ETAC syllabus is the large amount of content of chapters and sub-chapters that need to be learned and understood to answer cognitive domain questions in the examination.Furthermore, the history of the coronavirus disease (COVID-19) pandemic in 2020~2021 has had an impact on online learning and contributed to the decline of students' cognitive skills (Sihab et al., 2023).

Conclusion
This paper investigates the course performance in the cognitive and psychomotor domains between the non-ETAC and ETAC syllabus that were implemented in the Diploma in Electrical Engineering (Electronic) programme.The overall result indicates that by implementing the ETAC syllabus, drastic improvement was observed in the psychomotor domain, which means students are more skilled in solving practical problems.Nevertheless, in the cognitive domain, it has become a very big issue where the student is not well-versed in the fundamentals and theories related to electrical engineering.The cognitive domain plays an important role in the development of psychomotor abilities among students.Strengthening the cognitive domain can enhance the student's ability to develop profound understanding and strong technical skills.A lack of balance between both domains can hinder the overall skill development of engineering graduates.
This study provides useful information to the universities on how to guide the implementation of a more balanced approach on how to effectively align curriculum and practical skills to achieve the Programme Educational Objective (PEO) and produce graduates with strong conceptual bases and practical skills to be proficient electrical technicians or engineers.Below are some recommendations to consider when developing a new programme or revising the existing curriculum of the electrical engineering education programme at the diploma level.
1) The development of early engineering courses (basic courses) needs to be taught in the first semester before laboratory-embedded courses are offered.This is to enhance the student's awareness of the importance of high-level cognitive skills.
2) Enhance collaborations with industry by providing more hands-on training opportunities to help students with theoretical knowledge in real-world scenarios, thus promoting the integration of cognitive and psychomotor skills.
3) Set a high priority on developing cognitive abilities, such as analysis, synthesis, and evaluation, in the course assessment and learning activities to promote a comprehensive understanding of concepts and improve critical thinking skills.
4) Continuing and expanding the Science, Technology, Engineering and Mathematics (STEM) education at the school level is recommended to attract science and non-science students to better understand the field of electrical engineering.This can enhance the cognitive domain at an early stage.

Figure 2 .
Figure 2. Course vs Programme Outcome (PO) for fundamental engineering courses.

Figure 3 .
Figure 3. PO statement comparison between non-ETAC Syllabus and ETAC Syllabus for cognitive and psychomotor learning domains. below.

Figure 4 .
Figure 4. Background of statistical data.
Noor et al. (2020), and Alias et al. (theless, the percentage of the psychomotor domain increased (by 1.1%) from 83.68% to 84.76% and continued in excellent range.Reduction in the cognitive domain might be due to the assessment being solely dependent on the test and final examinations.Therefore, only a small percentage of students were able to cope with it and achieved good results.This finding is consistent withNoor et al. (2020), and Alias et al. ( ) are statistically significant.Table1shows the result of the Test of Between-Subjects Effects.As can be seen in the table, the learning domain was a statistically significant main effect of marks [F (1,49) = 81.867,p<.001].However, the syllabus did not affect the marks [F (2,49) = 1.022, p=.368].There was a significant interaction between the learning domain and syllabus [F (2,49) = 6.108, p<.004].Two (2) out of three (3) null hypotheses were rejected: there was a difference in the means of learning domain (cognitive and psychomotor), and there was interaction between learning domain (cognitive and psychomotor) and syllabus (non-ETAC and ETAC). syllabus

Table 2
Result for Estimated Marginal Means

Table 3
Tukey HSD Post Hoc Tests for Cognitive Marks