Attainment of Program Outcomes under Psychomotor Domain for Civil Engineering Undergraduate Students

Psychomotor domain involves physical movement, coordination and applied motor-skill areas, where substantial practices are needed in order to develop these skills overtime. Assessment of psychomotor domain can be carried out in aspects of speed, precision, distance, procedures. The attainment for program outcomes under psychomotor domain can be measured by using direct and explicit assessment. Majority of the lecturers lack the necessary practical experience to relate with fundamentals in order to practice and to give examples resulted in the students not being able to relate theory with practice and apply their knowledge to solve practical problems. This paper identifies the important attributes of students in attaining psychomotor skills in laboratory courses using a program outcome (PO) related to investigation aspects. Majority of the respondents indicated that they have acquired the necessary skills under psychomotor domain related to problem identification, procedure determination, apparatus usage, data collection, results interpretation and proposition of viable solutions. Direct attainment shows that the students passed the PO designed for the laboratory courses. Using relative important index (RII), important factors influencing students’ attainment on psychomotor skills are the condition of equipment in the laboratories, lecturers’ clear instructions before the student conduct experiment and the conducive environment to conduct experiment. A conducive environment with adequate equipment and supported by lecturers to carry out effective investigation give opportunity to the students to acquire and improve their psychomotor skills. This study is important to understand psychomotor elements attained by the students and how assessment is carried out effectively.


Introduction
In recent years, there has been an increasing interest regarding the accreditation process to measure graduate attributes in engineering program at Institute of Higher Learnings (IHLs). Accrediting body responsible to conduct the accreditation process and recognition award for engineering program in Malaysia is the Engineering Accreditation Council (EAC), Board of Engineers Malaysia (BEM). The accreditation process is essential to ensure quality future engineers with high attributes that can solve problems ethically and reducing risks in any solutions responsibly. Engineering education in Malaysia has been encouraged to implement education system called as Outcome Based Education (OBE) in curriculum development (Johari et al., 2005). Cognitive, affective and psychomotor are important components in the POs and play key roles in the aspects that will be evaluated as learning objectives' achievement (Zainudin, Ahmad, Ali & Zainal, 2012). Application of mathematical and natural sciences, knowledge, technology and techniques are the elements that need to be mastered by graduates in order to be a good engineer. Besides that, professional competency profile is necessary as it is a set of attributes which cannot be achieved without the trilogy taxonomy (International Engineering Alliance, 2013). For example, ability to solve complex engineering problem cannot be solved without basic knowledge (cognitive), instrumentation skills (psychomotor) and ability to take responsibility (affective). Markle and Banion (2014) opined that IHLs need to demonstrate the combination between cognitive and affective skills. For example, in communication, students will need to have the required knowledge and ability to read, speak and listen effectively. Without cognitive and affective skills, the information would not be effectively disseminate to the audience and will lead to miscommunication. So far, most researches paid more attention on cognitive and affective domain rather than psychomotor domain due to difficulty to assess the practical performance. Students' perception and ratings about the interesting and effective teaching methods was considered as one of the ways to suggest improvements in teaching and learning process (Sajjad, 2011).
Since, there is still insufficient data specifically emphasized the true attainment in graduate attributes for psychomotor domain this paper seeks to seek the students' perceptions on the attainment of the PO related to psychomotor and to identify the factors influencing the student's performance in attaining these attributes.

Literature Review
Outcome Based Education (OBE) adopts Bloom's educational learning theory which comprises of three main learning domains incorporated in the curriculum: the cognitive is on knowledge of and ability to work with information and ideas; affective is on the ability to organise, articulate, and live and work by a coherent value system relevant to the capabilities achieved through education; and psychomotor is on the ability to do and act relevant to the field of study (Ferris & Aziz, 2005). The paradigm of OBE has shifted from teacher-centred to student-centred learning. In order to meet the respective objectives, Program Educational Objectives (PEOs), Program Outcomes (POs) and Course Outcomes (COs) need to be aligned .
Psychomotor domain is related to skills' development which involve manual tasks and physical movement including the operation of the equipment (Rovai, Wighting, Baker, & Grooms, 2009). Psychomotor domain involves physical movement, coordination and the applied motor-skill areas, where substantial practices are needed in order to develop these skills overtime. All these skills, can be measured in the aspects of speed, precision, distance, procedures. The psychomotor skills of the students are commonly referred to physical abilities that are normally developed in the laboratory environment (Baharom, Khoiry, Hamid, Mutalib & Hamzah, 2015). Ferris and Aziz (2005) studied a psychomotor skills extension to Bloom's Taxonomy and presented a hierarchical taxonomy of psychomotor skills and discussed these skills specifically from the viewpoint of the needs of engineers. The psychomotor skills described by them are the practical aspects of the performance of the profession, rather than on the development of detailed physical skills (Ferris & Aziz, 2005). Another researcher, Shaaban (2013) studied on the importance of practical experience in civil engineering education, problems facing practical teaching, and successful practices in practical teaching. Thus, the students can benefit more when engineering courses are taught by instructors that have both academic and practical experience.
Malaysia has adopted the OBE in engineering education in the beginning of year 2000. Faculty of Civil Engineering, Universiti Teknologi MARA (UiTM) has implemented OBE in its Bachelor of Civil Engineering (EC220) program since 2006. One of the POs in the EAC Manual 2017 is as follow: Conduct investigations of complex problem using research-based knowledge and research methods including design of experiments, analysis and interpretation of data and synthesis of information to provide valid conclusions. In order to measure the psychomotor skills of the students, EC220 program has designed its curriculum using this PO as the graduate attribute. The psychomotor skills are evaluated through the subjects that mainly involve laboratory activities.
This study focuses on the practical test as a tool to measure the psychomotor domain. Practical test aims the students to demonstrate their ability to determine the problem to be explored, develop a procedure to investigate the problem, decide what data to collect and interpret the data in order to propose practical solutions. Students can gain knowledge and practical skills from the laboratory experiments and expose them to relevant engineering fields. The EC220 program adopts open ended activities in its laboratory courses, where students are free to develop their own experiments using investigative approach rather than following the prescribed manual guidelines. Implementation of an open-ended laboratory is one of the agreements with Engineering Accreditation Council (EAC) and Accreditation Board for Engineering and Technology (ABET) accreditation criteria requirements (Haron, Mohammad, Sam, Mustaffar & Yatim, 2013).
The emphasized on open ended laboratory has been set as the strength of the program in engineering education, since it give benefit in examining the creativeness and innovativeness, challenging the students on the anticipated level depth and understanding (Bolong et al., 2014). Graduates are expected to have achieved the following learning outcomes: the ability to select and use appropriate equipment, tools and methods; the ability to combine theory and practice to solve engineering problems; the ability to demonstrate understanding of applicable techniques and methods, and their limitations; the ability to demonstrate understanding of the non-technical implications of engineering practice; the ability to demonstrate workshop and laboratory skills (Rao, 2015). Different lab courses have different levels of open-endedness and three general areas that can be made open-ended which are concept, design and analysis and reporting. In concept area, the lecturer only provides the students a task with objectives in order to create this stage open ended. Once the concept of the experiment is in place, then the experiment is designed in accordance with the concept. At the end of the experiment, the lecturer will provide information about the analysis and reporting method thus facilitates the lecturers to reduce the learning for the students. All laboratory courses are depending to the level of openness which developed by Schwab (1962) and Herron (1971). There are four (4) levels of openness which are shown in Table 2.1. The problem, procedure and methods are provided to the students in order to achieve the solutions. The student performs the experiment and compare with the manual given 1 The problem and procedure are provided to the student. The students then interpret the data to propose the possible solutions. 2 The problem is provided to the student. The student need to develop procedures, decide on data collection and interpret data so that possible solutions can be proposed. 3 The student needs to choose the problem, develop procedures, decide on data collection and interpret data so that possible solution can be proposed.
In general, the assessment in laboratory courses are carried out through laboratory reports, presentation and practical test. Rubric is the regular tool that is used to assess psychomotor domain (Atiq & Rahmat, 2011). A sample of rubric that are being used as a tool using practical test have six (6) criteria according to the taxonomy with grading point from 1 to 5 as illustrated in the Table 2.2.
Six (6) levels of criteria based on problem identification, procedures, apparatus, data collection, results interpretation and proposition of viable solutions by the students are adopted. This assessment criteria have been used as the basis in the design of instrument to get the students' opinion on their PO attainment based on the above criteria as explained in the methodology section.

P1 -P2
Identify problem based on the scenario given.
Identify <30% of the problems.

Method
This section describes the research design and approach used to obtain opinion form a target respondents using questionnaires survey as the research instrument. The data was analyzed using descriptive analysis and statistics (reliability, normality, relative important index and inter-correlation analysis). The research design for empirical data in this study consists of the quantitative data obtained from the questionnaire survey. Quantitative study design is a well-structured, specific and has been tested for its validity, reliability and can be explicitly defined and recognized (Kumar, 2011). The questionnaire survey was used to determine the attainment of psychomotor skills and the factors influencing the attainment of the graduate attributes of civil engineering students at UiTM Shah Alam. The population is based on a sampling frame obtained from the Academic Affairs Office of Faculty of Civil Engineering UiTM, Shah Alam based on September 2017 -January 2018, which comprises of total 1231 students. The target respondents of 393 students were in their fourth year (semester 7 onwards) taking Final Year Project and Integrated Design Project courses.
The questionnaires in Section A asked on the respondents' background, in Section B the respondents are required to give their opinion on the factors influencing their performance to attain psychomotor skills and in Section C, acquires their opinion on the their ability to attain attributes related to psychomotor domain. Both sections used same measurement based on the level of agreement using a 5-point Likert scale: 1 -Strongly Disagree; 2-Disagree; 3-Moderately Agree; 4-Agree; 5-Strongly Agree. The influencing factors are statement related to the laboratory environment, equipment condition, equipment adequacy and lecturers' instructions to students in conducting experiment. While the attributes are the statement related to psychomotor domain on the ability of students to identify problem based on a given scenario, to determine correct procedures for investigating the problems, to demonstrate the usage of the apparatus to run the study, to collect data to gather important result, to interpret results leading to significant findings and finally to propose viable solutions to solve problems given.
Reliability analysis with Cronbach's alpha was calculated for the variables measuring the same dimension to provide evidence of reliability. The greater the degree of consistency and stability in an instrument, the greater its reliability where value above 0.70 was considered acceptable (Nunnally & Bernstein, 1978). A graphical (normality plot) and statistical (Skewness and Kurtosis) methods were used for evaluation of normality. Normality tests are used to determine whether a data set is modeled for model distribution. For Skewness and Kurtosis, if either of data value is close to zero, then the data set is normally distributed. Ultimately, determining the Relative Importance Index (RII) is crucial in this study to determine the ranked level of importance through the value of index. It is utilized specially for questionnaires that related to Likert scale. Eq. 3.1 shows the formula of RII that inserted into Microsoft Excel 2016 to calculate the index for sets of items.
Eq. 3.1 Where w is the weighting given to each factor by the respondent, ranging from 1 to 5. For example, 1 = number of respondents for Not Important, 2 = number of respondents for Less Important, 3 = number of respondents for Moderately Important, 4 = number of respondents for Important, 5 = number of respondents for Very Important. A is the highest weight (e.g 5 in the study) and N is the total number of respondents. The Relative Importance Index ranges from 0 to 1.
In this study, an inter-correlation analysis was carried out to describe the strength and direction of the linear relationship between two variables (Pallant, 2011); namely psychomotor PO attainment (dependent) and factors (independent). The analysis determines not only whether a relationship between variables exists, but also the degree of the relationship between them. If the nature of the data is normal thus, a parametric test is used to obtain and interpret a Pearson product-moment correlation coefficient (r), which is presented for interval level variables used in the questionnaire design. The value gives indication of both direction (positive and negative) and the strength of the relationships, taking on values from -1.0 to +1.0. In a positive correlation, the trend in both variables goes in the same direction, whether they increase together or decrease together, vice versa. Cohen (1998) suggests the following guidelines as shown in Table 3.1. In order to validate and support the findings from the survey, the raw marks that reflect the students' attainment of psychomotor skill through laboratory courses are extracted from an OBE assessment system known as myCOPO

Findings and Discussion
Out of 393 target respondents, 127 students have participated in the survey giving a response rate of 32.3%. The profiles of the respondents were analyzed according to semester, gender, age and laboratory courses taken. The results show about 56% are female respondents and the rest are male, 60% of them age between 24 to 27 years old and the rest (40%) age between 20-23 years old. More than 60% of them have taken Environmental Engineering with almost half of them have taken Hydraulics, Structure and Materials, and Geotechnical laboratory courses. The profile shows that the respondents were the final year students with similar percentage distribution based on gender, have taken the required laboratory courses that measure program outcomes under psychomotor domain and majority of them age above 24 years old.
The Cronbach's Alpha for variables obtained through four (4) items of questionnaires imposed to the respondents to identify the factors influencing students' performance on psychomotor skills (Section B) give coefficient of 0.900, while the coefficient for the level of attainment related to psychomotor domain (Section C) through six (6) items is 0.922. The statistics are greater than 0.7 for both sections which indicates that instrument used to measure the variable is reliable.

Respondents' Agreement on their PO Attainment of Psychomotor Skills
This section presents the respondents' level of agreement on their PO attainment on psychomotor skills, where findings are further elaborated using six (6) levels of attainment criteria. Figure 4.1 shows the percentage distribution of the respondents' level of agreement on their ability to identify problem based on a given scenario. About 64% of the respondents agreed that they have the ability to identify problem based on a given scenario and about 30% moderately agreed to this statement.  About 62% agreed that they have the ability to determine correct procedures for investigating problems. About 38% moderately agreed to this statement.

Ability to Demonstrate the Usage of Apparatus to run Experiment
As shown in Figure 4.3, about 64% of the respondents agreed that they have the ability to demonstrate the usage of the apparatus to run the experiment. About 33% moderately agreed that they are able to demonstrate the usage of apparatus. Only 3% were not confident that they have acquired that skill.  Figure 4.4 shows the percentage distribution of the respondents' level of agreement on their ability to collect data in order to obtain important results.

Fig. 4.4 Percentage Distribution of Respondents' Level of Agreement on Ability to Collect Data to
Gather Important Results About 67% agreed that that they have acquired the skill to collect data to gather important results with about 30% moderately agreed to the statement. Meanwhile about 3% of the respondents did not believe they have acquired such skills. Figure 4.5 shows the percentage distribution of the respondents' level of agreement on their ability to interpret results that lead to significant findings. About 63% agreed that they have acquired the ability to interpret results that lead to significant findings with about 31% moderately agreed to the statement. Meanwhile, about 6% of the respondents believed that they have not acquired such skill.

Ability to Propose Viable Solutions to Solve Given Problems
As shown in Figure 4.6, about 63% agreed that they have acquired the ability to propose viable solutions to solve the given problems with about 31% moderately agreed to the statement. Meanwhile about 6% of the respondents believed that they have not acquired the skill.  Table 4.2 shows ranking of statements on PO attainment of the students under six (6) levels of psychomotor domain. It shows that majority of the students are confident on their ability to collect data to gather important results and ranked this as the most important variable attributed to the PO classified as psychomotor domain. Next is their ability to demonstrate the usage of the apparatus to run the study, followed by their ability to interpret results leading to significant findings. However, they ranked their ability to identify problem as the least important.

Direct Attainment of Program Outcomes based on Psychomotor Domain based on Courses
This section presents the direct attainment of psychomotor skill through laboratory courses extracted from the assessment system known as myCOPO that indicates the true attainment of students' psychomotor domain. Figure 4.7 shows the percentage distribution of PO attainment by the students which are between 63% and 80% which are greater than 50% baseline.  Table 4.3 shows the ranking of the important factors influencing students' PO attainment on psychomotor skills. The condition of the equipment in the laboratories is ranked first, followed by the lecturers' instructions before the students conduct their experiment. The third important factor is the environment in the laboratory and finally ranked last is the adequacy of equipment in the laboratories. Thus, in the higher education environment, psychomotor learning can be included in training using specified equipment (Kasilingam, Ramalingam & Chinnavan, 2014).  Table 4.4 shows the inter-correlation between six (6) psychomotor attributes and the factors influencing the attainment of the attributes.

Table 4.4: Inter-correlation between PO attainment and Factors
A medium strength and positive correlation was found between "conducive environment" and "problem identification" based on a given scenario; r = 0.405, p < 0.05. It means that if the environment is improved in terms of its conduciveness, the problem can be identified in a better condition and vice-versa. A medium strength and positive correlation was found between the conducive environment and the determination of the correct procedure; r = 0.416, p < 0.05. It means that if the environment is improved in terms of its conduciveness, the correct procedure can be determined in a better condition and vice-versa. A medium strength and positive correlation was found between the conducive environment and the demonstration of the usage of apparatus; r = 0.404, p < 0.05. It means that if the environment is improved in terms of its conduciveness, the usage of apparatus can be demonstrated in a better condition and vice-versa. A medium strength and positive correlation was found between the equipment condition and the determination of correct procedure; r = 0.461, p < 0.05. It means that if the condition of equipment is improved, the correct procedure can be determined properly and vice-versa. A medium strength and positive correlation was found between the equipment condition and the demonstration of usage of apparatus; r = 0.413, p < 0.05. It means that if the condition of equipment is improved, the usage of apparatus can be demonstrated in a better way and vice-versa. Finally, a medium strength positive correlation was found between the equipment adequacy and the determination of correct procedure; r = 0.404, p < 0.05. It means that if the equipment adequacy is improved, the procedure can be determined in a better way and vice-versa.
The correlation analysis show positive relationships between procedures involved in open-ended activities with the external factors such as the environment conduciveness, equipment condition and adequacy, however, no positive relationship with lecturers' influence was indicated in the analysis. As been observed by Mishra, Barrans and Crinela (2009), psychomotor skills based teaching strategy results in better acquisition of hands-on skills. In addition, the environment in which teaching takes place can affect optimal performance (White, Rodger & Tang, 2016).

Conclusion
In the past, assessment of psychomotor skills was seen as being less important than assessment of knowledge and cognitive skills. Thus, this study was carried out based to determine the perceptions of students' on their psychomotor skills attainment through open-ended laboratory courses. The curriculum for the civil engineering program incorporates psychomotor attributes in laboratory courses and final year projects. An indirect measurement on level of attainment was carried out using questionnaire survey administered to a sample of 393 civil engineering students that have taken laboratory courses. General findings indicate that a majority of the final year students perceived that they have acquired the basic attributes in psychomotor skills through the open-ended activities designed by the lecturers, with a very small percentage of students believed that they have not acquired such skills. In addition, positive and medium strength relationships exist between openended procedures and factors related to laboratory environment and equipment but none with lecturers' influence. These findings was further validated using a direct measurement of psychomotor domain extracted from the assessment system known as myCOPO that shows the true attainment of students achieving the skills. This study has shown important attributes in the attainment of program outcomes to close the knowledge gap on psychomotor skills of engineering students. Thus, the findings of this study could help in guiding teaching for enhancement of psychomotor skills in Institute of Higher Learnings.