Science, Technology, Engineering and Mathematics (STEM) Professional Development: An Evaluation of Teachers’ Learning on Project - based Learning (PBL) Approach in Teaching

This study aims to identify the level of teachers’ learning (made up of sub components of knowledge, skill and attitude) acquired from professional development on project-based learning (PBL) approach. The study is undertaken in selected MoE schools that implement STEM PD and from then, use PBL method in the classroom. A total of 221 teachers participated in the study and data is analysed using SPSS Ver. 18.0. It was found in this study that two components of teachers’ learning; knowledge and attitude ac quired from PD attended were at a high level but another component; skill was found only moderate with each of their sub domains having variations from moderate to high level. Recommendations were made on bettering the implementation of PD focusing on the sub domains of teachers’ knowledge, skill and attitude hopefully resulting in desired PBL implementation.


Introduction
Students' outcomes hinges upon teachers who constantly update their knowledge of the subjects they teach along with pedagogical skills in order to prepare quality teaching and learning.Therefore, implementation strategy towards students' outcomes, outlined in the Malaysia Education Blueprint, prioritizes teaching and learning quality (MoE, 2013).Such a strategy can be achieved through PD as it is viewed as an avenue in providing teachers with new knowledge and skill in response to vast changes taking place in education requiring teachers to change (Randel et al., 2016).Taylor et al (2011) stated that previous studies had confirmed that not only does PD assist teachers in increasing teachers' knowledge and skill but also creating shift in the minds of teachers who could be resistant towards change in the education landscape.Reseachers concluded that teachers who are exposed to PD tend to effectively implement their lesson (de Jager, Reezigt & Vol 14, Issue 6, (2024) E- ISSN: 2222-6990 To Link this Article: http://dx.doi.org/10.6007/IJARBSS/v14-i6/20684DOI:10.6007/IJARBSS/v14-i6/20684 Published Date: 10 June 2024 Creemers, 2002;Kealey et al., 2000).Even Guskey, a renowned scholar in PD evaluation emphasizes PD in education contending that PD is among important components to elevate education quality.Realizing the magnitude of impact that PD can have in spearheading education quality, Ministry of Education Malaysia has taken appropriate measures such as implementing PD as an avenue to raise teachers' quality in enacting better and more meaningful teaching.Such an effort is a manisfestation of MoE's commitment in upholding teachers' profession since PD aims towards teachers' quality through improvement in their knowledge, skills and positive attitude (Azmi, 2016).
Studies have shown that PD focusing on improving teachers' knowledge and skills, which is continously implemented and is related to what teachers do, are critical in improving teachers' lesson and ultimately, students' outcomes (Althauser, 2015).Therefore, PD ideally should emphasize knowledge and skill acquisition among teachers.Knowles (2017) argued that PD not only improve teachers' knowledge and efficacy in the teaching the subject content but also have significant impact towards students' confidence in learning and subsequently academic achievement.
Nevertheless, in the context of integated STEM in the teaching of Science and Mathematics as a new endeavour to improve students' understanding of concepts, teachers knowledge and skill need to be enhanced for them to implement teaching (Ceylan & Ozdilek, 2015;Radloff & Guzey, 2016;Abdul Hadi, 2015;Rahayu et al, 2018).Thus, in order to realise teachers' role in this new context, PD has been identified as the platform to boost teachers' knowledge and skills as well as nurture positive attitude towards change in their exisiting teaching practice (MoE, 2013;Wojnowski & Pea, 2013;Lambert et al, 2018;Gardner et al, 2019).
Abd Khalil (2017) argued that PD should function as a mechanism to close the gap between the lack of teachers ability with the latest needs to implement integrated STEM.Similarly, in the context of STEM teachers capability, PD should factor teachers' needs in order to implement integrated STEM in class.Moore et al ( 2014) opined that quality integrated STEM requires updated knowledge and specific pedagogy on related STEM disciplines.
The implementation of efficient integrated STEM requires teachers to have knowledge about their STEM subjects (Eckman et al., 2016).Besides that, teachers need to have knowledge in teaching STEM concepts to students (Thibaut et al., 2018).Becker and Park (2011) argued that STEM, taught in an integrated manner will result in effective learning in students.In the context of STEM Education, Kermani and Aldemir (2015) stated that students are able to achieve a higher level of understanding when their learning is supported by quality teachers.Thus, such teachers can be produced through intervention programme since integrated STEM is relatively new.Based on this scenario, Chai (2018) emphasized that there is a need for an intervention for STEM teachers which provides justification to embrace new changes.Moore et al (2014) added that quality STEM teaching necessitates an updated knowledge and specific pedagogical skill on integrated STEM be made available to teachers.Hence, PD for STEM teachers, focusing on assisting them to improve students' Mathematics and Science learning is a valuable avenue in creating a signigicant change in students interest towards STEM.PD, therefore is seen as not only assisting teachers in designing integrated STEM lesson but also providing engaging learning experience for students (Avery, 2013).
As PD significance is proven as the main foundation for any kind of change in education (Fore et al., 2015;Desimone, 2009;Guskey, 2002;Avery, 2013) stated that PD needs to improve teachers' undertanding on the required knowledge and skill for teaching to provide students with effective STEM lesson.Especially for STEM education, PD exposes teachers to STEM concepts along with ways in implementing pertinent teaching approach in their classroom (Donna, 2012).Further, activities in PD helps prepare teachers to teach STEM content using a new approach (Brophy et al (2008), a departure from their existing teaching practice.
Besides, the significance of PD has been proven in many a study, among others one undertaken by Dare, Ellis dan Roehrig (2018) on teachers' implementation which discovers that teachers require PD in the implementation of interdisciplinary STEM teaching.Dan and Gary's (2018) interviewed teacher who had no previous teaching experience and foundation on integrated STEM and established the need for PD for them.This need exists based on their lack of knowledge on how to apply integrated STEM in teaching.
Professional development for STEM teachers in Malaysia is planned to fulfill the aspirational needs outlined in the Education Blueprint.In line with this, MoE Malaysia plans to implement professional development to further develop teachers' knowledge and skills in order to improve students' learning (MoE, 2013).STEM PD prepares training for teachers to ensure that they can deliver the curriculum effectively.And is implemented in three cascading levels, the first level from ministry to national trainers, followed by national trainers to the district trainers and finally from district trainers teachers, the implementors of STEM PBL in schools, selected to execute the aproach in class.This STEM PD exposes teachers to global emphasis towards STEM and current state of STEM efforts in Malaysia especially STEM Initiative and PBL as the selected teaching approach for Integrated STEM.
Then, if PD is viewed as the solution to issues and challenges in integrated STEM implementation, there exists a need to systematically evaluate the programme, a desired exercise to investigate the training and its after-effect, specifically looking at teachers' learning (knowledge, skill and attitude) which will influence teachers' behaviour in the implementation of STEM PBL in their lesson (Kirkpatrick, 2000).Fore et al (2015) highlights that past evaluation on PD has paid less attention to teachers' learning process, referring to what happens during PD.As in the context of this study, what is material is that whether knowledge, skill and positive attitude among teachers take place as a result of attending the PD.
Programme evalution needs to be undertaken given that this aspect is capable of providing information to MoE, if ignored decision made on the future of programme is not made based on empirical findings (Kirkpatrick, 1994: Patton, 1990: Stufflebeam & Shinkfield, 2007).Therefore programme such as STEM PD calls for properly planned evaluation to determine the extent it impacts the organisation.
This study therefore aims to evaluate STEM PD on teachers' learning acquired from PD with specific focus on their knowledge, skill and attitude.Specifically, the objectives are as follow: i.
Identifying teachers' knowledge on STEM PBL ii.
Identifying teachers' skills in implementing STEM PBL iii.
Identifying teachers' attitude towards STEM PBL The study is undertaken in selected schools to implement STEM PD and from then, use the PBL method in the classroom.Stratified random sampling is employed in with STEM subjects taught by the teachers namely Mathematics, Science as well Design and Technology as the stratum.A total of 221 teachers participated in the study.

Methodology
This research employs the survey design with sample drawn the population of teachers who attended STEM PD.A total of 221 teachers took part in the study and data is descriptively analysed using SPSS.

Findings and Discussion
Based on Research Objective 1, teachers' knowledge records a high mean score (mean=3.91,s.d=0.56).Details of the three sub constructs of knowledge are knowledge about STEM Initiatives by MoE (mean=3.91,s.d.=0.54),knowledge about STEM PBL needs (mean=3.94,s.d.=0.64), and knowledge about PBL steps (mean=3.87,s.d.=0.63), all of which record high mean score indicating teachers' knowledge gained from PD is high.The first aspect of knowledge is on STEM Initiative by MoE Malaysia as guiding document on the implementation of STEM programmes including PD.The document, highlights STEM to be implemented in the education system.STEM Initiative drives the implementation of teaching and learning of STEM which also provides rationale on the integrated manner approach using PBL as the selected method .Based on the findings in this study, teachers report that they gain knowledge on STEM that can be integrated in formal, informal and non-formal settings, knowledge on elements of S,T, E and M that can be integrated in STEM subject taught and knowledge on the suitability of the PBL method in the STEM subject taught.Such findings corroborate finding by Han (2015) that discovered PD as an avenue to maximise improvement in teachers' capability in implementing change in education.Nevertheless, there are areas to be improved in future STEM PD with regards to teachers' knowledge on STEM Initiative.These include emphasizing knowledge of the STEM Education definition, knowledge of STEM Education goals, knowledge of STEM Education concept as well as knowledge of activities planned under STEM Initiative.These aspects are crucial to be exposed to teachers as early as they step into PD to prepare them to accept the rationale that STEM Education is given significant focus by MoE and the rationale why they are, in turn required to attend PD.This argument supports findings by Joyce and Showers (2002) that teachers need to explore and undertand the rationale as to why change needs to be embraced and subsequently implemented as well as why they need to think and teach differently in order to implement change to their existing practice.Finding from this study also resonates with Han (2015) who discovered that PD is effective for teachers, at least to help them believe about change in education system.Through PD, teachers are able to identify any educational change requiring different capabilities compared to what they need in their existing practice.Han (2015) argues that PD is an effective platform in exposing teachers to important concepts related to STEM PBL and that the teachers agreed that changes in education such as the use of STEM PBL requires a set of pedagogical capabilities to be achieved through PD (Capraro & Slough, 2013;Newell, 2003;Ozel, 2013).
The second aspect is knowledge about STEM PBL needs which is shown to be at a high level.Teachers report that they have the knowledge about the need to implement STEM PBL and knowledge about the need to adapt to the new teaching method They also report that they have the knowledge on the need to integrate at least two elements of S, T, E and M and relate those elements to real life situation.However, as documented in this study, future PD may want to strengthen knowledge among teachers on adapting STEM PBL to assessment component, on using STEM PBL for students' exploration activities and on the exploration activities to be carried out via group work.These findings are parralel to what Han (2015) argued that most teachers agree that STEM PBL is an activity that is crucial and effective in stimulating student interest and enhancing their understanding of STEM concepts.Next, the third aspect of knowledge in this study is on steps to implement PBL.Findings in this study show that this aspect is found to be at a high level.Teachers report that they acquire knowledge on the need to guide students in exploring problems relating to real world situation and knowledge on the need to help students complete a project based on their participation in PBL.Aside from these, teachers also report that they know the need to guide students in stating problems that they wish to explore and strategising solutions accordingly.Nevertheless, there are aspects to be improved in future PD.Such aspects include equipping teachers' knowledge on how to guide students to further research the problem they wish to explore, knowledge on how to guide students acquire knowlege and subsequently deeper understanding of STEM concepts from participating in PBL activities as well as knowledge on how to guide students to reflect on both their exploration and groupwork activities.Considering that these aspects constitute steps in implementing PBL (Capraro et al, 2013), teachers ideally should possess knowledge about these steps in order to implement PBL.Findings above are parralel to argument that teachers' understanding of STEM PBL influences students' understanding of STEM contents and skills that they are expected to learn (Capraro et al., 2013).As put forth by Darling-Hammond (2000); Darling Hammond and Young (2002); Goldhaber (2002); Rice (2003); Wayne and Youngs (2003), students learn more from qualified teachers with considerable amount of knowledge and skills.
When compared to the first construct of learning which is teachers' knowledge about STEM PBL, found to be at a high level in this study, the same corresponding aspect under teachers' skill in implementing STEM PBL records a moderate mean score (mean=3.63,s.d=0.67).Details of the two sub constructs of skill are skill about STEM PBL needs (mean=3.63,s.d.=0.70), and skill about steps in implementing PBL (mean=3.63,s.d.=0.67), both of which record moderate mean score indicating teachers' skill gained from PD is at a moderate level.The first aspect of teachers' skill in this study is skill about STEM PBL needs which is found to be at a moderate level.In this aspect, teachers are found to have skill at a moderate level with regards to teaching STEM using PBL approach through groupwork, connecting STEM elements to real life situation and combining at least two elements in teaching using PBL approach.However, as documented in this study, future PD may want to strengthen skill among teachers on how to use PBL for students' exploration activities.Such findings are in line with that of Han (2015) who discovered in his study that STEM PBL is a relatively new approach and teachers do face challenges in implementing despite being exposed to skills needed to implement such an approach, justifying teachers' skill discovered in the present study to be at a moderate level.Nevertheless, for the benefit of future PD, some critical aspects on PBL need to be given focus and be equipped among teachers, among others, skills on connecting lesson using PBL with Learning Standards (Curriculum) and Performance Standard (Assessment).Besides that, focus on future PD should also aim to equip teachers with the skill to use PBL for students' exploration activities.Thus, teachers would acquire skills on STEM PBL needs to implement PBL in their lessons.These needs are parralel to argument put forth by Jones & Carter (2007) that apart from teachers' knowledge and motivation, their skill are also called for to implement a new teaching approach.The importance of the needs to equip teachers with this skill on PBL needs, evident in this study, is parralel to Rowan and Ball (2005) that skills in implementing certain approach have an impact on their teaching and eventually students' outcomes.
The second aspect of teachers' skill in this study is skill about implementing steps in PBL which is also found to be at a moderate level.Teachers reported that their skill is at a moderate level with regards to guiding students to implement project as the output of participating in PBL, as well as guiding students to reflect on groupwork activities.However, future PD needs to address aspects such as equipping teachers with skill to guide students to strategise solutions to problems they wish to research and skill to guide students to further research the problems they wish to explore.These aspects are steps in implementing PBL and it is desired that teachers be equipped with skills in these aspects (Capraro, 2013).In addition, aspects such as guiding students to acquire knowledge and further, deep understanding of STEM concepts from participation in PBL need to be strengthened in teachers in future PD.Besides, emphasis on teachers guiding students to reflect on their exploration activities in PBL needs to also be addressed in future PD.Especially with teachers' skill in guiding students acquiring deep understanding of STEM concepts as documented in this study, this aspect supports those discussed by Han (2015); Barron et al (1998) as well as Capraro, Capraro and Morgan (2013) PBL is implemented in education to encourage deep understanding among students.
Next, the third construct of learning, namely attitude records a high mean score (mean=3.68,s.d=0.59).Details of the three sub constructs of attitude are motivation (mean=3.73,s.d.=0.63), self concept (mean=3.65,s.d.=0.55) and attitude toward job satisfaction (mean=3.64,s.d.=0.69), all recording high mean score indicating the level of teachers' attitude gained from PD is high.Such findings support those of Al Salami, Makela and Miranda (2017) as well as those of Berlin and White (2009) whose findings indicate that teachers' attitude are positive towards integrated STEM.In this study, among other aspects discovered under teachers' attitude as a general construct are that teachers feel that it is their responsibility to implement PBL in their teaching and PBL is realistic to be implemented with their students.Nevertheless, under self concept, a sub construct of teacher's attitude a few aspects need addressing.These include improving teachers' perception that there is no problem in implementing PBL in a big class, found in the current study to be low.This finding supports that of Che Seman et al (2017) who documented that teachers faced problems in implemeting a new change in their teaching because of the big class size.Such self concept calls for a change via serious addressing in future PD so that teachers can overcome perceived challenges such as class size.Positive self concept sucessfully inculcated during PD is also indicative of a successful PD.Apart from the aspect discussed above, another aspect under teacher self concept also requires attention in future PD which is about teacher's perception on the need to implement PBL while at the same time teach students to the test as well as perception on PBL as burdening students.Such findings support those documented in Che Seman (2017) as well as Hand and Treagust (1994) about the exam-oriented education system in which teachers' duty in teacher-based learning is to deliver knowledge and finish the syllabus in preparing students for examination.Besides that, findings from the current study is also parralel to that of Herro and Quigley who documented that the inablity to have control of the syllabus stands in the way for the teacher to implement STEM approach in class.In addition to that, findings from the current study about STEM PBL burdening students also support those of Al Salami et al (2017); Asghar et al (2012); Bagiati and Evangelou (2015); Goodpaster et al (2012); Van Haneghan et al (2015) on teachers underestimating students' capabilities.Among the highlights in these studies are that teachers do not have the confidence in students having enough competency to solve problems in STEM PBL classroom that has eventually caused them to be unmotivated in learning.Consequently, teachers need to have a change in their self concept and this should be the focus in future PD in order for them to implement PBL without any hiccups.
Further, motivation, another subconstruct under teachers' attitude is also found to be at a high level.Teachers reported that their participation in PD has driven them to further search for new knowledge to implement PBL for a more effective teaching.Aside from that, teachers also reported that they are prepared to guide students towards meaningful learning via PBL.Nevertherless, for the betterment of future PD, efforts need to be in place to intensify motivation among teachers, especially on their excitement, desire, and determination in implementing PBL as well as the passion to apply knowledge about STEM PBL learnt in PD into their classroom.Such findings are parralel to that of Jones and Carter (2007) who discovered that motivation, apart from knowledge and skills, is required by teachers in implementing STEM PBL.Considering that motivation is seen as an important element in enabling teachers to apply the knowledge acquired from any PD into practice in classroom, visible in the current study, supports those documented by (Copriady, 2015;Nafukho et al., 2017;Ku Ahmad & Hassan, 2016;Thoonen et al., 2011;Weissbein et al., 2011).In those studies, motivation has been found to be contributive in influencing behavioral change in the classroom Next, attitude towards job satisfaction, another construct under teachers' attitude also discovered to be high in the current study.Teachers reported that they are happy to assist students to better understand STEM topics through PBL and are happy to reflect upon their teaching using PBL.Few aspects under this sub-construct call for emphasis in future PD such as correcting the perception of teachers who feel burdened in implementing PBL.Similarly, are the perception of teachers about PBL method compared to teaching students for examination as well as perception about their experience attending PD in order to effectively teach students using PBL.Findings from the current study regarding these aspects are parralel to Barak and Shakman (2008) as well as Rajendran (2001) discovering that teachers prefer to teach using the teacher-centred learning.

Conclusion and Recommendation
Based on findings from thist study, it can be concluded that PD on STEM PBL has been successful in creating knowledge, skill and positive attitude towards change in the hope of effecting change in teachers' existing practice.This is based on the range of moderate to high level of teachers' knowledge, skill and attitude on STEM PBL.However, aspects highlighted in discussion in the previous section can be taken into consideration to improve future PD especially in ensuring that teachers' learning during PD be maximised and later translated into the desired practice in the classroom as the ideal outcome desired in any PD.

Table 1
Mean score and standard deviation for teachers' knowledge level after STEM PD

Table 2
Frequency and percentages of teachers' knowledge about MoE's Strengthening of the

Table 3
Frequency and percentages of teachers' knowledge about STEM PBL needs

Table 4
Frequency and percentages of teachers' knowledge about PBL steps

Table 5
Mean score and standard deviation for teachers' skill level after STEM PD

Table 6
Frequency and percentages of teachers' skill about STEM PBL needs

Table 7
Frequency and percentages of teachers' skill about PBL steps

Table 8
Mean score and standard deviation for teachers' attitude level after STEM PD

Table 9
Frequency and percentages of teachers' attitude regarding self concept

Table 10
Frequency and percentages of teachers' attitude regarding motivation

Table 11
Frequency and percentages of teachers' attitude regarding their attitude towards job