The Contribution of Instruction in a Knowledge Mapping Technique to Students' Understanding of Texts

An experiment to assess the effects of instruction in a knowledge mapping technique on students’ learning performance in Economics was conducted in four secondary schools in the State of Selangor, Malaysia. Instruction in a note-taking technique served as a control condition, and a pre-test-post-test design was utilised. The study involved 135 Sixth-Form Economics students (24 males and 111 females) and four Economics teachers. The effectiveness of the technique was assessed by four types of assessment tests, and the tests were developed on the basis of Bloom’s taxonomy of educational objectives for the cognitive domain (Bloom, 1956), and followed the procedure suggested by Gronlund (1982). It was hypothesised that the knowledge mapping technique would assist students to perform better in the performance tests in comparison with the control group using the note-taking technique. The influences of gender, ethnicity, cognitive style and learners’ characteristics were also assessed. Furthermore, the research also investigated the effects of the technique on measures of students’ self-efficacy beliefs and cognitive effort investment.


Introduction
This study focuses on the influence of instruction in learning strategies upon the performance in Economics of Sixth Form Economics students in Malaysia. The issue of the study was investigated because students taking the Economics subject were not performing well in the Malaysian Higher School Certificate (STPM) examinations. Possible reasons for this poor performance have been subject to much discussion by Malaysian educationalists in recent years. The approach taken here is to consider whether the level of performance may be due, at least to some extent, to deficiencies in students' learning strategies and that their achievement might be improved by concentration on improving these strategies. In order to investigate this, a sample of sixth form students were taught a strategy for understanding an Economics text, and their performance on tests of their understanding of this material was compared with that of a control group who were not taught this strategy. In the remainder of the chapter, we discuss the education system in Malaysia in order to provide a context for this study, but before doing so we give a brief description of Sixth Form Economics studies.
Utilising effective strategies in teaching is important in order to provide the student with the necessary mental tools to examine, understand and criticise the economic issues involved. The difficulties in achieving the aims and objectives of the curriculum were discussed at many educational seminars. Most of these seminars focused on problems of teaching and learning and low achievement in subjects. One of the aspects that was identified as a major contribution to students' low achievement was their lack of ability in understanding the texts. Although students may have the ability to decode the texts appropriately and they possess the necessary word recognition skills (because they are Sixth Form students), they may lack the ability to effectively use cognitive strategies to facilitate comprehension of the material.
Cognitive strategies are concerned with how incoming information is processed and structured in memory. The strategies are important for comprehending Economics texts materials effectively. Ineffective learning strategies encourage students to rely on the surface structure of language and the regurgitation of facts, rather than developing ways of coming to understand important ideas and relationships in texts. In practice, most teaching methods implicitly encourage students in rote memorisation by identifying in detail what must be learned. Rote memorisation normally involves learning from texts with little intention to assimilate the information. Therefore, material learned by this method is not usually meaningful in relation to other stored information. This limits the student's capability to retrieve the knowledge at a later date or apply this knowledge in normal situations. Such a strategy is ineffective for learning where understanding is far more important than mere storage of information. Students are likely to come to the subject with the preconceived idea that objective and immutable methods exist to discover the right solutions. At this stage, they view learning as nothing more than a process of transmission of facts and truths from teacher to student.
By relying on ineffective ways of transferring knowledge and failing to teach learning strategies, educators, in essence, discourage students from developing and exploring new strategies. This limits students' awareness of their cognitive capabilities (Dansereau, 1978). This lack of awareness eventually limits the students' ability to perform well at university (Jarji et al., 1993) which requires new learning strategies. Furthermore, if the strategies that individuals have spontaneously adopted or have been taught do not match their own cognitive capabilities and cognitive styles (Riding, 1997), this may affect the student's learning preferences, interest in the subject, and motivation to learn.
This study aims to investigate the effect of teaching the knowledge mapping technique on learning performance compared to the effect of the note-taking technique. In specific, the first objective of the study is to investigate whether there are significant differences in improvement in performance between students who have been exposed to the two learning techniques; knowledge mapping technique and the note-taking technique. The second objective of the study was to examine whether these differences are influenced by certain student variables; gender, ethnicity and cognitive style. Finally, the study is designed to investigate whether learners' characteristics have a significant impact on the post-test scores, over and above the effects of treatment and pre-test scores.

Methods and Materials
The research involved a systematic manipulative experimental design which focused on how students handled a cognitive task that required the manipulation of mentally represented knowledge. Based on the premise that students' cognitive approaches towards learning text materials can be changed through training in learning strategies, this chapter describes the method used to examine the influence of training in the knowledge mapping technique on students' learning performance, self-efficacy beliefs and perceptions of cognitive workload in comparison with the influence of the note-taking technique on these measures. Because the note-taking technique utilised the typical approach that students normally use in schools, this group could be considered a control group.
Five instruments were used in this research: the Learning Performance Test developed by the present researcher; the Learning and Study Strategies Inventory (Weinstein, Schulte, & Hoy, 1987); Cognitive Styles Analysis (Riding & Cheema, 1991); Cognitive Workload Measurement (NASA, 1986); and Self-efficacy Belief (Geisler-Brenstein & Schmeck, 1996). All the instruments were initially tested by means of a pilot study to assess their reliability and suitability for the main study.
For the purpose of the study, a pre-test / post-test control group design was utilised to test the research hypotheses. The basic plan of the study is presented in Table 1. Note-taking Group R O1 C O2 This design is known as a true control group design (Cohen, Louis, & Lawrence Manion, 1994) and the subjects are allocated either to the experimental group and to the control group. The two groups were assessed both before the experimental treatment started (the pre-test) and after it had had time to produce an effect (the post-test). Comparison was made between groups and within the groups (male and female, Malay and non-Malay, low and high in prior attainment and Wholist-Analytic style). The dependent variables, independent variables and subject variables are listed in Table 2. The gains scores and the post-test scores are the dependent which assessed students' performance at two levels of understanding, i.e. the low level of understanding and high level of understanding. The two levels were chosen because the present researcher was interested in understanding whether the technique employed in this study was able to enhance learning ability in basic and more complex skills. A low level of understanding is defined in terms of requiring less cognitive ability on the part of students to perform at the knowledge and comprehension levels. A high level of understanding is defined in term of requiring greater cognitive ability to perform the application, analysis, synthesis and evaluation levels. The levels were derived from Bloom's taxonomy of educational objectives in the cognitive domain (Bloom, 1956).
To investigate the effect of the treatment variable on performance on tests of understanding an Economics text, the first hypothesis was constructed in terms of the null hypothesis that there are no differences in the gains scores between students exposed to, and instructed in, the knowledge mapping technique and those exposed to and instructed in the note-taking technique.
In addition to looking into the effect of the treatment variable (knowledge mapping technique and note-taking technique) on learning performance, the study also investigates the influence of gender, ethnicity and cognitive style on learning performance. Gender plays an important role in academic achievement in schools (Arnold, Wiethoff, & Houwing, 1996). An enormous body of research has investigated the relationships between gender and academic performance in many areas. Beside the achievement factor, the equality of opportunity is another important factor in education.
Ethnicity has also played an important role in education particularly in a multi-ethnic society like Malaysia. The issues of multi-cultural education have long been debated in Malaysia and inequality in achievement among different ethnic groups are issues that are always relevant to the country. Based on the important role of this factor in Malaysian educational system, this research incorporated ethnic group membership as a variable in the study.
Research on the influence of cognitive styles/learning styles on academic performance has been carried out by many researchers (Kagan, 1965;Pask, 1976;Riding and Cheema, 1991;Witkin, 1962). Most of these argue that understanding students' cognitive or learning styles can help teachers in developing instructional materials and in understanding students' learning preferences. In this research, the Wholist-Analytic cognitive style was employed because of its applicability to the subject of Economics.
To examine the influence of gender, ethnicity, and cognitive style, the second null hypothesis was developed which stated that gender, ethnicity and Wholist-Analytic dimension of cognitive style would have no influence on the learning performance.
The third null hypothesis was that the learners' characteristics would have no significant impact on the post-test scores. The characteristics of learners were classified by means of factor analysis, because of the large number of measures involved.
One hundred and thirty-five students were assigned to two groups, i.e. the knowledge mapping group and the note-taking group. Before the treatment, students in each group completed four tests in order to assess their level of understanding of the Economics text. This assessment was designed according to the classes of the taxonomy proposed by Bloom (1956), and the terms 'low level of understanding' and 'high level of understanding' refer to Bloom's categorisation. After attending either the knowledge mapping technique session or the note-taking technique session, each student was again tested on the same tests. On each occasion, these tests produced four scores as follows: 1. Score 1: Multiple Choice Question 1 or MCQ 1 (low level of understanding); 2. Score 2: Cloze test (low level of understanding); 3. Score 3: Multiple Choice Question 2 or MCQ 2 (high level of understanding); 4. Score 4: Essay test (high level of understanding).
The study utilized two approaches in assessing improvement in scores on the tests following the treatment, one based on the post-test scores and one based on measuring changes between the pre-test and post-test scores (the gains scores). Since there has been controversy over the use of change (or difference) scores to measure improvements in performance, for example in terms of poor reliability, a brief justification for this approach is now presented. In a detailed analysis of the issue, Overall and Woodward (1975) reached the following conclusion: Numerous writers have emphasised the unreliability of difference scores, which result from summation of measurement errors. While this may be a problem for certain types of correlational studies, it is not a cause for concern in the use of simple difference scores to measure treatment-induced change in experimental research. In supporting the above conclusion, Huck and McLean (1975) suggested that the analysis of difference scores was a more conservative procedure than the analysis of covariance. An investigation by Zimmerman and Williams (1982) indicated that, given certain assumptions about the reliability coefficients and standard deviations of pre-test and post-test scores, the reliability of the gains scores can be fairly high. In most cases, it is usually assumed that the high reliability of the gains scores occurs when the gains scores and the pre-test scores are negatively correlated. However, according to Zimmerman and Williams (1982), the gains scores might either be positively or negatively correlated with the pre-test scores, depending on the experimental procedure. Either or both might have high reliability. Furthermore, Norusis (1993) argued that by considering the differences between the two scores from the same test, it is possible to control for some of the initial differences among the students. After given serious consideration, this study will focus on gains score rather than post test scores. However, we will use post test scores for final hypothesis since the difference between Likert scale score are too small to see the differences.

Result
The Effect of Knowledge Mapping Technique and Note-taking Technique on Gains Scores The objective of the analysis reported in this section was to investigate the influence on learning performance of the knowledge mapping technique compared to the note-taking technique. This analysis serves as a test of Hypothesis I.
The gains scores and mean post-test scores are presented in Table 3. Data were analysed using a one-way analysis of variance (ANOVA) to determine whether there were significant differences in the gains scores of the knowledge mapping group and the note-taking group. Inspection of the pre-test and the post-test scores suggests that students using the knowledge mapping technique showed an increase in the MCQ1 (gains scores = 28.71) and Cloze test (gains scores = 8.15). Students using the note-taking technique also showed increases in the MCQ1 (gains scores = 20.92) and Cloze test (gains scores = 2.64).
Table 3 also suggests that the students employing the knowledge mapping technique showed an increase in the MCQ2 (gains scores = 11.25) and the Essay test (gains scores = 3.57). Students employing the note-taking technique also showed an increase in the MCQ2 (gains scores = 8.27) and Essay test (gains scores = 3.23). In order to test the significance of differences in gains scores between the two groups, ANOVA test were applied to the gains scores in each test.  The results showed that the knowledge mapping group outperformed the note-taking group in terms of gains scores for the tests designed as assessing low level of understanding (MCQ 1 and Cloze test). This implies that using knowledge mapping as a technique for learning material could produce higher gains in the MCQ1 and Cloze test compared to using the notetaking technique. Although the mean gains scores on the MCQ2 and Essay test were higher for the knowledge mapping group, the variation in scores between the groups was too small to demonstrate any significant differences between the groups.

The Contribution of Gender, Ethnicity and Cognitive Style on Predicting Gains Scores
The objective of the analysis was to examine whether gender, ethnicity and cognitive style would have any influence on gains in the performance tests. The gains scores according to gender, ethnicity and cognitive style for the knowledge mapping group and note-taking group are presented in Table 5, for each of the four tests. A series of two-way analyses of variance (ANOVA) were carried out. This analysis serves as a test of Hypothesis II.
In each ANOVA, one of the two factors was Treatment. In the first ANOVA, the second factor was Gender, i.e. a Treatment X Gender design. In the second ANOVA, the second factor was ethnicity. The ethnicity variable were coded as follows: • Z 1= 1 Malay, 0 Non Malay; • Z 2= 1 Chinese, 0 Non Chinese; • Z 3= 1 Indian, 0 Non Indian; In the third ANOVA, the second factor was cognitive style. The cognitive style variable was coded into two divisions; a Wholist would be a person with a score of less than 0.99, while a score above 1.00 would indicate an Analytic.
The results of the series of analyses of variance are presented in Table 6. The only significant main effect involved the difference between Malay and non-Malay students on the Cloze test (Malay mean = 7.32, non-Malay mean = 1.72, F = 5,951, p = .0167). With the large number of statistical analysis and the significance level of 0.05 (the level used for rejecting the null hypothesis with probability of making a mistake is 5 out of 100), the result of one significant finding could have occurred purely by chance. However, because the probability obtained in this analysis was close to the 0.01 level the chances of incorrectly accepting the null hypothesis were less than 1 in 100 and there was no need to use a Bronferroni correction with this analysis.  The results revealed no significant main effects of gender in any of the tests. The results also show no significant main effect of cognitive style in any of the tests. Furthermore, no significant main effect of Chinese and non-Chinese and Indian and non-Indian was found on any of the tests. None of the interaction terms involving the treatment variable was significant.

The Contribution of Learners' Characteristics on Predicting Post-test Scores
This section reports findings from analysis of the relative importance of the variables reflecting learners' characteristics in predicting post-test scores. The multiple regression analysis method was employed to test the contribution of the variables of learners' characteristics to the post-test scores. The post-test scores were entered into the linear regression equation as a dependent variable with factors scores for the three factors (Affective Strategy, Cognitive Strategy, Prior Attainment), the coded treatment variable and the pre-test scores as independent variables. The objective was to examine whether learners' characteristics have influence on the post-test scores over and above the effects of treatment and pre-test scores. The results of the four multiple regression analyses are presented in Table  7. The results indicate that none of the learners' characteristics variables has any significant impact on the post-test scores of any of the tests once the variables of pre-test scores and treatment have been entered into the equation. Analysis of pre-test scores (Section 5.4) had found that learners' characteristics (Affective Strategy, Cognitive Strategy and Prior Attainment) did have a significant relationship with the pre-test scores on the MCQ1, and the first two factors are associated with pre-test scores on the Cloze test. These relationships are not maintained for the post-test scores when the effects of treatment and pre-test scores are statistically controlled.

Discussion
The results of the study indicate that there were significant differences between the knowledge mapping group and the note-taking group in the gains scores for both the MCQ 1 and the Cloze test. However, no significant differences were found between the gains scores of the knowledge mapping group and the note-taking group in the MCQ 2 or the Essay test As the results, the knowledge mapping group outperformed the note-taking group in terms of gains scores for low level of understanding (MCQ 1 and Cloze test). This implies that using knowledge mapping as a technique for learning material could produce higher gains in the MCQ1 and Cloze test compared to using the note-taking technique. For the high level of understanding, although the mean gains scores were higher for the knowledge mapping group, the variation in scores between the groups was too small to demonstrate any significant differences between the groups.
The ANOVA test indicate that there were significant differences between the two groups in the gains scores for both tests. Analysis of the Beta weights for the treatment variable using multiple regression analysis also shows that the weights are significant for the MCQ1 test and the Cloze test. However, the Beta weights for the MCQ2 test and Essay test were not significant. Analysis of the trend of gains scores at different levels of pre-test scores revealed that low score in the pre-test gain higher gains scores in the MCQ1 and the Cloze test, compare to their counterpart with high score in the pre-test.
In looking into the contribution of gender, ethnicity and cognitive style to predicting the gains scores, the result revealed that the only significant main effect involved a difference between Malay and non-Malay on the Cloze test. No significant main effects were found involving gender and cognitive style. The results also indicated that neither gender or cognitive style had a significant relationship with any of the post-test scores.
In looking into the contribution of learners' characteristics to predicting the post-test scores, the findings revealed that none of the learners' characteristics had any significant impact on the post-test scores on any of the tests. The results also indicated that students in the knowledge mapping technique had higher mean score in all dimensions of self-efficacy belief (cognitive organisation, fact retention and critical thinking). The result also revealed significant differences in three out of six scales measuring students' perceived cognitive workload.
This research has utilised a general-purpose strategy for learning an International Trade topic from Economics textbooks. Dansereau (1985) argued that the most important strategy dimension is the degree of specificity. In other words, future research on learning strategies should focus on training individuals in specialised strategies that are applicable to only a specific class of tasks. Although the strategy dimension might produce a built-in restriction on the generality continuum, it appears to be more cost-effective. Installing a specialised strategy training programme for learning skills does not appear to require very substantial resources. The cost to the school is small and it is not too demanding of time.
It is also suggested that future research should try to develop a strategy midrange on the generality-specificity continuum. This kind of strategy should be able to be employed effectively and also be general enough to have a relatively wide range of applicability. Such a learning strategy should either be content independent or content dependent and should relate to the actual mental manipulation of information to facilitate the cognitive processes of abstraction, assimilation, organisation and retrieval of academic information. Future research into cognitive learning strategies should also focus on teaching the strategies to learners in the early years of intellectual development. This recommendation is based on the assumption that by successfully training children at an early stage to use a strategy, we can sometimes learn about the mechanisms involved in the ontogeny of strategy acquisition. The researcher believes that young children can also be successfully instructed to use many of the strategies previously thought to be available only to older children.
Future research should also focus on determining what aspect of the training may be used to increase the level of Bloom's cognitive domain, i.e. the application, analysis, synthesis and evaluation, beyond the knowledge and comprehension levels. By identifying what aspects of training are beneficial, researcher and educators may be able to use these strategy instruction techniques to increase students' level of understanding.
Future research should also examine the more recently developed models of learning style and establish links with a range of learning strategies. This will be likely to generate insight into the development of more extensive learning strategies. The behavioural approach, particularly the motivational aspect of learning, and its linkage with individual cognitive style and learning strategies, is another aspect that may provide much useful insight and knowledge. Academic achievement is very much valued in Malaysian society. Recently, teachers, schools, the district education offices, the State Education Department and the Ministry of Education have emphasised the need to improve students' learning performance as assessed by the standard formal examination. Based on this concern, the researcher carried out an experiment with the knowledge mapping technique to assess whether this technique could fulfil the objective of improving students' learning performance, specifically in the subject of Economics. Although the results were relatively moderate, it appeared that the technique did have the potential to improve students' ability in understanding an Economics text. More effort should be undertaken to identify the best method of teaching the technique to students. However, despite all these suggestions, the most important requirement is the willingness and co-operation of teachers to teach new learning strategies.