Higher-Order Thinking in Singapore Mathematics Classrooms

Yeo, Shu Mei Zhu, Yan Centre for Research in Pedagogy and Practice National Institute of Education, Singapore The Thinking School, Learning Nation program has been launched in Singapore since 1997, which gives impetus for the infusion of creativity and higher-order thinking skills into the curriculum as one initiative. However, there seems to be little research evidence to show how successful or effective we have been promoting higher-order thinking in daily mathematics teaching. This paper reports the extent to which higher-order thinking occurs in mathematics classrooms and the depth of such thinking at both primary and secondary levels. The data are collected by means of classroom observations of 118 mathematics lessons using the Singapore Pedagogy Coding Scheme developed by the CRPP, NIE. The analysis reveals that most of the time, students are engaged in activities focusing on factual/rote knowledge and procedural (how to) computation. The main mode of students knowledge manipulation is found to be regurgitation / copying / repeating of what is taught and the focus of knowledge criticism is often on the Truth. It is believed that higher-order thinking can be integrated into our mathematics classrooms to a greater extent. Implications for teaching higher-order thinking in mathematics lessons will be discussed. Introduction The Thinking School, Learning Nation program has been launched in Singapore since 1997. Part of the program is intended to drive the infusion of greater higher-order thinking skills to develop and fully realize students latent creativity. After eight years of implementation, to what extent and degree higher-order thinking skills have been successfully or effectively promoted in daily classroom teaching? The Third International Mathematics and Science Study (TIMSS) series consistently shows that Singapore students performed the best among all participating countries in mathematics at both grades four and eight levels over the years (Beaton, et al., 1996; Gonzalez et al., 2004; Mullis, Martin, Beaton, et al., 1997; Mullis, Martin, Gonzalez, et al., 2000). However, some researchers argued that those test items used in the TIMSS hard to measure students ability in higher-order thinking (e.g., Bracey, 2000; Wang, 2001). One of their concerns is that most test items used are multiple-choice questions, which might better measure students procedural knowledge and routine problem solving skills. Cai (1997) further suggests that more challenging and non-routine tasks, such as open-ended problems and problem posing problems, need to be included in order to fully assess students mathematics learning ability. The Singapore mathematics syllabus for both primary and secondary levels emphasizes the development of students ability in problem solving, which is the primary aim of mathematics instruction (Ministry of Education [MOE], 2000a, 2000b). Thinking skills and heuristics is defined as one of the five interrelated components that the attainment of mathematical problem solving ability is dependent on. The syllabus states that thinking is inherent in the subject. (MOE, 2000a, p. 17) It indicates that the development of thinking skills and mathematics learning should not be taught separately. Instead, thinking skills should be consciously integrated and reinforced in daily mathematics classroom teaching. This study is intended to investigate to what extent higher-order thinking occurs in Singapore mathematics classrooms as well as the depth of such thinking. It aims to provide readers with a broad and updated picture of higher-order thinking attainment in mathematics classroom teaching and learning. It also explores the implications and offers suggestions for the improvement of mathematics instruction focusing on the development of students higher-order thinking skills. Methodology To investigate to what extent higher-order thinking occurs in daily mathematics teaching and the depth of such thinking in classrooms, this study uses the data from Panel 3 of the CRPP Core Program which comprise of 118 coded mathematics lessons from 18 primary five classes and 19 secondary three classes by using the Singapore Pedagogy Coding Scheme. The analysis focuses only on specific items in the coding scheme, which we believe to represent the extent to which higher-order thinking occurs in mathematics classrooms. They are identified to be

framing (various types of classroom activities), major source of authoritative knowledge, students produced work, depth of knowledge, knowledge criticism, and knowledge manipulation. Data background The data analyzed in this study is based on 37 units of mathematics lessons observed at both primary five and secondary three levels from 36 schools. The classroom observation includes all streams at both school levels. In particular, the data from the primary level consist of EM1, EM2, and EM3 streams, whereas the data from the secondary level consist of Special, Express, Normal Academic, and Normal Technology streams. The term unit in the coding scheme is defined as a designated topic of the subject to be coded. For instance, area of triangle is a primary five mathematics unit that could be taught over three lessons; inequalities is a secondary three mathematics unit that could be taught over seven lessons. The 37 mathematics units analyzed in this report are made of 118 lessons observed, with an average of 3.2 lessons per unit. In the coding scheme, lessons are further broken down into a series of distinctive classroom activities, termed as phases. To illustrate, a mathematics lesson could start with a whole class lecture (coded as one phase) followed by students self-practice (coded as another phase). Consequently, the 118 mathematics lessons mentioned earlier contain 422 phases, with an average of 3.6 phases per lesson. It should be noted that a phase has a minimum duration of about five minutes. A profile of mathematics lessons observed is given in Table 1. Table 1 A Profile of Mathematics Lessons Observed Primary Five EM1 No. of Units No. of Lessons No. of Phases 5 15 64 EM2 12 38 132 EM3 1 6 54 SP 1 2 5

Secondary Three EXP 13 38 113 NA 3 14 42 NT 2 5 12

Total 37 118 422

Data processing All the observed mathematics lessons are coded using the coding scheme consisting of 33 items with a Likert Scale response format. The data are then collated together based on school levels (i.e., primary five & secondary three). After that, data for primary and data for secondary are analyzed separately at both phase and lesson levels using descriptive statistics, including frequency, average, and percentage. Due to the fact that the focus of this report is to delineate the extent to which higher-order thinking occurs in mathematics classrooms, only 6 out of the 33 items are included in this analysis, which are framing, major source of authoritative knowledge, students produced work, depth of knowledge, knowledge criticism, and knowledge manipulation. Results and Discussion In this section, the results of the study will be reported in the following sequence: patterns of classroom activity, where does the knowledge come from?, students in-class products, complexity of knowledge transfer, critique of knowledge, and process of knowledge transfer, which is parallel to the sequence of the items arranged in the coding scheme. Patterns of classroom activity Patterns of classroom activity are referred to phases termed in the coding scheme, which are defined as shifts between distinct classroom activities. Nine main different types of social arrangement of classroom instruction are identified in the coding scheme, such as whole class lecture (monologue), student demonstrations/presentations, individual seatwork, just to name a few. The data show that whole class answer checking, whole class lecture, and individual seatwork are the three major discourse structures in both primary and secondary mathematics classrooms. In particular, at primary five level, about 39% of the phases observed are doing answer checking with 32% and 13% on individual seatwork and monologue teaching respectively. The dominated discourse structure at secondary three level is also whole class answer checking, occurring in 41% of total phases observed. Differing slightly from the primary mathematics

lessons, more monologue teaching (19%) is observed in the secondary classrooms with less frequent individual seatwork (26%). It appears to us that Singapore mathematics lessons are mainly teacher-centered with low-level teacher-student or student-student interaction. In other words, the majority of the classroom activities are initiated by teachers with students acting in accordance. In fact, the data consistently reveal that the frequency of other patterns of classroom activities occurring in mathematics lessons is low at both school levels. At the primary level, only 3% of the phases observed include whole class elicitation and discussion and 3% involve student demonstrations/presentations. The corresponding percentages at the secondary three level are 8% on whole class elicitation and discussion and 1% on student demonstrations/presentations. The observation reveals that a typical mathematics lesson starts with teachers lecture-style talking followed by students individually working on some form of exercises and then teacher provides answer-checking and/or feedback. According to Productive Pedagogies Classroom Reflection Manual, the instructional process involved in these classroom activities is to simply transmit knowledge and practice procedural routines, which is identified as lower-order thinking (Education Queensland, 2002). Instead, higher-order thinking is referred to students combining facts and ideas and synthesizing, generalizing, explaining, hypothesizing, or arriving at some conclusions or interpretation, whereby teachers role should be to create an environment that allow students to experience, gain understanding, and discover new knowledge through the aforementioned processes. In other words, engaging students in constructing knowledge is one of the main features in higher-order thinking acquisition. Researchers comment that teacher-centered learning approach is effective in transmitting facts and prompting basic knowledge; however, it is not particularly effective in developing students higher-order thinking and problem solving abilities (e.g., Amundsen, Weston, Abrami, & McAlpine, 2003). Similarly, Howe and Warren (1989) point out that the availability of ample amounts of students active participation is an important factor to enhance the development of students higher-order thinking skills. Where does the knowledge come from? For this category, we are looking for the major source of authoritative knowledge that is delivered during mathematics lessons. That is to say, where does the knowledge come from? Does it come from teacher, or textbooks, or internet, or similar media? Or is it initiated by students or gained by students through their own exploration/investigation/ discovery? The observations at both primary five and secondary three levels show that the major source of authoritative knowledge is the classroom teacher. In fact, in 89% of the total phases observed in primary classrooms, the teacher plays the main role in knowledge transmission, whereas similar situation is found in secondary classrooms (77%). Use of textbooks and other similar sources as major source of knowledge transfer occurs in 4% of the phases observed at the primary level; the corresponding data at the secondary level is 13%. Therefore, it is somehow not surprising to note that the knowledge derived from students input at both school levels is not frequent (primary five: 1%; secondary three: 3%). The data above seem to indicate that mathematics lessons are tightly controlled by teachers, with very little resistance from students. Teachers appear to hold the authority to decide what to teach, how to teach, how students should go about learning, and what to be achieved. However, as warned by Hoetker (1969), teachers who too often impose their authority will certainly vitiate students learning and possibly its educational values. During the observations, we also have an impression that students are comfortable with teacher-directed learning environment, where they merely follow teachers instruction. There seems a common belief among students that whatever teachers say must be right and should be accepted without doubts. At the same time, teachers also seldom encourage students to question or offer their own thinking or opinions. Gradually, our students become passive in learning instead of actively involved in generating knowledge on their own. It is believed that passive learning only allows for simplistic understandings of concepts and it is difficult to bring forth the grasp of more complex knowledge (Kuersten, 1998). Interestingly, in the observations, we have also seen some teachers encouraging their students to raise questions, explain the strategies they used in problem solving, and share their ideas or thinking. However, students in those classrooms are not very responsive. Some of the teachers therefore go back to use more talks to continue to feed students with answers and solutions. We believe that in order to promote higher-order thinking, both teachers and students need to rethink of their roles in mathematics classrooms; teachers as facilitators to assist students in developing their learning and students taking responsibilities for monitoring their own learning.

Students in-class products Students in-class products are defined as students producing work during classroom learning, which could be in verbal, written, or other formats. To list a few, students products could be oral responses (short or sustained) and written work (fill-in-blanks, short answers, sustained writing). As reported earlier, whole class answer checking occurs most frequently in the mathematics classrooms observed. Consistently, short oral response as student produced work is found to carry a high percentage of occurrences during the period of classroom observations. At the primary five level, in 82% of the phases, students give short oral answers; at the secondary three level, students also often response to teachers questions via short verbal reply (54%). Most of the time, the observations show that the short answers merely consist of a single syllable, yes or no for instance. Very often, it is also seen that students simply provide a numerical answer to teachers questions without giving any substantial explanations how they get the answers. As a result, students producing sustained oral responses in mathematics classes has a very low frequency (primary five: 2%; secondary three: 4%). It is also observed that students repeatedly produce short written answers in class. Comparatively, primary students generate this type of products more regularly than their secondary peers (primary five: 78%; secondary three: 51%). Consistently, it has been noted that there is little emphasis on engaging students in sustained writing, such as giving explanations, justifications, reasons, arguments, views, opinions, etc. In fact, only 2 out of 59 (3%) primary five mathematics lessons observed are made to write more substantially. Similarly, secondary school students also seldom produce sustained written answers. In the observations, we find that our school teachers put a lot of effort in pre-designing worksheets for in-class use or homework. It is quite apparent in the descriptive data that students in 18% of the lessons observed at the primary level and 22% at the secondary level work on worksheets in mathematics classrooms. These worksheets are usually structured in a way merely meant for drill and rote practice and no much sustained writing is expected. In some cases, although tasks require students to give a certain amount of explanations and justifications, students produce one or two short phrases, or incomplete sentences. Additionally, the fact that a worksheet often consists of a large number of repeated exercises which need students to complete within a short period of time might also attribute students not writing sustained text. It is found that students at both primary and secondary levels also produce some multimodel text in mathematics lessons, although the frequency is not high (2% at both school levels). It is believed that this type of product requires students to integrate diverse knowledge across discipline and might give students the opportunities to use and make connection of knowledge learnt. Moreover, producing multimodel text needs students to source out all possible references, through which students could gain knowledge by themselves. As a matter of fact, classroom observations show that our students are capable to do so if given opportunities. Overall, we find that short oral responses and short written answers are the two dominant students output in mathematics classrooms. These forms of output seem to be effective as well as convenient in checking students mastery of factual knowledge and procedural skills. However, they may not successfully assess students higherorder thinking skills, including apply knowledge to new situations and solve non-routine problems. In fact, Kyle, Bonnstetter, and Gadsden (1988) describe that simply answering questions from the text and completing worksheets are focusing on low level cognitive skills. Furthermore, Wiggins (1992) argues that over-reliance on factual recall and routine procedural exercises will impede the development of learners higher-order thinking. School mathematics teachers need to provide opportunities for students to engage in activities that would allow them to express themselves more, do more sustained writing, and achieve higher level thinking. Complexity of knowledge transfer In this category, knowledge is classified into four different levels based on its complexity: factual/rote/basic, procedural/how to, advanced concepts, and relationships between facts and concepts. It seems that mathematics learning is associated with use of lots of notations and repeatedly applying algorithms. Although in the coding scheme, the first two levels of knowledge are coded separately, we find that they are closely related with insignificant differences in terms of knowledge transfer level in the mathematics context. Advanced concepts are referring to those beyond syllabus requirement, for example teaching the concept of probability at secondary three level. For knowledge dealing with Relationships between facts and concepts, students need to be able to apply what they have learnt in mathematics class to the real world situation. The classroom observations show that of the total phases observed, knowledge transfer in mathematics classrooms often deals with procedural/how to at both primary and secondary levels with occurrence varying from a little, sometimes, to almost always (primary five: 77%, secondary three: 87%). In these phases, we find that

knowledge transfer centering on procedural/how to happens in mathematics classroom with almost always in 88% at the primary level and 76% at the secondary level. Factual/rote/basic is the next major form of knowledge transfer found in the total phases observed (primary five: 36%, secondary five: 46%). Similarly, in these phases, at the primary level, knowledge transfer on factual/rote/basic occurs 69% almost always; the corresponding percentage at the secondary level is 47%. The other two levels of knowledge transfer (i.e., advanced concepts and relationships between facts and concepts) are also found in the mathematics classroom observations but they are less regular. In the primary mathematics lessons, 8% of the total phases observed shows knowledge transfer dealing with advanced concepts and 15% with relationships between facts and concepts. Similar results are obtained in the secondary classrooms. (advanced concepts: 3%, relationships between facts and concepts: 15%). The above data seems to suggest that the mathematics teaching in our schools emphasizes a lot on rote and procedural knowledge acquisition, which is often mechanical-oriented. Under such situations, students just need to memorize and repeatedly practice without much deep understanding of mathematics concepts. We believe that for the rote and procedural skills, once not frequently practiced, the knowledge learnt maybe easily forgotten compared to the knowledge that is acquired through deep understanding. Other researchers also gave similar comments on this type of low level learning. For instance, Wegerif (2002) maintains that the process of higher-order thinking is not mechanical and does not focus on procedural skills. Instead, to promote effective higher-order thinking in students, teaching should involve a variety of components, including problem solving strategies, reflective thinking, and opportunities for meaningful interaction. Rote and procedural skills are important and provide a ground floor in a learning hierarchy with each progressive level of learning building upon all lower levels (Howell & Dunnivant, 1992). However, merely focusing on memorization and drill-and-practice is not sufficient for students to move on to a higher-level learning. Classroom teachers could consider introducing students more advanced concepts where suitable and providing learning of relationships between facts and concepts, which will not only open students view but also give students opportunities to apply the knowledge they learnt. Although many researchers claim that higher-order thinking is hard to define, it is possible to recognize and teach (e.g., Thomas, 1987; Weferif, 2002; Whittington & Newcomb, 1991). Critique of knowledge This coding scheme also examines whether knowledge is criticized explicitly in mathematics classrooms. It includes students expressing their doubts, teacher/students making judgments, and spelling out possible mistakes, errors, or problems. Truth (i.e., knowledge is regarded as having fixed correct answer), Comparison (i.e., different sources or ideas are compared and contrasted), and Critique (i.e., the validity of the sources of knowledge is challenged) are three characteristics being identified to estimate the extent to which knowledge is critiqued. Furthermore, each of these characteristics is coded according to their degree of occurrences. Of the total phases observed in the primary five classes, knowledge is often (95%) presented as Truth; similarly, at the secondary three level, the corresponding percentage is 94%. In fact, knowledge taken as Truth at both school levels occurs 98% as almost always. It is also observed that knowledge being compared and contrasted happens 10% of the total phases at the primary level and 20% at the secondary level. However, the occurrence is not regular with almost always coded as less than 50% of the time (primary: 44%, secondary: 36%). Among the three characteristics, the validity of sources of knowledge being challenged taking place in the total phases observed is the least (primary: 8%, secondary: 11%) and most of the class time, it is coded as a little (primary: 70%, secondary: 61%). The above results indicate that there is little knowledge comparison and knowledge critique in the mathematics classrooms. It seems that knowledge is often transmitted mainly by the teacher, while students at the receiving end simply accept whatever the teacher says as Truth. By presenting information in this way, knowledge is regarded as being static and immutable (Education Queensland, 2002). If students always accept knowledge as a fixed body of information without doubts and questions, then they will fall into the mode of passive learning again, which keeps them at the lower order thinking level. In fact, critique, or being critical, does not means being negative; it could be to identify the differences and similarities, strengths and weaknesses, to make connections and relationships, to examine others perspectives, and to explore gaps in concepts and thoughts. By doing so, giving students opportunities to critically examine concepts, construct knowledge, criticize ideas, they will become producers of knowledge and allow them to engage in higher-order thinking (Education Queensland, 2002). Process of knowledge transfer Process of knowledge transfer refers to student managing, establishing and deconstructing knowledge in the mathematics classrooms. It is classified into four different characteristics, which are Reproduction (i.e.,

regurgitating / replicating / repeating of knowledge that is taught), Interpretation (i.e., providing plausible explanations to content being taught), Application/Problem solving (i.e., applying knowledge to solve mathematics questions and tasks in other disciplines), and Generation of Knowledge New to Students (i.e., gaining new knowledge by themselves). Similarly, each characteristic mentioned above is coded in terms of the degree of its concurrence. Reproduction is observed as the most prevalent form of knowledge manipulation in mathematics classrooms. At the primary level, we find that in 84% of the total phases observed students are merely regurgitating, replicating, and repeating what teacher teaches; similar situation happens at the secondary level (77%). This form of knowledge manipulation occurs 90% of the class time being almost always at the primary level and 82% at the secondary level. Application/Problem solving is the next highest coded form of knowledge manipulation in mathematics classrooms (primary: 22%, secondary: 42%) with occurrence at almost always in 55% of the class time at both school levels. Interpretation as a form of knowledge manipulation is observed in 16% of total phases in primary mathematics lessons and 24% in secondary lessons. In 75% of the class time at the primary level, this process of knowledge transfer takes place either a little or sometimes; similarly, the percentage at the secondary level is 81%. Generation of Knowledge New to Students taking place in the mathematics classrooms has the least occurrence at both school levels (primary: 7%, secondary: 9%) with an insignificant rate of recurrence. It is believed that higher-order thinking processes require the manipulation of information rather than the reproduction of knowledge (Grant, 1988). Similarly, in Blooms taxonomy, the first two levels of thinking processes including knowing (i.e., to recall and memorize knowledge) and understanding (i.e., to comprehend meaning) are considered low level, while the other four levels of thinking processes (i.e., application, analysis, synthesis, evaluation) are considered to be associated with critical thinking (Bloom, Englehart, Furst, Hill, & Krathwohl, 1956). Therefore, to achieve higher-order thinking, teachers should take students beyond recall and reproduction of knowledge to apply and evaluate concepts, arguments, ideas and performance (Leaman, 2002). Although reproduction may be effective in assessing students ability to recall information, define concepts, use algorithm, and solve routine problems, mere reproduction of prior knowledge, as claimed by Newman and Wehlage (1996), does not constitute authentic academic achievement. According to them, reproduction of knowledge does not involve the thoughtful use or application of knowledge found in authentic adult accomplishment. Unfortunately, it seems that conventional curricula and examinations emphasize too much on reproducing knowledge, which we believe must be changed. Summary and Conclusions This paper reports the results from a total number of 118 mathematics classroom observations at primary five and secondary three levels in year 2004, carried out by CRPP, NIE as part of a larger research effort in identifying types of pedagogical practices performed and kinds of knowledge emphasized in daily classroom teaching in Singapore. The focus of this analysis is to examine the extent and depth of higher-order thinking occurring in mathematics classrooms. It is found that the three typical forms of discourse structure in mathematics classrooms are answer-checking, whole-class lecturing, and students doing individual work, which are all teacher-directed. Consistently, the observations reveal that classroom teachers are major source of authoritative knowledge for students learning. Regarding the complexity of knowledge transfer, it is noted that mathematics teaching and learning often deals with routine procedural skills and basic concepts. Students rarely criticize explicitly what have been taught in classrooms, because knowledge is presented as Truth most of the time. Relating to this finding, the analysis also shows that students repeatedly regurgitate and replicate knowledge that is taught to them in mathematics learning. The data reported in this paper seems to imply that higher-order thinking has not been maximized in Singapore mathematics classrooms. Howe and Warren (1989) believe that higher-order thinking skills cannot be learnt well unless teachers emphasize it and use it on a continuing basis. To achieve higher-order thinking, students should be involved in the transformation of knowledge and understanding. This includes creating a communicating environment for students effective interaction, encouraging them to verify, question, criticize, and assess others arguments, engaging in constructing knowledge through various processes, and generating new knowledge through self-exploration. In the students perspectives, they also need to be aware that they must be an active learner taking initiatives and responsibilities in their own learning. As a matter of fact, research and studies over the years have shown that higher-order thinking can be taught, unlocked, nurtured, and developed (Lumsdaine & Lumsdaine, 1994). Therefore, as long as teachers consciously teach thinking skills and provide opportunities for interaction, the implementation and practice of higher-order thinking can be greatly improved in our mathematics classrooms.

References Amundsen, C., Weston, C., Abrami, P., & McAlpine, L. (2003). A faculty development approach that focuses on learning for the effective integration of technology in higher education. Retrieved 24 January, 2005, from http://doe.concordia.ca/cslp/Grants/RA-SSHRC_FDAFLEITHE.php Beaton, A. E., Mullis, I. V. S., Martin, M. O., Gonzalez, E. J., Kelly, D. L., & Smith, T. A. (1996). Mathematics achievement in the middle school year: IEAs Third International Mathematics and Science Study. Chestnut Hill, MA: TIMSS International Study Centre, Boston College. Bloom, B. S., Englehart, M. D., Furst, E. J., Hill, W. H., & Kratlwohl, D. R. (1956). Taxonomy of education objectives: The classification of educational goals. New York: David McKay. Bracey, G. (2000). The TIMSS final year study and report: A critique. Educational Research, 29(4), 4-10. Cai, J. (1997). Beyond computation and correctness: Contributions of open-ended tasks in examining U.S. and Chinese students mathematical performance. Educational Measurement: Issues and Practice, 16(1), 5-11. Education Queensland. (2002) Productive pedagogies classroom reflection manual. Queensland: The State of Queensland (Department of Education). Gonzales, P., Guzmn, J. C., Partelow, L., Pahlke, E., Jocelyn, L., Kastberg, D., et al. (2004). Highlights from the trends in international mathematics and science study (TIMSS) 2003. Washington, DC: National Centre for Education Statistics. Grant, G. E. (1988). Teaching critical thinking. New York: Praeger. Hoetker, J. (1969). Dramatics and the teaching of literature. Champaign, IL: National Council of Teachers of English / ERIC Clearinghouse on the Teaching of English. (ERIC Document Reproduction Service No. ED028165) Howe, R. W., & Warren, C. R. (1989). Teaching critical thinking through environment education. Columbus, OH; ERIC Clearinghouse for Science Mathematics and Environment Education. (ERIC Document Reproduction Service No. ED324193) Kuersten, A. K. (1998). The case of the speluncean explorers: Simulating the appeals court process. Unpublished manuscript. Retrieved 24 January, 2005, from http://www.wmich.edu/teachlearn/winter1999/teaching/facpub. html Kyle, W. C., Jr., Bonnstetter, R. J., & Gadsden, T., Jr. (1988). An implementation study: An analysis of elementary students and teachers attitudes toward science in process-approach vs. traditional science classes. Journal of Research in Science Teaching, 25, 103-120. Leaman, E. (2002). Action research investigating the integration of ICT and higher order thinking. Retrieved 24 January, 2005, from http://www.e-magine.education.tas.gov.au/innovating/Showcase/emilycs.doc Lumsdaine, E., & Lumsdaine, M. (1994). Creative problem solving: Thinking skills for a changing world. New York: McGraw-Hill. Ministry of Education. (2000a). Mathematics syllabus (primary). Singapore: Curriculum Planning and Development Division. Ministry of Education. (2000b). Mathematics syllabus (lower secondary). Singapore: Curriculum Planning and Development Division. Mullis, I. V. S., Martin, M. O., Beaton, A. E., Gonzalez, E. J., Kelly, D. L., & Smith, T. A. (1997). Mathematics achievement in the primary school years: IEAs Third International Mathematics and Science Study. Chestnut Hill, MA: TIMSS International Study Center, Boston College. Mullis, I. V. S., Martin, M. O., Gonzalez, E. J., Gregory, K. D., Garden, R. A., OConnor, et al. (2000). TIMSS 1999 international mathematics report: Findings from IEAs repeat of the Third International Mathematics and Science Study at the eighth grade. Chestnut Hill, MA: Boston College. Newman, F. M., & Wehlage, G. G. (1996). Authentic pedagogy boots student achievement. WCER Highlights, 8(3). Retrieved 24 January, 2005, from http://www.wcer.wisc.edu/publications/WCER_Highlights/Vol.8_No.3_Fall_ 1996/Authentic_Pedagogy.html Thomas, R. G., (1987). Higher order thinking: Definition, meaning, instructional approaches. Washington, DC: Home Economics Education Association. Wang, J. (2001). TIMSS primary and middle school data: Some technical concerns. Educational Researcher, 30(6), 17-21. Wegerif, R. (2002). Report 2: Literature review in thinking skills, technology and learning. UK: NESAT Futurelab, School of Education, Open University. Whittington, M. S., & Newcomb, L. H. (1991). Raising cognitive levels of college instruction. NACTA Journal, 36(2), 8-11. Wiggins, G. (1992). Creating tests worth taking. Educational Leadership, 49(8), 26-33.