Sie sind auf Seite 1von 13

Impact Evaluation of the Intel Teach to the Future Program in Malaysia by Associate Professor Toh Seong Chong Centre

for Instructional Technology and Multimedia Universiti Sains Malaysia

Cited as: Toh, S. C., (2009). Impact Evaluation of the Intel Teach to The Future Program in Malaysia. Oral presentation at the 13th UNESCO-APEID Conference on Education and World Bank-KERIS High Level Seminar on ICT in Education, Hangzhou, Peoples Republic of China, 15 17 November, 2009.

Abstract Intel Teach to the Future is a worldwide effort to train classroom teachers how to promote project-based learning and to effectively integrate technology into instruction. Since its launching in 2000 in Malaysia, more than 29,000 Malaysian teachers had undergone is training program. This paper presents findings from an impact study of this program in Malaysia. The study was conducted by a team of evaluators from the Universiti Sains Malaysia . A total of 2,095 survey forms were distributed to the teachers. The results showed that the program had opened the horizons of teachers to the rich potential of technology in the design of lessons that are motivating and support high-order thinking skills amongst students and has helped teachers to reflect on their practices, and ultimately integrate educational technology into their teaching. Additionally, more teachers from the rural schools implement their unit plan more than once compared to the teachers from urban schools. 65% of Malaysian teachers indicated that they are integrating technology in new ways upon completion of the training program. However, several weaknesses in this program were also highlighted, and the paper concludes with suggestions on how to optimize and integrate ICT skills in the teaching and learning in Malaysia schools. Introduction As we enter the 21st century, there has been considerable international attention given to the role that ICT can play in economic, social, and educational change. Equally it is creating enormous challenges, confronting countries with the need to rethink their educational and social systems (Kozma 2003). To participate in this global knowledge, Kozma stresses the need for students to leave school with a deeper understanding of school subjects and with the skills needed to respond to an unbounded but uncertain 21st century skills to use their knowledge, to think

critically, to collaborate, to communicate, to solve problems, to create and to continue to learn. Taking cognizance of this need, Intel has taken a worldwide initiative known as Intel Teach to the Future program to train classroom teachers on skills to promote project-based learning and to effectively integrate technology into instruction. This program adopted the train-the-trainer approach to deliver a technology-rich curriculum that emphasizes using commonly used Microsoft software tools to support students in conducting original inquiries and creating multiple representations of what they learn. A core group of teachers called Master Facilitators (MF) are first trained, and these teachers in turn become trainers to train more teachers from their respective schools in this program. To achieve this end, the program focuses on four specific goals of the Innovation in Education initiatives: enhancing the effective integration of technology in the classroom teaching, supporting the teaching of critical thinking skills, developing school leaders, and supporting educators working in informal learning environments. Since its launching in 2000 in Malaysia, more than 29,000 Malaysian teachers had undergone is training program. Is the program effective? To what extent has the program achieved its desired goals? To answer these compelling questions, there is a need to carrying out a comprehensive evaluation using a robust evaluation model. This paper presents findings from an impact study of the Intel Teach to the Future Program in Malaysia, conducted by a team of evaluators from the Universiti Sains Malaysia (USM). Stufflebeam CIPP Evaluation Model The Stufflebeam (2000) CIPP Evaluation Model was used in this evaluation study. CIPP is an acronym for Context, Input, Process and Product evaluation. In general, these four parts of an evaluation respectively ask, 1. What needs to be done? - The Evaluation of the Context 2. How should it be done? - The Evaluation of the Input 3. Is it being done? - The Evaluation of the Process 4. Did it succeed? - The Evaluation of the Products/Outcomes For the present impact study, further modifications were made to the CIPP model to accommodate the new demands of different variables. A comprehensive review of four components and their variables is given in Figure 1 below:Components

Intel Teach to the Future Objectives Human Resources Funding and Operations Targets Meeting Educational and Pedagogical Development Changes Projections for Future Developments Systems Capabilities Procedural Design for Implementing the Strategies Budget Allocations and Schedules




Human Resources Quality of Operations Front-End Analyses Supportive Systems Analyses Program Quality Feedback Strategies and Analyses Educational Development and Pedagogical Matching Interpretation of Worth /Merit of Program Products/Outcomes

Figure 1: Stufflebeam CIPP Evaluation Model A multi-dimensional approach was proposed for the conduct of the product evaluation. The following strategies are proposed for the product evaluation. (i) (ii) (iii) (iv) (v) (vi) Sampling of teachers for evaluation ITTF utilization. Sampling of ITTF operation at district/school levels for evaluation of delivery services. Sampling of products for evaluation of subject intents/teacher needs. Conduct interview, checks etc to determine data received in an actual situation. Conduct post-hoc interview with ITTF officers. Write preliminary reports on product evaluation.

This evaluation posed the following research questions, with a goal of generating insight into current program implementation and follow-up future Intel Teach to the Future courses.

To what extent does the Intel Teach to the Future training influence teachers understanding and utilization of the pedagogy and associated resources?

To what extent participating teachers who have gone through the Intel Teach to the Future training integrate technology in the classroom and make use of the resources? What are the barriers towards the implementation of this program? Data sources gathered in the course of this formative evaluation included surveys, observations, visits to schools and interviews. The Context of Intel Teach to the Future (ITTF) Training in Malaysia Malaysia consists of 13 States and the Federal Territories of Kuala Lumpur, Putrajaya and Labuan which is an island off the coast of Sabah over in East Malaysia. As of April 2006, about 29,000 teachers have undergone the Intel Teach to the Future (ITTF) training. Intel Teach to the Future is a worldwide training program that helps experienced and pre-service teachers integrate ICT into instruction and enhance student learning by supporting the teaching of critical thinking skills. The training program focuses on integrating the use of Microsoft software applications such as Microsoft Word, Publisher, PowerPoint,

Encarta and Excel into key learning areas. Forty hours of training was provided and divided into various modules. It emphasizes on the following objectives: helps teachers develop unit plans and portfolios that advance their skills in locating resources; creating student multimedia presentations, publications and websites; creating student and teacher support materials; developing implementation plans; and showcasing unit portfolios facilitates the effective use of ICT in the classroom supporting the teaching of critical thinking skills focuses on the ways students and teachers can use ICT to enhance learning through research, communication, and productivity strategies and tools emphasizes hands-on learning and the creation of work units and assessment tools that are linked to the Malaysian school curriculum and syllabus promotes engaging opportunities for students through access to ICT and project-based learning encourages teachers to work in teams, problem-solve and participate in peer reviews of their units. The training program for Intel Teach to the Future was prepared by the Institute of Computer Technology (ICT), Sunnyvale, California, U.S.A. In Malaysia, Universiti Sains Malaysia (USM) has been appointed to localize and adapt the contents to suit the needs of the Malaysian educational system, teachers and students. Results of Quantitative Analysis of Responses from Teachers The research team aggregates and analyzes all data from impact surveys submitted by participating teachers in three states from Malaysia, namely, the Federal Territory of Kuala Lumpur, Penang state and Malacca state. A total of 2,095 survey forms were distributed through post to the teachers from 78 schools. This constitutes about 10% of the 29,000 teachers. Key Findings On the whole, the survey data indicates that, this program has opened the horizons of Malaysian teachers to the rich potential of technology in the design of lessons that are motivating and support high-order thinking skills amongst their students. Subsequent to that, it helped Malaysian teachers to reflect on their practices, provides avenues for them towards reforming their practices, and ultimately integrate educational technology into their teaching. The key findings of this survey are given below:Successes: The ITTF program has reached even in remote urban Malaysian schools thus bridging digital divide concern Through very careful implementation, the program has reached many Malaysian schools both urban and rural. Even the remote Malaysian rural schools are not left behind. The Intel Teach to the Future program has become a familiar household word even amongst teachers in the remote

schools. It is heartening to note that in terms of unit plan implementation with the schools location (urban or rural), more teachers from the rural schools (31.3%) implement their unit plan more than once compared to the teachers from urban schools (24.3%). Clearly this program had bridge the digital divide between rural and urban schools. Teachers feel prepared to integrate technology after this training 65.15% of teachers participating in Intel Teach to the Future trainings reported that the ideas and skills they learned through the program would help them to successfully integrate technology into their students activities. These teachers valued the opportunity the workshops gave them to think about supporting their pedagogy with technology and creating usable materials. Teachers rated the training positively and reported feeling more prepared to integrate technology into their teaching after taking part in the program. Successful implementation of the unit plan designed during the Essentials Training Course is a key strategy for helping teachers meet the goals of the program. Overall, 50 % of the teachers reported having implemented all or part of their unit plan at least once; 25.9 % have used their unit plan multiple times. Teachers who had implemented their unit plans felt very strongly that their unit had been effective in helping them to meet their learning goals for their students, in terms of more in-depth understanding of a topic, students projects becoming more creative and increasing students motivation and involvement in the lesson. In a school in Kuala Lumpur that the research team visited, a high school teacher become so excited of her success in implementation of her unit plans that she upload her unit plans into the school server computer for other teachers in her school to adopt. Also this study found that there is a positive correlation between the teachers implementation the unit plan and frequency of usage of technologies and teaching methods emphasized in this training. Teachers who see the beneficial effects of unit plans tends to harness technology more often to carry out more technology-related activities in their teaching. The program is supporting teachers in integrating new student technology activities. 65.15% of Malaysian teachers indicate that they are integrating technology in new ways upon completion of the training program. These technology activities enable students to leverage the power of technology to gain a deeper conceptual understanding of subject matter and to produce new intellectual and creative works based on this understanding. Students participating in the program learn to find useful information on the Internet, solve science and mathematical problems, use tools to organize and generate information, and create presentations and other media products that incorporate pictures, designs, charts, and written text that they have developed. The skills have enable students to communicate effectively, work well in teams, learn continuously, and use digital technology. Teachers are increasing the use of Technology for Lesson Planning and Preparation

The Essentials Course also offers teachers the opportunity to experiment with new ways to use technology for lesson planning, preparation and school administration. These new ways include: How to use the Internet to find information and classroom resources, and

create teacher support materials. Innovative ways of using technology for planning, for administrative activities and Innovative ways to present information to students In this context, the question asked, Since completing your Intel Teach to the Future training, do you use textbook as a primary guide for instruction more often or less often? The teachers reported that they are now less dependent on textbook as the primary delivery medium According to this indicator, 17.9 % of teachers reported that they use textbook less, implying that they are using technology more in their lesson planning and preparation. Increasing innovative use of computers for administrative work after training The survey also asked participant teachers about changes in their use of computers for administrative purposes since completing the Essentials Course. This activity could include creating teacher tools like class rosters for attendance or grading, handouts or worksheets. 84 % of teachers who answered this question reported that they had increased their use of computers for administrative work since the training. Familiarity with the targeted teaching methods has a direct effect on the teachers follow up to training The survey shows that there is a direct relationship between participants prior familiarity with the targeted teaching methods and the optimal outcome measure of integrating new technology activities into their teaching. The survey data indicated that across all levels of prior familiarity, large numbers of teachers report integrating new technology activities into their teaching. However, teachers who are more familiar with the teaching methods does appear to facilitate more integration of new technology activities. Teachers with no prior knowledge show lower integration of student technology activities. Teachers who understand the relevance of the teaching methods presented in the training are more likely to integrate technology into their teaching and to experiment with new teaching methods. The data on the relationship of teachers perceptions of relevance and using new technology activities with their students suggests the importance of giving teachers time during their training to discuss whether and how they see connections between their current teaching practices and project-based, student-cantered approaches to teaching. Teachers who understand the relevance of the teaching methods are more likely to buy this approach and integrate them to their everyday teaching practiced. Students are becoming more motivated and actively involved after teachers adopt the ITTF program.

Students worked together more often after using the ITTF program. The ITTF program addressed students different learning styles. Students showed more in-depth understanding of content. Students better able to communicate their ideas and opinions with greater confidence. Students use computers at home to do their schoolwork. Computers are now ubiquitous in most Malaysian homes. Barriers: The four factors, namely (i) infrastructure, (ii) time constraint, (iii) over worked teachers and (iii) administrative support, emerge time and time again as the variables within schools as systems and learning organizations that make a difference in teachers ability to integrate technology into their teaching. Infrastructure Infrastructure is consistently raised as an issue by teachers working in a wide range of access conditions. In Malaysia, despite the governments efforts to provide the adequate infrastructural facilities to all schools, disparities still exist between rural and urban schools. It should be noted that despite the vast inequalities in technology and provision of resources in general, experiences of teachers from both ends of this spectrum, when taking part Intel Teach to the Future training, did not differ considerably; almost all teachers found it challenging to design projects based on inquiry learning and produce appropriate assessment strategies. Teachers from both well-resourced schools and teachers from schools who are not so well-equipped in terms of infrastructure wrestled with the pedagogy. Almost all teachers struggled with their own creativity when designing projects that were intended to encourage learners to think but fortunately almost all emerged with a considerable sense of achievement. Time Time constraints are consistently mentioned as an obstacle to technology integration by program participants. This is especially amongst Malaysia teachers where they are overwhelmed by other programs and priorities which take precedence over the all other educational innovations. Although lack of time is a difficult problem to resolve, there are ways that program design can address the issue. One way is to have a ITTTF user group when all MFs and PTs can exchanges ideas and offer solutions in a fast and effective manner. Also future ITTTF trainings should incorporate interactive training CDs to enable teachers to miss or not very sure about a particular part of the module to refer to again at his own pace. In addition, data suggest that offering incentives, especially in the early stages of the program, is essential for encouraging both MTs and PTs to dedicate their time to the training. Systematic exploration of the importance of incentives will better enable program staff to make decisions about how to allocate resources.

Weakness in the cascading model of Implementation The cascading model of implementation (also known as Train the trainer) entails training of a small group of trainers (Master Facilitators) who in turn deliver training to educators (Participating Teachers) in a school who then implement the methodology in their classrooms. This study found that crucial in the successful implementation of the ITTF program is that to what extent the training is actually carried out from the Master Fascinators level to the Participating Teachers level in the true sense where all the attributes on the program are actually taught and emphasized? In schools where MFs have strong support from the school administration to train the predicating teachers, the success rate of the program is very high. In schools where MFs receive only lukewarm support from the school administration, the MFs have an uphill task in selling the program to participating teachers of the school. Teachers in these schools would not take the training seriously. There were evidence of diluted content propagation and dubious methodology in some schools. Over-worked teachers In Malaysia, teachers put in long hours at school, contrary to public perception that teachers work only half day. Teachers spend an average of 13 hours a day or about 66 hours a week (five working days) inside and outside the classroom and not five to seven hours a day based on the school timetable. Their working hours is double that of a regular civil servant who is expected to put in 38 hours and 30 minutes per week, according to regulations under the Malaysian Public Service Department (PSD). The time spent by Malaysian teachers into nine areas, including curriculum, co-curriculum, community relationship and staff development. Of the 48 hours spent on curriculum, teachers spent about 15 hours or 27 periods a week on teaching. The rest is used for planning, research, preparation of teaching materials and notes; marking; replacement classes; extra classes; meetings and courses; discussions with parents and students; motivational programs; research; and academic co-curriculum. Other factors also directly or indirectly affected teachers workload. These include bureaucracy, weaknesses in the delivery system, transfers, class size, community expectations and co-curriculum activities. As the ITTF program needs time and effort to prepare the Essential Questions and Unit Questions, teachers are hard-pressed of time to carry out this endeavor. Slight mismatch between the original ITTF program design and the Malaysian Education context The reason of this mismatch could be due to the technology infrastructure available to teachers across urban and rural schools at the time of the study. Most Malaysian schools reported that they have Internet access in the school computer labs access only and not in the classrooms. Since the original program was designed initially to meet

the needs of teachers in the US, further localization and adaptation of the program needs to be done to meet challenges as the Essentials Course in Malaysian context. However, with the rapid proliferation of Wifi technology in Malaysia, this problem is expected to be reduced. Hawthorne effect There are some teachers who are experimenting with new technology tools and with new teaching practices. They have not truly internalized or truly grasp the essence of Essential Course and Unit plans. These teachers have not typically achieved a high level of proficiency in designing technology-rich lessons that really exploit the learning potential of these technology tools, or a depth of understanding of how specific uses of various technology tools can best support student learning. Superficial understanding of Unit Plans and Essential Lessons Some teachers view some activities as equivalent to one another, rather than seeing them as distinct work products associated with distinct learning goals. For example, one reason teachers often use only part of their unit plan (for example, the PowerPoint presentation but not the webpage design) is because of this perception. Therefore, the changes in teachers willingness and readiness to use technology with their students do not necessarily translate into significant changes in what or how their students are learning. Recommendations In order for the Intel Teach to the Future to be sustainable, the following recommendations are proposed:1. Provide more training in designing curriculum framing questions. The single most disappointing aspect of the ITTF Essential Question training so far has been the teachers lack of success in grasping the concepts of curriculum-framing questions. From our interview with teachers, the team discovered that the following skills are evidently lacking:
the lack of ability to distinguish between what are essential (critical)

questions and unit questions the lack of attention to unit questions that stimulate higher order thinking the lack of attention to the unit question in the portfolio documents Critical questions are confusing to participants. It is imperative that subsequent ITTF trainings should address this need. 2. Need to provide long-term nurturing follow-up activities to schools Evaluation findings to date suggest that Intel Teach to the Future is a valuable professional development program for Malaysian teachers working in a broad range of contexts. The program provides support for the development of teacher skills and knowledge in project-based learning and general ICT literacy, which initial evaluation findings indicate is crucial to its success. Thus far, the ITTF program in Malaysia

has provided substantial software and training to educational institutions. But to most school teachers, this looks like a one-off training with little or no thought to how to support the long-term use of those resources continuously. For the program to be sustainable, efforts must be to towards nurturing long-term and sustained relationships with governmental and educational stakeholders especially participating teachers in order to allow the program to take hold and flourish. 3. Teachers who have been through Intel Teach to the Future Essentials training program need two key forms of further support in order to help them move toward integrating technology in ways that will really have an impact on student learning. Firstly, more opportunities should be provided to refine the teachers assessment strategies for technology-rich student work. More powerful tools and strategies should be in-built in future Essential course training. These strategies should cover both the formative and summative aspect of assessment. Secondly, whilst the current Essentials course offers some opportunities to think about appropriate assessment, future Essential course training modules should provide more opportunities to teachers to enable them reflect critically about exactly what they expect technology tools can do to add to their students learning. 4. Teachers also need further training that is technology-rich but that is more intensively focused on enhancing teachers understanding of students metacognition ( how students learn) and how lessons and units can be designed to scaffold that learning process (designing a constructivist learning environment). From interviews with teachers, we received very positive feedback that they need further training on pedagogy in the Essentials course. This suggests that further emphasis on student learning and effective pedagogy would be well-received in future trainings and resources. They would also welcome training that enhances them to design technology-based constructivist-learning environment (that is learning environment with technologies to keep students active, constructive, collaborative, intentional, complex, contextual, conversational, and reflective). 5. In designing such follow-up training, it is important to care out proper needs assessment to take into consideration where teachers are starting from in their understanding of these pedagogical concepts, and teachers own instructional goals and curricular requirements. Standing on the essential course that was designed to build on what teachers already do in their classrooms, subsequent follow-up trainings should build on teachers initial experiences integrating project-based technology lessons by having them reflect on the opportunities and challenges they encountered and then revise existing lessons or create new ones to better support student learning. In this regard, proper needs assessment to ascertain what the teachers need in terms of skills, strategies and instructional design so that further training can be

customized to the teachers needs. To achieve this, it is well recognized that teachers need professional development, not only in technology skills and applications, but also in new pedagogical methods of incorporating technology into the classroom (Carlson and Gadio 2002, Snider, 2003). 6. Provide a repository server to deposit the Unit plans as Reusable Learning Objects. The Unit Plans developed in the Intel Teach to the Future training provide a rich repertoire of content for teachers. If deposited in the school server computer as Reusable Learning Objects, these materials instantly provide a very useful resource that all teachers in the school can share. The market is demanding a quicker and less-expensive way to build and maintain content. Other than RLOs, there are no other development strategies that have emerged promising a quicker time to market, reduced cost to produce learning, and a single maintenance source for whatever courseware that needs updating. Initially, the Intel Teach Website can provide this facility so that teachers worldwide can link their materials to it. It can be modelled like the MERLOT server. ( 7. Programs success anchors on the Masters Teachers and the School Headmasters. From our visits to schools and from survey feedback, we found that the linchpin of the programs impact is the Master Teachers and the School Headmasters. The Headmaster provides the administrative support whilst the Master Teachers are the innovators and early adaptors of innovation. Their hard work, their dedication to the program, and probably their previous status as leaders or experienced teachers within their districts is important to make it possible for them to extend the impact of this program, in many districts, beyond the individual teachers who participated in the training. 8. Establishing district relationships helps create conditions for success. In Malaysia, there is District Teachers Activity Centers (DTAC) in every district which provides teaching-learning support for schools in the vicinity of that district. The DTACs have a profound influence on the schools in that district. This fits very well with the Intel training model where the Master Teachers provide training for the Participating Teachers. Teachers from these district teachers activity centers should be enlisted as partners. The DTAC can then identify local educators to become Master Teachers based on an understanding of both the pedagogical and technical aspects of the curriculum. This is necessary to meet their professional development needs by recruiting and training as many Master Teachers as they needed to provide their local training. This strategy helped to establish cohorts of trained teachers in individual schools who were then able to support each other in their work, and helped to develop many Master Teachers into district-level technology

leaders with a deep understanding of how technology could be used to support instruction. 9. Strategies to help create an environment in which more Master Teachers would be able to expand this programs impact effectively within their school districts. The program itself and associated trainings need to be represented to all teachers and administrators as a coherent sequence of trainings that guide teachers through a process of developing the skills and knowledge they need to use technology to support student learning. When districtlevel administrators do pay attention to Intel Teach to the Future, they do so because this program demonstrates to them that technology training can be focused on improving instruction and supporting student learning, and that it can be more than just technical training. There is considerable variation among the schools represented in this report regarding the maturity of their Intel Teach to the Future programs, and in the structure and design of their programs. Intel Teach to the Future program staff needs to take cognizant of this fact and ensure that evaluation is used to help all of us understand what success can look like in a variety of cases. Continuing to invest in Master Teachers through further training and possibly through other forms of support such as mini-grants for research or to extend local training opportunities is important both to maintain Master Teachers connections to Intel and to build their professional knowledge and stature within their districts. CONCLUSION Intel Innovation in Education innovation initiative has provided 29,000 Malaysian teachers with very interesting and innovative training on how to integrate technology in teaching. From interviews conducted with school teachers, the consensus amongst Malaysian teachers is that they are very grateful for this noble and altruistic effort of Intel. It has given the pervasive perception that Intel is an asset to the community. Evaluation findings to date suggest that Intel Teach to the Future is a valuable professional development program for teachers working in a broad range of contexts. The program provides ongoing and continuous support for the development of teacher skills and knowledge in project-based learning. However, for continual sustainability, much still needs to be done. Future work should focus on the following:- (i) development of Interactive CDs with video clips on unit plans procedures so that teachers can revise the steps at their own pace, (ii) development of Reusable Learning Objects for good unit plans, have these meta-tagged and deposit them on Intel Innovation in Education website, (iii) encourage teachers to have e-groups to exchange ideas and solve teachinglearning problems, (iv) explore specific ways to heighten motivation through good interactivity design in unit plans, (v) making deliberate efforts to provide ongoing professional development and support to Master Facilitators. Reference

Carlson, S. and C. T. Gadio. 2002. Teacher Professional Development in the Use of Technology, in Haddad, W. and A. Drexler (eds). Technologies for Education: Potentials, Parameters, and Prospects. Washington DC: Academy for Educational Development and Paris: UNESCO . Kozma, R. (2003). Technology and classroom practices: An international study. Journal of Research on Computers in Education, 36 (1), 1-14. Snider, S. (2003). Exploring technology integration in a field-based teacher education program: Implementation efforts and findings. Journal of Research on Technology in Education 34 (3), 230-249. Stufflebeam, D.L. (2000). The CIPP model for evaluation. In D.L. Stufflebeam, G. F. Madaus, & T. Kellaghan, (Eds.), Evaluation models (2nd ed.). (Chapter 16). Boston: Kluwer Academic Publishers. Bio-data of Writer Dr. Toh Seong Chong is an associate professor from the Centre for Instructional and Multimedia, Universiti Sains Malaysia. His research interests include Multimedia Project Management, Web-based Design and Development, On-line Testing Systems, Rapid Prototyping of Courseware, Constructivist Learning Environment and developing Reusable Learning Objects. He has successfully carried out several multimedia training projects for multi-national companies as well as the university. He was awarded the Innovative Staff of the School of Educational Studies, USM in 2002 and the USM Excellent Educators Pioneer Award in 2003. His e-mail is