Jl.

of Technology and Teacher Education (1999) 7(1), 75-81

Contextual Variables in a Technology-Based Teacher Education Project

DAVID KUMAR College of Education, Florida Atlantic University 2912 College Avenue Davie, Florida 33314, USA david@fau.edu JAMES ALTSCHULD College of Education, The Ohio State University 29 West Woodruff Avenue Columbus, Ohio 43210, USA

In this paper the application of a synthesized program evaluation model emphasizing contextual/environmental factors is described. The model recognizes that the life cycle of projects is the core of what is to be evaluated, and the life cycle is embedded in an environment of personal, social and economic factors. Therefore evaluation of large scale education programs and projects should focus on factors such as organizational climate, evidence that the environment was supportive of change and experimentation, administrative interest in and knowledge of the project, and the availability of resources to facilitate users to adopt new products and processes, and so forth. The evaluation model was applied to study the interactive technology-based teacher education project of Vanderbilt University. The findings revealed that the following factors worked together for project success and they are: (a) conducive environment for innovation and development in the use of interactive technology for teacher education; (b) faculty interest and commitment to integrate technology into their instruction both inside and outside education; (c) administrative support, and: (d) strong technical support.

76

Kumar and Altschuld

In this paper a contextual approach to evaluation of educational technology in teacher education is discussed. The study applied a context-specific model of program evaluation (Altschuld & Kumar, 1995) to evaluate Vanderbilt Universitys National Science Foundation (NSF) funded interactive media teacher education project, Improving Science Education: A Collaborative Approach to the Preparation of Elementary School Teachers. Technology has gained considerable access to education, and often educators are expected to accept technology on face value with less or no sufficient supportive evidence of its quality, impact and physical and procedural requirements for successful implementation. According to Edmunds (cited in Branscum, 1992) we have a long way to go (p. 84) before educational technology is integrated into instruction. While technology makes inroads to education, science education is undergoing rapid growth due to the proliferation of scientific information and developments in theories of learning and problem solving. These changes have collectively increased the complexity of classroom science instruction. One of the ways of reforming science instruction would be to revise science teacher education programs with special emphasis being given to the use of technology for classroom instruction. A case in point, interactive videodisk (IVD) technology, has been a center of attention in science education. The use of IVD for teaching methods by Goldman and Barron (1990), and for teaching content by Vitale and Romance (1992) are examples of successful utilization of educational technology in preservice science education. While in isolated instances it seems that technology and science teacher education are allied to reform education, there is no guarantee that this alliance will improve preservice teacher learning. A lack of suitable frameworks for evaluating the effectiveness of technology programs in science education is a matter of concern. Most of the existing evaluation models in science education tend to focus on end products and do not seem to provide a sound basis for comprehensive evaluation efforts with emphasis on contextual and supportive factors (Altschuld & Kumar, 1995). DESCRIPTION OF A CONTEXT-BASED EVALUATION MODEL Evaluation, primarily of a formative nature, regarding the systematic development of curricula, products, and processes is evident in the writings of Pines (1980), Mayer and Stoever (1978), and, to a lesser degree, Small (1988). While the evaluation of developmental steps is prominent in Pines, Mayer and Stoever, and Small, it was not solely confined to those authors.

Contextual Variables in Technology-Based Teacher Education

77

Welch (1974), Virginias Standards of Learning-Science (SOL-Science) by Exline (1985), and the Evaluation of a Child-Structured Curriculum by Espejo, Good and Westmeyer (1975) have developmental components. Also it should be noted that the works of Exline (1985), Exline and Tonelson (1987), Welch (1974), and Shell, Horn, and Severs (1986) in one form or another stress contextual evaluation. At the contextual level, Exline (1985) portrays the SOL-Science, as a comprehensive approach to evaluation. One of the notable aspects of SOL-Science is the recognition of the need for a buy in period for administrators, parents, teachers and other stakeholders. Consequently, six critical support componentsadministrative support, learning environment, teacher preparation, community involvement, fixed facilities, and instructional materialswere identified and must be in place for educational programs to be successful. That is, quality in science education is a function of the degree to which programs are supported and perceived positively by relevant constituencies. Welch (1974) emphasized that contextual evaluation would not only be of value in understanding the system but also in determining needs. Over two decades ago, Stufflebeam (1971) suggested incorporating contextual factors in program evaluation. From this point of view, the evaluation model proposed by Altschuld and Kumar (1995) recognizes that the life cycle of a program or project in science education is seen as the central core of what is to be evaluated. Since that life cycle exists in a complex mix of social, economic and personal factors, evaluations of large scale science education programs should focus on variables such as organizational climate, evidence that the environment was supportive of change and experimentation, administrative interest in and knowledge of the program, and the availability of resources to enable users to adopt new products and processes. For a description of the evaluation model used in this study, see Altschuld and Kumar (1995). For an extensive analysis of the evaluation that was conducted, see Kumar and Altschuld (in review). PROCEDURE The evaluation of the interactive media teacher education project of Vanderbilt University proceeded as follows. (For a description of the Vanderbilts interactive media project see Barron, Joesten, Goldman, Hofwolt, Bibring, Holladay, & Sherwood (1993).) An in-depth examination of background documents was carried out, looking for major issues and potential questions that would seem to be important for conducting site visits and

78

Kumar and Altschuld

a contextual evaluation of the project. After document review, the ideas emerging from the documents were distilled into a set of key interview questions for the project staff and students. The content of those interview questions included administrative support, financial resources, physical facilities, understanding and knowledge of the project and program, availability of resources, fit of the program into the environment, attitude, and enthusiasm. A few examples of questions used in the interview protocols were as follows: In your judgment, what are or would be the long-term outcomes of the project and the use of interactive media; on students, on the faculty/ academic environment?; What did you feel that you learned from the interactive media portion of the program? Please be as specific as you can in your answer; If another teacher education institution or another area were to adopt a project like this, what specifically would you recommend that they do or consider, and why? (Kumar & Altschuld, 1996, p. 36-39). The sample selected for interviews consisted of the project director, university administrators, faculty members inside and outside science education, a school administrator, students currently enrolled in the projectbased teacher preparation program at Vanderbilt University, graduates currently teaching in Nashville area schools, a video editor, and a computer programmer. The interviews took place during 1994-1995 academic year. The evaluators taped all interviews in addition to taking individual notes. Each team member analyzed his interview notes independently. While one team member had tapes transcribed and conducted his analysis from the transcriptions and the notes he had taken, the other member typed interview notes and placed all comments and responses together by question. Hence, by working independently, a degree of independent verification of results was reached. Then, using the following analysis strategy, interview data was examined to create meaning out of the interviews: Coding and the ConstantComparative Method; Describing the Initial Data Categories; Looking Across the Categories for Emergent Explanatory Themes; and Seeking Verification for the Explanatory Themes (in a second set of interviews conducted approximately six months after the first.). The evaluation study generated two sets of results: Initial Data Categories (IDC); and Emerging Explanatory Themes. The Initial Data Categories contain a class of responses that were consistent and repeated throughout the interviews as identified through individual analyses of data. Points of overlaps and differences were examined and a consensus list of key variables was constructed. The following are IDCs from faculty, staff and university administrator interviews: Student learning; Student opportunities to

Contextual Variables in Technology-Based Teacher Education

79

learn; Instruction; Effects upon instructional staff; and Institutional effects. The Emergent Explanatory Themes on the other hand are concepts that span across the Initial Data Categories. They explain connections and ideas, and provide a picture of the overall context of the interactive media project of Vanderbilt University. They are: Administrative support; Osmosis/Permeation of project ideas into the environment; Student perception on campus; Student perception in schools; Technical support; Organizational climate; and Critical Mass (especially in regard to the number of individuals participating in the development of interactive media instructional products). EVALUATIVE CONCLUSIONS The evaluation study led to the following conclusions. Both current students and former graduates strongly perceived that the interactive mediabased science methods courses influenced their understanding of discovery science and reflective teaching and helped them to integrate content into methodology and teach hands-on science. They also felt that real classroom situations often did not meet their expectations of classrooms portrayed by the project videos. The project seems to receive considerable administrative support within the university environment in terms of financial assistance, technical support, motivation, and incentives for more faculty involvement throughout the university. The project had permeated from science education into other disciplines. Faculty members outside science education (e.g., social studies education, biology) have adopted the interactive media approaches to teaching. A nourishing environment is critical for successful development and implementation of innovative ideas that call for considerable changes away from traditional classroom practices to educational technology-based instruction in preservice education. Vanderbilt University has provided such a favorable environment for the NSF interactive media project in the form of strong administrative support and readily available technical assistance as apparent in the evaluation. The university administration has committed itself to maintain familiarity with the project and to promote project activities. For example, high ranking university officials have taken an interest in presenting the project at national forums and to potential funding agencies. Along with the administrators, the faculty members, graduate students and other staff of the interactive hypermedia project were dedicated to the project resulting in a critical mass. Faculty members outside science education began to adopt the innovation in their classes because they believed

80

Kumar and Altschuld

the project ideas and materials provided a timely opportunity to improve their teaching practices. In addition, in comparison to the high developmental costs, the project materials are not very expensive, are well organized, and integrate well into established courses in teacher education. The semistructured interview approach taken in this evaluation of Vanderbilt Universitys interactive media science teacher education project has shifted the nature of the study from product- to context-based qualitative evaluation of contextual (or environmental) variables. To improve the evaluation, the terms context and contextual variable need to be defined in depth. Also, the interviews could have focused a bit more on contextual variables, and some interview questions could have been streamlined. On the other hand, the evaluation reported here helps to clarify the idea that looking at the context of development is necessary to gain a clearer and full picture of large-scale innovative educational technology projects in science education. References
Altschuld, J.W., & Kumar, D.D. (1995). Program evaluation in science education: The model perspective. New Directions for Program Evaluation, No. 65, 5-17. Barron, L.C., Joesten, M.D., Goldman, E.S., Hofwolt, C.A., Bibring, J.B., Holladay, W.G., & Sherwood, R.D. (1993). Improving science education: A collaborative approach to the preparation of elementary school teachers. A final report to the National Science Foundation under grant number TPE-8950310. Nashville, TN: Vanderbilt University. Branscum, D. (September, 1992). Educators need support to make computing meaningful. It is time to wield technology wisely. Macworld, pp. 83-88. Espejo, M., Good, R., & Westmeyer, P. (1975). Evaluation of a child-structured science curriculum using the intellectual models of Piaget and Guilford. Journal of Research in Science Teaching, 12(2), 147-155. Exline, J.D. (1985). Virginias long range plan for science education. NSTAs Supplement of Science Education Suppliers. Exline, J.D., & Tonelson, S.W. (1987). Virginias science education program assessment model resource guide. NSTAs Supplement of Science Education Suppliers. Goldman, E., & Barron, L. (1990). Using hypermedia to improve the preparation of elementary teachers. Journal of Teacher Education, 41(3), 21031. Kumar, D.D., & Altschuld, J.W. (1996). Evaluation of educational technology in preservice teacher education. (A report to the AERA Grants Program). Davie, FL: Florida Atlantic University.

Contextual Variables in Technology-Based Teacher Education

81

Kumar, D.D., & Altsculd, J.W. Evaluation of an interactive media science teacher education project: A contextual approach . (in review). Mayer, V.J., & Stoever, E.C. (1978). NAGT crsustal evolution education project: A unique model for science curriculum materials development and evaluation. Science Education, 62(2), 173-179. Pines, L.A. (October, 1980). A model for program development and evaluation: The formative role of summative evaluation and research in science education. A paper presented at the annual conference of the International Congress for Individualized Instruction, Windsor, Canada. Shell, D.F., Horn, C.A., & Severs, M.K. (October, 1986). We dothey do: A model for practical service program evaluation. A paper presented at the annual meeting of the American Evaluation Association, Kansas City, MO. Small, L. (April, 1988). Science process evaluation model. A paper presented at the annual meeting of the American Educational Research Association, New Orleans, LA. Stufflebeam, D.L. (1971). Excerpts from evaluation as enlightenment for decision-making. In D. L. Stufflebeam (Ed.), Educational evaluation: Theory and practice, (pp. 143-150). CA: Wadsworth Publishing Co. Vitale, M.R., & Romance, N.R. (1992). Using videodisk instruction in an elementary science methods course: Remediating science knowledge deficiencies and facilitating science teaching attitudes. Journal of Research in Science Teaching, 29(9), 915-928. Welch, W.W. (1974). The process of evaluation. Journal of Research in Science Teaching, 11(3), 175-184.

Notes
This research was supported by the American Educational Research Association which receives funds for its AERA Grants Program from the National Science Foundation and National Center for Education Statistics (U.S. Department of Education) under NSF Grant #RED-9255347. Opinions reflect those of the authors and not necessarily those of the granting agencies. A version of this paper was presented at the International Conference of the Society for Information Technology and Teacher Education, Association for the Advancement of Computing in Education, Orlando, Florida, April 1997.