Assessment in Schools Technology Education and ICT

A Jones, B Cowie and J Moreland, University of Waikato, Hamilton, New Zealand

2010 Elsevier Ltd. All rights reserved.

Introduction
In this article we address the issues related to school-based assessment of student learning in technology, including information and communication technology (ICT). We begin by explicating the nature of technology and the characteristics of technology education and then explore the relationship between curriculum, assessment and pedagogy. A sociocultural perspective on the nature of the discipline, curriculum, learning and assessment, and their associated interrelationships underpins our analysis.

Nature of Technology and Technology Education

People use technology to intervene in the world to expand their possibilities, applying both intellectual and practical resources. Technology encompasses a broad range of activities, including the transformation of energy, materials, and information. Technology is not only about artifacts, but also how and why those artifacts are developed and the impact they might have on people and our world. It encompasses more than ICTs, such as computers, the web, and e-mail; it is about products, systems, and environments. It can include electronics and control technology, materials technology, food technology, structures and mechanisms technology, and production and process technology. Technology is included as a curriculum area in many countries in all continents and regions, including Europe, Asia, Pacific, Africa, and the Americas, where increasing the levels of technological literacy is seen as of intrinsic value for individual development, as a particular insight into culture and for the betterment of society. The most compelling personal reason for studying technology is that it is a major and, some would argue, a determining feature of the world we inhabit. As part of culture, young people need to be introduced to technological practice so that they can understand its nature and be able to participate in it at some level. If technology is indeed a determining feature of the world we inhabit, it follows that young people, as future citizens, need to understand how it shapes the world.

learning in that subject. In the case of technology, there is a need to accommodate its complex multidimensional nature. When we look at the development of technology education many of its roots can be traced to traditional technical subjects that emphasized only skills teaching. More recently, it has been linked with and subsumed by science and also conflated with ICT. Consequently, technology education has struggled to establish itself as a school subject with its own identity. The primary focus of technology is to intervene in the made world to extend human capabilities as opposed to understanding the world. Technology encompasses both technological knowledge and technological practice and their interrelationships. Technological knowledge and practices are context dependent. They are associated with and structured by objects, artifacts, and tools in action embedded in social and cultural practices. Its characteristics as practices, as well as a body of knowledge that are crucial to technology education. The uniqueness of technological knowledge, processes, and skills has not always been recognized in general education. Technology education needs to be concerned with involving students with the identification, exploration, and solving of technological problems. Technological problems encompass multiple and interrelated conceptual, procedural, societal, and technical aspects. A comprehensive technology curriculum includes an understanding of the nature of technology, technological knowledge and understanding, technological practices, and the relationship between technology and society. This complexity poses a number of challenges to traditional approaches to assessment.

Assessment and Views of Learning in Technology and ICT

This article offers an analysis of assessment in technology and ICT that is grounded in a sociocultural view of learning. Individual and cognitive notions of learning and achievement have dominated the thinking about assessment. This has contributed to the predominance of individual testing as well as to a focus on written assessment tools in tightly controlled environments. Such approaches are inappropriate for technology and technology education. Social views of learning endorse the view that knowledge is socially constructed and context dependent. Human actions are situated within a historical, cultural, and institutional setting. Sociocultural theory proposes that knowledge emerges through social and cultural activity where tools,

Technology Education and Assessment

Assessment models must accommodate the specific features of the subject and the characteristics of teaching and

311

312

Educational Assessment - Assessment in Domains

artifacts, and systems mediate thinking, action, and interaction. From this viewpoint, technology assessment is a situated social and cultural activity that cannot be separated completely from the classroom or from ongoing student teacher interaction.

Assessment of Technology Including ICT

Assessment brings with it connotations of testing but international research suggests that formative assessment, now often referred to as assessment for learning, is one of the keys to enhancing student learning and engagement. In formative assessment, teachers and students use assessment information to enhance student learning within the classroom and on a daily basis. In summative assessment, teachers and students use assessment information to sum up what has been learned. Traditionally, distinct summative assessment tasks have been used to generate data for this purpose. Formative assessment and summative assessment do not need to be seen as independent practices. Effective formative assessment practices can contribute to summative assessment. When formative and summative assessment practices mutually inform each other, teacher assessment practice can become seamless and optimally productive. Effective summative practice reviews learning and can contribute to decisions about further teaching and learning and thus can have a formative function for both teachers and students. Role of the Teacher in Classroom Assessment Teachers cannot design and evaluate valid assessment tasks or interact formatively unless they have a clear sense of the ideas of the subject. Teachers need to have an appreciation of the nature of the discipline, its organizing concepts, mediational tools, cultural values, and symbolic and language systems. For teachers to be effective technology assessors, they require clear understandings of the complex multifaceted nature of technology. Valid teacher assessment in technology needs to encompass: 1. conceptual (knowledge and understanding of relevant concepts and procedures); 2. procedural (knowing how to do something, what to do, and when to do it); 3. societal (aspects related to the interrelationship between technology and groups of people); and 4. technical (skills related to manual/practical techniques) aspects. Teachers need a clear sense of the conceptual terrain and a pedagogical sense of the understandings the students will bring. Teachers cannot provide experiences and activities that guide and monitor student progress toward the understanding of ideas if they themselves do not know

what the ideas are. Successful facilitation of student teacher formative interaction requires a flexible grasp of the subject matter being explored. With sound content and pedagogical knowledge, teachers can respond to student ideas both formatively and summatively. Often in technology lessons, students design and create an artifact or a virtual solution in response to a scenario undertaken over several days. The long-term nature of such technology tasks poses particular issues for assessment. One way to help with building connections, continuity, and coherency is to think about these aspects when planning. Teachers can first define the macro-task, that is, the overall task. This macro-task needs to encompass the technological conceptual, procedural, societal, and technical learning outcomes. Then teachers can arrange a series of interrelated subtasks, meso-tasks, which are mutually important for achieving a solution to the macrotask. Micro-tasks may also be planned by teachers and are more localized tasks, embedded within meso-tasks. The macro-, meso-, and micro-tasks form a connected network that provides structure, support, and direction for students. In technology, it is essential that students work iteratively when designing, making, and testing. Often in technology classes, the design process is treated as a series of steps. These steps can become ritualized with lessons structured around each step so that students undertake the process in a stepwise fashion, giving rise to a veneer of accomplishment. When teachers include opportunities for students to distil out the essential criteria for the creation of a technological artifact, and these criteria are used as the foci, students are able to work iteratively across designing, making, and testing process. Research indicates that when teachers plan for technological, conceptual, and procedural goals, they pay less attention to social and managerial aspects in the classroom, such as the need to take turns, work in groups, work independently, and finish on time. Technology goals can be expressed in terms of the knowledge, design processes, and technical skills. After planning appropriate technology goals, teachers are able to identify appropriate tasks that afford opportunities for students to accomplish the goals. Consideration of the demands and affordances of tasks is essential for planning for the incorporation of assessment for learning strategies, including the provision of feedback. By knowing the ideas and skills inherent in the tasks, teachers can be more clear about their focus for assessment. Teachers may anticipate students possible actions and ideas and they may rehearse how they might respond. Rehearsal enables teachers to ascertain the technology knowledge and skills required, the suitability of the activities for their students, and to foreshadow potential problems. By prior testing the teachers are more aware of the demands, both conceptually and technically of the tasks, and can interact more confidently and effectively with students.

Assessment in Schools Technology Education and ICT

313

Effective assessment in technology, both formative and summative, focuses on the multifaceted and multimodal nature of technology. Effective assessment accommodates multiple modes such as drawing and modeling, not just talk, but to communicate and develop ideas. Technologists draw, make models and prototypes to develop and test ideas. In technology, teachers typically engage students in tasks that have a practical aspect. This provides students with access to multiple modes for developing, representing, and communicating their technological ideas. Drawing, modeling, and manipulating materials can contribute to, and are integral to, teachers and students exploring tasks and negotiating ideas together. ICTs provide a tool in technology for representing ideas. They can also engage learners in building understanding through the collaborative construction of an artifact or a shareable product in a virtual environment. They can construct a virtual reality, play with animated puppets, or build a three-dimensional (3D) model of solar system. It is important that students are allowed to use these multiple modes to represent their ideas in assessment settings.

Classroom Interactions and Assessment for Learning

Classroom interactions and feedback are central to teaching technology formatively. One of the strengths of technology lies in the way that ideas and concepts can be expressed in concrete, practical ways. Through exploration of technological ideas through talking and designing, students can begin to see for themselves what they know and can do, and how well they know and can do it. By listening to and interacting with students, a teacher can provide feedback that suggests ways in which students can improve their learning. Feedback, whether from teacher or students, is useful to both teachers and students in providing information that enables both teachers and students to modify the teaching and learning activities in which they are engaged. There are a variety of design and technology pedagogical approaches that can contribute to students learning through formative interactions and assessment for learning practices. These include: comparing and contrasting technologies, categorizing and grouping examples of technology and recognizing exceptions, and making predictions about technology activities. Examining existing technologies helps students distil the features of the technology. What is essential in these activities is that the teacher makes it clear to the learners that the activity is to explore what they think rather than for them to guess the answer the teacher has in their head. This process helps the teacher and the students to identify their current understandings and what they might need to know more about and where they might go next. Categorizing and grouping examples of technology and recognizing exceptions and talking about them are

activities that challenge students to justify their classifications in terms of technological ideas. These types of activities enable learners to test their understandings about the nature of technology and about particular technologies. Incorporating prediction activities encourages students to apply their knowledge and understanding to future situations, through which they can then either test or realize their ideas. Problems may not be foreseen by the students, and therefore need to be dealt with on the spot by themselves, their peers, and/or the teacher. How teachers deal with the emerging problems students face as they engage with the task at hand, impacts on students learning. Hence, how teachers engage with students formatively strongly affects how students undertake technological processes and their learning in technology. One strategy that teachers have used successfully is to keep the key goals (learning intentions) upfront so that students can identify and clarify their own problems within the bounds of the learning goals. When students are engaged with drawing, designing, and making activities, it is important for the teacher to work alongside them so that they can interact with the students emerging ideas. Leaving the students to work unassisted conveys the message that design and technology is a self-help activity. Teachers need to model techniques, processes, and procedures in front of students. The process of formative assessment includes students finding out about their learning, recognizing, reacting, and evaluating their learning from their own viewpoint or others. When students monitor their own progress, evaluate their strengths and weaknesses, and devise strategies for improving their learning, their commitment to learning is strengthened. Technological specifications, intended learning, and success criteria align strongly with the technological process and self-assessment process. Key to the success of both peer- and self-assessment in technology is teachers and students discussing about technology together, as this provides students with the language to discuss technological concepts and processes. When teachers target conceptual and procedural aspects and talk about them with their students, more opportunities are afforded for students to learn about these technological aspects and include them in their work. When teachers concentrate on technological learning outcomes and make these clear to the students as outcomes for them to achieve, students are able to review their work and undertake self-assessment focused on these aspects. Technological concepts and procedures require embedding in learning activities through constant articulation and exploration, so that students begin to understand their strengths and weaknesses and how to deal with them. Engaging students in conferring, consultative conversations related to technological aspects, along with the provision of continual support, reinforcement, and overt cueing by teachers, leads to effective student understandings. In technology, teachers need to help

314

Educational Assessment - Assessment in Domains

students focus on the conceptual aspects inherent in the processes and procedures involved in reaching solutions. Often a practical focus can overshadow conceptual aspects. When students are encouraged to review their accomplishments in technology in an ongoing manner they are able to do this throughout the technological process, rather than undertaking self-review only as an endpoint summative process. Discussion plays an important part in the classroom assessment. Through discussion students are able to help each other, clarify and develop ideas, communicate ideas and solutions, critique, and seek help. Class and peer discussion provides the teacher with opportunities to notice, recognize, and respond to students ideas as they are brought to light. ICTs can allow for group collaboration over time and space. Classroom Summative Assessment Teachers are responsible for summing up and reporting on student learning. Teachers can summarize formative assessment information during and at the end of a teaching sequence. This information can be used to improve teaching and inform students on where to go next. Accumulated formative assessment information can assist in developing a richer picture of the sum of student learning and achievement, one that includes both the process and product of their learning. Summative assessment as a summing up of learning across time, across individuals, and across classes can support a formative function. Effective formative assessment practices can also contribute to summative assessment, and vice versa. This can be of benefit to teacher, students, and their parents. Student performance in technology is often assessed by an evaluation of the end product. When this is the case, the product should first be assessed holistically against criteria pertaining to its form, function, and quality. This judgment needs to be followed by identifying strengths and weaknesses. This way the whole can be evaluated and also how the individual elements work together. Judgments need to be a balanced reflection of the whole of student learning, which are then backed with detail. Such summative assessment includes information that can be used to set the next direction for learning. At the same time as looking back at what students have accomplished, teachers also look forward to what they might accomplish next. This fits with a sociocultural perspective that is concerned with learning and achievement over time.

2. a vehicle for supporting and enhancing the teaching, learning, and assessment of other curriculum subjects; and 3. a subject in its own right as part of technology education. The first focus is associated with current concerns that students leave school with the knowledge and skills to take an active part in society, that is, with a focus on information and ICT literacy. The second focus is associated with research and development that seeks to utilize the affordances of ICT to enhance student motivation, make teaching more relevant, incorporate multimodal items in lesson materials, represent and communicate ideas, and to generate, store, and analyze student assessment data for formative and summative purposes. The latter role is currently related to ICT as an area of study computing, software design, gaming, and development of ICT-based systems. ICT now lies at the heart of most activities that constitute social inclusion. ICT use is ubiquitous in what it means to be socially, economically, culturally, and politically involved in current society. In this section, we elaborate on the second view of ICT as the first is beyond the scope of this article and the third has been covered in the previous section on technology and technology education generally. ICTs can enhance teaching and learning through a contribution to formative and summative assessment. They can be used for assessment purposes at the national, school, class, and individual level. National electronic (e)-assessment banks provide access to well-designed assessment items that allow teachers to make comparative judgments in real time to provide evidence of learning which can be used both formatively and summatively. Not only are assessment banks online but ICTs also are useful for developing electronic (e)-portfolios to document learning and achievement in a multimodal way. E-portfolios can contain written work, photographs, video, audio, and other digital media. These allow both teachers and students to document the process and products of learning as well as the teacher feedback and the student response to this. This type of e-portfolio can be used in national assessment and moderation processes. Digital evidence is easy to share, easy to search, and easy to ask analytical questions of. This means that teachers and schools are able to ask probing questions about the nature of the learning of their students and to think about the implications of this for their teaching and learning programs. School and class e-portfolios allow for ease of sharing between teachers and provide a productive forum for discussion between teachers to compare and contrast student work to explore possibilities and potential for learning through questioning each other on how students have been supported to achieve to a high standard. Teachers are able to expand their teaching repertoire to enhance student learning. Students are also

Assessment and ICT

There are three broad overlapping views of the role of ICT in schools, namely, ICT as: 1. a set of skills or competencies to be mastered in an information-rich society;

Assessment in Schools Technology Education and ICT

315

able to see the potential in the learning tasks and what might be possible for them. For example, the use of photographs as a story line of images adds to student confidence and the sense that they are making progress. Social technologies are on the rise for classroom use. Students can immerse themselves in contexts that challenge and extend their understanding and use these social networkings as part of peer-assessment processes. These social networking ICT environments allow for multiple perspectives in assessment for learning and assessment of learning, including peers, teachers, students, and their families. As ICTs become more integrated into classrooms the process of assessment has the potential to become more dynamic, multimodal, and interactive.

assessment can be used formatively. ICTs have the potential to enhance classroom assessment practices through the provision of additional modes of representing, recording, and reviewing information on student learning process and products. Assessment is often constructed as an external evaluation of learning. However, school-based classroom assessment has a greater influence on student learning and perceptions of a subject. Valid and reliable classroom assessments in technology need to represent and reflect the multidimensional and multimodal nature of technology. When this occurs the students experience of teaching, learning, and assessment is enriched.

Further Reading Conclusion

Technology encompasses a very broad range of activities, including the transformation of energy, materials, and information. It encompasses products, systems, and environments. Technology is a multifaceted discipline with a strong multimodal aspect. Technology education requires students to develop and use knowledge, procedures, skills, and ethical/value/societal sensibilities to create technological solutions. A sociocultural view of learning accommodates the social, cultural, and historical dimensions of technology. Assessment models need to reflect this complexity. Too often assessment models do not reflect the characteristics of the subject under consideration. Classroom assessment for technology can have two main purposes: formative and summative. For teachers to assess in both a valid and reliable way, they need to have an understanding of the nature of technology as well as insights into its knowledge and practice. Technology is often taught and learned in long-term tasks requiring that teaching and assessment establish a sense of continuity and coherence. Effective planning not only addresses the multifaceted long-term nature of technology tasks but also accounts for how students and teachers might be actively involved in the assessment process. Clear learning goals and a shared understanding of the specifications of the technology solution enable teacher, student, and peer formative assessment. When teachers systematically collect formative evidence, this can serve a summative function and valid summative
Black, P. and Wiliam, D. (1998). Assessment and classroom learning. Assessment in Education 5(1), 774. Cowie, B. (2005). Pupil commentary on assessment for learning. In Black, P. (ed.) Special Issue: Formative Assessment. Curriculum Journal 16(2), 137151. Gess-Newsome, J. and Lederman, N. (eds.) (1999). Examining Pedagogical Content Knowledge. Dordrecht: Kluwer Academic Publishers. Gipps, C. (1999). Sociocultural aspects to assessment. Review of Educational Research 24, 353392. Jones, A. and Moreland, J. (2004). Enhancing practicing primary school teachers pedagogical content knowledge in technology. International Journal of Technology and Design Education 14(2), 121140. Kimbell, R. A., Stables, K., and Green, R. (1996). Understanding Practice in Design and Technology. Buckingham: Open University Press. Loveless, A. and Ellis, V. (eds.) (2001). ICT, Pedagogy and the Curriculum: Subject to Change. London: Routledge/Falmer. Means, B., Roschelle, J., Penuel, W., Sabelli, N., and Haertel, J. (2003). Technologys contribution to teaching and policy: Efficiency, standardisation, or transformation? Review of Education Research 27, 159182. Moreland, J., Cowie, B., and Jones, A. (2007). Assessment for learning in primary technology classrooms. Design and Technology Education: An International Journal 12(2), 3748. Roth, W. -M. (1995). From Wiggly structures to Unshakey towers: Problem framing, solution finding and negotiation of course of actions during a civil engineering unit for elementary children. Research in Science Education 25(4), 365382. Shulman, L. S. (1987). Knowledge and teaching: Foundations of the new reform. Harvard Educational Review 57, 122. Staudenmaier, J. (1985). Technologys Storytellers: Reweaving the Human Fabric. Cambridge, MA: The Society for the History of Technology/MIT Press. Wells, G. and Claxton, G. (eds.) (2002). Learning for Life in the 21st Century. Oxford: Blackwell.