EDUC 5104: LEARNING TOOL METRIC DEVELOPMENT & IMPLEMENTATION

Scale Justification Template

Student Names: Harry Blyleven, Heidi Milovick, and Giovanni Mirarchi Title of Scale: Evaluation of Web-based Learning Tool - Instructor Number of Themes: Theme 3 Number of Items: 14 Learning Theory (if applicable) (cite theory) Situated Learning the learner is placed in the context specific to the content. Instructional Design (if applicable) (cite principle) Coherence Principle clearly communicating the desired learning goals set the tone for the rest of the learning object. Clear goals will aid the evaluator to ensure that there is no extraneous material, or decoration, that does not support the learning goals. Research-Based (If applicable) (cite specific reference) Refer to Kay & Knaack paper (Kay & Knaack, 2008) Of the reasons for using a WBLT, teaching a new concept, (8%), explore a new concept after a formal lesson (6%) or homework (4%), while low, together are significant and warrant that learning goals are clearly communicated. Also as many teachers (46%) allowed students to explore the LO on their own it is important that purpose and goals are readily known.

Scale Item (please add rows as needed)

1. The learning goals are clearly communicated.

D. Petrarca, UOIT, EDUC 5104G, Winter 2012 (modified by Harry Blyleven, Heidi Milovick, and Giovanni Mirarchi)

Theme

Scale Item (please add rows as needed)

Learning Theory (if applicable) (cite theory)

Instructional Design (if applicable) (cite principle) Modality Principle effectively using a variety of learning modes will allow the user to better process the material presented. Multimedia Principle the use of a variety of learning modes also supports the Multimedia Principle by including words and graphics that support understanding. Coherence Principle Mayer and Moreno's (2003) concept of weeding calls for WBLTs to remove content that is not key to learning goals

Research-Based (If applicable) (cite specific reference) Refer to Kay & Knaack paper

2. A variety of learning modes are effectively used (eg. Audio, visual, text etc.)

3. The content (eg. text, diagrams etc,) presented is accurate.

4. The content is clear and coherent (eg. spelling, grammar, etc.).

Coherence Principle Mayer and Moreno's (2003) concept of weeding calls for WBLTs to remove content that is not key to learning goals

Vargo et al (2003) stress the importance of content accuracy Because of the risk of students being misinformed by inaccurate content... As they continue to discuss accuracy, more emphasis is placed upon having multiple reviewers to help ensure content is accurate. Vargo et al (2003) stress the importance of content accuracy Because of the risk of students being misinformed by inaccurate content... As they continue to discuss accuracy, more emphasis is placed upon having multiple reviewers to help ensure content is accurate.

D. Petrarca, UOIT, EDUC 5104G, Winter 2012 (modified by Harry Blyleven, Heidi Milovick, and Giovanni Mirarchi)

Theme

Scale Item (please add rows as needed)

Learning Theory (if applicable) (cite theory)

Instructional Design (if applicable) (cite principle)

5. Overall, the learning process is enhanced by the WBLT.

Research-Based (If applicable) (cite specific reference) Refer to Kay & Knaack paper Kay & Knaack (2008) state that Ultimately, though, the goal of the LOEM (Learning Object Evaluation Metric) is to help educators increase the pedagogical impact of technology in their classrooms. The LOEM can offer helpful guidance to educators who wish to select effective learning objects. The summary presented parallels the desire of this criterion to help select effective WBLTs that enhance learning. Kay & Knaack (2008), states that while the impact of learning may vary, even within the same classroom, that , WBLTs, when used, do significantly increase student performance.

LEARNER INTERACTIONS

6. Learners control the pace of information/activities.

Jonassen, HernandeqSerrano, & Choi (2000), reconceptualized the meaning of learning as intention-actionreflection cycles (p. 126). By providing learners with flexibility to control the pace of their learning they have the time to reflect on their actions.

D. Petrarca, UOIT, EDUC 5104G, Winter 2012 (modified by Harry Blyleven, Heidi Milovick, and Giovanni Mirarchi)

Theme

Scale Item (please add rows as needed)

Learning Theory (if applicable) (cite theory)

Instructional Design (if applicable) (cite principle)

Research-Based (If applicable) (cite specific reference) Refer to Kay & Knaack paper

7. Information/activities accommodate different levels of learning.

According to the segmenting principle, dividing the WBLT into manageable segments helps to avoid cognitive overload in the learner. According to ARCS Model of Motivation Design (Keller), a WBLT should engage learners in relevant tasks that build confidence by providing them with the information they need to be successful, and leave them feeling satisfied with what they achieved. Social Development Theory facilitate learner construction of knowledge by providing guidance through feedback to learner responses as they work within their zone of proximal development

Haughey and Muirhead (2005) suggest the importance of including a few challenge levels to scaffold student learning and reinforce learning that occurs in initial activities.

8. Practice activities that foster learning are provided.

9. Sufficient feedback is provided to support student learning?

According to Brown and Voltz (2005) providing rich feedback in a timely manner enhances learning by allowing students to further develop their skills and knowledge.

10. Clear guidance is provided for the learning

Kay and Knaack (2007b) suggest that clear instructions are an important aspect of a WBLT. We chose to include it in our tool as we feel that clear instructions and guidance are also essential for a good WBLT

D. Petrarca, UOIT, EDUC 5104G, Winter 2012 (modified by Harry Blyleven, Heidi Milovick, and Giovanni Mirarchi)

Theme

Scale Item (please add rows as needed)

Learning Theory (if applicable) (cite theory)

Instructional Design (if applicable) (cite principle)

Research-Based (If applicable) (cite specific reference) Refer to Kay & Knaack paper

11. The layout is clearly

organized.

The coherence principle applies to this question, as the tool should not include any material that does not support the instructional goal. Extra graphics, audio or words that do not contribute to the instructional goal should not be included. A good WBLT ensure that only necessary items are included in the layout

12. The tool is simple to use.

Concannon et al (2005) say that negative comments from previous studies focussed on problems with navigation. Kay and Knaack (2007b) also suggest that an organized layout is very important. Both studies are our reasoning in including a question on the layout, as it seems to play an important role in the tools use Both Kenny et al (1999) and Schoner et al.(2005), suggest that ease of use is an important component of a good WBLT. We concur with both studies and included ease of use in our scale. Kay and Knaak (2007b) reported that teachers agree that learning objects were beneficial tools to use and helped with their understanding. It is helpful to have feedback on how the tool does this.

OPEN-ENDED QUESTIONS

13. What, if any, did you see as the strength(s) of this WBLT?

D. Petrarca, UOIT, EDUC 5104G, Winter 2012 (modified by Harry Blyleven, Heidi Milovick, and Giovanni Mirarchi)

Theme

Scale Item (please add rows as needed)

Learning Theory (if applicable) (cite theory)

Instructional Design (if applicable) (cite principle)

Research-Based (If applicable) (cite specific reference) Refer to Kay & Knaack paper

14. What, if any, did you see as the weakness(es) of this WBLT?

McCormick & Li (2005) and Kay & Knaack (2007b) both commented on the teacher perspective of using learning tools. McCormick & Li (2005) noted that some teachers reported technical problem in their use. The scale should include any weaknesses so that any problems are specifically noted.

D. Petrarca, UOIT, EDUC 5104G, Winter 2012 (modified by Harry Blyleven, Heidi Milovick, and Giovanni Mirarchi)

References Brown, A. R. & Voltz, B. D. (2005). Elements of effective e-learning design. The International Review of Research in Open and Distance Learning, 6(1). http://www.irrodl.org/index.php/irrodl/article/view/217/300 Concannon, F., Flynn, A. & Campbell, M. (2005). What campus-based students think about the quality and benefits of e-learning. British Journal of Educational Technology, 36(3), 501-512. Haughey, M. & Muirhead, B. (2005). Evaluating learning objects for schools. E-Journal of Instructional Science and Technology, 8(1). http://www.ascilite.org.au/ajet/ejist/docs/vol8_no1/fullpapers/eval_learnobjects_school.htm Jonassen, D. H., Hernandez-Serrano, J., & Choi, I. (2000). Integrating constructivism and learning. In J. Spector & T. Anderson (Eds.), Integrated and Holistic Perspectives on Learning, Instruction and Technology (pp. 103-128). Netherlands: Kluwer Academic Publishers. Kay, R. H., & Knaack, L. (2008). An examination of the impact of learning objects in secondary school. Journal of Computer Assisted Learning, 24(6), 447-461. Retrieved from http://search.proquest.com/docview/621610141?accountid=14694 Kay, R. H. & Knaack, L. (2007b). Teacher evaluation of learning objects in middle and secondary school classrooms. Manuscript submitted for publication. Kenny, R. F., Andrews, B. W., Vignola, M. V., Schilz, M. A. & Covert, J. (1999). Towards guidelines for the design of interactive multimedia instruction: Fostering the reflective decision-making of preservice teachers. Journal of Technology and Teacher Education, 7(1), 13-31. McCormick, R. & Li, N. (2005). An evaluation of European learning objects in use. Learning, Media and Technology, 31(3), 213-231. Schoner, V., Buzza, D., Harrigan, K. & Strampel, K. (2005). Learning objects in use: Lite assessment for field studies. Journal of Online Learning and Teaching, 1(1), 1-18. [verified 28 Oct2008] http://jolt.merlot.org/documents/vol1_no1_schoner_001.pdf Vargo, J., Nesbit, J.C., Belfer, K., Archambault, A., Learning Object Evaluation: Computer Mediated Collaboration and Inter-Rater Reliability, International Journal of Computers and Applications, Vol. 25, No. 3, 2003, pp. 198-205.

D. Petrarca, UOIT, EDUC 5104G, Winter 2012 (modified by Harry Blyleven, Heidi Milovick, and Giovanni Mirarchi)