Star Child Website Evaluation

Homeschool Instructor Evaluation For Academic Purposes

Submitted By Robin Armstead

27 JULY 2012

Summary
The Star Child website is being evaluated in this project. The original website was created by two K-12 teachers in conjunction with the Outreach Director at NASAs Goddard Space Flight Center. The website serves the purpose of being a space exploration center for students in K-8. There is content that includes readings, audio, activities, and lesson plans for instructors. The main objective of the website is to support learning about the solar system and the universe for the target audience. The purpose of the evaluation was to examine the effectiveness of the website as a tool for homeschool educators of students in K-5. The participants are chosen based on this set of criteria. They are asked to review the Star Child Website for at least 15 minutes and then take the survey. Evaluation participants were solicited via email and a discussion forum for homeschool families. They completed the survey using a web link. The survey was created, delivered, and data collected and analyzed using the Survey Methods website. The evaluation found there is a need to update and overall the website. The technical quality, authority, affiliation, and objectivity of the website received the highest regard. The main issues presenting themselves were the currency, design, and content. It is suggested that these areas should receive the most attention if the website is revised.

Website Description
The Website
The Star Child website is intended to be used by educators, parents, and students to assist with learning about the solar system and different items in the universe. The website was created as collaboration between NASA and two Georgia school teachers. It was launched in the early 1990s and was continually maintained on a monthly basis until December 2003. There were some minor updates made in 2006, including a brief statement reclassifying Pluto as a dwarf planet. The website can be used as a teaching or learning tool. In addition to having basic information about the solar system and universe, there is also a glossary and learning activities that can be used by educators, parents, and students. The content of the website was written for grades K-8. It is broken up into two separate levels.

Website Objectives
The objectives were derived from the content on the website. Provide a resource for parents, educators, and students to learn about the solar system and universe Provide activities that reinforces student learning Provide multiple learning opportunities through the use of varied activities and presentation of materials

Provide additional resources for learning Create an interest in the solar system and the universe Foster continued interest in the solar system and universe through monthly activities Provide additional activities for educators via ideas and lesson plans that can be used in the classroom

Website components
The website is free and anyone can use it that has access to the internet. It is maintained by NASA. It is composed of several parts in order to facilitate its goals. Content- The basic content of the website used as a knowledge base for learning is divided into two levels, labeled levels 1 and 2 on the website. Each level is divided into four sections; Solar System; Universe; Space Stuff; and Glossary. The difference between the two levels is the language and activities; some basic topics are the same, with level two having additional and advanced topics and information. Search Area-There is an area that allows the users to search for specific information on the website. Star Child History- This section details several awards and accolades that the website has received between the years of 1996 and 2003. In The Classroom-This section has lesson plans and activities that instructors can use in their classrooms. The content is broken up between level one and level two. There are also printable activity books that can be used with each level. Other Languages-There is links to websites that have translated the Star Child website into German, Italian, and Portuguese. There is also a space for Spanish stating it would be available soon, however there is no link. Question of the Month-There is one question presented on the opening page. There are several months of questions in this area in sequential order in the archive section. Other Good Places-This page provides links to several websites that discuss the solar system, universe, outer space, and the space program.

Evaluation Method
Participants
The participants for the survey evaluation were chosen based on the objectives of the evaluation. Each participant had to have experience as a homeschool educator of students in grades K-5. This group was chosen based on their unique experience in the education arena and the necessity for homeschool instructors to have quality resources to reinforce the educational environment. The survey link was posted in a Homeschool discussion forum and emailed to participants who fit the criteria. There are several members of the Star Child Team that is responsible for the content and maintenance of the website. The two primary creators are Joyce Dejoie and Elizabeth Truelove. Dejoie is a 6th grade science teacher in Evans, GA. She has a BA in psychology from UCLA and A Master in Education from

Augusta State University. Truelove teaches 8th grade Science in Marietta, GA. She received her teaching credentials from Paine College. The project leader is Dr. Laura Whitlock. She served as the Educational Outreach Director for the NASAs Goddard Space Flight Center when the program was created. She is currently an Associate Professor of physics at Louisiana State University.

Procedures
During the evaluation period, the website appeared to function has it always has. The best of the evaluators knowledge, there were no updates or revisions made to the website. The participants were instructed to review the content of the website for at least 15 minutes and then proceed to the survey and answer the questions.

Data Sources
The data source for the evaluation is the survey filled out by the participants after reviewing the website. Survey-the data from the participants was collected using both rank data and the Likert Scale. The surveys were sent out using the Survey Methods automated email tool and a survey was posted in the Homeschool.com discussion forum. The participants filled out the survey on the internet and the data was collected by the automated feature. The items on the survey were as follows o General information Authority Affiliation Purpose Objectivity Learning Process Audience Currency o Content o Design o Technical Quality

Results
There were a total of 18 participants in the evaluation survey. The results of the data are summarized in the tables below. They are divided into 4 sections; general information, content, design, and technical quality.

General Information
There was not much variation on in the areas of Authority, Affiliation, Objectivity, or Currency. The results of those questions are summarized below.

Authority-94.44% or 17 participants responded that the site has an author listed with appropriate credentials. 5.56% of the respondents stated the author is listed with no credentials. Affiliation-100% or 18 participants responded that the site is supported by a reputable institution. Objectivity-100% or 18 participants reported that the site is free from bias and advertising. Currency-88.9% or 16 participants reported that the information had not been revised in 18 months or no date located. 11.1% or 2 participants reported that information has not been updated in the last year but the information quality is good. There were mixed results in the Purpose, Learning Process, and Audience questions. The questions and answers are summarized in the tables below. Purpose-Please rate the sites purpose Purpose Respondents Percentage Unclear 1 5.56% Implied or Inferred 1 5.56% Somewhat clear 5 27.78% Clear 11 61.11% The purpose of the site is not completely clear. A majority of the respondents, 16 (88.89%) stated that the purpose was clear or somewhat clear while two respondents stated that is was either implied or inferred or unclear. Learning Process-Please rate the sites learning process Learning Process Does not challenge learner in any manner Does not challenge the learner but has good factual and resource information Challenges the learner but is not engaging Challenges the learner to use higher order thinking and is engaging Respondents 1 6 7 4 Percentage 5.56% 33.33% 38.89% 22.22%

There are mixed results about the challenge that the site presents to the learners. A majority, 11 (61.11%) respondents replied that the website information challenged the learner; however 7 (38.89%) responded that it was not challenging. Additionally 7 respondents stated that the material was not engaging.

Target Audience-Please rate the sites capacity to match the content to the target audience Target Audience Is not appropriate for my audience Written for a level above but provides useful information Is appropriate for my audience level and has useful information Is appropriate for my audience and suitable for academic use Respondents 1 4 8 5 Percentage 5.56% 22.22% 44.44% 27.78%

A majority 13 (72.22%) respondents stated that the website was appropriate for the audience level and was useful for information or academic purposes. Only 5 (27.78%) respondents stated it should be used for academic purposes.

Content
Please rate each statement about the content of the site Strongly Agree 3(16.67%) 9(50%) 3(16.67%) 11(61.11%) 14(77.78%) Agree 3(16.67%) 5(27.78%) 2(11.11)% 5(27.78%) 3(16.67%) Neutral 2(11.11%) 3(16.67%) 4(22.22%) 0(0%) 0(0%) Disagree 3(16.67%) 0(0%) 1(5.56%) 1(5.56%) 0(0%) Strongly Disagree 7(38.89%) 1(5.5%) 8(44.44%) 1(5.56%) 1(5.5%)

Content is accurate and factual Content is educational appropriate Content is free from errors Content is age appropriate Content is free from stereotypes and bias

There is no clear agreement about the accuracy and factual content of the website. Almost half 7 (38.89%) disagree that the content is accurate and factual. Likewise, about half strongly disagree that it is free from errors. Most respondents strongly agree or agree that the content is age appropriate and free from stereotypes and bias.

Design
The site is consistent throughout The site holds the learners attention The site uses colors and fonts well The site is visually appealing Strongly Agree 14(77.78%) 7(38.89%) 7(38.89%) 7(38.89%) Agree 4(22.22%) 2(11.11%) 4(22.22%) 4(22.22%) Neutral 0(0%) 3(16.67%) 5(27.78%) 4(22.22%) Disagree 0(0%) 3(16.67%) 2(11.11%) 1(5.56%) Strongly Disagree 0(0%) 3(16.67%) 0(0%) 2(11.11%)

There were no clear cut answers about the design of the site. All of the respondents strongly agree or agree that the site is consistent. A little less than half 7 (38.89%) respondents think the site holds the

learners attention, uses colors and fonts well, and is visually appealing. There was strong disagreement in the holding learners attention questions and visually appealing. Of all the survey questions, this topic presented the most neutral answers.

Technical Quality
Animation and graphics are used well Audio is well used and works properly The site is easy to navigate The site loads quickly Strongly Agree 11(61.11%) 13(72.22%) 12(66.67%) 13(72.22%) Agree 2(11.11%) 4(22.22%) 5(27.78%) 5(27.78%) Neutral 4(22.22%) 1(5.56%) 1(5.56%) 0(0%) Disagree 1(5.56%) 0(0%) 0(0%) 0(0%) Strongly Disagree 0(0%) 0(0%) 0(0%) 0(0%)

Most respondents agreed that the site presents technical quality. Of all the questions, 4 (22.22%) people respondent with neutral and 1 (5.56%) with disagree in when asked to rate the animation and graphics. No one strongly disagreed in any of the categories.

Discussion
The purpose of this study is to evaluate the effectiveness of the website to meet its objectives for homeschool educators. In order to measure the effectiveness, the evaluation examined factors relating to the content, user perception, design, and ease of use. All of these factors work together in order to create the users experience. Despite the fact that the site is hosted and maintained by NASA and has won numerous awards, there were many elements of the site that the participants did not rate highly. There are modifications that can be made to improve the site for the learners.

General Information
Although the information across the evaluation survey is somewhat scattered in different areas, it is consistent in the areas of Authority, Affiliation, Objectivity, or Currency. An overwhelming majority of respondents agreed that the website had the authors listed appropriately, is managed by a reputable organization, and is objective. Likewise, an overwhelming majority agreed that the website that the website is not kept current or there was no date listed. In these categories, the website has done extremely well, however should work to update the website regularly or place a date on the website indicating when the material was last updated. The purpose of the website is mostly clear to all the respondents. When examining the contents, it is easy to derive that its purpose is to present educational information about the solar system and universe. Only 2 (11.11%) of respondents thought the websites purpose was inferred, implied, or not clear. It may be helpful to add a list of objectives or uses to the website in order to make the purpose crystal clear.

The website was found by 11 (61.11%) to challenge the students and 7 (38.89%) to not challenge the students. Furthermore, only 14 (77.78%) found that the website was not engaging. Engagement and challenging the learners are important components to learning. The website developers and content creators may want to consider revising the activities or content on the site. Recommendation prompts in the form of statements, questions, or videos to engage the learners interest. More interactive activities or ones integrated directly into the content may help. Most respondents felt the information was appropriate for their audience. There are two levels of information and activities; however the content is written in such away that it is practical for each group to be subdivided into more parts or easily adapted. The activities can also be modified if necessary for a particular grade or to focus on a small part within a topic.

Content
The content is clearly free from bias and age appropriate with an overwhelming majority strongly agreeing or agreeing so. There is some discrepancy over whether the content is free from errors, educational appropriate, factual and accurate. Upon closer inspection, there may be some issues over the use of Pluto on the website. While the content has been updated to reflect Plutos demotion as a full planet to a dwarf planet, this has not been clearly updated in the activities and teaching materials. Additionally, the other dwarf planets are not given the same attention as Pluto as to suggest that the updated content was superficial.

Design
The results from the design portion of the website are varied. A majority of respondents agree that the website design is consistent throughout but there is not agreement about its visual appeal, holding the attention of the learner, and use of the colors and fonts. These categories are very subjective and speak to the individuals own perception. The initial site was created in the early 1990s and is reported to be overhauled in 1996. It may be possible that the site could use another overhaul that incorporates technology and graphic design that is more consistent to what respondents view and interact with on a daily basis. This would draw more users to the site so that it can fulfill its objectives.

Technical Quality
A majority of the participants agreed or strongly agreed that the audio worked well, the site was easy to navigate, and it loaded quickly. The biggest spread was in the use of the animation and graphics categories. Although this question was in relation to the technical quality, it is possible that there is some overlap about this issue. In general, the website has excellent technical quality and changes do not need to be made in this area.

Overview
The area that stood out the most was the content and design section questions. The content developers should consider doing a more in depth review of the content. Likewise, there seems to be issues with the content being engaging and holding the learners attention. This could be related to the design issues that the respondents showed varied results. Presently, educationally based websites must take into account both the written content as well as graphic design. Both elements are necessary to engage the

learner and hold the attention. These two areas should be considered together when the website receives its next overhaul, which is highly suggested. The purpose of the study is to evaluate the effectiveness of the website in terms of its effectiveness for homeschool students in K-5. After reviewing the results, the website meets many of its objectives. The most important elements that support these objectives are the authority supporting the website in both information and hosting. It is imperative that the website, in totality is updated regularly. Besides the issue with Pluto, it appears as if the Question of the Month has not been updated since December 2003. This can cause the user to be concerned about the currency and care of the website. In order to improve the quality of the learning process and addressing the target audience, the designers should consider using several different learning strategies for the students. These strategies should include approaches based on not only traditional methods but also emerging ones like differentiated instruction and brain based learning. Modifying these areas or adding more content will support the goals of providing multiple learning activities and addressing a wide audience grade range.

Project Costs
Personnel Robin Armstead-7 days at $500 per day Communications Internet Communications Supplies and Materials Survey Software Total Cost

$3500.00 $25.00 $9.00 $3534.00

The evaluator charges a flat rate of $500 per day. During the evaluation, the internet was the main source of communication in relation to anything in reference to the survey at a cost of $25.00. The evaluator used a website called Survey Methods in order to create and deliver the survey as well as analyze the results. There is a monthly cost associated with the website at $9.00.

Appendices Appendices A1-B

Appendix A Links
Appendix A1 Website Link The website being evaluated can be found using the following link http://starchild.gsfc.nasa.gov/docs/StarChild/StarChild.html Appendix A2 Survey The survey can be found using the following link http://www.surveymethods.com/EndUser.aspx?D5F19D84D5958982D0 Appendix A3 Survey The website used to support the evaluation process can be found using the following link https://www.surveymethods.com/Index.aspx?out=1

Appendix B Correspondence
There were two correspondence pieces used to invite participants. The first was an email sent out through the Survey Methods system. The system automatically added the information link to the bottom of the survey. The second was posted in a discussion forum on the HomeSchool.com website. It included a manual inserted web link at the bottom. Subject: Please Help Robin Armstead With Her Grad School Project Greetings My name is Robin Armstead. As most of you know, I was a homeschool parent for several years and also tutored children. I am currently in Graduate School at Boise State University. I am attempting to complete a project for my 505 Evaluation Class. If you have time, I would like you to review a website and complete a short evaluation survey. It should take no more than 20 minutes of your time altogether. Below is a link for you to click. It will give you further instructions and links to the website and survey. I really appreciate you taking the time to help me complete this project .If you have any questions please feel free to contact me via email or phone. Have A Great Day Robin Armstead 770.885.8593