Empirical Article Evaluation #2 Quantitative Methods EDUR/FRER 7130: Educational Research Online Course, Fall 2012 Leisha Cavallaro

Georgia Southern University

EDUR/FRER 7130: Educational Research

Empirical Article Evaluation Guide (20pts.) Qualitative Methods

Reference information (in APA style):

I. STATEMENT OF THE RESEARCH PROBLEM

1. In your own words, what is the research problem being investigated in this study? The research problem in this article is educators trying to offer alternative programs for students who are struggling by adapting their curriculum to meet the needs of their students while still conforming to current educational practices (Groth, 1998).

II. LITERATURE REVIEW (be sure to consider date of publication)

2. How would you evaluate the literature review with regard to recency, relevance to the research problem, clarity, depth, and organizational structure? Based on the current year, the literature review is a little out of date; however, relative to the year this article was published the researcher utilized a lot of research that was close to its publication date of 1998 as well as some outdated information such as research from 1983 and 1972. The depth and clarity of the research was extremely well done and I think it had a good flow from the background to the research problem to give the reader a good understanding of the problem at hand.

III. DESCRIPTION OF THE CONTEXT

3. Please describe the context in which this study was conducted.

This study was done in a more natural context than that of a typical experiment. The data was collected through a participant-observational approach where some of the information came from the researcher observing while others came from student interaction, reports, and definitions. 4. Was the context well-suited to the stated purpose of the study? Why/why not? Due to the nature of this study, attempting to figure out how students experience plays into their alternative programming, I think this context was effective and well-suited for the purpose of the study. The study isnt intended to find a definite answer, but rather be more exploratory and interpretive which is conducted well through the natural context and the participant-observational approach. The researcher can observe behaviors of students as well as interview them and gather information from participants to figure out the effectiveness and personal experience of students which is the goal of this study.

IV. METHODOLOGY

5. What type of qualitative design is this (e.g., ethnography, phenomenology, case study, grounded theory, critical study)? How did you know? This study was done with a mainly phenomenological qualitative design, with a hint of enthographic design. The fact that this study focuses on participants in their nature state gave away the hint of ethnography. The researcher stressed the importance of the experience a student has and a lot of his data collection methods are focused on figuring out the feelings and thoughts that students have while in alternative programs in school. This type of study that focuses on capturing the perspective and experience of a participant is backbone to the phenomenology qualitative design. Participants 6. Were the participants clearly described?

Very Clear

Mostly Clear

Mostly Unclear

Very Unclear

Though the research described the demographics of the participants the researcher only stated that the students came from AAA program; he did not mention what school this program is from but he did provide a school year. 7. Did the authors convince you that the selected participants were appropriate and sufficient to inform the purpose(s) of the study? Yes No

8. Why or why not? The fact that the researcher didnt have a steady rate of students nor did he provide the area in which they were chosen I was a little skeptical about the situation. He did, however, convince me of the diversity of the students given their ethnicities and various backgrounds and experiences they had with school which was the intent of this study. I just personally would have liked to know a little more about their age, school, and location for instance.

Data gathering tools and techniques? 9. Were the tools/techniques used for data collection clearly described?

Very Clear

Mostly Clear

Mostly Unclear

Very Unclear

The ability to replicate this data collection wouldnt be exactly on point. There were no specific interview questions listed despite the excerpts from interviews given and there was no real description or operational definition of when the researcher observed the students and what exactly he was observing. He did provide a list of what he did but details were not really given so the replication again would be difficult. 10. Did the authors establish the validity of the data collection techniques used? 11. Explain your answer to #10. The apparent lack of consistent interview questions makes the validity and reliability of the data collection techniques a little below average. There were no specific interview types or observation behaviors defined. The idea of this study was to be exploratory and interpretive so it may not be necessary to the goals of the study, however, I do think he could have provided a few more techniques and details to establish a little more reliability and validity. On the other hand the researcher was very detailed about the topics of which he assessed throughout the AAA class context. Yes No

V. DATA ANALYSIS

12. What technique(s) was/were used to analyze the data? Why do you think the particular technique(s) was/were chosen?

The type of data collected was not quantified and therefore the analyzing was left up to the interpretation of the researcher, which was explicitly stated in the beginning of the study. This technique was chosen due to the lack of quantitative data available as well as the context and design of this study. The purpose was to be exploratory and interpretive while obtaining direct feedback from students to assist in the interpretation of the effectiveness of the AAA program. 13. Were the data analysis techniques clearly described?

Very Clear

Mostly Clear

Mostly Unclear

Very Unclear

14. Did the authors establish the validity of the data analysis techniques used? Yes 15. Explain your answer to #14.

No

Im not sure if the intent of this study and design would allow for the establishment of validity of the data analysis. Due to the fact that it is interpretive in nature, each person can have their own biases as well as opinions on the data collected through observation and participant interaction and feedback. VI. SIGNIFICANCE of the STUDY

16. Based on the author's arguments, how would you rate the practical significance of the study?

Very Significant

Mostly Significant

Mostly Insignificant

Very Insignificant

Explain: I think that there is a major problem presented from the literature review. Whether or not this program would be effective for every student still remains a question but the new idea of trying it as well as considering the experience and feedback of students can really make an impact on these alternative programs. More research should be conducted and maybe try to obtain quantitative data as a form of more validity and reliability, but outside of that I do think this study could have practical significance if utilized properly.

17. Based on the author's arguments, how would you rate the theoretical significance of the study?

Very Significant Explain:

Mostly Significant

Mostly Insignificant

Very Insignificant

Due to the fact that this is an interpretive study it can be outwardly rejected, however, it does pose a new theoretical perspective that many havent considered before. The author posed a challenge and problem with current theoretical perspectives on alternative programming for struggling students, so his study and proposition of a new theory can have a big impact on future programming.

VIII. STRENGTHS and WEAKNESSES

1. What are the three strongest points of this study? The theoretical significance is a key strong point of this study as it challenges current theories and practices and presents a possible new idea. The recognition of a problem is another strong point. The study did a good deal of research and literature review finding a specific problem that he wanted to address. The in depth discussion on specific aspects about the AAA program was also a pretty strong aspect of this study. 2. What are the three weakest points of this study? The ability to replicate the study is extremely weak due to the lack of specific information on data collection as well as rubrics for interviews. The information about participants was pretty good but the lack of supplying their age and location seems like a pretty important piece that was not included. The data analysis technique was mainly interpretive. Though this was the intent of the study I would have liked to have seen an operational definition or some form of quantitative data to do some comparisons or some other form of results to better support the findings.