Running head: CRITICAL ANALYSIS OF A SCHOLARLY ARTICLE 1

Critical Analysis of a Scholarly Article:

“Understanding Cognitive Engagement in Online Discussion: Use of a Scaffolded, Audio-based

Argumentation Activity”

California State University Monterey Bay

IST 520

Team I: Russ Fleming, Gwen Hansen, Stacey Knapp, Shwetha Prahlad, Lei Sun

April 9, 2019
CRITICAL ANALYSIS OF A SCHOLARLY ARTICLE 2

Table of Content

Introduction 3

Research Procedures and Methods 5

Research Results and Discussion 6

Summary 8

Reference 10
CRITICAL ANALYSIS OF A SCHOLARLY ARTICLE 3

Introduction

This paper analyzes the article “Understanding Cognitive Engagement in Online

Discussion: Use of a Scaffolded, Audio-based Argumentation Activity” by Eunjung Grace Oh

and Hyun Song Kim. The purpose is to “explore how to engage in asynchronous online

discussion through the implementation of an audio-based argumentation activity.” The research

is a descriptive, phenomenological qualitative applied research study. A phenomenological study

is a “form of qualitative research in which the researcher attempts to understand how one or

more individuals experience a particular phenomenon” (Johnson and Christensen, 2017). The

researchers conclude that audio-based argumentation promoted a higher level of cognitive

engagement as “peer-led dialogic argumentation” requires cognitive processes that are not

accounted for in text-based online discussions. They also found that providing a framework of

argumentation guidelines led to students thinking more critically about complex issues, both in

their own posts and in their responses to peers. Therefore, the researchers draw a very positive

conclusion from implementing a scaffolded, audio-based online argumentation in asynchronous

online discussion.

The researchers propose that if online discussions are not well-constructed nor well-

facilitated, then they do not promote deep thinking and do not adequately support critical

thinking goals in online interactions. In an effort to create more productive discussion-based

learning experiences and promote cognitive engagement, this study adopted “scaffolding of

audio-based argumentation activities.” The study examines how adult learners engage in

asynchronous audio-based argumentation activities within online discussions and compares this

relatively new technology with traditional, text-based discussion forums. The study examines
CRITICAL ANALYSIS OF A SCHOLARLY ARTICLE 4

how adult learners engage in asynchronous audio-based argumentation activities within online

discussions. The researchers seek to promote learner cognitive engagement through discussion

activities. Three research questions are clearly stated. Specifically, how learner discourse is

characterized, how learners engage, and how learners perceive and evaluate the activity.

Oh and Kim built the study upon the standpoint that learner’s cognitive engagement can

be improved by participating in online asynchronous discussion activities. To further improve

adult learner’s thinking skills, instructors need a more effective cognitive engaging strategy.

Researchers compared a group of graduate student’s performance in both text-based discussion

and audio-recorded arguments and then conducted interviews. Johnson and Christensen (2017)

claim that qualitative research focuses on studying the participant’s experience and problems

relying on collecting qualitative data. Therefore, this study is considered as qualitative research.

In the class, the instructor used two forms: online discussion and audio based discussion

using VoiceThread for audio based discussion. The research attempted to confirm or deny two

prongs when using text-based asynchronous learning platforms: limited learners’ participation in

online discussions and a perceived lack of depth in thinking demonstrated in online discussions.

In comparing an audio-based argumentation with a text-based modality, the researchers

propose the following hypothesis: Does a designed and implemented “scaffolded, audio-based

argumentation activities” lead to increased learner engagement as compared to a text-based

pedagogical strategy in graduate-level online discussion forums? The author’s study presented

and supported their findings through research of the following.

The research could only be supported if the comparison on what existed (low student

engagement), to what was theorized (does an audio-based modality increase learner engagement
CRITICAL ANALYSIS OF A SCHOLARLY ARTICLE 5

both on quantity and quality?). The hypothesis could be and was ultimately confirmed through

measured data points using qualitative modalities. Scientific modalities also exist but, the authors

did not use True experimental or Quasi-Experimental research methods.

Research Procedures and Methods

Based on existing empirical research, Oh and Kim agreed that a strategic format of

discussion is needed to improve the adult learner’s level of cognitive engagement in eLearning.

They assumed that implementing audio- or video-based technology into scaffolded

conversational argumentation could be very effective to make such improvement. The authors

recruited six graduate students who enrolled in the same online course. Those students were

asked to participate in two text-based discussion forums and two audio-based argumentations.

Oh and Kim conducted a content analysis by studying learner’s level of thinking skills, the

components of arguments, and the overall quality of argumentation. To understand the student’s

experience and perspective of audio-based argumentation, the authors also conduct interview

analysis with some participants.

The researchers used “qualitative analytical methods” to analyze participants’ skills,

knowledge, and experiences in online discussions. The researchers in analyzing participants’

thinking skills and arguments, used a “content analysis method”. Using a MAC Nvivo 11

analysis tool, the authors were able to create data points based on three areas: levels of thinking

skills, components of arguments, and the overall quality of argumentation.

Bloom’s (1956) Taxonomy of Cognitive Learning was used in the analysis for all

collected data. The response rate of 37 text-based messages to 59 audio-based recorded messages

was analyzed. The researches were aware that subjectivity could impact the analysis of the data
CRITICAL ANALYSIS OF A SCHOLARLY ARTICLE 6

points. As a result, all components were assigned a unit as a component of the argument

structure. A mathematical flaw could occur in the final analysis as only the two researchers

needed to reach agreement in assigning any value point. That being said, the data that follows is

used in support of team-first decisions and analysis of the case study.

The measurements were valid and reliable. In total, 287 semantic units were identified.

Table 3 presents the description of the components of an argument and example postings. Lots of

studies were consulted for this research project (see references) and several, including those that

produced counter results, were included in the discussion. The researchers explained in detail

both the process and the participants involved in the study. There were no special circumstances

that would preclude the study from replication other than the fact, perhaps, that the study was

conducted with a small graduate school cohort and all participants were involved in the

educational field, either as teachers or as education professionals. The scaffolds and discussion

prompts were clearly illustrated in both text and visuals, and researchers explained both indirect

(interviews) and direct (qualitative assessment) data collection procedures in detail.

Research Results and Discussion

The results of the qualitative data analyses support the findings of the study in that the

researchers conclude that “using a scaffolded, audio-based argumentation activity, students

have demonstrated higher levels of thinking skills, important components of arguments to

substantiate their claims, and greater cognitive effort during discussion” (Oh & Kim, 2016, p.

42). The researchers explain that one of the goals of their study is “to promote learners’

cognitive engagement in the discussion activities.”

CRITICAL ANALYSIS OF A SCHOLARLY ARTICLE 7

The researchers do provide a reasonable explanation for their findings, namely that using

the audio tool first as an “ice-breaker activity” prior to the argument-based audio discussion post

and “the Scaffolded Online Dialogic Argumentation (SODA) framework created by Kim and

Oh (2014)” as well as breaking the larger group down into “pairs” produced a more cognitively

rich debate environment over the text-based discussions created by the same participants.

However, the researchers implied but did not clarify, that these same scaffolds were provided for

the text-based discussion questions.

Moreover, some learners may prefer written response over voice narration due to verbal

competency, speaking skills, and/or how they prefer to formulate their thoughts. Instead, the

text-based discussion questions seemed to be more simply constructed as the researchers explain,

“the instructor posted discussion topics and open-ended questions, and then the students

shared their thoughts and commented on the thoughts of others.” So the findings are murky

given that the audio-based projects were supplemented with additional scaffolds and the text-

based discussion forums were not. Therefore, we would argue that the researchers did not draw

reasonable implications for practice from the findings because 1) there was no control group, 2)

the scaffolding supporting the audio-based assignment may have had the greatest impact on the

findings, 3) the sample size was very small with only six, graduate-level participants and

therefore may not be repeatable given a larger, undergraduate group. In addition, only four

participants were interviewed about their preferences and experiences despite there being six

participants in the study, and there was no explanation as to why two of the participants did not

complete the interview.

CRITICAL ANALYSIS OF A SCHOLARLY ARTICLE 8

According to the authors, participants reported the pressure of greater accountability for

internalization of their own arguments and the participants felt that they were having

conversations with “a real person” and that audio as a modality allowed more “intimate

expression, emotion, and characters” (p.38). However, the researchers fail to acknowledge that in

verbal argumentation, there is an increased potential for emotional reactions that would

potentially need to be moderated by instructors and in a larger educational context, this may

prove to be very difficult.

Indeed, the researchers do point out that a study by Hew and Cheung (2013) that did not

have these scaffolds resulted in the opposite conclusion in a group of undergraduate students.

The researchers point to cultural implications for this variation in results, but do not examine the

‘apples and oranges’ implications of the difference in the scaffolds provided by the audio-based

assignment or their own small sample size of advanced (graduate-level) learners; indeed, they

seem to generally overlook the fact that these enhancements could be a major contributor to the

positive results reported by their qualitative analysis. Still, their findings are meaningful to

educators, broadly speaking, in that they do illuminate the importance of two variables in opting

for audio-based discussions: 1) the size of the cohort (small is better) and 2) that enhanced

scaffolding will improve the audio-based argumentation experience for learners.

Summary

The purpose of the study was to increase cognitive engagement in the discussions by

using audio-based argumentation strategy. The researchers believe that online discussions were

not providing a deep level of understanding or engagement from students. They covered quite a

few data points to prove that cognitive engagement considerably increased with the audio-based
CRITICAL ANALYSIS OF A SCHOLARLY ARTICLE 9

argumentation over a text-based discussion. However, the larger question here would be if the

purpose was satisfied due to the audio-based argumentation itself or due to the instructional

methods used in the activity.

We were not able to conclude that entirely, as there was no control group and the data

sample was very small. The study did not account for different kinds of learners, their individual

cognitive comprehension, their preference of writing over speaking, or their ability to articulate

their points into an argument using recorded audio. The study has not revealed the level of

influence that the culture, language, communication skills, or the cognitive abilities of each

student had on the results, nor has the study included a randomized undergraduate student

populations as all of the study participants were graduate-level students pursuing degrees in

education-related fields, which may have skewed results.

Despite some shortcomings in the methodology, overall we found the article to be useful

to educators in that it provides an interesting analysis of alternatives to traditional text-based

forums that leverage new technological tools. In addition, we would recommend further study

using “true experimental or quasi-experimental” (Salkind, 2009). Research methods that include

larger sample size, a control group, and the addition of equal development of the argumentation

scaffolding in the text-based discussion forum in order to better discern if the higher results in

the audio-group stemmed from the audio-modality or from the scaffolding provided only to this

group and not the other.

CRITICAL ANALYSIS OF A SCHOLARLY ARTICLE 10

Reference

Heick, T. (2018). What Is Bloom's Taxonomy? A Definition For Teachers. Retrieved

from https://www.teachthought.com/learning/what-is-blooms-taxonomy-a-

definition-for-teachers/

Johnson, R. B., & Christensen, L. (2016). Introduction to Educational Research. In

Educational Research: Quantitative, Qualitative, and Mixed Approaches (6th

ed.). Thousand Oaks, CA: SAGE Publications.

Oh, E.G., & Kim, H.S. (2016). Understanding Cognitive Engagement in Online

Discussion: Use of a Scaffolded, Audio-based Argumentation Activity,

International Review of Research in Open and Distributed Learning, 17(5), 28-

48.

Salkind, N. J., (2009). Exploring Research, The Role and Importance of Research,

Exploring Research (7th ed.). Pearson Prentice Hall.