Education Tech Research Dev (2008) 56:241264 DOI 10.

1007/s11423-007-9054-5 RESEARCH ARTICLE

A catalyst for teaching critical thinking in a large university class in Taiwan: asynchronous online discussions with the facilitation of teaching assistants
Ya-Ting C. Yang

Published online: 2 August 2007 Association for Educational Communications and Technology 2007

Abstract This study was designed to investigate the effects of teaching critical thinking skills (CTS) in a large class through asynchronous discussion forums (ADFs) with the facilitation of teaching assistants. A pretest and posttest quasi-experimental design with a comparison group was employed to validate the effectiveness of the proposed approach. The independent variable was the structured ADFs with two levelswithout Socratic dialogues, and with Socratic dialogues, modeled and facilitated by the teaching assistants via structured ADFs, while the dependent variable was the students levels of CTS as measured by two different evaluations: (a) the California Critical Thinking Skills Test, to holistically examine students gains in their CTS, and (b) the Coding Scheme for Evaluating Critical Thinking in Computer Conferencing, to investigate students interaction patterns and the depth of their critical thinking (CT) demonstrated via the ADF. The evaluation data were collected from 278 college students in Taiwan. The qualitative analysis provided a detailed description of how students discussions moved from the lower to the higher phases of CT. Results indicated that an inspired instructor and some energetic teaching assistants who use Socratic dialogues during small-group online discussions can successfully develop students CTS in a large university class. Keywords Critical thinking Computer-mediated communication General education Large class size Graduate teaching assistants

One of the greatest experiences for students in higher education is to have the opportunity to think freely and challenge other students ideas with their own. This ability to think critically and argue logically is a precious, invaluable asset that benets students as they journey through life. Thus, the hallmark of higher education is to teach and develop students critical thinking skills (CTS) (Barnett 2004; Facione 2007; Garrison et al. 2001;
Y.-T. C. Yang (&) Institute of Education and Centre for Teacher Education, National Cheng Kung University, No. 1, Ta-Hsueh Rd, Tainan 701, Taiwan e-mail: yangyt@mail.ncku.edu.tw

123

242

Y.-T. C. Yang

Yang et al. 2005; Yeh 2006). Ennis (1985) offered a widely accepted denition of critical thinking (CT) as reasonable, reective thinking that is focused on deciding what to believe or do (p. 46). Several teaching strategies, such as cooperative learning, reciprocal teaching (Brown and Palincsar 1989), communities of practice (Wenger 1998), case study pedagogy, and the integration or the direct teaching of CTS, have been proposed to help promote CTS (Daud and Husin 2004). Meanwhile, King (1990) and Taba (1966) have suggested that the level of thinking that occurs within a given situation is inuenced by the level of questions asked. That is, thoughtful questions play an important role in inducing students higher-level cognitive processes, which include self-reection, revision, social negotiation (Brown and Palincsar 1989; Wenger 1998), and conceptual change of student misconceptions, all of which are integral to CT. However, when students are asked to generate questions on their own, factual rather than thought-provoking questions are generally posed (Dillon 1988; Flammer 1981; King 1990). Socratic questioning is one of the most popular and powerful teaching approaches to use in guiding students to generate thoughtful questions that will foster their CTS (Maiorana 19901991; Paul 1995). Instead of providing direct answers, the Socratic questioning approach stimulates students minds by continually probing the subject with thought-stimulating questions (Paul). Through interactive discussions that include rational dialogue and questioning between the instructor and students and among students, Socratic questioning can facilitate students CTS by the exchange of ideas and viewpoints, giving new meaning to content, exploring applications to problems, and providing implications for real-life situations (Yang et al. 2005). However, in a traditional classroom, discussions are often hindered due to limited class time and unequal access of interaction (e.g., a small number of students dominate the discussion). If collaborative inquiry tasks are assigned outside of class time, it is difcult if not impossible for the instructor to monitor collaborative discussions and guide students through the thinking process. However, the advancement in information technology has changed the way teaching and learning are traditionally conducted. For instance, emerging technologies such as asynchronous discussion forums (ADFs) (i.e., text-based computermediated communication tools) have become promising tools that extend opportunities for interactive discussions outside the classroom, allowing students to compose their thoughts in writing, and at the same time, providing opportunities for the instructor to moderate such discussions (Duffy et al. 1998). It has been argued that appropriately designed online discussions can facilitate interaction between the instructor and learners and among the learners themselves. However, while student participation is necessary for successful online discussions, it is hardly sufcient. In an unstructured online discussion where no facilitators organize or guide students discussion, students may talk for hours or post many messages without learning anything of substance. Therefore, the use of well-organized and well-facilitated discussions within structured online forums is just the component needed in higher education to achieve the larger instructional goal of developing students CTS (Duffy et al. 1998; Harasim et al. 1995). At the same time, further research is required in order to address questions about the components of truly successful online discussions using structured ADFs, and to determine whether structured ADFs can promote the development of students CTS. Angeli et al. (2003) investigated the quality of ADFs. The results of their study showed that students online exchanges consisted largely of personal experiences and showed little evidence of CT in unstructured online discussions. Moreover, a large number of students had difculty in understanding how to approach discussions meaningfully (Ellis and Calvo 2006). Also, greater online interaction did not lead to more in-depth interaction (Sing and

123

Teaching critical thinking in a large university class

243

Khine 2006) or signicantly higher performance for students just making passing grades (Davies and Graff 2005). It was suggested that the lack of an assessment strategy or welltrained mentors to guide, model, and evaluate students online contributions might have inuenced the quality of messages posted (Gilbert and Dabbagh 2005). Related to this research, Gunawardena et al. (1997) studied the quality of online interactions in a list-serve debate format, from both the interactive and cognitive dimensions. Based on constructivist learning theories, they developed an assessment tool, the Interaction Analysis Model (IAM), to assess the exchanges made among class members and the way these exchanges moved from the lower to the higher phases of CT. In a related study, Newman et al. (1995) presented a content analysis method to measure the depth of CT in face-to-face and computer-supported group learning. Both Gunawardena et al.s and Newman et al.s studies were able to provide some evidence that CT can occur in computer conferencing and that such processes occur as a direct result of exchanges among participants. Using case study and qualitative research methods, Bullen (1998) and Salmon (2002) likewise concluded that reection-on-action and CT development can be demonstrated through computer conferencing. Other related research includes that of Jeong (2003) and Walker (2004), which showed that structured ADFs can support the development of CTS. These studies suggest that with appropriate assessment strategies and well-trained mentors for guiding student learning, students CTS can be fostered and demonstrated through structured online discussions. Using a quasi-experimental design, Yang et al. (2005) made a rst step toward ascertaining the effectiveness of structured ADF discussions in developing students CTS. Positive gains in students CTS and attitudes provide empirical evidence that instructional designs incorporating asynchronous online discussion and interaction can be effective and conducive to the development of CTS. Most of the above research was conducted in small regular classroom contexts or in distance learning environments. Although ADF discussions tend to become the norm in these learning environments, the difculty of increasing students CTS in large classes is the challenge most instructors face when using this strategy, such as in many general education courses (i.e., those with more than 100 students). This is because an open discussion may be impossible to manage or may greatly limit students participation. As a result, it becomes easy for most students to assume a passive role, merely recording the facts that the instructor conveys in the lecture, thus leading the students to a shallow understanding of the course materials. The typical solution to this problem is to have students pair up to discuss a question or problem for 5 min and then come back for a fullgroup discussion. Unfortunately, the questions asked in this short period of time are usually very brief and shallow. With the advent of networked computers and Internet technology, new teaching methodologies are, therefore, needed so that students do not fall prey to the inadequacies of traditional studentteacher-based models of interaction (Preston and Shackelford 1998). ADF tools seem to extend opportunities for interaction, provide all students with a chance to contribute, and motivate students to compose their thoughts in writing. To extend prior research and explore applications of long-used teaching and learning methods in online instruction, the current study, based on Yang et al. (2005) study, thus takes a further step by applying both quantitative and qualitative analyses to assess the impact of improving students CTS via ADFs in a large class. The major goal was to investigate whether students CTS would improve after they participated in Socratic dialogues as taught by the instructor and then as modeled and facilitated by the teaching assistants via ADFs in a large university class. Based on the above goal, the specic research questions are:

123

244

Y.-T. C. Yang

1. In a large university class, will students levels of CTS improve after they participate in Socratic dialogues, as modeled and facilitated by the facilitator via ADFs? 2. In a large university class, will students who participate in Socratic dialogues, as modeled and facilitated by the facilitator in ADFs, have higher levels of CTS than those who participate in ADFs, where Socratic dialogues are not fostered?

Method Design and participants A pretest and posttest quasi-experimental design with a comparison group was employed to answer the above two research questions. Students enrolled in Electricity and Life, an undergraduate-level general education course offered by the Electrical Engineering Department at a large university in Taiwan, participated in the study. The purpose of this course was to foster understanding by non-engineering students of the fundamental concepts and terminology for electricity that are used frequently in daily life. The contents included eight units: (1) nature and origin of electricity, (2) electricity and magnetism, (3) classication of batteries, (4) direct current (DC) generators and DC motors, (5) alternating current (AC) generators and AC motors, (6) transformers, (7) concepts of circuit, and (8) characteristics and applications of electrical power. The course typically draws a diverse group of students from different elds of study, such as Statistics, Economics, Business, History, Chinese Literature, Medicine, and Biochemistry. There were two sections for this course: Section I (experimental group) and Section II (comparison group). These two sections were randomly assigned to experiment/comparison groups and had the same instructor, course content, schedule, examinations, and online discussions to complete. Two credit hours were awarded for successful completion of this course. Initially 150 and 142 students enrolled in this course in Sections I and II, respectively. Excluding students non-responses and unintentional skips on some items of the test/ survey, 278 complete sets of data were collected: 145 in Section I (49% males and 51% females) and 133 in Section II (51% males and 49% females). The students ages varied from 19 to 22, with an average age of 19.55 years old in Section I and 20.16 years old in Section II. All were full-time students, had previous computer experience, and had access to the Internet at home or at the university.

Independent and dependent variables The independent variable was the structured ADFs, which refer to a variety of organized dialogues in which a facilitator leads the students and facilitates exercises that can help bring about more productive conversations to achieve the desired instructional goal via online discussions. The structured ADFs have two levels: without Socratic dialogues, and with Socratic dialogues, modeled and facilitated by the facilitator via structured ADFs. The dependent variable was the students levels of CTS as measured by the Chinese version of the California Critical Thinking Skills Test (CCTST) (Form A) (Facione 1990a, 1992) and class discussions on the ADF. The CCTST is discipline neutral and aimed at college undergraduates, graduate students, and adult professionals. The 34 multiple-choice items in the CCTST cover the

123

Teaching critical thinking in a large university class

245

domain of the ve cognitive skillsanalysis, evaluation, inference, deductive reasoning, and inductive reasoning. One point is given for each correct answer, and the maximum possible score is 34. In addition, the test has two parallel conceptually and statistically equivalent forms that are suitable for a pretestposttest research design. The internal consistency reliability (KuderRichardson 20) of the published version of Form A is KR20 = .70, and of Form B is KR-20 = .71. As to the construct and content validity, the CCTST is based on the conceptualization of CT articulated in the Expert Consensus Statement on College Level Critical Thinking (Facione 1990b), known as the Delphi report. This concept was supported by an independent replication research study of policymakers, employers, and academics, which was conducted at Penn State University and sponsored by the US Department of Education. A sample question from CCTST, Form A (Facione 1990a, 1992) is as follows: Passage: The micro-organisms in this pond are of the kind which generally reproduce only in water with a temperature above freezing point. Now its winter time and this pond is solid ice. So if there are any micro-organisms of the kind we are researching in the pond, they arent reproducing right now. Assuming all the supporting statements are true, the conclusion of this passage A = could not be false. B = is probably true, but may be false. C = is probably false, but may be true. D = could not be true. The content of the class discussion posted on the ADFs was analyzed qualitatively by using the Coding Scheme for Evaluating Critical Thinking in Computer Conferencing, which is discussed in detail in the Data Analysis section.

Procedure Training of instructor and teaching assistants (TAs) There were one researcher, one instructor, and ve-experienced TAs in this study. All of the TAs were graduate students in electrical engineering. The researcher conducted the training for the instructor and TAs to ensure that they were capable of teaching and modeling Socratic dialogues during the online discussions. The training had four 2-h sessions during which the researcher introduced the concept of CT, explained the rationale for using it, and discussed and modeled the use of the Six Categories of Socratic Questioning Prompts (see Fig. 1), which are different but correlated categories of questioning prompts to guide students in exploring ideas or statements in depth and breadth. The categories included questions that (a) are for clarication, (b) probe assumptions, (c) probe reasons and evidence, (d) are about viewpoints or perspectives, (e) probe implications and consequences, and (f) are about the question. The instructor and TAs practiced Socratic questioning prompts to guide the students in exploring ideas or statements in depth and breadth during the pilot study (one semester long), which was the same course as the main study, but one semester earlier than the main study. The main tasks of the TAs were to focus on the quality of the students interaction, including the assumptions, reasoning, and evidence provided by the students. For example, the TAs needed to recognize common errors in reasoning (such as making unsupported assertions and using anecdotal evidence) and praise students correct use of reasoning (such as focusing on

123

246

Y.-T. C. Yang

f. Questions about the Question Why is this question important? How can we find out? What does this question assume? Is this the same issue as ____? Is the question clear? Do we understand it? Do we all agree that this is the question? To answer this question, what questions would we have to

a. Questions of Clarification What do you mean by ___? What is your main point? Could you give me an example? Could you explain that further? Why do you say that? How does this relate to our discussion (problem, issues)?

b. Questions That Probe Assumptions What are you assuming? All of your reasoning depends on the idea that ____. Why have you based your reasoning on ____ rather than ____? You seem to be assuming ____. How would you justify taking this for granted? Is it always the case? Why do you think the assumption holds here?

Categories of Socratic Questions

e. Questions That Probe Implications and Consequences What are you implying by that? When you say ____, are you implying ____? s But if that happened, what else would happen as a result? Why? What effect would that have? If this and this are the case, then what else must also be true?

d. Questions about Viewpoints or Perspectives You seem to be approaching this issue from ____ perspective. Why have you chosen this rather than that perspective? How could you answer the objection that ____ would make? Can/did anyone see this another way? What would someone who disagrees say? What is an alternative? How are Kens and Marys ideas alike? Different?

c. Questions That Probe Reasons and Evidence How did you come to believe that? Why did you say that? How do you know? What led you to that belief? Why do you think that it is true? Do you have any evidence for that? But is that good evidence to believe that? What are your reasons for saying that? Are these reasons adequate? Is there reason to doubt that evidence? What other information do we need? What would change your mind? Can someone else give evidence to support that response? By what reasoning did you come to that conclusion? How could we find out whether that is true?

Fig. 1 Six categories of Socratic questioning prompts (summarized from Paul 1995, pp. 341344)

empirical evidence for a theory and the ability to integrate personal values with evidence). The instructor closely monitored the interaction and offered pedagogical support for the TAs. The preliminary results of the pilot study indicated that the instructor and TAs were successfully trained to teach and model Socratic dialogues during the online discussions, resulting in higher CT levels of students at the end of the pilot study. Small-group structure One possible solution to overcome the problem of a big class is to divide and conquer. At the beginning of the semester, the researcher randomly divided the whole class of 145 and 133 students (in the comparison and experimental groups, respectively) into small groups of 1315 students. There were a total of 20 small groups (10 groups in each

123

Teaching critical thinking in a large university class

247

Comparison Group Experimental Group

O1

O2

O3

O4

O5

O6 O6

O1 X1 O2 X1 O3

X 1 O 4 X 1O 5

First half of the semester

Second half of the semester

Note: X1 = independent variable (the teaching and modeling of Socratic questioning in ADFs); O1 = scores on CCTST; O2, O3, O4, and O5 = CTS demonstrated in online discussions 1, 2, 3 and 4, respectively; O6 = scores on CCTST at the end of the semester.

Fig. 2 Pretest and posttest quasi-experimental design

research group) and ve TAs in this study, with each TA facilitating two-group discussions in the comparison group and another two in the experimental group. Online discussions At the beginning and end of the semester, the students were required to take the 45-min CCTST in class (see Fig. 2). Two structured ADFs were separately set up for the comparison and for the experimental groups. The instructor posted four specic discussion topics on both ADFs for students to complete during the semester. Discussions 1 and 2 (O2 and O3) took place during the rst half of the semester, while Discussions 3 and 4 (O4 and O5) took place during the second half of the semester. Of these four discussions, two were case studies (Discussions 1 and 3), whose topics were home electrical appliances with anion function, and electrical ground: why three prongs? respectively. The other two were debates (Discussions 2 and 4), whose topics were Are electromagnetic waves/radiation that come from the electrical appliances (e.g., cell phones, computers, microwave ovens, and hair dryers) harmful to human health? and pros and cons of constructing a nuclear power plant, respectively. Each discussion lasted 2 weeks. During the discussions, the TAs asked the students to identify and post at least one argument or strong example to support their comments about the discussion issue. Each student was also required to respond to at least one other students posting by exploring the issue at hand and widening the discussion. Finally, at the end of the discussion, the students were asked either to summarize the points that were made during the discussion or to write a short reection on the discussion. Student participation in the discussion counted for 20% of the total course grade in both research groups. In the comparison group, the students could interact with each other and ask questions of their peers, instructor, or TA on the structured ADF, but the TAs did not attempt to facilitate the discussion. In the experimental group, at the beginning of the semester, the instructor taught and provided the students the Six Categories of Socratic Questioning Prompts (see Fig. 1). Both the TAs and students adapted the prompts to generate thought-provoking questions by lling in the blanks with specic content linked to the topic covered. Following are some examples (with the course content underlined) of the Socratic questions that the TAs modeled and asked so as to help students examine their thinking from a case study of electrical ground: why three prongs? and a debate of the pros and cons of constructing a nuclear power plant. (a. Questions of clarication) Student A: There are many disadvantages of nuclear power generation, so I oppose building nuclear power plants.

123

248

Y.-T. C. Yang

TA: Could you give us some examples of the disadvantages of nuclear power generation? (b. Questions that probe assumptions) Student A: Because computers are more sophisticated than hair dryers, we usually see grounds with computers more than with hair dryers. TA: You seem to be assuming that if the appliance is more sophisticated (e.g., computer vs. hair dryer), the ground is more needed for that appliance. How would you justify taking this for granted? (c. Questions that probe reasons and evidence) Student A: Recent surveys have indicated that there is currently more support for construction of nuclear power plants than for halting construction. TA: What is the source of your information? What other information do we need? (d. Questions about viewpoints or perspectives) Student A: We need to have grounds for computers or washing machines more than for hair dryers because the former are more sophisticated than the latter. TA: You seem to be approaching this issue from the sophistication of the appliance perspective. Why have you chosen this rather than the prevention of electric shock perspective? (e. Questions that probe implications and consequences) Student A: I would support a nuclear power plant. As demand for electricity soars, the pollution produced from fossil fuel-burning plants is heading towards dangerous levels. Unlike nuclear power plants, coal, gas, and oil burning power plants are already responsible for half of the worlds air pollution. Burning coal produces carbon dioxide, which depletes the protection of the ozone. TA: When you say Unlike nuclear power plants, air pollution., are you implying that nuclear power plants have less or no pollution? Or that they cause a different kind of pollution rather than air pollution? If so, what effect would that have? (f. Questions about questions) Student A: Im on the con side of the debate. Can you imagine a nuclear power plant turning into a mushroom cloud? Images of the exploded reactor represent an almost primal fear of technology out of control. TA: Do we all agree that this is the main question (imagining the explosion of a nuclear power plant or having a nuclear fear) for deciding whether we should construct a nuclear plant or not? Why? Throughout the online discussions, the TAs attempted to explicitly model Socratic questioning during the ADF. Then the students practiced these questioning techniques by composing comments, examining their own thinking, and challenging others in the ADFs.

Data analysis The quantitative data, the scores on the CCTST, were analyzed using two-way mixed design ANOVA to identify whether there was a difference between the comparison and experimental groups on the CCTST, and whether there was a difference between the CCTST pretest and posttest in each group. The qualitative data, online class discussions on the ADF, were analyzed using the Coding Scheme for Evaluating Critical Thinking in Computer Conferencing (see Table 1). The coded qualitative data were then further

123

Teaching critical thinking in a large university class Table 1 Coding scheme for evaluating critical thinking in computer conferencing Part A: Interaction analysis model (IAM) Phase I: Sharing/comparing of information A. B. C. D. E. A statement of observation or opinion A statement of agreement from one or more other participants Corroborating examples provided by one or more participants Asking and answering questions to clarify details of statements Denition, description, or identication of a problem

249

[IA] [IB] [IC] [ID] [IE]

Phase II: Discovery and exploration of dissonance or inconsistency among the ideas, concepts, or statements advanced by different participants A. B. C. Identifying and stating areas of disagreement Asking and answering questions to clarify the source and extent of disagreement [IIA] [IIB]

Restating the participants position, and possibly advancing arguments or consideration in [IIC] its support by references to the participants experience, literature, formal data collected, or proposal of relevant metaphor or analogy to illustrate point of view Negotiation or clarication of the meaning of terms Negotiation of the relative weight to be assigned to types of argument Identication of areas of agreement or overlap among conicting concepts Proposal and negotiation of new statements embodying compromise and co-construction Proposal of integrating or accommodating metaphors or analogies [IIIA] [IIIB] [IIIC] [IIID] [IIIE]

Phase III: Negotiation of meaning and/or co-construction of knowledge A. B. C. D. E. A. B. C. D. E. A. B. C.

Phase IV: Testing and modication of proposed synthesis or co-construction Testing the proposed synthesis against received fact as shared by the participants and/or [IVA] their culture Testing against existing cognitive schema Testing against personal experience Testing against formal data collected Testing against contradictory testimony in the literature Summarization of agreements Applications of new knowledge Metacognitive statements by the participants illustrating their understanding that their knowledge or ways of thinking (cognitive schema) have changed as a result of the conference interaction Relevance Relevant statements to the issue discussed Importance/signicance Important/signicant points/issues Totally unimportant, trivial points/issue Novelty Provide new information, ideas or solutions that have never been mentioned (even if they are not important or useful) Accuracy [IVB] [IVC] [IVD] [IVE] [VA] [VB] [VC]

Phase V: Agreement, statement(s), and applications of the newly constructed meaning

Part B: Analysis model for analyzing depth of critical thinking R+ R+ I+ I+ I N+ N+

R Totally irrelevant statements to the issue discussed

N Repeat what has already been said without any further exploration A+

123

250 Table 1 continued A+ A J+ J+L+

Y.-T. C. Yang

The references/literature used or information/data collected to support the participants position are accurate and true The references/literature used or information/data collected to support the participants position are clearly false Justication Provide a logical statement of opinion, agreement or disagreement with supporting reasons/ examples/justications/proof

J+L Provide an illogical statement of opinion, agreement or disagreement with supporting reasons/ examples/justications/proof J C+ Statement with simple agreement, disagreement or alternative opinions without elaboration Critical assessment

C+L+ Critical assessment/evaluation of ones own previous statements/reection or others contributions toward the issue discussed with logical thinking process C+L Critical assessment/evaluation of ones own previous statements/reection or others contributions toward the issue discussed with illogical thinking process C Uncritical or unreasoned acceptance/reject

analyzed using a chi-square test to investigate whether the students demonstrated CTS from the online class discussions.

Coding scheme for evaluating critical thinking in computer conferencing Constructivist learning theories are becoming widely accepted in all elds of education, including the application of technology to teaching and learning. This interest is related to the capacity of the computer to provide an interactive environment that creates an effective means for implementing constructivist strategies that would be difcult to accomplish in other media (Driscoll 1994, p. 376). Two widely accepted constructivistlearning theories are critical constructivism and social constructivism (Kanuka and Anderson 1998; Young 1997). While critical constructivism assumes that knowledge is constructed as an integration of internal contradictions resulting from environmental interactions, the assumption of social constructivism is that knowledge is generated through social intercourse, and through this interaction, we gradually accumulate advances in our levels of knowing. In this view, meanings emerge from the patterns of our social experiences that occur over time in a contextual, situated, and continually changing synthesis. Based on a constructivist paradigm, Gunawardena et al. (1997) developed the Interaction Analysis Model (IAM) for the evaluation of the process of knowledge construction that occurs through social negotiation in computer-mediated communication (p. 400). This model is designed to detect evidence of knowledge construction and is focused on assessing the quality of interactions in a CMC environment from both the interactive and cognitive dimensions. As shown in Table 1 Part A, their model includes ve phases of interactions and 21 subcategories for use in the study of critical discussions in case study and debate forums. Gunawardena et al. theorized that the active construction of knowledge moves through ve phases, and that although every instance of socially constructed knowledge may not progress linearly through each successive phase, these phases are nonetheless consistent with much of the literature related to constructivist

123

Teaching critical thinking in a large university class

251

knowledge creation. Based on these phases, a model was developed that could be used to analyze the construction of knowledge (constructivism) in computer conferencing transcripts. These ve phases are summarized as follows (Kanuka and Anderson 1998, p. 65): 1. Phase I. Sharing/comparing of information. In everyday transactions, this might take the form of ordinary observations, statements of problems, or questions. This phase may include an observation, opinion, agreement, corroborating example, clarication, and/or identication of a problem. 2. Phase II. Discovery and exploration of dissonance or inconsistency among the ideas, concepts, or statements advanced by different participants. This phase concerns inconsistency between a new observation and the learners existing framework of knowledge and thinking skills. Operations which may occur in this phase might include identication of differences in understanding of terms, concepts, schemas, and/ or questions to clarify the extent of the disagreement. 3. Phase III. Negotiation of meaning and/or co-construction of knowledge. This phase includes negotiation or clarication of the meaning of terms, identication of areas of agreement, and proposal of a compromise or co-construction. 4. Phase IV. Testing and modication of proposed synthesis or co-construction. Events that occur in this phase include testing against an existing cognitive schema, personal experience, formal data experimentation, or contradictory information from the literature. 5. Phase V. Phrasing of agreement, statement(s), and applications of the newly constructed meaning. This phase encompasses summarizing agreement(s) and metacognitive statements that illustrate new knowledge construction and application. The focus of IAM (1997) was to understand and describe the processes of negotiating the meaning, knowledge co-construction, and conceptual change of student misconceptions in a collaborative online discussion environment. All of such processes are important to promote CT in online discussions (Marra et al. 2004; Yang et al. 2005). According to Gunawardena et al. (1997), movement from one phase to the next shows that knowledge is constructed by the process of social negotiation. The transcript analysis procedure involves reading and coding the messages into one or more of the ve phases. Newman et al. (1995) provided a content analysis method to measure CT in computermediated group learning, as shown in Table 1 Part B. To measure the frequencies of specic critical (+) and non-critical () skills demonstrated in the discussions, they developed a set of paired CT indicators (x+), such as R+ and N+, where x represents CT indicators such as relevance (R) and novelty (N). The symbol x+ represents the count of positive statements of a CT indicator, while x is the count of negative statements in a transcript. Statements from the discussion transcripts were analyzed and scored across the list of indicators. Once the scripts were marked, the total for each + or indicator was counted, and a depth of CT ratio (x ratio) was calculated for each of the CT indicators:x ratio x x =x x ; converting the counts to a 1 (all uncritical) to +1 (all critical) scale. For example, 90 positive statements of relevance (R+) and 10 negative statements of relevance (R) were found in one transcript. The CT ratio of relevance (x ratio) is 0.80 as computed by: (90 10)/(90 + 10). This measure was designed to be independent of the quantity of participation, reecting only the quality of the messages. While the aim of Newman et al.s coding scheme was to measure the amount and type of CT taking place in group learning in order to check on the possibilities of using computer

123

252

Y.-T. C. Yang

conferencing to promote deep learning, they did not attempt to measure the interactive dimension taking place in the discussion. Gunawardena et al. (1997) and Newman et al. (1995) both focused on the problem of assessing the quality of online discussions. Although the latter provided focused and segmented coding on certain potential indicators of CT, the former provided a more holistic view of discussion ow and knowledge construction (Marra et al. 2004). Thus, the tag codes with critical (+) and uncritical () valences used in Newman et al.s coding scheme (see Table 1, Part B) were incorporated into Gunawardena et al.s interaction model (see Table 1, Part A) to form a new coding system called The Coding Scheme for Evaluating Critical Thinking in Computer Conferencing for the current study (see Table 1) (Yang et al. 2005). The combined coding system evaluates whether the students discussion moves from the lower to the higher phases of CT, and whether the students demonstrate more in-depth CTS within their online discussions.

Training session for establishing inter-rater reliability Two ratersthe researcher and the instructor, followed the coding manual, detailing the criteria and procedure for assigning a code for the different units. The raters reviewed and negotiated the denitions of 21 CT categories and 14 different tags in the coding theme one by one. The raters separately evaluated and assigned ratings to the rst online discussion in the pilot study. Using Miles and Hubermans (1994) inter-rater reliability formula (number of agreements/(total number of agreements + disagreements)), the initial inter-rater reliability coding had an 83.66% agreement. When differences occurred, two raters discussed the discrepancies in the coding results until a consensus was reached. An independent coding of the second discussion achieved a 91.35% inter-rater reliability. Based on this rating, both the accuracy and reliability of using this coding instrument met the general check-coding standard, which requires a 90% range (Miles and Huberman).

Coding procedure One of the rst tasks in the coding procedure was to parse the discussion transcripts into a unit of analysis, that is, the portion of communication that would be used as the smallest unit to analyze. A unit of analysis was determined to be a phrase, sentence, paragraph, or message, which illustrates any one of the indicators (see Table 1). The researcher and instructor discussed, negotiated, and then parsed the discussion transcripts into units of analysis together. Then they independently rated each unit across the Gunawardena et al. (1997) category of interactions and all six of Newman et al.s (1995) criteria, if applicable. The following examples are used to explain how each unit of the online postings was coded. For example, if Student A started a new (N+) discussion (IA), which was relevant (R+) and important (I+) to the discussion issue but the statement was inaccurate (A) and without supporting arguments (J), the coding was [IA/R+/I+/N+/A/J]. Please note that the indicator, critical assessment (C), was not suitable in this case because Student A was the rst person to post a message in the discussion forum; hence, no other students statements or opinions were available for him to evaluate. Another example is Student B, who critically evaluated Student As message using a logical thinking process (C+J+) and replied by stating a contradictory viewpoint (IIA) that no one had mentioned before (N+). In addition, Student Bs statement was accurate (A+), relevant (R+), and important (I+),

123

Teaching critical thinking in a large university class

253

with logical supporting arguments (J+L+). Therefore, it was coded [IIA/R+/I+/N+/A+/ J+L+/C+J+]. After the coding, we counted the total number of units of analysis that occurred in each of the phases and subcategories according to the Coding Scheme for Evaluating Critical Thinking in Computer Conferencing (see Table 1, Part A). In addition, similar to what Newman et al. (1995) had done, the total for each + or indicator was counted, and a depth of CT ratio (x ratio) was calculated for each criterion (see Table 1, Part B). Both the researcher and the instructor coded all of the four online discussions in the comparison and experimental groups and calculated the inter-rater reliabilities for each discussion separately. Using Miles and Hubermans (1994) formula, the inter-rater reliabilities for the eight coded online discussions ranged from 91.25% to 94.38%.

Results California critical thinking skills test (CCTST) A two-way mixed design ANOVA was performed to identify whether there was a difference between the comparison and experimental groups on the CCTST, and whether there was a difference between the pretest and posttest of CCTST in each group. The mean scores of CCTST (see Fig. 3) on the pretest were 16.79 (SD = 3.58) and 17.21 (SD = 3.36) in the comparison and experimental groups, respectively. On the other hand, the mean scores on the posttest of CCTST were 16.92 (SD = 3.84) and 18.23 (SD = 3.72) in the comparison and experimental groups, respectively. The CCTST scores were then analyzed using a 2 2 mixed design ANOVA in which the instructional group was a between-subjects factor, and occasion was a within-subjects factor. The instructional group was divided into two levels (comparison group and experimental group), and occasion was also divided into two levels (pretest and posttest). The results indicate that both the main effects of instructional group, F (1, 276) = 4.87, p = .03, g2 = .02, and occasion, F (1, 276) = 8.96, p < .001, g2 = .03, and the interaction between them, F (1, 276) = 5.26, p = .02, g2 = .02, were all signicant. Because the interaction effect was signicant, follow-up comparisons of means were conducted. Results revealed that the posttest mean (M = 18.23) in the experimental group was signicantly higher than the pretest mean (M = 17.21) in the experimental group (F (1, 276) = 16.27, p < .001, g2 = .06), but there were no differences in the comparison group (F (1, 276) = .21, p = .65, g2 = <.001). In addition, while no signicant difference

CCTST Scores

Fig. 3 The means of pretest and posttest CCTST scores of the two research groups

18.5 18.23 18 17.5 17.21 17 16.79 16.5 16 Pretest Posttest 16.92

experimental group comparison group

123

254

Y.-T. C. Yang

(F (1, 552) = 1.01, p = .32, g2 = <.001) was found between the pretest CCTST mean of the comparison (M = 16.79) and experimental groups (M = 17.21), the posttest CCTST mean score of the experimental group (M = 18.23) was signicantly higher than that of the comparison group (M = 16.92), F (1, 552) = 8.26, p = .04, g2 = .02.

Online discussions As shown in Table 2, there were a total of 3,244 and 5,732 units of analysis in the rst half of the discussion in the comparison and experimental groups, respectively, and 3,275 and 5,965 units of analysis in the second half, respectively. The number of units of analysis for each discussion was about the same. For example, 5,965 units of analysis in the second half of the semester in the experimental group included 2,971 units in the third discussion (case study) and 2,994 units in the fourth discussion (debate). On average, while each comparison group participant contributed 44.96 units of analysis ((3,244 + 3,275) units/145 participants), each experimental group participant contributed 87.95 units ((5,732 + 5,965) units/133 participants), respectively. On average, there were 2.15 units of analysis for one posting. The number of units of analysis, in general, was approximately proportionally equivalent to the number of messages. That is, the more units of analysis in a discussion, the more student postings were generated. Therefore, the above results imply that students in both the comparison and the experimental groups participated actively in the structured online environment (on average, 44.96 and 87.95 units per student per semester, respectively). However, when the TAs modeled and challenged students CTS during the online discussion in the experimental group, the students seemed to be more motivated to participate, and the discussions tended to be more interactive. Comparison group In the comparison group, 78, 8, 2, 1, and 11% of the analysis units were coded as Phases IV, respectively, in the rst half of the structured ADF. In the second half, 72, 11, 4, 2, and 12% of the units were coded in the respective phases (see Table 2). The results of the chi-square test, v2(4, n = 6,519) = 33.25, p < .001, / = .07, show that during the second half of the ADF, the discussion tended to gradually move toward higher phases (see Fig. 4). Moreover, the depth of CT ratios (see Table 3 and Fig. 5) for each indicator in the second half of the ADF was slightly higher (v2(1, n = 18,335) = 19.68, p < .001), / = .03, than in the rst half: R (relevance) (86% vs. 90%), I (importance) (85% vs. 89%), N (novelty) (61% vs. 66%), A (accuracy) (63% vs. 67%), J (justication) (45% vs. 50%), and C (critical assessment) (46% vs. 48%). Although there was no instructional intervention in the comparison group, the students also improved the quality of their online discussion in the second half. This result probably occurred because the social construction of knowledge in the structured ADF induces students higher-level cognitive processes, such as reection, social negotiation, and conceptual change in student misconceptions.

123

Teaching critical thinking in a large university class

255

Table 2 Results of asynchronous online discussions by interaction category for comparison group and experimental group Interaction category First half of the semester (Discussions 1 and 2) # of unit of analysis Comparison group Phase I IA IB IC ID IE Sum Phase II IIA IIB IIC Sum Phase III IIIA IIIB IIIC IIID IIIE Sum Phase IV IVA IVB IVC IVD IVE Sum Phase V VA VB VC Sum Total Phase I IA IB IC ID IE Sum Phase II IIA IIB IIC Sum Experimental group 2,894 205 86 335 86 3,606 384 340 71 795 13.87 62.91 2,203 179 90 341 88 2,901 465 597 103 1,165 19.53 1,960 16.76 48.63 6,507 55.63 2,013 124 96 198 87 2,518 178 77 5 260 24 2 32 13 2 73 5 7 12 10 6 40 25 112 216 353 3,244 10.88 100.00 1.23 2.25 8.01 77.62 1,843 130 90 206 89 2,358 255 93 7 355 36 7 55 15 3 116 7 8 18 10 8 51 27 128 240 395 3,275 12.06 100.00 748 11.47 6,519 100.00 1.56 91 1.40 3.54 189 2.90 10.84 615 9.43 72.00 4,876 74.80 % Second half of the semester (Discussions 3 and 4) # of unit of analysis % Total %

123

256

Y.-T. C. Yang

Table 2 continued Interaction category First half of the semester (Discussions 1 and 2) # of unit of analysis Phase III IIIA IIIB IIIC IIID IIIE Sum Phase IV IVA IVB IVC IVD IVE Sum Phase V VA VB VC Sum Total 116 14 156 74 5 365 23 21 56 35 15 150 95 41 680 816 5,732 14.24 100.00 2.62 6.37 % Second half of the semester (Discussions 3 and 4) # of unit of analysis 145 31 296 115 13 600 42 26 84 42 21 215 108 72 904 1,084 5,965 18.17 100.00 1,900 16.24 11,697 100.00 3.60 365 3.12 10.06 965 8.25 % Total %

100.00% 90.00% 80.00% 70.00% 60.00% 50.00% 40.00% 30.00% 20.00% 10.00% 0.00% Phase I 8.01% 10.84% 2.25% 3.54% Phase II Phase III 77.62% 72.00%

comparison group (1st half) comparison group (2nd half)

12.06% 10.88% 1.56% 1.23% Phase IV

Phase V

Fig. 4 Online discussion results by interaction categories for the comparison group

123

Teaching critical thinking in a large university class Table 3 Depth of critical thinking by indicator for comparison group and experimental group Tag Code First half of the semester (Discussions 1 and 2) Number of codes Comparison group R (Relevance) I (importance) N (novelty) A (accuracy) R+ R I+ I N+ N A+ A J (justication) J+L+ J+L J C (critical assessment) Total Experimental group R (Relevance) I (importance) N (novelty) A (accuracy) R+ R I+ I N+ N A+ A J (justication) J+L+ J+L J C (critical assessment) Total C+L+ C+L C 4,431 98 3,612 93 2,343 123 3,521 204 3,229 27 218 724 20 48 18,691 82.83 85.90 89.05 90.02 94.98 95.67 4,881 59 3,631 43 2,256 45 3,706 123 3,354 23 85 748 18 21 18,993 90.09 93.76 93.58 96.09 97.66 97.61 9,312 157 7,243 136 4,599 168 7,227 327 6,583 50 303 1,472 38 69 37,684 86.45 89.82 91.34 92.95 96.31 96.68 C+L+ C+L C 1,932 148 1,910 156 1,341 325 1,245 285 1,088 147 269 161 15 45 9,067 45.70 44.68 62.75 60.98 84.90 85.77 2,043 109 2,036 119 1,360 278 1,250 248 1,202 153 243 168 17 42 9,268 48.02 50.44 66.89 66.06 88.96 89.87 3,975 257 3,946 275 2,701 603 2,495 533 2,290 300 512 329 32 87 18,335 46.88 47.65 64.80 63.50 86.97 87.85 Depth of CT ratio (%) Second half of the semester (Discussions 3 and 4) Number of codes Depth of CT ratio (%) Total

257

Number of codes

Depth of CT ratio (%)

Note. Depth of CT ratio = (positive indicator negative indicator)/(positive indicator + negative indicator)

123

258
100.00% 89.87% 85.77% 80.00% 84.90% 88.96%

Y.-T. C. Yang

comparison group (1st half) comparison group (2nd half)

90.00%

70.00%

66.06% 60.98%

66.89% 62.75%

60.00% 50.44% 50.00% 44.68% 40.00% Relevance Imoprtance Novelty Accuracy Justification

Fig. 5 Online discussion results by CT indicators for the comparison group

Experimental group In the experimental group, 63, 14, 6, 3, and 14% of the analysis units were coded in Phases IV, respectively, in the rst half of the ADF discussion (see Table 2). In the second half, 51, 20, 10, 4, and 18% of the units were coded in Phases IV, respectively. Figure 6 shows that the students discussion progressively and signicantly moved from the lower mental functions (Phase I) to the higher mental functions (Phases IIV), v2(4, n = 11,697) = 248.29, p < .001, / = .15. Moreover, the depth of CT ratios (see Table 3 and Fig. 7) for each indicator in the second half of the ADF was signicantly higher (v2(1, n = 37,684) = 149.00, p < .001,
100.00% 90.00% 80.00% 70.00% 62.91% 60.00% 50.00% 40.00% 30.00% 20.00% 10.00% 0.00% Phase I Phase II Phase III Phase IV Phase V 19.53% 13.87% 6.37% 10.06% 2.62 % 3.60% 18.17% 14.24% 4 8.63%
experimental group (1st half) experimental group (2nd half)

Fig. 6 Online discussion results by interaction categories for the experimental group

123

Teaching critical thinking in a large university class

100.00%

259

95.67%96.68%

94.98%

96.31% 92.95% 90.02%

90.00%

experimental group (1st half) experimental group (2nd half) 91.34% 89.82% 89.05% 85.90%

80.00%

70.00%

60.00%

50.00%

40.00%

Relevance

Imoprtance

Novelty

Accuracy

Justification

Fig. 7 Online discussion results by CT indicators for the experimental group

/ = .06) than in the rst half of the ADF: R (relevance) (96% vs. 98%), I (importance) (95% vs. 98%), N (novelty) (90% vs. 96%), A (accuracy) (89% vs. 94%), J (justication) (86% vs. 94%), and C (critical assessment) (93% vs. 90%). These results show that during the second half of the ADF, the positive impact of the teaching and modeling of Socratic questioning via the ADF increased over time in the experimental group. Comparison group versus experimental group When comparing the quality of interaction between the comparison and experimental groups (see Fig. 8), a statistically signicant difference was found, v2(4, n = 18,216) = 709.30, p < .001, / = .20. The results show that the instructional treatment was signicantly associated with the quality of interactions: students who participated in Socratic dialogues had a higher quality of interaction than students who participated in the comparison group.
100.00% 90.00% 80.00% 70.00% 60.00% 50.00% 40.00% 30.00% 20.00% 10.00% 0.00% Phase I 9.43% 2.90% Phase II Phase III 16.76% 8.25% 3.12% 1.40% Phase IV 16.24% 11.47% 55.63% 74.80%

comparison group (total) experimental group (total)

Phase V

Fig. 8 Online discussion results by interaction categories for the two research groups

123

260
100.00% 96.68% 92.95% 90.00% 87.85% 86.97% 91.34%

Y.-T. C. Yang

comparison group (total) experimental group (total)

89.82% 86.45%

80.00%

70.00% 63.50% 60.00% 64.80%

50.00%

47.65%

40.00% Relevance Imoprtance Novelty Accuracy Justification

Fig. 9 Online discussion results by CT indicators for the two research groups

In addition, as shown in Fig. 9, the analysis of the depth of CT indicated that both the comparison and experimental groups had high R (relevance) (88% and 97%) and I (importance) (87% and 96%) ratios. The participants, in general, brought in relevant materials and seemed to have adopted a serious style when taking part in the discussions. There were notable differences for N (novelty) (64% vs. 93%), A (accuracy) (65% vs. 91%), J (justication) (48% vs. 90%), and C (critical assessment) (47% vs. 86%). The chi-square test of the relationship between the instructional treatment and the depth of CT demonstrated that the structured ADF was signicant, v2(1, n = 56,019) = 2,275.93, p < .001, / = .20. Depth of overall CT in the experimental group was higher than that in the comparison group. Discussion In this study, we investigated the effects of teaching CT in a large class through ADF with the facilitation of teaching assistants. The results of the data analyses are discussed in detail in the following sections. California critical thinking skills test (CCTST) Results indicated that the comparison group did not signicantly enhance their CTS levels after actively participating in the online discussions. However, the students in the experimental group signicantly improved their levels of CTS and also outperformed the comparison students. This outcome probably occurred because teaching and modeling of Socratic questioning via ADFs stimulated students to exchange their thoughts and evaluate their understanding of the material through being critically challenged by each other and by their TAs in the experimental group. These results conrm those from the studies of Brown and Palincsar (1989), Wenger (1998) and Yang et al. (2005), which indicated that learning

123

Teaching critical thinking in a large university class

261

is accomplished as students interact with their peers or instructor and benet from combining their levels of expertise, offering support, distributing the thinking load, and confronting alternative points of view. The results indicate that learning is also accomplished as students elaborate, clarify, and justify their personal responses/solutions.

Online discussions In the structured ADF experienced by both the comparison and experimental groups, the students demonstrated very high levels of interaction, a social interaction that reduces students reliance on the passive viewing mode of learning. Interpretations of the patterns for each group follow separately. Comparison group Results show that the comparison students improved the quality of their online discussion in the second half relative to the rst half. Thus, even without the instructional intervention (Socratic dialogues), the students also improved the quality of their online discussion in the second half. These results disconrm the ndings of some previous studies (Davies and Graff 2005; Sing and Khine 2006) in which greater online interaction did not lead to more in-depth interaction or better learning performance. The students in the comparison group seemed to ask and answer questions more critically to clarify ideas and negotiate meaning as well as to identify areas of agreement or overlap among conicting concepts. These behaviors might have been fostered by students learning in the structured ADF how to negotiate the meaning and co-construction of knowledge through the social learning experience. Their classmates feedback provided benecial information and questions, which made them re-analyze some of their answers, eventually increasing the quality of the discussion and their thoughts. Although there were positive results in the online discussions when the students worked as a team in the comparison group, the individual CTS of students did not improve as reported in the CCTST section. This outcome implies that although the students improved their CTS when the TAs and the students probed the meaning, justication, or logical strength of other students statements in the group discussion, they did not improve their CTS when working independently on their CCTST. Thus, without the external prompts, stimulus, or guidance of the instructor and TAs in teaching and modeling CTS, an interactive dialogue alone cannot effectively increase students CTS over a short period of time. More time and practice might be needed for students to learn and practice such skills from their interaction with classmates before these skills become self-monitoring and independent. Experimental group When students participated in Socratic dialogues, as modeled and facilitated by the facilitator via ADFs during the rst half of the semester, they learned how to use Socratic questioning prompts to articulate and argue their positions in detail. Thus, students provided evidence to support their ideas, opinions, and answers and to evaluate their own and others decisions during the rst half of the ADF. These results corroborate those of previous studies although the latter were conducted in different learning environments (Bullen 1998; Jeong 2003; Salmon 2002; Walker 2004; Yang et al. 2005). With the practice and

123

262

Y.-T. C. Yang

experience of thinking critically, students seemed to think more logically and thoroughly before posting their messages in the second half of ADF, resulting in a better quality of critical discussion. This result suggests that the positive impact of the teaching and modeling of Socratic questioning via the ADF increased over time in the experimental group. Comparison group versus experimental group When comparing the quality of students online discourse to reveal the process of CT between two research groups, we found that the experimental group demonstrated signicantly higher phases of interaction and a higher depth of CT ratios than did the comparison group. In the experimental group, Socratic questioning was taught and modeled to stimulate students minds by continually probing the subject with thought-provoking questions. The students therefore seemed to think longer before posting their messages. As a result, the number of negative statements for CT indicators (such as novelty, accuracy, justication, and critical assessment) was reduced, which greatly increased the depth of CT ratios for these indicators. These results suggest that instructional intervention (teaching and modeling of Socratic questioning) should be employed during structured online discussions to encourage learners to critically engage in learning issues. Conclusions Contributions of the study Faced with todays ever-changing information technology and knowledge economy, testing the validity of claims made by journalists, politicians, researchers, and other citizens is a constant challenge. When citizens work together to solve public problems, a lot of effort goes into understanding the problem and deciding what to do or believe about it. This type of interaction requires excellent CTS. Thus, the importance of CT and the teaching of such skills have been widely emphasized (Daud and Husin 2004; Ennis 1985; Yeh 2006). However, relatively few researchers in the past decades have tried to teach CT in a large university class. Because of the number of students in a large class, it is difcult to foster CT using the traditional didactic approach to teaching. With the rapid development of computer and network technologies, much research has revealed that structured ADFs have tremendous potential to foster communities of inquiry and trigger a critical spirit (Angeli et al. 2003; Duphorne and Gunawardena 2005; Sharma and Hannan 2004). Therefore, this study was designed to investigate an instructional strategy (i.e., teaching CTS with the facilitation of TAs via ADFs) that can be used to overcome obstacles in the development of CTS in a large class setting. The empirical results indicate that with an instructor who teaches CTS at the beginning of the class and TAs who adopt Socratic questioning prompts to model CTS during small online group discussions, students CTS can be successfully developed in a large class environment. In addition, unless the instructor and the TAs play a pedagogical role in teaching, modeling, and prompting Socratic dialogues, an interactive dialogue alone cannot effectively help students become independent critical thinkers over a short period of time. Finally, based on the results of this study, we strongly recommended that intellectual online discussions be designed where students can enjoy high quality discussions and voice their thoughts, and where an instructor can guide, model, and develop students CTS, one of the most important skills for citizens in a democratic society.

123

Teaching critical thinking in a large university class

263

Limitations and suggestions for future research There are several limitations that might have affected the outcome of this study. The rst limitation of this study is that the students levels of CTS were measured by the results from the CCTST and their contributions to the online discussions. Future studies should include interviews to probe more in-depth opinions, comments, or attitudes on students reactions to the teaching and modeling of Socratic Questioning via structured WBBs. A second limitation is that the sample used in this study was a convenience rather than a random sample. Still, the ndings of this study can be transferable to similar contextsa large undergraduate level general education course at a large university. Third, CT is a mental process that seeks to clarify as well as evaluate the action and activity that one encounters in life. In general, good critical thinkers should possess both CTS and CT dispositions (CTD), which refer to the willingness, desire, and disposition to base ones actions and beliefs on reasons (Siegel 1988). Facione (2007) mentioned that CTS and CTD might be two separate things in people: Being skilled does not assure one is disposed to use CT, and being disposed toward CT does not assure that one is skilled. This study only investigated the mental processes but not the thinking dispositions (i.e., willingness/inclinations). Additional research is needed to focus on how both to develop students CTS and to nurture students consistent internal motivation to use these skills. This effort will, thus, be a complete endeavor in cultivating students CT.
Acknowledgments The funding for this research was provided by the National Science Council of Taiwan, ROC under Grant NSC92-2520-S006-001. We extend our special thanks to National Cheng Kung University (NCKU) for supporting us through a free and reliable e-learning system to make this study a success. The Academic Affairs Division of NCKU has selected this course as having the best design and implementation of an e-learning course in 2004.

References
Angeli, C., Valanides, N., & Bonk, C. J. (2003). Communication in a web-based conferencing system: The quality of computer-mediated interactions. British Journal of Educational Technology, 34(1), 3143. Barnett, R. (2004). Learning for an unknown future. Higher Education Research and Development, 23(3), 247260. Brown, A. L., & Palincsar, A. S. (1989). Guided, cooperative learning and individual knowledge acquisition. In L. B. Resnick (Ed.), Knowing, learning, and instruction: Essays in honor of Robert Glaser (pp. 393 451). Hillsdale, NJ: Erlbaum. Bullen, M. (1998). Participation and critical thinking in online university distance education. Journal of Distance Education, 13(2), 132. Daud, N. M., & Husin, Z. (2004). Developing critical thinking skills in computer-aided extended reading classes. British Journal of Educational Technology, 35(4), 477487. Davies, J., & Graff, M. (2005). Performance in e-learning: Online participation and student grades. British Journal of Educational Technology, 36(4), 657663. Dillon, J. T. (1988). Questioning and teaching: A manual of practice. New York: Teachers College Press. Driscoll, M. P. (1994). Psychology of learning for instruction. Toronto, ON: Allyn and Bacon. Duffy, T. M., Dueber, B., & Hawley, C. (1998). Critical thinking in a distributed environment: A pedagogical base for the design of conferencing systems. In C. J. Bonk & K. S. King (Eds.), Electronic collaborators: Learner-centered technologies for literacy, apprenticeship, and discourse (pp. 5178). Mahwah, NJ: Lawrence Erlbaum Associates. Duphorne, P. L., & Gunawardena, C. N. (2005). The effect of three computer conferencing designs on critical thinking skills of nursing students. The American Journal of Distance Education, 19(1), 3750. Ellis, R. A., & Calvo, R. A. (2006). Discontinuities in university student experiences of learning through discussions. British Journal of Educational Technology, 37(1), 5568. Ennis, R. H. (1985). A logical basis for measuring critical thinking skills. Educational Leadership, 43(2), 4448.

123

264

Y.-T. C. Yang

Facione, P. A. (1990a, 1992). The California critical thinking skills test (CCTST): Form A (1990) and form B (1992). Millbrae, CA: California Academic Press. Facione, P. A. (1990b). Critical thinking: A statement of expert consensus for purposes of educational assessment and instruction. Research ndings and recommendations. ERIC Document Reproduction Service No. ED315423. Facione, P. A. (2007). Critical thinking: What it is and why it counts. Millbrae, CA: California Academic Press. Flammer, A. (1981). Towards a theory of question asking. Psychological Research, 43, 407420. Garrison, D. R., Anderson, T., & Archer, W. (2001). Critical thinking, cognitive presence, and computer conferencing in distance education. American Journal of Distance Education, 15(1), 723. Gilbert, P. K., & Dabbagh, N. (2005). How to structure online discussions for meaningful discourse: A case study. British Journal of Educational Technology, 36(1), 518. Gunawardena, C., Lowe, C., & Anderson, T. (1997). Analysis of global online debate and the development of an interaction analysis model for examining social construction of knowledge in computer conferencing. Journal of Educational Computing Research, 17(4), 397431. Harasim, L., Hiltz, S., Teles, L., & Turoff, M. (1995). Learning networks: A eld guide to teaching and learning online. Cambridge, MA: MIT Press. Jeong, A. (2003). Sequential analysis of group interaction and critical thinking in online threaded discussions. The American Journal of Distance Education, 17(1), 2543. King, A. (1990). Enhancing peer interaction and learning in classroom through reciprocal questioning. American Educational Research Journal, 27(4), 664687. Kanuka, H., & Anderson, T. (1998). Online social interchange, discord, and knowledge construction. Journal of Distance Education, 13(1), 5774. Maiorana, V. P. (19901991). The road from rote to critical thinking. Community Review, 11(12), 5363. Marra, R., Moore, J., & Klimczak, A. (2004). Content analysis of online discussion forums: A comparative analysis of protocols. Educational Technology Research and Development, 52(2), 2340. Miles, M. B., & Huberman, A. M. (1994). Qualitative data analysis: An expanded sourcebook. Thousand Oaks: Sage. Newman, D., Webb, B., & Cochrane, C. (1995). A content analysis method to measure critical thinking in face-to-face and computer supported group learning. Interpersonal Computing and Technology: An Electronic Journal for the 21st Century, 3(2), 5677. Paul, R. W. (1995). Critical thinking: How to prepare students for a rapidly changing world. Santa Rosa, CA: Foundation for Critical Thinking. Preston, J., & Shackelford, R. (1998). A system for improving distance and large-scale classes. ACM SIGCSE Bulletin Inroads, 30(4), 193198. Salmon, G. (2002). Mirror, mirror, on my screenExploring online reections. British Journal of Educational Technology, 33(4), 379391. Sharma, P., & Hannan, M. (2004). Scaffolding critical thinking in an online course: An exploratory study. Journal of Educational Computing Research, 31(2), 181208. Siegel, H. (1988). Educating reason: Rationality, critical thinking, and education. New York: Routledge Chapman & Hall. Sing, C. C., & Khine, M. S. (2006). An analysis of interaction and participation patterns in online community. Educational Technology & Society, 9(1), 250261. Taba, H. (1966). Teaching strategies and cognitive functioning in elementary school children (Cooperative Research Project No. 2404). San Francisco: San Francisco State College. Walker, S. A. (2004). Socratic strategies and devils advocacy in synchronous CMC debate. Journal of Computer Assisted Learning, 20, 172182. Wenger, E. (1998). Communities of practice: Learning, meaning, and identity. Cambridge, UK: Cambridge University Press. Yang, Y.-T. C., Newby, T. J., & Bill, R. L. (2005). Using Socratic questioning to promote critical thinking skills through asynchronous discussion forums in distance learning environments. The American Journal of Distance Education, 19(3), 163181. Yeh, Y. (2006). The interactive effects of personal traits and guided practices on preservice teachers changes in personal teaching efcacy. British Journal of Educational Technology, 37(4), 513526. Young, G. (1997). Adult development, therapy, and culture: A postmodern synthesis. New York: Plenum Press.

Ya-Ting C. Yang is an Assistant Professor at the Institute of Education and Center for Teacher Education, National Cheng Kung University, Taiwan. Her research interests are in the eld of educational research and the role of information and communication technologies for improving teaching and learning processes.

123