Research Express@NCKU - Articles Digest (Volume 3 Issue 7)

1. Introduction

John Dewey stated that the central purpose of education is learning to think. As part of education, learners need to develop and learn to effectively apply critical thinking (CT) skills to their academic studies, to the complex problems that they will face in their professions, and to the critical choices they will be forced to make as a result of the information explosion and other rapid technological changes. In general, CT has been defined as “reasonable, reflective thinking that is focused on deciding what to believe or do.” Research has suggested that face-to-face verbal interaction, which includes rational dialogue and questioning among learners and the instructor, accelerates the acquisition of knowledge and skills while fostering CT development. Unfortunately, in distance education, learners are separated by distance and/or time from their instructor and peers; thus, learner/instructor as well as learner/learner interactions can be significantly hindered. However, with advances in technology in the World Wide Web (WWW) and computer-mediated communication (CMC), researchers can now explore new ways to make distance education a truly interactive experience and, by extension, improve the development of CT skills through the use of distance education modalities. Thus, the goal of this study was to examine the impact of structured online discussions on the improvement of learners’ CT skills in different phases of learning. Learners’ attitudes toward learning and enhancing CT skills via a structured WBB (Web-Based Bulletin Board) were also investigated. Based on the above goal, a quasi-experimental design was employed to test the following two research hypotheses:
1. Learners who participate in structured WBB discussions will demonstrate CT skills at higher levels than learners who participate in unstructured WBB discussions.
2. Learners who participate in structured WBB discussions will show more positive attitudes toward learning via WBBs than learners who participate in unstructured WBB discussions.

2. Method

2.1. Participants and independent variable

The participants were 23 distance learners (see Table 1) enrolled in the same online course (with three sections) in an undergraduate veterinary distance learning course at a large US university. The independent variable, WBB discussions, was divided into two levels: unstructured and structured.
•Unstructured WBB discussions (Treatment I): Similar to email, which many instructors use as an educational tool for communication/feedback in their instruction, the unstructured WBB discussions referred to Web-based discussions where a threaded WWW discussion forum was provided in which learners could ask questions, interact with each other, and obtain feedback from their peers or their instructor, but where no discussion topics were posted.
•Structured WBB discussions (Treatments II and III): The structured WBB discussions consisted of four posted discussion activities that provide learners with guidance in asking CT questions and evaluating their understanding of the course material. For example, during the CT modeling, the instructor demonstrated and prompted the use of thought-provoking questions such as: What is my point of view on the issue? Is there supporting evidence for my conclusions? Are those reasons adequate? By what reasoning did you come to that conclusion? Learners were required to contribute their ideas and thoughts to each exercise and were encouraged to think critically about the course content and the questions asked by their instructor and peers. Those within Treatment II also had the instructor model CT questioning during the second half of the semester (for two discussion activities), while within Treatment III, the CT questioning was modeled during the first half of the semester (for two discussion activities).

The impact of structured versus unstructured discussion was examined through a comparison of the quality of responses from participants in Treatment I with the quality of responses from participants in the other two treatments. Moreover, a comparison of Treatment II with Treatment III allowed for the examination of the effect of the instructor’s CT modeling at different points of time as well as its long-term impact.

Table 1. Experimental settings

2.2. Dependent variables

The two dependent variables were the learners’ levels of CT skills and their attitudes toward learning via WBBs (see Table 2). Learners’ levels of CT skills were measured via the California Critical Thinking Skills Test (CCTST) (Facione, 1990, 1992) as well as the Interaction Analysis Model (see Table 3) (Gunawardena, Lowe, & Anderson, 1997).

Table 2. Attitude survey

Table 3. Interaction Analysis Model

3.Data Analysis

Both quantitative and qualitative methods of data analysis were used in this study.
•For the quantitative data, a 2-way mixed design ANOVA was performed to identify whether differences between the results of the CCTST and the participants’ attitudes toward learning via WBBs existed among the three research groups.
•From a qualitative perspective, learner responses were categorized using the Interaction Analysis Model and analyzed using chi-square tests. Two raters—the instructor and the researcher—discussed, negotiated, and then together parsed the discussion transcripts into units of analysis. They then independently rated each unit across category of interactions. The following examples are used to explain how each unit of the online postings was coded. For example, if Learner A started a new discussion, the coding was [IA]. If Learner B evaluated Learner A’s message and replied by stating a contradictory viewpoint, it was coded [IIA]. Based on Miles and Huberman’s (1994) formula, the inter-rater reliabilities for the online discussion analyses ranged from 90.79% to 100%.

4. Results and discussion

The results of the data analyses showed both positive and negative findings in the testing of the research hypotheses. The main findings included:
(1).According to the CCTST scores, the learners who joined in structured WBB discussions (Treatments II and III) significantly improved their CT skills after the instructor facilitated discussion activities to help bring about more productive conversations via the WBB.
(2).From an analysis of the quality of learners’ online discourse to reveal the “process” of CT, learners who participated in a structured WBB discussion (Treatments II and III) demonstrated CT skills at a higher level than learners who participated in an unstructured WBB discussion (Treatment I). In addition, increased levels of interaction positively impacted the levels of CT within this structured WBB discussion environment. However, if WBBs are employed as an educational tool (similar to email) for learners to use in voluntarily interacting with their peers and instructor for feedback (Treatment I), they might not devote enough time and effort to actually develop the needed CT.
(3).In a structured WBB discussion environment, the sequence of the study procedure alternated between Treatments II and III conditions with the instructor’s modeling at different points in the semester. The results indicated that with the same training time, if the instructor started to model and challenge learners’ CT skills at the beginning of the online discussion (Treatment III) rather than in the middle of the semester (Treatment II), learners seemed to be more motivated to participate, and the discussion tended to be more dynamic and interactive. Furthermore, learners maintained their CT skills after the instructor facilitated CT questioning (Treatment III).
(4).From the attitude survey (see Table 2), the findings (see Figure 1) suggest that the experimental groups felt significantly more positive toward the use of WBBs in helping them learn better because the WBBs allowed them to ask questions and receive feedback from their peers more easily, to better share and compare viewpoints with their peers, and to better examine and justify their own opinions. In addition, the WBBs enabled learners to better negotiate and integrate the different opinions expressed by their peers and enabled the instructor to better observe and enhance the CT skills in the structured online discussion. Thus, after critically discussing the course materials on the structured WBB, the experimental groups had more positive attitudes than did the comparison group toward the use of WBBs for increasing the productivity, efficiency, and quality of their learning.

Fig. 1. Item-by-item means of post-course attitudes toward learning via WBBs.

5. Conclusion

Critical thinking is an important issue in higher education, and educators have continued to focus on the development of CT in learners. Structured WBB discussion has recently been used to improve learners’ CT skills; however, its effectiveness has not yet been examined experimentally. This study, using a quasi-experimental design, has been a first step toward ascertaining the effectiveness of structured WBB discussions in developing learners’ CT. Two different methods have been used to examine learners’ CT: (a) quantitative method: CCTST to investigate learners’ changes in their general CT skills, and (b) qualitative method: Interaction Analysis Model to investigate learners’ interaction patterns in different phases of the content-based class discussion. The combination of the two instruments allowed the authors to thoroughly study learners’ CT on both macro and micro levels. Positive gains in learners’ CT skills and attitudes provide empirical evidence that instructional designs incorporating WBB discussion and interaction can be effective and conducive to the development of CT.

References:

Facione, P. A. (1990, 1992). The California critical thinking skills test (CCTST): Form A (1990) and From B (1992). Millbrae, CA: California Academic Press.
Gunawardena, C., Lowe, C., & Anderson, T. (1997). Analysis of global online debate and the development of an interaction analysis model for examining social construction of knowledge in computer conferencing. Journal of Educational Computing Research, 17(4), 397-431.
Miles, M. B., & Huberman, A. M. (1994). Qualitative data analysis: An expanded sourcebook. Thousand Oaks: Sage.