Lars Leader
Department of Curriculum & Instructional Technology Valdosta State Uni- versity U.S.A.
lfleader@valdosta.edu
Lorraine Schmertzing
Department of Curriculum & Instructional Technology Valdosta State Uni- versity U.S.A.
lschmert@valdosta.edu
PROMOTING STUDENT COLLABORATION IN AN ONLINE COURSE
Introduction
Collaboration (Dunlap, 1999) and problem-based learning (Robinson, 1993) have emerged in recent years as important strategies for improving learning in high- er education. Collaborating to solvie the complex tasks presented by problem-based learning is frequently associated with better use of higher-level thinking skills, such as planning, reflection and metacognition (O’Malley, 1995). Incorporating technol- ogies such as asynchronous discussions, computer-mediated communication in online courses is also capable of promoting critical thinking (Barab, Tomas, & Mer- ril, 2001). In an online graduate research course frequently offered over the years, we have provided students with problem-based projects that combine the advantages of both collaborative teamwork and an online environment.
Although working online for team projects can provide the practical communica- tion benefits of distance education and enhance the collaborative process, the dy- namics of working in an online team may entail the problems that are commonly found in face-to-face teams. A frequent complaint made by members of project teams is that other members don’t “pull their weight.” Individuals may reduce their efforts when performing in a group, a form of behavior also called “social loafing”
(Falchikov & Goldfinch, 2000).
To attempt to address the problem of social loafing and encourage better group participation in our online classes, we introduced strategies intended to promote collaboration. In Phase I of a series of studies, students in an intervention group anonymously evaluated themselves and other team members on their collaborative efforts. In a subsequent survey, those students who had done the peer/self-evaluation increased their frequency of communications within the team. Attempting to provide a more effective intervention in Phase II of the study (Schmertzing & Leader, 2003), we used open-ended questions for the evaluations in order to encourage in students more reflection and more recognition of the strengths and weaknesses of their col- laborations. Student survey results for this phase showed the our intervention had no
positive effect on attitude towards collaboration. The low level of student response here suggested that the students may have felt threatened by the intervention, which was lengthy, serious, and fairly personal.
Based on the results of the Phase II study, we decided that in the next phase we would create another intervention that used a problem-based teamwork scenario to encourage discussion among participants about what makes good collaboration also to and measure the transference of students’ evaluations of the discussed scenario to their own team collaborations. The main research question in this phase of the study was “When online project team members respond to a problem-based teamwork scenario, what are the effects on team member’s attitudes toward their collaboration and the effectiveness of their group work?”
The Study
The participants in this study were graduate students in an evaluation and re- search methods course at a regional southeastern U.S. university. The course is a requirement for Masters and Educational Specialist students in our Instructional Technology program. We have described in a previous report the student course- work that is involved in our study:
Among the course requirements were three major projects in which the students worked collaboratively and were assessed by a group grade on each project. The first project involved each team observing an activity, writing descriptive notes on it, and interviewing one of the participants. The team report of this qualitative field- work included coded excerpts, data display, and interpretation of the meaning of behaviors in the observed activity. The second project was a team evaluation of an educational or training program. This mixed-method evaluation included testing, observation, and a survey. The third project was a proposal for an instructional in- tervention to address a concern within the evaluated program. The product for this project was a PowerPoint presentation that had been developed by the team. The students self-selected the members of their two- or three-person teams. (Leader &
Schmertzing, 2002, p. 1771)
The participants in Phase III were students from the two sections of the evalua- tion and research methods course during the Spring 2004 semester. These students were undertaking the same three major projects completed by the students in our previous studies.
The Phase III intervention was a scenario the researchers had created. One of the researchers randomly assigned half of the student teams, i.e. five teams (n=11), to an intervention group. After the first project, the same researcher distributed the scenar- io to students in the intervention group for them to read and then discuss. The sce- nario was a case based on several problems that we had witnessed in student teams from previous semesters. In the scenario, two teammates work on an assignment for an online class. While completing mutually agreed-upon components of the assign- ment, teammates did encounter conflicts, including late and inadequate completion of tasks, lack of agreement on conclusions from their research, and misunderstand- ings due to poor communication. At the end of the scenario, two questions are pre-
sented: “Where should the students go from here?” and “How could they have avoided this situation?” The intervention participants were asked to discuss their answers to these questions in an online forum created for their group on the course Web site. They were also informed that the professor who was grading their work in the course would not have access to the forum, nor would he know of their participa- tion in the study until the course was over.
Dependent measures in Phase III were student responses to the same pre- and post-surveys as were used in previous phases of our study:
The surveys consisted of 20 Likert-style questions. Some questions asked about skills in using online communication tools. An example was “What is your present level of skill using chat room discussion?” Other questions asked about the frequen- cy of interaction using these tools, for example: “While working with your team- mate(s), how often did you communicate by WebCT personal mail?” Another type of question asked about teamwork. For example, one question was “How often did problems concerning the responsibilities that members of your team were to assume affect the quality of your project?” An open-ended question at the end of each sur- vey asked students to comment on team project work in the course. (Leader &
Schmertzing, 2002, p. 1771)
The students responded to the pre-survey after the first project and responded to the post-survey after the third project. They responded anonymously, using an iden- tification code to match their two survey forms. Eight of the participants did not respond to one of the two surveys and were therefore not included in the final analy- sis of data. Of the 23 remaining students, six were males and 17 were females. The students in the intervention group responded to the intervention scenario a few days after they had responded to the pre-survey – and before they began work on the second project.
Likert-response survey data were analyzed using the SPSS General Linear Mod- el. Tests included a multivariate analysis for the between-subjects factor of engaging in the scenario discussion and a repeated-measures multivariate analysis for the within-subjects factors of responses to pre- and postsurvey questions, along with testing for interaction between these factors. Follow-up univariate tests were per- formed when multivariate significance was indicated. An alpha of .05 established significance. All data from the intervention scenario responses and the open-ended survey questions were qualitative. Therefore, these data were coded thematically, analyzed for frequency of responses, and examined both within and outside the context of the collaborative teams (Miles & Huberman, 1994).
Results
Before analyzing student responses to the surveys, we looked at the messages that students had posted in the forum established for discussing the scenario (provid- ed to the intervention group). Our analysis identified a few common themes. Strate- gies that the responding students frequently proposed for improving teamwork in the situation described by the scenario related to improving team communication meth- ods and making an effort to establish a plan and/or timeline early in the project. In
response to a lengthy posting in the discussion forum, one student summarized the consensus of many when she wrote, “You alluded to the fact that effective commu- nication is the key during a research process. I totally agree with you. Students A and B should have set up a schedule for communicating throughout the process from the beginning. They also should have delegated specific assignments.” Other, less common, yet still repeated, strategies included calling on the professor to mediate, changing partners, and taking responsibility for one’s role in a team. The preferred method of communication that students proposed was often meeting face-to-face.
However, when this was mentioned another student would routinely point out the logistical problems with that solution and propose the online chat as an alternative.
We wished to identify any differences as regards survey responses between those students who performed the scenario activity before completing the second and third course projects and those students who did not do the scenario activity. First, a sta- tistical analysis of responses to the questions with Likert-style response options was done. Significant results were obtained for within-subjects effects. The repeated- measures analysis of responses to survey questions indicated a(n overall) significant difference from the pre-survey to the post-survey, F(7,15)=8.77, p=.004. That is, a main effect was observed for the students as a whole class responding to the survey before and, then, after class projects. Follow-up univariate tests revealed significant differences, at the p=.05 level or better, between the pre-survey and post-survey with four of the 15 survey questions. Two of these survey questions concerned perceived skills at using online communication tools. Most students believed their skill with the discussion board posting at the beginning of the course was “a lot” (M=3.66), whereas all of them believed this at the end of the course (M=4.00). Likewise, stu- dents believed that their skill at using chat rooms had increased from “some”
(M=2.99) to “a lot” (M=3.52). No significant pre-survey to post-survey differences were found for students’ perceived skills with use of e-mail or Microsoft Word.
Two other survey questions with significant pre-post differences in student re- sponse concerned the frequency of use of different communication modes. On the average, students reported that they communicated by e-mail with teammates “3-6 times per week” while working on the first team project but only “1-2 times per week” while working on the follow-up projects. In contrast, the frequency of meet- ing with teammates face-to-face increased, from “less than once per week” on the first project to “1-2 times per week” on the following projects. No significant pre- survey to post-survey differences were found for frequency of communicating by telephone or chat room.
The multivariate tests also revealed that for the between-subjects factor of engag- ing in the scenario activity an overall significant amount of interaction occurred with the within-subject factors in pre-post survey question responses, F(7,15)=4.12, p=.03. Follow-up univariate tests also revealed significant interactions (at the p=.05 level), for engagement in scenario activity with pre-post responses on two survey questions. One question was, “From your experience with your teammate(s), how comfortable are you with working as a member of a team on projects in this class?”
The students who responded to the scenario indicated that that they were “comforta- ble” (3 on the response scale), though not “very comfortable” (4 on the scale), with
an increase from M=3.36 after the first project to M=3.46 at the end of the course. In contrast, students in the non-scenario group were also “comfortable” after the first project (M=3.33) but had become “uncomfortable”(M=2.58) by the end of the course. The other question that revealed a significant interaction was, “How effec- tive do you believe the team project work in this class is for your own learning of research and evaluation?” After the first project, students who responded to the scenario indicated they considered the team project work to be “effective” (M=3.18).
At the end of the course, they believed the team project work had been “very effec- tive” (M=3.55). On the other hand, the students in the non-scenario group also found the project work to be “effective” (M=3.08) for the first project. However, the opin- ions of these non-scenario respondents about the effectiveness of the project work had become more negative by the end of the course (M=2.58).
The last question on the surveys was, “Would you prefer to work individually on these projects rather than as a team?” The majority of the students (70% on the pre- survey; 56% on the post-survey) responded “No” rather than “Yes”, indicating that their preference is to work as a member of a team.
The other four survey questions asked about the frequency of specific types of problems with collaboration that might affect project quality: working as a member of a team, communicating with each other, assuming individual responsibility, and dealing with differences of opinion. No significant differences either pre-post or between treatment groups, were indicated via analysis of these four survey ques- tions.
We then analyzed qualitative responses to the survey statement that requested students to provide “any comments you have on team project work in this course.”
Responses to this open-ended question were first categorized according to how stu- dents had responded to the question “Would you prefer to work individually on these projects rather than as a team?” Of the group who engaged the scenario, 33%
of the students shifted from preferring to work with a team to preferring to work alone on the 7070 course projects; and of the students who did not work with the scenario, 27% reported the same shift. Interestingly, in the pre-survey all of the students who later shifted from team preference to an individual work-style prefer- ence focused comments on the usefulness and value of teamwork. Among these responses were, “I like working with teams because my peers help me get to know if I am working in the right direction,” and “Even though at times it is difficult to or- ganize and plan when more than one person is responsible for a project, in the long run it is a better experience because in the real world teams are often most success- ful.” However, the intervention and non-intervention groups differed on their post- survey responses in that the teams that had worked with the scenario identified con- cerns related to the self (e.g., “It seems that I may have learned more if I had com- pleted the project myself.”), while those who did not respond to the scenario identi- fied problems with others as their reasons for no longer wanting to do group work (e.g., “My team member(s) did not help.”). This focus on other team members as the source of difficulties was also apparent in our earlier research (Schmertzing & Lead- er, 2004).
The tendency to blame others for group work problems was a theme that was al- so seen in the post-survey responses of members of teams that had not worked with the scenario and who reported that they preferred individual work to team work both before and after the intervention. In fact, 46% of the non-scenario group fell into this category, with comments like, “I don’t feel that my teammate worked as hard as I did,” and “I think most people find it easier to just do all the work themselves if they have an unresponsive, ‘bad’ partner.” Other comments this group of participants routinely made were related to the difficulty of meeting and scheduling with others.
Only one person who responded to the scenario stated a preference for individual work in both the pre-survey and post-survey. This student’s only comment was, “I really appreciate alternating teamwork assignments with individual ones. It was a good break.”
The majority of the students who worked with the scenario (56%) reported to prefer teamwork over individual work both before and after the intervention. One student from this group commented in the post-survey, “I can’t simply say that teamwork does not work due to one bad experience; I look forward to future team- work and the learning that comes with it.” Students who maintained their positive attitude toward the use of teamwork throughout the course began by commenting about how much they “enjoy” teamwork or how useful the teamwork can be. The idea that when the people work, the team works well, was recurrent in both those students who responded to the scenario and those who did not respond to it.
Discussion
In this study, we set out to discover what happens to team member attitudes to- wards collaboration as graduate students completed an online project in teams, then discussed a problem-based teamwork scenario, and then did more team projects. In earlier phases of this on-going study, team members evaluated various aspects of their own and their teammates’ collaborations. In the current phase, our focus was on how team collaboration might be affected by participant evaluation of collabora- tion problems (in a hypothetical case). We had hoped that this non-threatening envi- ronment would increase the level of engagement and depth of thought of the partici- pants through discussion of how to effectively improve online collaboration. As explained by Hass and Hass (1998), providing students with an opportunity to re- flect on the various criteria that make for an effective collaborative effort may sug- gest to them avenues for improving the group effort. That such reflection can bring about changes in group members’ attitudes appears to have been confirmed with the intervention in our study.
Participation in discussing the scenario resulted in significant changes in atti- tudes for two of the more comprehensive items on the survey. Students who re- sponded to the scenario became more comfortable with teamwork by the end of the course, whereas those who did not discuss the scenario became less comfortable as members of a team. Similarly, the intervention group became more positive in their opinions about the effectiveness of team project work for their own learning while the non-intervention group became more negative about team effectiveness. These
were two of the more general attitudes probed by the survey. For more specific sur- vey questions, such as how often differences of opinion on a team affected the quali- ty of their projects, the scenario discussions had little impact. This lack of attitude change for more specific group dynamics was rather surprising, considering the patterns we found in students’ responses on the survey’s open-ended question. The post-survey open responses of students in the intervention group mainly focused on concerns about their own actions, whereas the post-survey responses of non- intervention students were directed towards problems with their teammates. Overall, students who responded to the problem-based teamwork scenario maintained a more positive outlook on teamwork at the end of the course, even if the team they were working on did not perform up to the student’s desired expectations.
In their online discussions of the scenario, students proposed a number of strate- gies for improving teamwork. A common suggestion was that team members should establish effective communication, such as through setting up a schedule at the be- ginning of the project for communicating and delegating assignments. A number of students emphasized the importance of taking responsibility for one’s role in a team.
In general, these discussions indicate that the students thoughtfully considered the scenario and addressed it with practical strategies.
The strategies proposed by students in the scenario discussions suggest further possibilities for promoting collaboration online. Our future research may incorporate some of these strategies. We intend to continue to study how the collaboration among students in online courses may be improved. As more teaching and learning occurs in the online environment, accompanying attention and effort should be di- rected to ways to improve such online teamwork.
References
Barab, S., Thomas, M. & Merril, H. (2001). Online learning: From information dissemination to fostering collaboration. Journal of Interactive Learning Research, 12(1), 105–143.
Dunlap, J. C. (1999). Rich environments for active learning on the web: Guidelines and examples. Proceedings of WebNet 99 World Conference on the WWW and Internet. Association for the Advancement of Computing in Educa- tion.
Falchikov, N., & Goldfinch, J. (2000). Student peer assessment in higher education:
A meta-analysis comparing peer and teacher marks. Review of Educa- tional Research, 70, 287–322.
Haas, A. L., & Haas, R. W. (1998). The use of self-rating and peer-ratings to evalu- ate performances of student group members. Journal of Marketing Edu- cation, 20, 200–210.
Leader, L. F., & Schmertzing, L. C. (2002). Peer and self-evaluation for improving student collaboration in online courses. Proceedings of E-Learn 2002(01), 1770–1773.
Miles, M. B., & Huberman, A. M. (1994). Qualitative data analysis (2nd ed.) Thou- sand Oaks, CA: SAGE Publications.
O’Malley, C. (1995). Computer supported collaborative learning. Berlin: Springer.
Robinson, V. (1993). Problem-based methodology: Research for the improvement of practice. Oxford: Pergamon Press.
Schmertzing, L. C., & Leader, L. F. (2004). Improving student collaboration in online courses using reflective partner evaluations. Society for Infor- mation Technology and Teacher Education International Conference 2004(1), 1027–1032.
