Computer-Mediated Argumentation in Higher Education: Developing Discussion Skills via Roles

University of Birmingham

R.M.Pilkington@bham.ac.uk

ABSTRACT

Many authors advocate providing opportunities for critical discussion to develop students’ abilities to reason in a specialist subject. Computer Mediated Communication (CMC) offers some distinct advantages as well as disadvantages for debate. Research in analyzing CMC dialogue is revealing that effective discussions involve different kinds of role-taking including community building and management roles as well as argumentation roles. Our hypothesis was that making post-graduate students aware of all three kinds of role would lead to wider independent adoption of these roles by students. Based on student perceptions and further analysis by the researcher some preliminary results are presented.

Keywords

Computer Mediated Communication, argumentation, discourse, collaborative learning

INTRODUCTION

Constructivist theories of learning originally developed to explain conceptual development in children have more recently been applied to adult and expert knowledge construction through participation in social networks including online discourse communities (Ekeblad, 1998). Many authors advocate providing opportunity for interaction and discussion through collaborative learning with, through or around computers as a means of developing conceptual understanding and intellectual skills (Crook, 1998). Computer Mediated Communication (CMC) offers some distinct advantages as well as disadvantages for debate. Advantages are claimed to include greater inclusivity and participation by students, particularly by those typically under-represented in face-to-face debate (Sullivan & Pratt, 1996; Warschauer, 1996). Disadvantages include the need to develop new skills to manage multiple parallel threads of discourse in ways that enable coherence and focus to be maintained (Herring, 1999). This can create an additional load for the tutor who needs to develop new facilitative skills (Pilkington, Bennett & Vaughan, 2000; Kuminek & Pilkington, 2001). Explanations for the success (or otherwise) of CMC have often failed to account for when and why some forms of collaborative interaction are more successful than others (Wegerif & Mercer, 1996). Current research in dialogue analysis is beginning to reveal what makes for effective discussion. Based on this research, at least three kinds of role-taking seem to be needed. These include:

The DESIGN OF The Role-TAKING EXERCise

A "role play" activity was designed and used with a group of postgraduate students undertaking a module on ‘Learning and the New Technologies’. This course caters for both face-to-face and distance learners and is supported by a WebCTä ¹ Virtual Learning Environment (VLE). As part of the course all students take part in a weekly synchronous CMC chat debate, facilitated by the tutor and based on set reading (weekly online course-notes plus two set research papers on the topic for that week). This debate takes place at two different times each week to enable as many students as possible to attend. There are therefore two groups, the Monday group and the Tuesday group. Students choose which of the two to attend on a weekly basis but the Monday group regulars tend to be mainly native English speakers whilst the Tuesday group regulars are more likely to be non-native speakers of English. Both groups attract both face-to-face and distance learners. In addition to these online debates the tutor sets exercises, the instructions for which are also online, to accompany each weekly topic. These are designed to take approximately 90 minutes in total and are designed to be done with a partner either in group classroom work or flexibly out of regular classroom hours. Face-to-face students elect to take a two hour class with the tutor each week within which the exercise is incorporated. Distance learners complete the exercises in their own time with a partner. Both kinds of learner post their collaborative answers to the asynchronous online discussion board (though distance learners are more likely to do so since for them "class attendance" is judged on the basis of postings for distance learners).

The role-taking exercise was to take place in weeks seven and eight of the course after the students had become familiar with the form of the synchronous online chat debates. Students completed the reading for the regular debate in the normal way. Students read the course notes for the topic that week which was on the nature of CMC communication and read two research papers (Robertson, Good & Pain, 1998; Wegerif & Mercer, 1996). With their partner or small group, students debated the kinds of discussion skills required for collaborative learning based on their own experience. They were asked what ground rules for effective discussion were suggested by Wegerif & Mercer (1996) and what discussion skills identified by Johnson & Johnson (1994) translated into sentence openers by McManus and Aiken (1995) were adapted and reported by Robertson, Good & Pain (1998). Having generated this material as a basis for discussion, students were asked to list the kinds of roles they thought students and tutor should take in online debate to maximize its effectiveness and to group these according to whether they were community building, management or argumentation roles. Papers by Veerman, Andriessen & Kanselaar (2000) and by Berzsenyi (1999) were suggested as additional helpful references for this task. Students were to post their answers for this part of the exercise to the discussion board before taking part in the regular online chat debate that week. For this chat debate they were additionally given (at random) one of six individual and secret roles (to be their special responsibility) to take in the chat. These roles were:

After the chat they were to discuss their impressions of using the synchronous chat tool in the asynchronous online discussion forum but were not to disclose their role until after they had completed the exercise in week eight. For this exercise they were asked to review an anonymised version of the chat transcript posted online by the tutor after the debate and, using a table also provided online, attribute roles to participants for a portion of the transcript. Based on the result, they were to answer the following questions: Who do you think is the tutor and why? What is the balance of roles and what does this say about how the dialogue might have been improved do you think? Can you identify any students who were adopting a particular role most strongly? What do you think is the role given by the tutor for each participant?

Face-to-face students completed this part of the exercise in the weekly class for week eight whilst distance learners did this with their partner. Having compiled their answers students again posted their results to the asynchronous discussion forum. In synchronous debates following the role-playing exercise students were not asked to adopt specific roles in the discussion - they were free to choose their own level and type of participation - as had been the case before the role-taking exercise. Based on the students’ own perceptions, together with further analysis by the researcher, results are now presented concerning the effectiveness of the exercise in supporting the students in developing their independent discussion skills.

RESULTS

Community Building Roles and Balance of Participation

In this section the results of the role-taking exercise on community building are discussed through a discussion both of student perceptions and a comparison of role-taking before during and after the role-taking exercise.

Balance of Participation – Inclusivity of the Dialogue

The balance of participation is indicated by comparing the total number of words produced by each participant (end column in each of figures 1-6). A comparison of the proportion of words produced by each member of the group in a debate taking place before the role-taking exercise shows that in both debates in week four the tutor, Lana tends to dominate, producing 42% - 45% of the total words. Lana is the active tutor whose role it is to facilitate the debate. Throughout the course Cherry also acts as an assistant tutor – however for the purposes of the debate Cherry can be regarded as an observer since her principal role was to give assistance to students who asked privately for help on matters outside the context of the debate. However towards the end of the course, in week ten there are relatively fewer private help requests and Cherry joins in the debate a little. In week four Vince and Olive between them produce a third of the words in the Monday group. The remaining participants in the Monday group participate little. In this group Olive and Jake are the only distance learners, the rest are face-to-face learners. All but Shirley are native speakers of English.

Apart from the tutor and her assistant only Faith, Beth and Dean are native speakers of English in the week four Tuesday group. In this group Rudolph, Dean and Fred share a third of the dialogue with the remaining students participating little. In the Tuesday group, Lauren, Mae, Doris and Ruth are distance learners whilst the remainder are face-to-face learners. There is some movement between groups but the Tuesday group is generally larger and more diverse. However, in week ten some of the students studying abroad are returning to their home countries early and the Monday group is larger than usual whilst the Tuesday group is smaller.

The chat debate from the role-taking exercise (week seven) and another chat debate on a different topic three weeks after the role-taking exercise (week ten) were also analyzed. During the role-taking exercise participation was more inclusive and the tutor dominated less. The tutor’s contributions in week seven were down approximately 15% to between 27% and 31% of the total words. By week ten this had crept back up to 33% in the Monday group and 41% in Tuesday group. The space left by the tutor in week seven was filled by other students. Ruth, Doris and Shirley, in particular, took more active roles in the Tuesday group whilst Vince and Olive dominated less and shared more with Clark and Gerry in the Monday group. However, when recording their impressions of the discussion, Vince still felt that Vince and Olive dominated the Monday discussion in week seven whilst in the Tuesday group, Beth and Helen felt strongly that others ought to wait until everyone’s point had been addressed and not move on too quickly. Beth also thought that Lana still dominated the discussion. In week 10 it is difficult to draw firm conclusions since several regular members of the Tuesday group joined the Monday group, reversing the usual size of the groups. Moreover, Cherry takes a more active role in the debate in week ten’s Tuesday group. Overall though it would appear that the balance of participation improved as a result of adopting the roles approach and this improvement seemed to be partially sustained in week ten.

Figure 1: Balance of role-taking in the Monday group, week 4

Figure 2: Balance of role-taking in the Tuesday group, week 4

Adoption of Community Building Roles

Community building roles are roles four, five and six in the figures 1-6. It can be seen that overall the role-taking exercise brought about an increase in the number of students talking such roles. Shirley was given the secret community building role of giving positive feedback to others (role five) in week seven. In week four Shirley showed below average or average role-taking across all categories of role. During the role-taking exercise she improved her participation to average in number of words produced (and above average in number of turns) but did not explicitly improve on her giving of positive feedback. However, she did increase the number of simple "yes" agreements without further explanation or comment and these may have been an attempt to play the role that were not counted in the analysis. In week ten she raised her level of participation so much that she was above average on all categories of role-taking except negative feedback which had a zero instance. Gerry, also given the role of giving positive feedback, showed a raised number of these in week seven compared with week four and above the average in the group but Olive appeared to a more natural encourager in the Monday group performing above average in all community building roles in week seven and, unlike Gerry, maintaining this in week ten. Ruth, given the role of inviting others into the dialogue (role four) increased the number of times she took this role in week seven but slipped back a little in week ten. Olive commented in the discussions that she was pleased to see others had thought she had shown a high rate of giving positive feedback as she liked to think she did that generally but felt that she had actually done more content building and not the management of focus role she had been given. In contrast, Vince showed a strong increase in community building roles in week seven which was maintained in week ten despite his given role being that of management focus.

Figure 3: Balance of role-taking in Monday group, week 7

Figure 4: Balance of role-taking in Tuesday group, week 7

Managing the Dialogue and Responsibility for Focus

The management and focus role-taking is illustrated by column three in figures 1-6. From the discussion in the Monday group it was clear that more than any other role, students believed this role was the responsibility of the tutor. Olive commented that the tutor needed to "lead without seeming to lead" and Vince felt that the role of the tutor often included the pressure of time-keeping as well as facilitating. Gerry felt that someone had to be "in charge" to start the discussion. Two students not taking part in the debates themselves but who also contributed to the discussion forum commented that they thought Lana was the tutor because "s/he is continually trying to keep the others on task".

Figure 5: Balance of role-taking in Monday group, week 10

Figure 6: Balance of role-taking in Tuesday group, week 10

Beth felt that adopting the roles helped the dialogue to stay focused. In contrast, Faith thought that adopting the roles made it more difficult to have a meaningful discussion. Beth became quite frustrated with the tensions between focusing on the task issues and addressing the issues which came up in response to questions. She thought that the tendency to move on before addressing the questions of others had been detrimental to useful content building. She commented that if everyone’s ideas were to be considered then they couldn’t discuss so many issues in the time allotted. This may in part reflect the group struggling to find the right balance as they took on new responsibility in managing the discussion for themselves.

The role-taking exercise had a marked effect on students’ adoption of the focusing role. In week four the tutor produced 65% of the focus role-taking in the Monday group and 100% of the focus role-taking in the Tuesday group. In week seven the tutor shared responsibility for this role with both Olive and Vince. Olive produced 40% of such roles, Lana the tutor 31% and Vince 20%. Both Olive and Vince had been given the management of focus role as their secret role. In the Tuesday group, Beth was given the role and produced 26% of focus role-taking whilst the tutor produced 36% and Doris given an argumentation role, produced around 17%. This improved sharing of responsibility for the focus of the discussion was sustained in week ten’s Tuesday group where once again the tutor produced 44% of this role, but the tutor returned to producing 75% of the instances of focus role-taking in the Monday group. This perhaps indicates the need for more prolonged scaffolding of this role to encourage students to adopt this role independently.

Developing Critical Discussion – Exploratory Dialogue

The key argumentation skills are those involving the taking of roles one, two and seven. Role one is that of challenging others to provide evidence and pointing out alternatives or contraindicating evidence. Role two is an inquiry role which asks for explanations and clarifications. Role seven, building content, is the offering of information spontaneously or to answer an inquiry or challenge. In week four the tutor provides between 27% and 42% of challenges. This reduces to between 18% and 21% in week seven and between 21% and 25% in week ten. Accompanying this change there is a shift towards students becoming more critical in debate. In the Monday group Olive, Gerry, Clark, Vince and Jake all show higher levels of challenging in week seven and continue to show higher overall levels in week ten (except for Clark who was absent). In the Tuesday group there is a similar story with a generally higher level of exploratory talk in week seven. In particular, Shirley improves markedly in challenging, inquiring and building content in week seven and this is further improved upon in week ten. Doris who was given role one (challenge) increases her challenging from 0 to 12% in week seven and to 20% in week ten. Lauren and Helen, both given role two (inquire) in week seven raised their inquiries from 0 in week four to 3% and 6% respectively in week seven but were both absent in week ten. Beth, Doris, Shirley and Faith raised their number of inquiries even more despite not being given this role as their secret role. Doris raised her level of question asking from 4% to 23% (from 1 to 16 questions) but dropped back to 10% in week ten, whilst Beth raised her level of question asking from 0 to 19% but was absent in week ten. No one person was given the role of content building as their secret role but all understood that it was important to contribute in this way. However the general level of content building fell as the adoption of other roles increased. (The combined totals for content building for Monday and Tuesday groups are week four 220, week seven 196 and week ten 164). This is not as bad as it sounds since as students address the points of others by asking for more explanations they make fewer independent points resulting in more meaningful debate. Parallel threads are developed but they are threads not a collection of independent and enigmatic points.

Summary And Conclusions

Our hypothesis was that making students aware of the need for all three kinds of role, community building, management of task focus and argumentation would lead to wider adoption of these types of role by individuals and, over time, independent adoption of these roles in future debate. This preliminary evaluation of the role-taking approach gives cautious support to this hypothesis. The role taking exercise seemed to be effective in prompting at least some students to change their behaviour in ways that appeared to lead to more inclusive, mutually supportive and yet more critical debate. Impressions were that the debate during the role-taking exercise was more focused, but students did not continue to take responsibility for focusing the debate after the role-taking exercise finished and some of the benefits to the meaningfulness of the dialogue may then have been lost. This indicates that management of the dialogue had not been learnt as an independent discussion skill. It is likely that had students been instructed to take roles over more weeks and if roles had rotated around students so that all students practiced all roles, then the effects would have been more marked and sustained. Some (mainly infrequently attending) students did not improve their role-taking whilst others did so only for the duration of the role-taking exercise, "slipping back" to near previous levels later. The role-taking exercise did, however, demonstrate that, over time, synchronous CMC debate can become more effective both in terms of building a community and strengthening the criticality of debate. Students exposed to such debate over time improve their ability to participate meaningfully in debate. Moreover, as Olive said it can be "all so inviting! And fun."

ACKNOWLEDGMENTS

The teaching described in this paper took place on the Master of Education programme at the University of Leeds, UK and was devised and undertaken by Rachel Pilkington and Aisha Walker. Additional initial pilot work was undertaken by Peter Kuminek.

REFERENCES

Berzsenyi, C. A. (1999). Teaching interlocutor relationships in electronic classrooms. Computers and Composition, 16, 229-246.

Burnett, R. E. (1993). Decision-Making During the Collaborative Planning of Coauthors. In A. Penrose & B. Sitko (Eds.), Hearing Ourselves Think: Cognitive Research in the College Writing Classroom. Oxford University Press, Oxford (pp. 125-146).

Crook, C. (1994). Computers and the Collaborative Experience of Learning. Routledge, London.

Ekeblad, E. (1998). Contact, community and multilogue - electronic communication in the practice of scholarship. Paper presented at the Fourth Congress of the International Society for Cultural Research and Activity Theory, Denmark, June 7-11. URL: http://w1.314.telia.com/~u31414381/writings/iscrat98/cocomu.html [last accessed 15/02/02].

Herring, S. (1999). Interactional coherence in CMC. Journal of Computer-Mediated Communication, 4, 4. URL: http://www.ascusc.org/jcmc/vol4/issue/4/herring.html [last accessed 11/12/01].

Johnson, D. W., & Johnson, R. T. (1994). Learning Together and Alone. Prentice Hall, Englewood Cliffs, New Jersey.

Kuminek, P. A. and Pilkington, R. M. (2001). Helping the tutor facilitate debate to improve literacy using CMC. Proceedings of IEEE International Conference on Advanced Learning Technologies, ICALT 2001 (Madison, USA. 6-8th August), 261-262.

Katz, S., O'Donnell, G. and Kay, H. (2000). An approach to analyzing the role and structure of reflective dialogue. International Journal of Artificial Intelligence in Education. Part I of the Special Issue on Analysing Educational Dialogue Interaction, 11, 320-343.

Kneser, C., Pilkington, R. and Treasure-Jones, T. (2001). The tutor's role: an investigation of the power of exchange structure analysis to identify different roles in CMC seminars. International Journal of Artificial Intelligence in Education. Part II of the Special Issue on Analysing Educational Dialogue Interaction, 12, 63-84.

Kolb, D. A. (1984). Experiential Learning: Experience as the Source of Learning and Development. Englewood Cliffs, NJ: Prentice-Hall.

Pilkington, R. M., Bennett, C. L. and Vaughan, S. (2000). An Evaluation of Computer Mediated Communication to Support Group Discussion in Continuing Education. Educational Technology and Society: Special Issue on on-line Collaborative Learning Environments, 3, 3, 349-359.

Pilkington , R. and Walker, A. (in press). Using CMC to Develop Argumentation Skills in Children with a Literacy Deficit. In J. Adriessen, M. Baker, & D. Suthers (Eds.), Arguing to Learn: Confronting Cognitions in Computer-Supported Collaborative Learning Environments. Kluwer Academic, Amsterdam.

Robertson, J., Good, J. and Pain, H. (1998). BetterBlether: The design and evaluation of a discussion tool for education. International Journal of Artificial Intelligence in Education, 9, 219-236.

Sullivan, N. and Pratt E. (1996). A comparative study of two ESL writing environments: A computer-assisted classroom and a traditional oral classroom. System, 29, 4, 491-501.

Walker, S. A. and Pilkington R. M. (2001). Facilitating computer-mediated discussion classes: a study of teacher intervention strategies. Proceedings of IEEE International Conference on Advanced Learning Technologies, ICALT 2001 (Madison, USA. 6-8th August), 253-256.

Warschauer, M. (1996). Comparing Face-to-Face and Electronic Discussion in the Second Language Classroom. CALICO Journal, 13, 7-26.

Wegerif, R. and Mercer, N. (1996). Computers and reasoning through talk in the classroom. Language and Education, 10, 1, 47-64.

Veerman, A. L., Andriessen, J. E. B. and Kanselaar, G. (2000). Learning through synchronous electronic discussion. Computers and Education, 34, 269-290.

WebCT is a registered trademark for more information see URL: http://www.webct.com/ 15/01/02