ABSTRACT

This learning and teaching development project, funded by the University of Sheffield, is being carried out to enhance the learning experience of participants on a professional development Masters programme in Health Informatics. The project involves the introduction of problem-based, collaborative learning to the programme, through the vehicle of an extended, multimedia case study. The paper discusses the pedagogic rationale for this initiative, describes its main features and offers a reflection on a number of educational and project management issues that have arisen from the first phase of development, piloting and evaluation. The authors conclude by noting that although the project has entailed significant development investment, its impact already appears far-reaching in terms of both educational and staff development benefits.

Keywords

Problem-based learning; collaborative learning; multimedia; WebCT; professional development

INTRODUCTION

This paper reports on a learning and teaching development project, funded by the University of Sheffield, which is being carried out to enhance the learning experience of participants on a professional development Masters programme in Health Informatics. The project involves the introduction of problem-based, collaborative learning to the programme, through the vehicle of an extended, multimedia case study. In what follows we discuss the pedagogic rationale for this initiative, describe its main features and reflect on a number of educational and project management issues that have arisen from the first phase of development, piloting and evaluation.

The programme

The MSc in Health Informatics was launched in October 2000 by the Department of Information Studies, the School of Nursing and Midwifery and the School of Health and Related Research at the University of Sheffield. It is a specialist, professional development programme that aims to enable healthcare practitioners and managers to develop their knowledge and skills in health information management and technology through flexible, part-time study alongside work. The programme is delivered principally via distance learning, using WebCT (a virtual learning environment), plus some paper-based materials and a number of day schools. It is multidisciplinary in nature, recruiting participants from a wide range of healthcare professions, and aims to maximise opportunities for critical reflection on experiences of professional practice, integration of existing knowledge and experience with new perspectives, and practical workplace applications of learning. Although it was initially targeted at professionals working within the UK National Health Service, it is attracting increasing interest from practitioners and organisations in the private healthcare sector.

The programme is based on six taught modules followed by a research dissertation. The first four core modules, taken sequentially over four semesters, are as follows:

From its inception, day schools were an important (and highly appreciated) part of the distance learning experience on the programme. Scenario based activities, designed to draw on the collective expertise of the group for collaborative problem solving, immediately proved an effective means of stimulating productive, multidisciplinary exchange and of focusing on the interconnections between themes addressed in different modules. Attendance at day schools also helped to provide some degree of group cohesion within the cohort. However, the contributions that individuals made to discussion and group work at day schools were not carried over into the WebCT environment. WebCT supports both synchronous and asychnchronous text-based discussion, through its chat and bulletin board facilities; a number of discussion questions were broadcast weekly to the participant group with the intention of stimulating debate on issues relevant to the twelve units of each module. However, there was notable variation in levels of response. Bulletin board exchanges tended to be dominated by a small minority, while other participants mainly observed, and little use was made of the synchronous chat facility. Moreover, levels of participation in online discussion declined significantly towards the deadlines for individual coursework (Bacigalupo et al., 2002). Module coordinators and tutors made efforts to draw in the ‘watchers’, for example by summarising discussions, inviting further contributions from individuals, posing provocative questions, and reducing on-line group size with the aim of decreasing pressures that might be experienced in a larger forum and increasing possibilities for peer support. However, these strategies had only limited impact. The programme team concluded that the question of how to stimulate willing engagement of all participants in online discussion, in the way that happened face-to-face at day schools, remained unresolved (Bacigalupo et al., 2002).

The project

The project discussed in this paper arose out of this early experience. It is introducing a new emphasis on collaborative, problem-based learning within the programme by using an extended, multimedia case study scenario that is integrated into, and explicitly links, the first four core modules. The case study and its associated learning tasks are designed to enable participants to:

test theoretical frameworks within a progressively more complex practice scenario, drawing on multidisciplinary perspectives within the learning group;

address inter-connected themes on the programme in a more sustained and developmental way;

reflect critically on their own experience of professional practice in the light of the experiences and perspectives of other participants, as well as relevant theory and research;

develop awareness and skills of relevance to effective practitioner learning in the networked environment.

The design and implementation of the case study also entails significant learning outcomes for the staff involved. By taking an action research approach to the project we aim to improve our own educational understandings and practice in the context of networked learning, and also to offer an educational and technical exemplar that will be of interest to others who are developing on-line professional development programmes.

The project is being taken forward by the multidisciplinary group of academic staff who teach on the programme, in collaboration with colleagues from the University’s Learning Media Unit and from the Educational Informatics Research Group in the Department of Information Studies. A development officer with knowledge of the Health Informatics programme and WebCT as an external tutor, as well as a background in health information management research, was employed on a full-time basis for a period of eight months at the start of the project (April, 2001). This was a key ‘link’ position within the team, involving collaborative work with academic staff to research and devise the case study scenario and to develop learning tasks, and with colleagues in the Learning Media Unit to develop aspects of the technical platform.

EDUCATIONAL AND TECHNICAL DESIGN

In this section, we discuss the rationale for the case study approach and highlight the main educational and technical design features of the initiative.

Pedagogic assumptions

In reviewing the pedagogic assumptions that underpin this initiative, there is some danger of presenting too neat a picture of our collective understandings about networked learning and teaching. As educational practitioners with very diverse backgrounds in terms of academic discipline and the use of technology in teaching, our experiences and understandings necessarily differ. Engaging with educational issues as a project team has been a developmental process rather than a ‘once and for all’ step when the idea of creating a multimedia case study was mooted by the programme team. In working on the project we have needed to explore and negotiate our views of educational purposes and practice collaboratively, and the project has triggered a process of reflection that is still on-going. Nevertheless, the aim with which we set out - to enhance the dialogic and social dimensions of participants’ distance learning experience on the Health Informatics programme - reflects a shared, broadly constructivist perspective on effective support for practitioner learning (e.g., Brophy, 2001; Duffy and Jonasson, 1992; Goodyear, et al., 2000; Grabinger and Dunlap, 1995); it also relates to approaches to networked learning that have been developed successfully in other contexts by members of the project team (e.g., see Levy, 1999). This means that we take as our starting-point the view that learning is a process of active knowledge construction that arises out of personal experience and inter-subjective negotiation of meaning within ‘situated’ social contexts. We view individual learners’ prior experiences and understandings as an important resource for both personal and others’ learning, and we assume that like other professionals, health practitioners and managers are capable of, and responsible for, developing the knowledge that will guide their professional judgements and actions. In the light of these assumptions, we aim to offer plenty of opportunity on this Masters programme for learners to participate in directing their learning experiences, to reflect critically on their prior understandings, experiences and practice, and to engage in dialogue, debate, feedback and experimentation that is relevant to their professional circumstannces and interests. We recognise the importance of ensuring that learning tasks are meaningful in relation to the concerns and complexity of real-life practice contexts, and that distance learners can gain educational benefits from social interaction and support from peers.

Research-based case scenario

Case studies are recognised as a powerful method for exploring the application of theoretical principles in the practical context of workplace environments, and are particularly suited to team-based problem solving that requires multiple perspectives to be negotiated and synthesised (e.g., Grabinger, Dunlap and Duffield, 1997). In developing a case study for this project, it was important that it should be sufficiently rich to address a range of issues covered by the programme, and sufficiently authentic, timely and familiar to participants to strike a resonance with their own workplace experiences. We therefore decided to develop an extended scenario grounded in research into IT-related change management in healthcare, as required for Local Implementation Strategies (LIS) in support of the NHS Information Management and Technology Strategy, Information for Health (Burns, 1998). The case scenario was researched principally through interviews with staff involved in current LIS projects within Sheffield Health. Interviews focused on issues related to themes covered in the first four modules of the MSc programme and on key problems, challenges, issues and decision scenarios associated with the LIS projects as identified by the practitioners involved. The research also involved selection of primary and secondary sources for case documents, and consultation with external tutors and participants on the MSc programme.

The case narrative that has been developed in this way does not directly reproduce real-life situations but draws on them to chart the progress of a fictional General Practice, ‘Treetops Medical Centre’, through the planning and implementation of a project to ‘go paperless’; that is, to move from paper-based to computer-based medical records, and to develop new approaches to evidence based healthcare using electronic information resources. The Treetops change management process entails, among other things: identifying information needs of healthcare professionals, patients and administrators within the Practice; selecting a new computer system; agreeing protocols for dealing with different types of information; developing evidence-based practice skills; making the transition from paper-based to electronic health records; training staff; and, ‘going live’ with the new computer system. The case narrative follows the Treetops project, and the parts played within it by a number of fictional health practitioners and managers, through a number of stages that reflect the focus of the first four modules on the Health Informatics programme. The case is topical, in that all course participants will be experiencing the impact of change of this kind in their own workplaces, but it is devised in such a way as to provide a generic framework that will facilitate cost-effective future updating, rather than complete revisions.

The case is to be presented in multimedia format as a sequence of problem scenarios and decision points, using text, graphics, sound, animation and video. The aim is to convey a nuanced presentation of human and interpersonal factors as well as technical factors, for example by using video sequences to illustrate key points of decision, tension and conflict in the Treetops project. WebCT provides the technical platform for accessing the multimedia case, and for carrying out the group activities that are associated with it; multimedia elements of the case scenario, as well as some text-based material, will be made accessible via this ‘virtual learning environment’ (VLE) on CD ROM.

Problem-solving and reflective learning tasks

The case study provides a vehicle for active, collaborative, networked learning, through a series of problem-solving and reflective tasks that have been designed to link with the evolving storyline of the Treetops case. These tasks, which were developed in tandem with the case study scenario, replace the non-assessed discussion questions (and quiz-based feedback strategies) that were used initially on the programme. The tasks require participants to engage in multidisciplinary, critical analysis of a complex and evolving organisational project, and aim to encourage information exchange, sharing of experiences, discussion of problems, and group decision-making as regards solutions. Working in self-directing groups of four, with the support of module tutors, participants are asked to negotiate their perspectives through on-line discussion.

Assessment

The case study tasks are embedded into each module through assessed work. Participants build up individual portfolios comprising a series of assignments in which they document and critically review the case study decisions made by their group, and also reflect critically on the collaborative learning process. This strategy aims to provide an opportunity for learners to reflect on each stage of the case study as it evolves, to receive on-going feedback on their perspectives, and to provide support for the development of awareness and skills in networked learning.

Project schedule

The project is being taken forward in three main development phases encompassing the following main activities:

Phase 1: Educational and preliminary technical design (April 2001 – October 2001).

Researching the case study content; devising the case study scenario; producing text-based materials.

Designing case-related, problem-based learning tasks within all four modules.

Deciding on pedagogic strategies (e.g., arrangements for organising and facilitating on-line group-work, assessment procedures) and staff and tutor training (use of WebCT to support group-work).

Designing WebCT environment to support access to case study materials and group-work.

Phase 2: Prototype implementation and multimedia production (October 2001 – September 2002).

Pilot implementation, monitoring and formative evaluation of educational approach in all modules (case scenario, problem-based learning tasks, text-based materials, facilitation and assessment strategies).

Pilot implementation and evaluation of technical design (WebCT platform).

Production of multimedia elements (graphics, video, sound, animation) and further text-based materials.

Phase 3: Implementation and evaluation of full multimedia case study (October 2002 – December 2003).

Implementation and evaluation of revised pedagogic model and full multimedia case materials in all modules.

· Dissemination activities, including workshops with input from all those involved in the design, delivery and use of the case and production of Web-based guide to the design, development, implementation and evaluation process.

It is worth noting that this represents a revised version of our original schedule. We had planned to design and produce the full multimedia case study in a period of 8 months, and to implement and evaluate it in the academic year 2001 - 2002. For reasons highlighted below this proved unrealistic. At the time of writing, the case scenario and problem-based learning tasks are being piloted in a ‘skeleton’ text-based format within the programme’s WebCT environment (Phase 2). This provides an opportunity to evaluate the pedagogic framework before progressing to the second stage of the project when multimedia elements will be added to enhance the narrative richness of the case. The more extended piloting process is proving to have a number of benefits in terms of pacing the development of new practice in teaching and assessment.

REFLECTIONS ON DEVELOPMENT AND IMPACT

Evaluation of the project is being undertaken by means of an action research approach that involves a number of cycles of planning, implementing, monitoring, reflection, stakeholder feedback and further refinement of the educational model and materials. Overall, the evaluation strategy focuses on issues related to:

The project is still at a relatively early stage and therefore what follows is based on a preliminary cycle of monitoring and reflection on the development process and on implementation of the case study in the first module of the programme (2001-2). Some key themes that have already emerged include the following:

Online, multidisciplinary interaction

Our initial experience is that the case study has proved far more effective than previous strategies in stimulating participation in online discussion, and has been successful in carrying the benefits of the day schools over into the more isolated periods of distance learning. Almost all participants have contributed significantly to the case study work in the first module, and we have observed sustained group interaction throughout this period, using both asynchronous and synchronous communication facilities. (The case study seems to have provided sufficient stimulation for groups to complement asynchronous discussion with ‘real-time’ online meetings, which participants had previously perceived as problematic from a logistical point of view). It has been very encouraging to see the considerable effort time and energy being invested in the case study tasks. Moreover, the portfolios that are being developed show a clear synthesis of material from varied perspectives brought to bear by individuals on the case study problems within online interactions. We believe that the fact that the case study portfolios are assessed, whereas the previous discussion questions were not, is a contributory motivational factor here. A further factor in the more successful use of the online medium may be the introduction to WebCT received by this cohort, based on a networked learning skills induction model developed by colleagues (Baptisa Nunes, McPherson and Rico, 2001).

Early participant feedback suggests that the emphasis on collaborative group-work online is perceived as positive in a number of ways. In particular, learning in this way is experienced as engaging and motivational – participants see the commitment to a group task as a good motivator for ensuring that time is set aside for part-time, distance study. At the same time, whilst worthwhile, online group work is not experienced as entirely painless. Organising collaboration within the module timescales requires a good deal of effort in terms of managing asynchronous and synchronous communications, adding to the workload associated with case study tasks. This issue proved particularly problematic for members of the first cohort of the programme, who were introduced to the case study tasks in the fourth module after a year of study on the programme. On reflection, we recognise that to introduce a new emphasis on collaborative, problem-based learning to a group already accustomed to a different approach may require higher levels of support and encouragement.

Flexibility

One of the key purposes of distance learning delivery is to provide flexibility for participants to study and work concurrently. The introduction of collaborative online work to this programme, however, has created a new tension in the logistics of delivery. There is now a need to maintain flexibility for individual learners in tandem with an appropriate structure for collaboration within relatively constrained timescales. Our awareness of this tension grew during the initial implementation of the case study in the first module. The dependencies built into the case study tasks, between individuals within a group, and between groups, meant that we have had to be more liberal in allowing for contingencies and deadline extensions. For example, the first task required individuals to produce individual reflections to be discussed subsequently by their group as a whole. Each group then considered the deliberations of other groups. Educationally, we believe such dependencies are desirable. Logistically, the needs of one participant for an extended deadline can have significant knock-on effects on the work of the whole group and even of the cohort. We have no solution to this problem at present, apart from the strategy of generous timescale scheduling for group tasks, and setting group deadlines comfortably in advance of those for individual coursework.

Case content

We are concerned to ensure that the content of the case scenario is sufficiently realistic and rich to be meaningful to learners who already have a sophisticated awareness of the NHS workplace from a variety of perspectives. Our aim has been to produce material that enables exploration of the way in which theoretical principles relate to ‘real world’ healthcare situations, and we have drawn participants’ attention to the authenticity of the case by explaining its foundation in research into real-life experiences. Early observation and feedback suggests that the case study has had a practical impact in assisting learners to make links between theory and practice, and that their perception of the case is that it is meaningful - for some, even when delivered in skeleton, text-based format, the Treetops Practice already feels ‘real’. In the light of this, it will be interesting to examine the further impact of the introduction of multimedia elements in the next phase of development.

Programme assessment

In the process of developing the multimedia case study, it has become more central to the programme’s structure than was originally envisaged. This is reflected in particular in the significant adjustment that has been made to the programme’s previous assessment model. Our original intention had been to defer formal (summative) feedback on portfolio tasks until submission of the full case study portfolio at the end of the fourth module, and to maintain individual coursework (essays) as the principal form of assessment for each module. However, early feedback from participants has indicated that they desire rapid feedback on performance of the case study tasks. We have therefore adjusted our original plans and decided to incorporate case study assignments into the formal assessment for each module. The fact that the case study has become firmly embedded into the programme in this way is, we believe, a positive outcome of the project’s development process. Nevertheless, restructuring module assessment in this way increases the number of pieces of assessed coursework and the administration associated with assessment, thereby also increasing the workload of members of the programme team. The implications of these changes will be evaluated later on in the project.

Tutoring and programme management

The case study has also had an impact on tutoring roles and workload in the online environment, in that there is a greater need for active facilitation of online discussion and group work. Participants perceive the tutors’ role in this respect as critical to the success of their own online collaborative work. The new emphasis on online tutoring alters the original programme model significantly and has training implications. We are seeking to establish through the evaluation process what, from both tutors’ and participants’ perspectives, makes for effective facilitation in this context.

A further result of the case study initiative has been increased input to the programme curriculum by external tutors. Development of the case study has benefited from their experience and expertise, and we believe that this is improving integration of academic and professional expertise in support for learning.

Project management

The main project management issue so far has been the amount of time involved in co-ordinating, planning and taking forward collaborative project-work of this type and scale. The development officer’s role in this process has been essential; however, the project has also involved a significant level of input from academic and support staff in a number of departments, working for the first time together on a fairly complex educational design and development task (more complex, perhaps, than was originally recognised). It is difficult to know what the solution to the time problem is, particularly for academic staff for whom the project represents significant additional workload without release from other activities. In particular, much more time was needed for case study scenario development and for collaborative educational design than originally anticipated. The need to develop an integrated narrative has challenged us to think holistically about the programme, and a good deal of time was spent during the project’s early months in ‘thinking the project through’ - involving an iterative process of discussion about educational purposes and methods before reaching shared understandings and decisions, and designing closely linked learning tasks collaboratively in connected, but different, subject areas. Thus, although the over-arching framework for the project was establis hed prior to its start, the full educational model has been developed through a lengthy period of interaction within the team. Whilst this is a rewarding and essential part of the project – in fact, the most important part – it is nevertheless a time-consuming process with significant resource implications.

CONCLUSION

In this paper, we have outlined an innovative initiative in online, collaborative, problem-based learning for professional development. We have drawn attention to the main features of the initiative and to the fact that whilst it has entailed significant development investment, its impact already appears far-reaching in terms of both educational and staff development benefits.

REFERENCES

Bacigalupo, R., Bath, P., Booth, A., Eaglestone, B. and Procter, P. (2002) Studying health informatics from a distance: issues, problems and experiences, Health Informatics Journal, 7.3/7.4 (in press).

Baptista Nunes, J.M., McPherson, M. and Rico, M. (2001) ‘Constructivist instructional design and development of a networked learning skills (NICLS) module for continuing professional education distance learning’, In: Proceedings of the ED-MEDIA 2001-World Conference on Educational Multimedia, Hypermedia and Telecommunications, Tampere, Finland, June 25-30, 2001.

Brophy, P. (2001) Networked learning. Journal of Documentation, 57, 130-156.

Burns, F. (1998) Information for health: an information strategy for the modern NHS 1998-2005. A national strategy for local implementation. Leeds: NHS Executive.

Available at http://www.nhsia.nhs.uk/def/pages/info4health/contents.asp

Duffy, T.M. and Jonasson, D.H. (1992) Constructivism and the technology of instruction: a conversation. Hillside, NJ: Lawrence Erlbaum.

Goodyear, P. (ed). (2000) Effective networked learning in higher education: notes and guidelines. University of Lancaster, Centre for Studies in Advanced Learning Technology. Available at http://csalt.lancs.ac.uk/jisc/

Grabinger, S. R. and Dunlap, J. C. (1995) Rich environments for active learning: a definition. ALT-J, Journal of the Association for Learning Technology, 3, 2, 5-34.

Grabinger, S.R., Dunlap, J.C and Duffield, J.A. (1997) Rich environments for active learning: problem-based learning. ALT-J, Journal of the Association for Learning Technology, 5, 2, 5-17.

Levy, P. (1999) An example of Internet-based continuing professional development: perspectives on course design and participation. Education for Information, 17, 45-58.

Levy, P. (2002) ‘Researching networked learning and teaching: a case study in practitioner knowledge construction’, in this volume.