Chris Roberts, Mary Lawson, David Newble, Ashley Self
Department of Medical Education, University of Sheffield
Abstract
The University of Sheffield Medical School is undergoing a major revision of its medical curriculum in the light of reviews by both the professional regulatory body, the General Medical Council (GMC) and by the QAA. A networked learning environment has been developed in conjunction with other medical schools, which aims to facilitate, manage and support the delivery of the curriculum to medical students at their respective universities. The Sheffield NLE is considered central to the medical school's plan to implement and evaluate the new curriculum.
Keywords
Medical education, managed learning environment, problem-based learning, curriculum database.
Background
In a recent survey 17 out of 21 (of a possible 23) UK medical schools have instituted or are about to institute a Virtual Learning Environment (VLE) to support teaching and learning in undergraduate medicine. Of the 21 schools, 15 are developing systems in-house or with other institutions, 5 are using commercial systems and one had yet to decide (Cooke 2001). We will describe how Sheffield has set about building a VLE in conjunction with other medical schools and begun to embed it within its own teaching and learning strategies to create its own unique Managed Learning Environment.
At the University of Sheffield there are 1,012 full time undergraduates in medicine with 220 staff contributing to the teaching. Over 750 clinicians (consultants and general practitioners) provide clinical teaching in hospitals and the community. The curriculum at Sheffield had in the past been fairly traditional with a lecture based pre-clinical component based on medical sciences followed by a clinical component largely based in hospital settings.
A major impetus for curriculum change had arisen from a visit by the medical profession's regulatory body, the General Medical Council (GMC, 1999) and the QAA (1998) Subject Review. As a consequence a two staged strategy emerged for the curriculum review process: a refinement of the current curriculum and a plan for a more radical revision with a target start date of 2003. The major revision of the curriculum is under way and is based around four common themes:
· Increased community-based learning (i.e. students going out to see patients in the community)
· Development of an outcome-based curriculum framework: the aim is to produce graduates who are able to fulfil their role as junior doctors in the National Health Service (NHS) and who also possess the generic skills expected of students attending a research-led university.
· Development of a case / problem-based learning educational pedagogy (PBL) based on the seven jump approach (Schmidt, 1983)
· Increasing student choice so that students can select a minimum of 15% of the curriculum in the form of special study modules.
Although co-ordinated by a curriculum review team drawn form a wide range of scientists and clinicians, the design of the new curriculum has emerged collaboratively by involving all the major stakeholders at open meetings of teaching staff, the community and with students. There has been a commitment to wide consultation with the intent of developing a greater sense of ownership of the curriculum than has been achieved in the past.
The development of a networked learning environment (NLE) is one mechanism the school is developing to address the challenges of curricula delivered increasingly at a distance (e.g. to students whose time is spent in General Practice Surgeries or in out-patient clinics as opposed to hospital wards). The Sheffield Networked Learning Environment (NLE) has been developed with a consortium of Universities from the UK. The University of Newcastle has led the group who had the common goal of wishing to facilitate learning for medical students using a networked learning environment. The School of Medicine and Biomedical Sciences, University of Sheffield joined the consortium as a non funded partner within the Teaching and Learning Technology Programme Phase 3 (TLTP3) Project Number 86 Facilitated Network Learning in Medicine and Health Sciences. In an environment characterised by declining resources, there are huge educational implications largely focussed around delivery, quality and equity of access to teaching facilities. Improved communication between faculty, staff and students has already been achieved through effective use of information technology.
Project aims
The overall aim of the Sheffield NLE project was to develop and disseminate integrated communication and information technology (C&IT) based approaches to facilitate, manage and support the delivery of the curriculum to medical students at the University of Sheffield.
The Sheffield Model for a networked learning environment
The objectives for the NLE project have emerged collaboratively between the students, staff, as well as being informed from the emerging literature from other higher educational settings. The ethos of the team creating the technology of the NLE is encompassed in a 70:30 rule. For developing working software 30% of the effort is to develop the software itself, whereas the other 70% is about describing and refining the information systems as they already exist within the institution. This contrasts starkly with projects where software is installed and institutions have to change the way they work to use the software.
We will briefly describe the issues that arise from the project objectives and indicate how we intend to meet the challenges set by them. The NLE project has much akin with an action research approach for change and development (Zuber-Skerritt, 1991) with cycles of planning, acting, observing and reflecting within a community of like minded professionals committed to improvements in the curriculum.
Creating a positive impact of the NLE on institutional teaching and learning strategy.
At the core of any Managed Learning Environment (Everett, 2001) is the integration of the virtual learning environment with the teaching and learning strategy of the educational institution. The cycle of planning and implementing a new medical curriculum is slow and may take 10 or more years to complete. However curriculum planners need to be aware of the possibilities that technology can offer in setting the framework for any new curriculum. Ideally the process of creating an MLE is an iterative process between the curriculum planners and the software developers so that the cycles of planning and implementing change inform both the curriculum and the tools that support it within the NLE.
Refining and developing management and administration processes
Curriculum database
The heart of the NLE is based around a curriculum database. This contains all of the course handbooks, which are produced by module co-ordinators, and can be uploaded by clerical assistants, following basic training. (Interestingly prior to the NLE the curriculum had never existed as a collection of handbooks in one place.) The course handbooks contain the module aims, objectives course outlines, methods of assessment, and recommended reading, There is the facility for staff to upload supplementary learning resources directly from their own desktop for example presentations, word processed documents or addressees of web based resources. This process requires basic IT skills (akin to saving a document to a hard drive) and no requirement for programming skills such as HTML. A search engine allows rapid access to the curriculum database. A long-term aim is to be able to match outcome objectives designed at the strategic level to learning objectives as delivered by the teachers to students.
Communications system
Because the NLE plugs in to the student database within the central university management information system, the NLE 'knows' which student is logged in. It is possible to create a portal system so that the student is logging on to a personalised "my curriculum" so they know what is coming up in terms of content, why in terms of learning objectives, when in terms of the timetable and how in terms of the learning methods used. There are forums for year groups to discuss both academic and social issues, and notice boards for staff to communicate information to students.
Student placement software
A large administration team currently ensures students are placed in suitable clinical placements across the region of North Trent using paper-based systems and the occasional Excel spreadsheet. A pilot study is underway which will allow students to select on-line both their placements and ultimately their special study options. The sytem is already operational within the Newcastle NLE and allows considerable savings in administration time whilst increasing student choice and equity.
Supporting quality assurance administration
A monitoring system is being established to evaluate the implementation of the curriculum and to support a process of continuous improvement. Currently a number of home grown paper based evaluations are collected to monitor teaching effectiveness. Pilot studies at Sheffield suggest that students are happy to enter teaching evaluation data directly into web based forms. We are currently looking to a system which allows both a reliable (ideally with Cronbach's alpha above 0.8) and valid evaluation of teaching quality but which also allows the collection of qualitative data. The "Students' Evaluation of Educational Quality (SEEQ)" is one such system that is showing some early promise (Coffey and Gibbs, 2001). However there are clearly issues around the ethics of holding and using student data even for evaluation purposes. These concerns have recently become more acute with new NHS research governance guidelines suggesting that students should be regarded in the same light as human subjects when it comes to giving informed consent for research, including evaluation research.
Teaching and student support
The NLE will extend the range of curriculum support services available to undergraduate students, teaching staff and support staff based both on campus, in central teaching hospitals, district general hospitals and on community placements. Interestingly within Sheffield the students have set up their own website by organising themselves into teams responsible for different aspects of content of the website. The students have done much good work in introducing teaching staff to may of the issues of networked learning.
Staff development within an NLE
This will include issues such as management of institutional change, strategic and operational planning, changing roles and responsibilities, re-training and awareness raising for all staff. As the new curriculum review is beginning to reach the stage of detailed planning at the operational level, there is a clear need to dedicate additional resources to staff development
Delivery of learning materials
Enhancing teaching innovations
It is intended that the instructional approach in the new curriculum will consist of a spine of problem, case and patient-based integrated learning activities. Structured problem based learning, which is delivered via the NLE, may be an effective mechanism to address the problems now faced by educators who would need high staff to student ratios to deliver traditional face-to-face teaching, but without the necessary resources. Students are encouraged to work in tutorless groups within the lecture theatre to work out the learning objectives of carefully designed cases, for example a woman with painful legs. The students use the NLE to point them to number of prepared learning resources including formative multiple choice questions, multimedia learning objects such as x-rays, microscope slides etc. Students then bring back to the second session a more synthesised understanding of the patient problem based on their research into the learning objectives.
Students will be expected to become progressively more self-directed, aided by an increasing reliance on the NLE. However the challenge remains in shifting a culture both amongst students and staff towards problem-based learning from a more didactic traditional approach. A culture shift evidenced in an evaluation from first year students studying the problem based learning case; "Edna - a pain in the legs." Whilst one student suggests that
The knowledge gained was not in one discipline i.e. anatomy, biochemistry etc but was' integrated into one exercise. This makes learning more relevant.
another suggests
I didn't feel that I learned anything really as too much of the time was taken up by deciding what to focus on and how to present it. Hence the information that was actually absorbed was minimal. It would have been better in a lecture.
Provision for "virtual cohorts"
The move to more community based teaching where students may access the NLE from widely dispersed sites provides a natural test bed for self-directed collaborative learning. There is already some experience of the use of electronic whiteboards to support the brainstorming activity within problem based learning (Jill Gordon -personal communication) at the University of Sydney. In Sheffield where groups work together face to face within reasonable distance from the central campus there may be no need for a virtual classroom approach. This is in contrast to Australia where vast distances may separate students on placements.
Assessment
The electronic student record
There is a considerable interest within the medical establishment to move towards portfolio-based assessment. Their predominant use in medical education has been for formative assessment, often being a vehicle for encouraging a component of reflection. More recently the use of portfolios has been advocated for summative purpose for example in the final examination at The University of Dundee medical school (Davis et al, 2001) and for the revalidation of doctors (GMC, 2002).
Preparations are in hand to use an electronic portfolio system to assess professional development which occurs as a vertical theme through the new curriculum Students will be required to produce an electronic portfolio to evidence their own development of professional values and behaviours.
A multiple choice question engine has been developed for formative assessment of knowledge. The international standards for MCQ testing in medical education support both one choice from n, and extended matching questions. Efforts are being made to develop a bank of suitable questions. Such a task may well be the subject for collaboration with other medical schools.
IT Infrastructure
The University of Sheffield has committed its networked learning strategy to the purchase and support of a commercial package; WebCT. It has several hundred staff designing or implementing courses, including staff from biomedical sciences, who in addition to teaching pure science students have a major input into the first two years of the medical curriculum. The University has a central computing service, which aims to support the institutional teaching and learning strategy. The medical school NLE therefore receives no institutional support or funding from the central university. Funding for the NLE currently comes from the NHS via a Service Increment for Teaching (SIFT) levy.
Student access to services
Most NHS sites both within the community and hospitals and which includes all of the student clinical placements are connected by the NHSnet, which is large Intranet connected to the Internet via a firewall. The NHSnet and JANET communities have traditionally not mixed for cultural reasons rather than technical. Local agreements for both district general hospitals and GP surgeries are in place to allow students and community based staff access to the NLE. However Teaching Hospitals remain fearful of the misuse of inadequate computing facilities by students to the detriment of clinical staff.
Development of cross-institutional architectures for the delivery of materials and distributed and collaborative information
To facilitate teaching and learning students and staff need access to a high speed, dependable regardless of their location and timing of access. At the University level we continue to promote a view that their should be integration and interoperability at the technology level, so that what students and staff see on screen is driven by educational prerogatives rather than by which commercial package, for example WebCT, is currently most popular. We continue to promote local collaboration between NHS and University communities to break down the cultural barriers to joint working on the issues.
Research/Evaluation
A key desire of the NLE project team is to evaluate the acceptability, effectiveness and efficiency of the NLE systems. In any meeting where the curriculum comes up, there is a perceived need amongst clinicians and medical scientists to see evidence that it works elsewhere before accepting any major changes to the curriculum. This is no difference form other higher education settings but the kind of evidence they traditionally 'believe in is in the randomised controlled trial format. Action research is considered a weak form of evidence. The challenge then is to develop a research methodology which provides best medical education evidence.
References
Coffey, M., and Gibbs, G. (2001) Reliable student feedback: a replication of the factor structure of the SEEQ. Assessment and Evaluation in Higher Education. 26, 89-93
Cook, J. (2001) The Role of the Virtual Learning Environments in UK Medical Education: JTAP Report No: 623 Institute for Learning and Research Technology, University of Bristol.
Davis, M.H., Friedman, M., Harden, R.M., Howie P., Ker, J., McGhee, C., Pippard, M.J., and Snadden, D. (2001) Portfolio assessment in medical students' final examinations Medical Teacher 23, 357-366
Everett, R. (2001) 'MLE Information pack' http://www.jisc.ac.uk/mle/reps/infopack.html (accessed february21 2002
General Medical Council. (1998) Report of the visit to the University of Sheffield Faculty of Medicine. http://www.gmc-uk.org/med_ed/visits/sheffield.htm
General Medical Council. (2002) Revalidation. http://www.gmc-uk.org/revalidation (accessed 31 January 2002).
Schmidt (1983) Problem based learning: rationale and description Medical Education 17, 11-16
The Quality Assurance Agency for Higher Education (1999). Subject Review Report Q60/99 http://www.qaa.ac.uk/revreps/subjrev/institution_indexes/uni_of_sheffield.htm
Zuber-Skerritt, 0. (1996) Emancipatory action research for organisational change and management development. In: Zuber-Skerritt, O. (Ed.) New Directions in Action Research, London: Falmer.