The effect on staff perceptions of online learning when using a non-traditional approach to staff development

University of Surrey

ABSTRACT

Staff development plays a crucial role in the adoption of new technology. However, the appropriate requirements for effective staff development in terms of emphasis, content and delivery are often not met.

This paper identifies the key factors identified within current research and proposes a course methodology and design that addresses these issues. The evaluation focuses on the success of the methodology and the role of the individual participant’s pedagogical style with regard to the adoption of new technology. The paper concludes that while a traditional didactic pedagogical approach (in both the individual participant and in the design of staff development courses) may not affect uptake, it may result in inappropriate adoption.

Keywords

staff development; adoption of new technology; staff perception of online learning; online tutoring.

INTRODUCTION

The adoption of new technologies to support teaching and learning is a high priority for many higher education institutions, with the Dearing report providing a national driver in 1997. The inclusion of "exploitation of communications and information technology in the service of managed improvements in learning and teaching" (HEFCE, 2000) as a target within institutional learning and teaching strategies has subsequently ensured that this issue is kept on the institutional agenda.

HEFCE has recognized that staff development is vital in exploiting new technologies and maintaining good adoption rates. ("Meet specific staff development and training objectives that not only equip staff to meet their current needs but also prepare them for future changes, such as using new technologies for learning and teaching." (HEFCE, 2001)). The challenge is to make sure that the staff development opportunities available address the correct needs and are delivered in an appropriate way to ensure effective adoption.

Requirements for staff development in new technologies

Recent literature has highlighted four key factors that are pertinent to the issue of staff development regarding new technologies:

Emphasis on pedagogy, not technology

Staff development courses about new technologies often address the acquisition of technical skills, rather than the pedagogical aspects and implications of using learning technology. "The emphasis on the mastery and use of the technology has been the primary focus of many staff development activities" (de la Harpe et al, 2002).

However, the pedagogical considerations are important, as highlighted by the TALiSMAN report (Alexander, 1999): "the introduction of learning technology creates new opportunities and challenges, tutorial support and student administration….arguably this is a significant part of the pedagogic implications of using learning technology, and it may be no surprise that it is not represented in this survey, given the small number of items that deal with any pedagogic implications of using LT". The report concludes "there is a need for a range of staff development activities addressing the evaluation of learning technology and its pedagogic dimension".

Therefore, to encourage non-superficial adoption of learning technology, there is a need for staff to be engaged at a pedagogical level in the first instance, with the technological training provided once that has been explored.

Emphasis on individual pedagogy

However, focusing on pedagogical issues is not necessarily straightforward – whose pedagogical approach is used? ("The use of ICT in HE is complex and widely debated. For example, there appear to be no simple answers about how best to use technology within courses … All this poses particular problems for those involved in professional development in HE, since it means that there can be no definitive body of knowledge for those new to this topic." (Smith and Oliver, 2002).

The implication is that there need to be opportunities for the teacher to explore their individual concept of teaching and learning: "Academic development is most likely to succeed when the teacher’s own beliefs about teaching and learning provide the starting point" (Errington, 2001). The individual is more likely to use the technology if they can see the applicability to their situation and methodology, and adopt and adapt as they fit.

However, Mayes argues that the emerging pedagogical consensus in adopting new technology is around constructivism and the ‘most effective educational way of using online technology, therefore, is to focus on supporting the learner’s involvement in collaboration, authentic tasks, reflection and dialogue" (Mayes, 2001). Therefore, a teacher with a more traditional concept of teaching may see less applicability and feel less positive about using new technology than a teacher with a more student-centred approach.

Learner experience crucial

In developing personal teaching methodologies and styles, staff often draw upon their experiences as a learner, both good and bad. However, very few academic staff embarking upon the use of new technologies have been in this position. They are unable to draw on ‘the learner experience’ but approach new technologies as a novice. An important part of staff development must surely give access to this experience. As Gilly Salmon found with new tutors, they "truly needed to experience, much as their own students would, the pitfalls and potential of CMC if they were to e-moderate effectively" (Salmon, 2000).

Context of professional development

In designing staff development, the special requirements of professional development also need to be considered, as it differs from traditional learning. Milligan (1999) lists the following:

"Motivation is different (everyday demands of work are likely to take precedence)

Learners will be part time and will almost certainly require a flexible timetable

Examination will be atypical (will require participant to draw on their own experience)

Peer learning and mentoring will take a significant role."

This paper will address the following:

Is it possible to design a successful staff development course (i.e. one that encourages staff to adopt new technologies) that also addresses the above factors and requirements (pedagogical focus; individual teaching beliefs; experience as online learner; professional context)?

Assuming that teacher’s own beliefs about teaching and learning are the starting point for academic development, how will different teaching approaches influence the response to and success of the course for the individual?

DESIGN AND IMPLEMENTATION

Aims and methodology

With the increasing use and interest in Virtual Learning Environments, it was decided to focus the staff development course around the online learning aspect of new technologies.

The main aims of the course were for staff to develop a realistic perception of online learning (to address the lack of online learner experience) and to have an increased belief in the potential of online learning in their own teaching practice. The pedagogical approach used would be social constructivist, to provide a model of ‘good practice’ in accordance with the ‘emerging pedagogical consensus’ identified by Mayes.

In order to achieve these aims, the objective was to encourage staff to go through the following process:

The model (fig. 1) is an adaptation of Kolb’s (Kolb, 1984) learning cycle. Although entry to the cycle can usually be at any point, in this case it was specified as the ‘experiencing’ stage. The experience and reflection stages takes place as a learner, with the conceptualization and application stages taking place as a teacher. Where this model does not reflect Kolb’s cycle is that academic staff may often not have the opportunity to experience online learning as student more than once, and so cannot complete the cycle.

Fig. 1: Adaptation of Kolb’s 4-stage model as used in the staff development course

However the cycle could be completed and repeated if an as outer loop is introduced, with the stages of experience, reflection, conceptualization and application being repeated as a teacher throughout all the 4 stages (fig. 2.).

Fig. 2: Further adaptation of Kolb’s model with outer loop as used in the staff development course

The methodology also draws on Argyris’ (Argyris, 1999) double-loop theory (where a mismatch in action and consequences is addressed by examining the underlying governing variables before altering action). Creating an authentic online learner experience may include experiencing failure as the learner struggles with the new environment. Although valuable, it could potentially reduce the likelihood of increased belief in potential of online learning. As Argyris observes: "Because many professionals are almost always successful at what they do, they rarely experience failure. And because they have rarely failed, they have never learned how to learn from failure…their ability to learn shuts down precisely at the moment they need it most"). In fostering a realistic perception of online learning, the aim was to enable staff to examine and alter their misperception of the governing variables of online learning and thus alter their actions accordingly to become more effective online learners and teachers.

Design and implementation

With reference to the specific professional needs, the course methodology was implemented as a fully online course lasting 8 weeks, to provide an authentic online learning experience and allow part-time flexible study. Individual and collaborative activities were included, to provide an authentic and wide-ranging experience and allow peer interaction and feedback. The course material was about the role of the online tutor, thereby allowing learning to take place through content as well as process, and providing coherence of medium and message.

The course was not accredited, an authentic detail that one would expect in most online learning courses. However, this decision then avoided the negative "achievement" motivation, allowing the focus to be on the learning. This echoes Biggs’ view that if "students build up a good knowledge base, achieve success in problems that are significant, and build up a feeling of "ownership" over their learning, motivation follows good learning as night follows day".

EVALUATION

Scope of evaluation

The evaluation aimed to assess the overall success of the course aims and methodology by addressing the questions:

"Is this non-traditional approach to staff development successful in:

engendering a realistic perception of online learning and

increasing belief in the potential of online learning?"

The evaluation would also look at individual responses in comparison to their personal pedagogical beliefs do address the question:

"Do individual pedagogical beliefs affect the outcome of participating in this non-traditional course?

Methodology

Data was collected pre, during and post course using a range of methods to allow triangulation of data: self-assessment, quizzes and tests; learning contracts, written questionnaires, focus groups and graphical representation. Data was collected from staff from both the learner and teacher perspective, which was crucial to exploring all stages of the adapted Kolb’s model. Qualitative data was particularly important in gathering impressions, perceptions and opinions.

Graphical data collection

Graphical representation was chosen as being effective in gathering impressionistic data which may be harder to articulate verbally. It was also easier to see patterns between individual staff responses.

This form of data collection was used in two instances, the first being a continuum of teaching styles ranging from traditional didactic teaching at one end to student-centred participatory at the other (adapted from Brandes and Ginnis, 1986). Staff were asked to place themselves on the scale.

The second instance of data collection was the use of a reaction graph, where are participants asked to draw a graph showing negative and positive feelings (on the vertical axis) over the length of the course (on the horizontal axis). The graphs illustrate learner engagement and reaction to the course. The graphs were drawn both electronically using a drawing package (when used during the course) or by hand (in the post-course discussion sessions), and as such can be crude in their representation.

Data selection

The following data was used in the evaluation:

FINDINGS

Participant profile

The course has run on three occasions to date. The instance used for this evaluation took place during September to November 2001.

There were 12 participants from a range of academic schools, all of whom self-selected to participate in the course. 3 participants dropped out almost immediately, another 2 after completing half the course, with the ream 7 completing most or all of the course. 9 sets of complete data were collected and used in this evaluation of which 1 was from an ‘early leaver’, 2 were from the participants who completed half the course, and the reamining 7 from those who completed most or all of the course.

The participants rated themselves as follows:

IT skills: 3 good, 5 average, 1 poor

5 had used electronic discussion boards

1 had taught online previously (but not studied)

1 had studied online (but not taught)

Was the experience realistic?

For the participants to have an authentic learning experience, it was important that they engaged with the course on more than just a superficial level. Seven of the reaction graphs and accompanying comments indicated a high level of engagement over the length of the course, with two showing a low level of engagement. Within the seven that showed engagement, there were three patterns:

A positive start followed by one or more negative dips, and a positive finish as illustrated in fig. 3.

A positive start followed by a slow decline; comments showed that interest was maintained throughout the course but pressure of work and inability to contribute induced negative feelings.

An indifferent start followed by a steady increase in interest and positive feelings towards the course.

Fig. 3. Record of learner experience using a reaction graph from staff development course

Interestingly, an informal record from learner experiences on an accredited online course (University of Lancaster’sNetworked Learning Programme, September – December 2000) using the same reaction graph method produced the following comparable record:

Fig. 4: Record of learner experience using a reaction graph from the Networked Learning Programme, University of Lancaster.

Did perceptions change on reflection?

8 participants reported that their perception had changed having been on the course, with the other participant responding no change because he already had a realistic perception prior to the course. The following comments support that there has been shift in perception:

"Learning would be an easy experience and managing time wouldn’t be difficult" shifted to "Learning is difficult (but enjoyable) and managing time can be hard."

"Vague, gimmicky, education on the cheap" shifted to "The problems and solutions showed that it is by no means cheap and potentially very precise."

"I thought it was a bit like distance learning but with technology" shifted to "it is a unique learning environment"

Could participants conceptualize the impact on teacher practice?

The participants were asked to list the important factors to consider when designing and running an online course. Most were able to identify at least one key factor that would contribute to the effectiveness of an online course such as:

"provide flexibility, not just in terms of time and space, but take into account learning styles"

"know whether what you want to teach is suitable for online"

"know when to apply different tutoring styles"

"engagement of students is crucial"

Were participants convinced by the potential and keen to adopt?

When asked how they would use and apply what they had learnt on the course, six participants had definite plans, two were not sure and one did not respond.

The participants were then asked if they were more or less convinced about the potential of online learning: six were more convinced, two felt the same, and one did not respond. Of the two that felt the same, one felt that she had not had sufficient time to explore the all the issues, while the other still felt that online learning was ‘an alternative where face to face teaching isn’t possible – I still think it is second best’. Overall, the participants appeared more likely to use online learning than prior to the course.

Do individual pedagogical beliefs affect response and outcome of the course?

Seven of the participants placed themselves closer to the ‘student-centred participatory’ end of the teaching style continuum, with some variation depending on whether the student group was undergraduate or postgraduate. The other two placed themselves closer to the ‘traditional didactic’ end. One of them further articulated his teaching approach within the online discussion:

"Indeed as a dinosaur of impeccable credentials I've always thought that the obvious ways

are probably the best...if you want someone to know something - tell them...don't expect

them to discover it all over again by long winded discussions."

Of the seven student-centred teachers, 5 showed an increase in their belief in online learning. Of the two traditional teachers, one did not respond but the other also showed an increase in his belief in online learning. This may indicate that different pedagogical beliefs do not necessarily affect the adoption of online learning or alienate the more traditional teachers. However, although there may be an "emerging pedagogical consensus" based on constructivism, it may not be adopted by all and the teacher-centred approach may merely be replicated within the electronic environment.

Conclusions

The sample used was small and therefore any conclusions are drawn within that context, with further samples needed to confirm these initial findings,

For most participants, the course methodology and non-traditional course design appeared to be successful in achieving the aims of the course (namely, to encourage staff to adopt new technologies and to implement it using an appropriate pedagogical approach) while still addressing specific professional development requirements. However, this level of achievement was assisted by the prior pedagogical beliefs of the participants in student-centred learning.

Where the participant held a more traditional view of teaching, the second aim is less likely to be achieved, leading to the possibility that a less appropriate pedagogical style may be applied when adopting new technology. "The conceptions that lecturers hold need to be addressed, and if necessary, lecturers helped to change these so they are more in line with a learner-centred, process-orientated constructivist approach to teaching and learning." (de la Harpe et al, 2002) It is recommended that teaching and learning beliefs are addressed prior to the course and independently of new technology.

ACKNOWLEDGMENTS

Rhiannon Williams, Centre for Applied Research in Educational
Technologies, University of Cambridge for co-design of the course and introducing the concept of reaction graphs.

Martin Oliver, Kim Isroff, Holly Smith, Education and Professional Development, University College London for support in publishing this research.

Annette Odell, Centre for Learning Developments, University of Surrey for advice and encouragement.

Evan Dickerson, Centre for Learning Developments, University of Surrey for additional tutoring support.

REFERENCES

Alexander, W. (1999) TALiSMAN review of staff development courses and materials for C & IT in teaching, learning and assessment, Heriot-Watt University. Available electronically http://www.talisman.hw.ac.uk/CIT review

Argyris, C. (1999) On Organisational Learning. Blackwell, Oxford.

Biggs, J. (199) Teaching for quality learning at university: what the student does. Society for Research into Higher Education; Open University Press.

Brandes, D., and Ginnis, P. (1991) A Guide to Student-Centred Learning. Simon and Schuster, Herts.

De la Harpe, B. and Radloff, A. (2002) The importance of teacher conceptions of teaching and learning for facilitating student learning online: changes in conceptions following an online course about developing courses online. Proceedings of the 2001 9^th International Symposium Improving Student Learning: Improving Student Learning Using Learning Technologies. Oxford: OCSLD (in press).

Errington, P. E. (2001) The influence of teacher beliefs on flexible learning innovation in traditional university settings. Innovation in Open and Distance Learning, ed F Lockwood and A Gooley, Kogan Page, London.

HEFCE (2000) Institutional learning and teaching strategies. Available electronically http://www.hefce.ac.uk/pubs/circlets/2000/cl15_00.htm

HEFCE (2001) Rewarding and developing staff in higher education. Available electronically at http://www.hefce.ac.uk/Pubs/hefce/2001/01_16.htm

Kolb, D. (1984) Experiential Learning: Experience as the Source of Learning and Development. Prentice-Hall, New Jersey

Mayes, T. (2001) Learning technology and learning relationships. Teaching and Learning Online ed J Stephenson. Kogan Page, London.

Milligan, C. (1999) The Role of Virtual Learning Environments in the Online Delivery of Staff Development, Heriot-Watt University. Available electronically http://www.icbl.hw.ac.uk/jtap-573

Salmon, G. (2000) E-moderating: The Key to Teaching and Learning OnLine, Kogan Page, London.

Smith, H. J. & Oliver, M. (2002) University teachers' attitudes to the impact of innovations in Information and Communication Technology on their practice. Proceedings of the 2001 9^th International Symposium Improving Student Learning: Improving Student Learning Using Learning Technologies. Oxford: OCSLD (in press).