Introduction

Research has traditionally been rewarded and recognised more than teaching at leading universities such as UNSW. One reason for this disparity is that measures of research excellence are widely used and accepted. In contrast, there are no generally accepted measures of teaching excellence. 

myEducation Portfolio is derived from a 2017-2018 UNSW Scientia Education Investment Fund project grant awarded to the Scientia Education Academy. This project aims to build on existing literature and evaluation tools to develop measures of educational excellence at UNSW that can be used to provide essential feedback to staff about their teaching performance. These can then be used to evaluate and reward excellence via teaching awards and academic promotion.

Members of the Scientia Education Academy collaborated with national and international experts in educational evaluation to develop appropriate measures for use at UNSW. Such measures are standards-based, thereby providing transparency regarding the level required at UNSW to achieve educational excellence across a number of criteria. The measures and associated performance standards, guidelines and exemplars across a variety of disciplines will be further refined following feedback from students and staff, including Heads of School and Associate Deans Education in all Faculties at UNSW. The measures will then be piloted to support reflective practice in education and its recognition in teaching awards and academic promotion.

Aims

myEducation Portfolio is intended to deliver the following outcomes:

1. Provision of a repository for UNSW staff to document their achievements in education, as well as a means to showcase those achievements for the purposes of sharing good practice, career development, academic promotion, teaching awards and fellowships;
2. Development of performance standards, guidelines and exemplars for an education portfolio for UNSW staff;
3. Development of guidelines for the evaluation of education portfolios, useful for reviewing career development, as well as applications for teaching awards and academic promotion.
4. An integrated measure of educational excellence at UNSW, with the education portfolio incorporating outcomes of student surveys of teaching quality and peer review of teaching practice.
5. Establishment of a community of practice that enables and supports academic staff in standards-based evaluation of educational excellence at UNSW.

Theoretical Background

Attempts to evaluate teachers and teaching in higher education date back many years, and have always proved to be problematic (Marsh, 2007). Existing literature reveals controversy regarding the reliability and validity of student feedback surveys, peer review of teaching, education portfolios and student learning outcomes as measures of educational excellence (see Marsh, 2007; Gunn and Fisk, 2013). Even the concept of ‘teaching excellence’ in higher education is ill-defined and controversial (Gunn and Fisk, 2013; Wood and Su, 2017). Indeed, many academics consider educational excellence to be ‘unmeasurable’ (Wood and Su, 2017). Nevertheless, policy measures such as the UK Teaching Excellence Framework are intended to raise the standard of teaching across all universities. In the current global environment for higher education, it is appropriate that institutions such as UNSW develop rigorous measures to recognise and reward excellent teaching.

But how can the multi-dimensional concept of educational excellence be measured? Ideally, valid measures of the inputs (qualifications and professional development), process (teaching practice) and outputs (i.e. student learning) of education would all be incorporated. Evaluating student learning as a measure of teaching is clearly of great importance. However, many contextual and institutional factors affect students’ learning, hence the influence of the teacher is difficult to isolate. Indeed, Gibbs (2016) asserts that measures of learning gains would be the most appropriate indicator of quality teaching, but such measures are not yet available. Furthermore, better measures of the learning and teaching process, such as student engagement surveys, require further development.

Marsh (1982) developed the SEEQ (Student Evaluations of Educational Quality) questionnaire, through which nine valid and reliable dimensions of effective teaching were characterised, and have since been demonstrated to be robust and stable across multiple disciplinary and cultural contexts (Marsh and Roche, 1994):

1. Learning/academic value;
2. Lecturer enthusiasm;
3. Organisation and clarity;
4. Group interaction;
5. Individual rapport;
6. Breadth of coverage;
7. Examinations/grading;
8. Assignment/reading; and
9. Workload/difficulty.

Marsh (2007) concludes that the SEEQ and other validated student surveys are the most robust metrics to evaluate teaching. However, the SEEQ instrument has now largely fallen out of use, with most universities having their own unit evaluation instruments (Boud 2017, personal communication), e.g. myExperience at UNSW. Furthermore, there are known issues with student ratings of teachers, particularly gender bias. This phenomenon has been observed for many years, and was recently elegantly demonstrated in a controlled study by McNell and colleagues (2015) in the online environment, as well as by Fan and colleagues (2019) at UNSW. Therefore, corroborating measures of excellence in teaching are required.

Marsh (2007) contends that peer review of teaching is somewhat less reliable than student surveys as a measure of teaching excellence, unless there is direct observation of teaching behaviours utilising well-defined criteria. Such an approach has been developed by Crisp and colleagues (2009).
While student surveys and peer review of teachers have roles to play in evidencing educational excellence, they do not provide a comprehensive view of educational practice, either alone or in combination (Beckmann, 2016). More holistic measures of educational excellence are therefore required (Gibbs, 2008). In that regard, the SEEQ dimensions map well to the criteria for teaching excellence awards originated by the Australian Learning and Teaching Council (2008), except for the final criterion (Devlin and Samarawickrema, 2010). Interestingly, Marsh (2007) acknowledged that the dimension of scholarship would add to the valid assessment of effective teaching. Further, Devlin and Samarawickrema (2010) advocate modifying and expanding the criteria below to include measures of student engagement and educational leadership:

1. Approaches to teaching that influence, motivate and inspire students to learn;
2. Development of curricula and resources that reflect a command of the field;
3. Approaches to assessment and feedback that foster independent learning;
4. Respect and support for the development of students as individuals; and
5. Scholarly activities that have influenced and enhanced learning and teaching.

 The above criteria also fit well with Ramsden and colleagues’ (1995, p.24) listing of the qualities of good teachers:

• Good teachers are also good learners; good teaching is therefore dynamic, reflective and constantly evolving.
• Good teachers display enthusiasm for their subject, and a desire to share it with their students.
• Good teachers recognise the importance of context, and adapt their teaching accordingly; they know how to modify their teaching strategies according to the particular students, subject matter, and learning environment.
• Good teachers encourage learning for understanding and are concerned with developing their students’ critical thinking skills, problem-solving skills, and problem-approach behaviours.
• Good teachers demonstrate an ability to transform and extend knowledge, rather than merely transmitting it.
• Good teachers set clear goals, use valid and appropriate assessment methods, and provide high quality feedback to their students.
• Good teachers show respect for their students; they are interested in  their professional and personal growth, encourage their independence, and sustain high expectations of them.

The UK Higher Education Academy (2011) developed a professional standards framework for teaching and supporting learning in higher education. That framework distinguishes three dimensions for educators: areas of activity (e.g. designing and implementing learning activities and assessment); core knowledge (e.g. disciplinary and pedagogical understanding); and professional values (e.g. respect for learners and continuing professional development). Educators must demonstrate that they fulfil criteria within each of the dimensions in order to become a Fellow of the Higher Education Academy. Descriptors of the levels of attainment for each dimension enable classification into Associate Fellow, Fellow, Senior Fellow and Principal Fellow. Movement between classifications requires evidence of progressively deeper and broader contributions to the institution and the higher education sector. This framework has been reported to support reward and recognition of teachers at UK universities, as well as impacting positively on institutional support for educational excellence (Turner et al., 2013).

Gunn and Fisk (2013), in their review of the literature for the UK Higher Education Academy, list the following dimensions of individual excellence in teaching practice, which differ somewhat from the professional standards framework:

Professor Denise Chalmers and colleagues (2014) developed the Australian University Teaching Criteria and Standards, which have been adopted by a number of institutions. The AUTCAS criteria include:

1. Design and planning of learning activities
2. Teaching and supporting student learning
3. Assessment and giving feedback to students on their learning
4. Developing effective learning environments, student support and guidance
5. Integration of scholarship, research and professional activities with teaching and in support of student learning
6. Evaluation of practice and continuing professional development
7. Professional and personal effectiveness

Based on the literature cited above, the most appropriate method to evaluate dimensions of teaching excellence would be to utilise education portfolios, in which academic staff present evidence of achievement and reflection related to each dimension. Standards-based evaluation of teachers’ education portfolios will utilise a consensus set of dimensions, selected from those generated by the Higher Education Academy (2011), Chalmers (2014) and the UNSW Academic Expectations Framework. If metrics derived from student surveys of teaching quality and peer review of educational practice are incorporated within the portfolio, this could provide an acceptable measure of individual teaching quality at UNSW. Integrating student survey data and peer review outcomes into the education portfolio will help to ensure that no measure is viewed in isolation, as well as embedding the education portfolio within existing structures at UNSW.

Progress / Outcomes / Next steps

Commencing in July 2017, the project team consulted with higher education experts in Australia and overseas, using a Delphi process to determine the dimensions of teaching practice to be incorporated in a UNSW education portfolio. This Delphi process incorporated academics (n=65) from a variety of disciplines at UNSW, across Australia and internationally, to ensure that institutional and disciplinary differences in conceptions of educational excellence are acknowledged and incorporated into design of the UNSW education portfolio. The first round of the Delphi process identified 13 dimensions of effective teaching practice in higher education. Respondents in the second round of the Delphi process (n=58) prioritised 4 of those dimensions for incorporation into an education portfolio. Already, a community of practice has been established by this process.  

The first round of the Delphi process identified the following 13 dimensions of effective teaching practice in higher education:

1. Demonstrates up to date disciplinary knowledge, and applies teaching methods that display an understanding of how that knowledge can be effectively learned
2. Designs and plans effective curricula and learning activities
3. Designs and sequences appropriate assessment tasks together with constructive, actionable feedback
4. Inspires and engages students in learning
5. Promotes collaboration, active learning and critical thinking
6. Communicates effectively with students (listening, answering questions and explaining concepts)
7. Promotes reflection and self-regulation in learners
8. Creates inclusive, safe and positive learning environments
9. Uses technology innovatively and effectively to promote learning
10. Demonstrates educational scholarship
11. Demonstrates commitment to professional development in education
12. Demonstrates professional and ethical conduct in education
13. Demonstrates educational leadership

The second round of the Delphi process and further feedback from the project advisory group and peers at UNSW resulted in four dimensions of effective teaching practice, each associated with a number of criteria: