This publication includes a small selection of the case studies, the theoretical model and the seven principles, and the workshop plans for using the materials. The project team explored feedback issues with higher education institutions (HEIs) across Scotland.
1. The Higher Education Academy Enhancing student learning through effective formative feedback Charles Juwah, Debra Macfarlane-Dick, Bob Matthew, David Nicol, David Ross and Brenda Smith
2. About the authors Charles Juwah is Senior Educational Bob Matthew, a civil engineer by discipline, is David Ross is a chemist by discipline and has Development Officer, and Deputy Head, at the now director of the Teaching and Learning been at Abertay for nineteen years, the last nine Centre for the Enhancement of Learning and Service at the University of Glasgow having of which as Director of the Centre for the Teaching (CELT), The Robert Gordon University, previously taught civil engineering in Ireland and Enhancement in Learning and Teaching. An Aberdeen. His responsibilities include academic England. In his present post he has responsibility experienced quality assurance assessor, he has and professional development, quality for supporting the enhancement of learning, been involved in the strategic development of the enhancement and educational research. His teaching and assessment within the university. new enhancement-led model of quality current research interest focuses on the He is also associate dean (research) within the assurance in Scotland, and was also Convenor promotion and enhancement of the quality of faculty of education at Glasgow. He is currently of the Universities Scotland Educational learning, online learning, assessment and the chair of Universities Scotland Educational Development Sub-Committee from 2001-2003. feedback practices, postgraduate research Development Committee and a member of He is currently Chairman of the Universities supervision and the use of portfolios and Universities Scotland Learning and Teaching Scotland/QAA Joint Working Group on personal development planning in enhancing committee. In his spare time he is passionate Implementing PDP (ELF Project). His research learning and skills development. about growing rare varieties of potatoes, interests are in Change Management in Quality Japanese maples, and climbing Scottish Enhancement, Enhanced Learning Frameworks Debra Macfarlane-Dick, the Student Enhanced mountains. and Student Assessment Feedback. Learning through Effective Feedback (SENLEF) Project Officer, is an educational developer at the David Nicol is an educational consultant within Brenda Smith is Head of the LTSN Generic University of Glasgow. She joined the Teaching the Centre for Academic Practice, University of Centre (now Higher Education Academy) which and Learning Service as a University Teacher in Strathclyde. He works with academic is dedicated to developing enhanced teaching, 2002 having previously worked in training and departments/faculties on educational assessment and learning practice in the UK HE development. In 2003, she moved into her improvement projects in teaching, learning and sector. This involves working with 24 Subject current role as Employability Development assessment in both online and face-to-face Centres, and working at strategic level with key Adviser in the Careers Service at Glasgow. In her environments. Current projects include change agents in universities and other key current role she has responsibility for supporting supporting the implementation of an institution- national network groups such as faculty the implementation of the University’s wide virtual learning environment (VLE), developers, quality agencies and the UK funding employability strategy by working collaboratively evaluating a JISC/NSF-funded digital libraries in bodies. She has acted as a consultant and has with staff, students and employers to encourage the classroom initiative, and work on strategic facilitated workshops in many different countries and develop initiatives and learning materials to change in relation to e-learning. Recent including South Africa, the Philippines, Colombia, enhance student employability. Outside work, research publications have focused on the social Azerbaijan, Lithuania, the West Indies and Debra enjoys art (as an art historian by dimensions of e-learning, learning objects, Malaysia. She has written widely on learning and discipline), keeping tropical fish and gardening. electronic classroom feedback systems, shared teaching, including work on Research, Teaching workspaces in learning and on risk and cost- and Learning, Assessment, Resource Based benefit analysis in relation to e-learning. Learning and Lecturing to Large Groups.
3. Contents 1. Introduction 2 2. Rethinking formative assessment in HE: a theoretical model and seven principles of good feedback practice 3 3. Case studies 15 Case study 1 – Feedback as deviations from a ‘sound standard’ 16 Case study 2 – Combining formative and summative assessment on 18 a continuous basis Case study 3 – Feedback in interactive lectures using an electronic 21 voting system Case study 4 – Enhancing effectiveness and efficiency in student 23 feedback Case study 5 – Using immediate feedback in class: the New 25 Approaches to Teaching and Learning in Engineering (NATALIE) Project Case study 6 – Self and peer-assessment of written work in English 28 Literature Case study 7 – Portfolios and feedback: enhancing self-assessment 31 Case study 8 – Work to win: formative feedback on demand 33 4. Plans for possible workshops 36 5. References 38 6. Acknowledgements 41 June 2004 - The Higher Education Academy Generic Centre Enhancing student learning through effective formative feedback 1
4. 1 Introduction Student Enhanced Learning through Effective This publication includes a small selection of the The project team hopes that these might be Feedback (SENLEF) was a project funded by the case studies, the theoretical model and the seven useful to both educational developers and LTSN Generic Centre (now Higher Education principles, and the workshop plans for using the academics when they are attempting to enhance Academy) to develop a resource for practitioners materials. their practice in the area of giving learners wishing to improve their feedback practice to effective feedback. It is worthwhile reiterating the seven principles of students or get some new ideas on how to good feedback practice that we have identified. enhance their current practice. These are listed below. The idea for the project came from the Universities Scotland Educational Development 1. Facilitates the development of self- Committee. The project team explored feedback assessment (reflection) in learning. issues with higher education institutions (HEIs) 2. Encourages teacher and peer dialogue across Scotland. The outcomes from the project around learning. are available on the Higher Education Academy 3. Helps clarify what good performance is Generic Centre web site at (goals, criteria, expected standards). and include the following: 4. Provides opportunities to close the gap between current and desired performance. • A series of case studies 5. Delivers high quality information to students about their learning. • A theoretical model 6. Encourages positive motivational beliefs • Seven principles for good effective practice and self-esteem. • A questionnaire for others to contribute 7. Provides information to teachers that can further case studies be used to help shape the teaching. • Workshop plans for using the materials. These principles are explained in greater detail in the next section. 2 Enhancing student learning through effective formative feedback The Higher Education Academy Generic Centre - June 2004
5. 2 Rethinking formative assessment in HE: aprinciples theoretical model and seven of good feedback practice Dr David Nicol, University of Strathclyde not the sole province of the teacher. Peers often self-esteem. A number of writers have argued Debra Macfarlane-Dick, University of Glasgow provide feedback – for example in group-work that feedback is under-conceptualised in the contexts – and students generate their own theoretical literature in HE and elsewhere, and This section explores how higher education feedback while engaging in and producing that this makes it difficult to design effective institutions might use assessment more academic work (see below). Formative feedback practices or to evaluate their effectively to promote student learning. assessment also provides information to teachers effectiveness (Yorke, 2003; Sadler, 1998). While Assessment provides a framework for sharing about where students are experiencing difficulties there has been a move over the last decade to educational objectives with students and for and where to focus their teaching efforts. conceptualise ‘learning’ from a constructivist charting their progress. However, it can generate perspective (Laurillard, 2002, for example), feedback information that can be used by This section summarises the research on approaches to feedback have, until recently, students to enhance learning and achievement. formative assessment and feedback. It includes remained obstinately focused on simple This feedback information can also help teachers the following: ‘transmission’ perspectives. Teachers ‘transmit’ realign their teaching in response to learners’ • A conceptual model of the formative feedback messages to students about strengths needs. When assessment serves these purposes assessment/ feedback cycle and weaknesses in their work assuming that it is called ‘formative assessment’. It is argued these messages are easily decoded and turned that formative assessment should be an integral • Seven principles of good feedback practice: these are drawn from the model into action. In contrast, in this paper, students are part of teaching and learning in HE and that and a review of the research literature assumed to construct actively their own ‘feedback’ and ‘feed-forward’ should be understanding of feedback messages from tutors. systematically embedded in curriculum practices. • Some examples of good practice Moreover, these messages are assumed to be strategies related to each principle. Formative assessment aids learning by complex and difficult to decipher (Higgins, Hartley generating feedback information that is of benefit There are two central arguments within this and Skelton, 2001; Ivanic, Clark and to students and to teachers. Feedback on section (i) that formative assessment and Rimmershaw, 2000). performance, in class or on assignments, enables feedback should be used to empower students as The conceptual model and the seven principles students to restructure their understanding/skills self-regulated learners and (ii) that more presented in this paper are intended as tools that and build more powerful ideas and capabilities. recognition should be given to the role of teachers might use to analyse and improve their However, the provision of feedback information is feedback on learners’ motivational beliefs and own formative assessment and feedback practices. June 2004 - The Higher Education Academy Generic Centre Enhancing student learning through effective formative feedback 3
6. A conceptual model teachers give students feedback information on possess some of the same evaluative skills as (b) – that is, how their performance compares to their teacher. For many writers, this observation In a review article, Black and Wiliam (1998) drew the standard – but that this feedback often falls has led to the conclusion that as well as focusing together over 250 studies of formative short of what is actually necessary to help on the quality of the feedback messages, assessment with feedback carried out since 1988 students close the gap. For example, such teachers should focus their efforts on spanning all educational sectors. The studies information might be difficult to understand (such strengthening the skills of self-assessment in that formed part of their meta-analysis were as a comment that ‘this essay is not sufficiently their students (Yorke, 2003; Boud, 2000). ecologically valid in that they were drawn from analytical’) and especially if the learning goal (a) real teaching situations. Black and Wiliam’s Figure 1 presents a conceptual model of has not been fully assimilated in the first place. analysis of these studies showed that feedback formative assessment and feedback that Black and Wiliam (1998) further elaborate on this resulted in positive benefits on learning and synthesises current thinking by key researchers communication issue when they discuss the links achievement across all content areas, knowledge into this topic (Sadler, 1983, 1989; Black and between the way a feedback message is received and skill types and levels of education. One of Wiliam, 1998; Yorke, 2003; Torrance and Pryor, and what students do with that message. the most influential papers underpinning the 1998). The figure is based on a model of Black and Wiliam review, and the writings of other ...those factors which influence the feedback and self-regulated learning originally researchers, is that by Sadler (1989). Sadler reception of a [feedback] message and the published by Butler and Winne (1995). A key personal decision about how to identified three conditions necessary for students respond…[include]….beliefs about the feature in the model that differentiates it from to benefit from feedback. The student must: goals of learning, about one’s capacity to commonplace understandings of feedback is that respond, about the risks involved in the student is assumed to occupy a central and • Possess a concept of the goal/standard or responding in various ways and about what active role in all feedback processes. They are reference level being aimed for learning should be like (p21). always actively involved in monitoring and • Compare the actual (or current) level of Any model of feedback must take account of the regulating their own performance both in terms of performance with that goal or standard way students make sense of, and use, feedback their goals and in terms of the strategies being • Engage in appropriate action which leads information. More importantly, however, is used to reach those goals. to some closure of the gap. Sadler’s argument that for students to be able to In the model, an academic task set by the compare actual performance with a standard, and Sadler argued that in many educational settings teacher (in class or set as an assignment) is the take action to close the gap, they must already starting point for the feedback cycle. 4 Enhancing student learning through effective formative feedback The Higher Education Academy Generic Centre - June 2004
7. Engagement with the task requires that students draw on prior knowledge and motivational beliefs and construct a personal interpretation of the requirements and properties of the task. Based on this internal conception, they formulate their own task goals (which may be different from those of the teacher) and engage in actions to achieve these goals by applying tactics and strategies that generate outcomes. Monitoring these interactions with the task and the outcomes that are being cumulatively produced, generates internal feedback. This feedback is derived from a comparison of current progress against internal goals or standards – gaps are identified (between progress and goals) and further actions are taken to close these gaps (Sadler, 1989). This self- and the path of learning (Butler and Winne, feedback information from external sources. The generated feedback information might lead to a 1995). teacher’s feedback response (based on their re-interpretation of the task or to the adjustment monitoring and assessment of student In the model, external feedback to the student of internal goals or of tactics and strategies. performance) must be interpreted and might be provided by teachers, peers or others Students might even revise their domain internalised by the student before it can influence (placement supervisor, for example). However, knowledge or beliefs which, in turn, would subsequent action (Ivanic, Clark and students are always actively engaged in feedback influence subsequent processes of self- Rimmershaw, 2000). This has important processes. First, they generate aspects of their regulation. If external feedback is provided, this implications for feedback processes in HE. If own feedback as they monitor performance and additional information might augment, concur or students are always involved in monitoring and identify and make sense of gaps while carrying conflict with the student’s interpretation of the task assessing their own work, then rather than just out tasks. Second, they interpret and filter June 2004 - The Higher Education Academy Generic Centre Enhancing student learning through effective formative feedback 5
8. thinking of ways of enhancing the teacher’s ability 7. Provides information to teachers that can ability to self-assess and self-correct. be used to help shape the teaching. to deliver high quality feedback we should be In the conceptual model, the student or learner is devising ways of building upon this capacity for The following sections provide the rationale for always engaged in monitoring gaps between self-regulation (Yorke, 2003). each principle in terms of the conceptual model internally set task and personal goals and the and the associated research literature. Brief outcomes that are being progressively produced. Seven principles of good feedback examples of how these principles might be This monitoring is a by-product of purposeful practice applied are also suggested. engagement in a task. However, in order to build on this process, and the student’s capacity for From the conceptual model and the research literature on formative assessment it is possible 1. Facilitates the development of self-regulation, teachers should create more to identify some broad principles of good self-assessment in learning formal and structured opportunities for self- monitoring and the judging of progression to feedback practice. A provisional list might include Over the last decade there has been an goals. Self-assessment tasks are a good way of the following seven. increasing interest in strategies that encourage doing this, as are activities that encourage 1. Facilitates the development of self- students to take a more active role in the reflection on both the processes and the products assessment (reflection) in learning. management of their own learning (see Nicol, of learning. 1997). Black and Wiliam (1998) make the 2. Encourages teacher and peer dialogue around learning. argument that ‘a student who automatically Research shows that direct involvement by follows the diagnostic prescription of a teacher students in assessing their own work, and 3. Helps clarify what good performance is without understanding of its purpose will not learn’ frequent opportunities to reflect on goals, (goals, criteria, standards expected). (p54) while Sadler (1989) argues that the purpose strategies and outcomes are highly effective in 4. Provides opportunities to close the gap of formative assessment should be to equip enhancing learning and achievement (McDonald between current and desired performance. students gradually with the evaluative skills that and Boud, 2003). Moreover, if the skills of self- 5. Delivers high quality information to their teachers’ possess. These writers are assessment are developed progressively over the students about their learning. concerned that an over-emphasis on teacher course of an undergraduate degree this would 6. Encourages positive motivational beliefs assessment might increase students’ support a model of higher education where and self-esteem. dependency on others rather than develop their students are prepared for lifelong learning (Boud, 2000). 6 Enhancing student learning through effective formative feedback The Higher Education Academy Generic Centre - June 2004
9. An important aspect of self-assessment involves 2. Encourages teacher and peer student must play in constructing meaning from helping students both to identify standards/criteria dialogue around learning feedback messages. that apply to their work and to make judgements While research shows that teachers have a One way of increasing the effectiveness of about how their work relates to these standards central role in helping a develop student’s own external feedback and the likelihood that the (Boud, 1986). capacity for self-assessment in learning, external information provided is understood is to feedback from other sources (such as tutors or conceptualise feedback more as a dialogue peers) is also crucial. Feedback from tutors and rather than as information transmission. Examples of structured reflection peers provides additional information that helps Feedback as dialogue means that the student not and/or self-assessment are varied and might include students: challenge students to reassess their knowledge only receives initial feedback information but also (1) requesting the kinds of feedback and beliefs. Teacher feedback also serves as an has the opportunity to engage the teacher in they would like when they hand in authoritative external reference point against discussion about that feedback. This is shown in work; which students can evaluate, and self-correct the conceptual model by the two-way arrows that (2) identifying the strengths and weaknesses in their own work in their progress and their own internal goals. link external processes to those internal to the relation to criteria or standards student. The idea that feedback encourages before handing it in for teacher In the conceptual model (figure 1), for external dialogue is considered good practice by many feedback; feedback to be effective it must be understood writers on assessment. For example, Freeman (3) reflecting on their achievements and internalised by the student before it can be and selecting work in order to and Lewis (1998) argue that the teacher ‘should used productively. Yet in the research literature compile a portfolio; try to stimulate a response and a continuing (4) setting achievement milestones (Chanock, 2000; Hyland, 2000) there is a great dialogue – whether this be on the topics that for a task and reflecting back on deal of evidence that students do not understand formed the basis of the assignment or aspects of progress and forward to the next the feedback given by tutors (for instance, ‘this stage of action; students’ performance or the feedback itself’ report is not logically structured’) and are (5) having students give feedback on (p51). Discussions with the teacher help students each other’s work (peer feedback) therefore not able to take action to close the gap to develop their understanding of expectations also helps support the development (that is, he or she may not know what to do to and standards, to check out and correct of self-assessment skills (for make the report more ‘logical in structure’). example, Gibbs, 1999). misunderstandings and to get an immediate External feedback as a transmission process response to difficulties. involving ‘telling’ ignores the active role the June 2004 - The Higher Education Academy Generic Centre Enhancing student learning through effective formative feedback 7
10. Unfortunately, with large class sizes it can be work in relation to standards) which can be 3. Helps clarify what good difficult for the teacher to engage in dialogue with transferred to the assessment of their own work performance is students. Nonetheless, there are ways that (‘I didn’t do that either’, for example). Fourthly, teachers might increase feedback dialogue even peer discussion can be motivational in that it Students can only achieve a learning goal if they in these situations. For example, by reporting encourages students to persist and gives a understand that goal, assume some ownership of feedback in class and structuring break out yardstick to measure their own performance it, and can assess progress (Sadler, 1989; Black discussions of feedback or by using classroom against (see Nicol and Boyle, 2003). Finally, it is and Wiliam, 1998). In the model (figure 1), technologies that collate student responses in sometimes easier for students to accept critiques understanding the goal means that there must be class and then feed the results back visually as a of their work from peers rather than tutors. a reasonable degree of overlap between the task histogram. This feedback can act as a trigger for goal set by the student and the goal originally set teacher-managed discussion (for example, Nicol Good examples of feedback dialogue in class by the teacher. However, there is considerable and Boyle, 2003). include: research evidence to suggest that there are often (1) providing feedback using one-minute mismatches between tutors’ and students’ Another source of external feedback are the papers (Angelo and Cross, 1990); conceptions of goals and of assessment students themselves. Peer dialogue is beneficial (2) reviewing feedback in tutorials where students are asked to read the feedback standards and criteria. to student learning in a variety of ways. First, comments they have been given and discuss students who have just learned something are Hounsell (1997) has shown that tutors and these with peers – they might also be asked often better able than teachers to explain it to to suggest strategies to improve performance students often have quite different conceptions their classmates in a language and in a way that next time; about the goals and criteria for essays in is accessible. Second, peer discussion exposes (3) asking students to find one or two undergraduate courses in history and psychology examples of feedback comments that they students to alternative perspectives on problems and that poor essay performance is correlated found useful and to explain how they helped. and to alternative tactics and strategies. with the degree of mismatch. In a similar vein, Other ways of using feedback dialogue in a Alternative perspectives enable students to revise planned way, for assignments, might involve: Norton (1990) has shown that when students or reject their initial hypothesis and construct new (1) having students give each other were asked to rank specific assessment criteria knowledge and meaning through negotiation. descriptive feedback on their work in relation for an essay task they produced quite different to published criteria before submission; rankings from those of their teachers. Weak and Thirdly, by commenting on the work of peers, (2) group projects. students develop objectivity of judgement (about incorrect conceptions of goals not only influence 8 Enhancing student learning through effective formative feedback The Higher Education Academy Generic Centre - June 2004
11. what students do but also the value of feedback Hence there is a need for strategies that 4. Provides opportunities to close the information. If students do not share (at least in complement written materials and simple verbal gap part) their tutor’s conceptions of assessment explanations. An approach that has proved According to Yorke (2003) two questions might be goals (criteria/standards) then the feedback particularly powerful in clarifying goals and asked regarding external feedback. First, is the information they receive is unlikely to ‘connect’ standards has been to provide students with feedback of the best quality and second, does it (Hounsell, 1997). In this case, it will be difficult ‘exemplars’ of performance (Orsmond, Merry and lead to changes in student behaviour? Many for students to evaluate gaps between required Reiling, 2002) alongside other resources. researchers have focused on the first question and actual performance. Exemplars are effective because they define an but the second is equally important. External objective and valid standard against which One way of clarifying task requirements (goals/ feedback provides an opportunity to close the gap students can compare their work. criteria/standards) is to provide students with in the learning process between the current written documents embodying descriptive learning achievements of the student and the Strategies that have proved effective in statements that externalise assessment goals clarifying criteria, standards and goals goals set by the teacher. If feedback information and the standards that define different levels of therefore include: is not turned into action soon after it is produced achievement. However, many studies have (1) providing better definitions of requirements then this is a missed opportunity. As Boud notes: using carefully constructed criteria sheets and shown that it is difficult to make explicit performance level definitions; The only way to tell if learning results from assessment criteria and standards through (2) providing students with exemplar feedback is for students to make some written documentation or through verbal assignments with attached feedback; kind of response to complete the feedback descriptions in class (Rust, Price and O’Donovan, (3) increasing discussion and reflection about loop (Sadler, 1989). This is one of the criteria and standards in class; most often forgotten aspects of formative 2003). Most criteria for complex tasks are difficult (4) involving students in assessment exercises assessment. Unless students are able to to articulate; they are often ‘tacit’ and where they mark or comment on other use the feedback to produce improved unarticulated in the mind of the teacher. As Yorke students’ work in relation to defined criteria work, through for example, re-doing the notes: and standards; same assignment, neither they nor those (5) workshops where students in collaboration giving the feedback will know that it has Statements of expected standards, with their teacher devise their own assessment been effective (Boud, 2000, p158). curriculum objectives or learning outcomes criteria for a piece of work; are generally insufficient to convey the (6) combinations of the above five have proved In the conceptual model (figure 1), Boud’s richness of meaning that is wrapped up in particularly effective. arguments about closing the gap can be viewed them (Yorke, 2003, p480). June 2004 - The Higher Education Academy Generic Centre Enhancing student learning through effective formative feedback 9
12. in two ways. First, closing the gap is about gap, especially in the case of planned supporting students while engaged in the act of assignments. Invariably they move on to the next production of a piece of work. Second, it is about assessment task soon after feedback is received. providing opportunities to repeat the same ‘task- While not all work can be resubmitted, many Specific strategies to help students use external feedback to close the gap are: performance-feedback cycle’ by, for example, writers argue that resubmissions should play a (1) to increase the number of opportunities allowing resubmission. External feedback more prominent role in learning (Boud, 2000). In for resubmission; should support both processes: it should help addition, the external feedback provided to (2) for teachers to model the strategies that students to recognise the next steps in learning students often focuses on identifying specific might be used to close a performance gap in class (for example, model how to and how to take them both during production and errors rather than providing constructive advice structure an essay when given a new for the next assignment. about how performance relates to standards and question); about how to make improvements in subsequent (3) teachers might also write down some Supporting the act of production requires the ‘action points’ alongside the normal tasks; and even when corrective guidance about generation of concurrent or intrinsic feedback that feedback they provide. This would identify how to improve is given, students often do not for students what they should do next time students can interact with while engaged in an fully understand it or know how to turn it into to improve their performance; assessment task. This feedback would normally action. (4) a more effective strategy might be to be built into the task (a group task with peer involve students in identifying their own interaction is an example here) or the task might action points in class based on the be broken down into components each feedback they have just received. This would integrate the process into the associated with its own feedback. Many forms of teaching and learning situation and involve electronic feedback can be automatically the students more actively in the generated to support task engagement (multiple generation and planned use of feedback. choice, FAQs). Providing feedback at sub-task level is not significantly different from other forms of feedback described in this paper. In HE, most students have little opportunity to use directly the feedback they receive to close the 10 Enhancing student learning through effective formative feedback The Higher Education Academy Generic Centre - June 2004
13. 5. Delivers high quality information to to academic learning but that are more abstract ‘this essay was well-structured . . . However . . .’) students about their learning and difficult to define (strength of argument, for arguing instead that descriptive information about example). performance in relation to defined assessment Another finding from the research is that a great criteria is better received by students and is more deal of external feedback given to students is not Students might also receive too much feedback, likely to be acted upon. of good quality: it may be delayed, not relevant or making it difficult to decide what to act on. In the informative, or overwhelming in quantity, and so literature on essay assessment, researchers It has become common practice in recent years on. Good quality external feedback is defined as have tried to formulate guidelines regarding the to provide feedback sheets with assessment information that helps students trouble-shoot their quantity and tone of feedback comments. For criteria as a way of informing students about task own performance and take action to close the example, Lunsford (1997) has advocated requirements and of providing consistent gap between intent and effect. In the model providing only three well thought out feedback feedback in relation to expected goals. However, (figure 1) processes internal to the student comments per essay. Moreover, these the construction of such feedback sheets does (shown by the dotted line) are strongly influenced comments should indicate to the student how the not always encourage students to engage with a by contextual factors in the environment over reader experienced the essay as it was read – task in a way desired by teachers. Sadler (1983) which the teacher has considerable control. The ‘playing back’ to the students how the essay has argued that the use of such criteria sheets teacher sets the task, assesses performance and worked – rather than offering judgemental often has unwanted effects. For example, if there provides feedback. Research shows that in each comments. Such comments help the student to are a large number of criteria (12–20) they may of these areas there is considerable scope for understand the difference between his or her convey a conception of an assessment task (an improvement. intentions and the effects. Comments should essay, for instance) as a list of things to be done always be written in a non-authoritative tone and (‘ticked off’) rather than a holistic process – Feedback needs to be relevant to the task in where possible, they should offer corrective something involving the production of a coherent hand and to student needs. Despite this, advice (both about the writing process as well as argument supported by evidence. So as well as research shows that feedback information is often about content) instead of just information about being responsive to student needs, teachers about strengths and weaknesses in handed-in strengths and weaknesses. should also consider whether the instruments work or about aspects of performance that are they use to deliver feedback are commensurate easy to identify (such as spelling mistakes) rather Other researchers have argued against following with the expected goals and task requirements. than about aspects that are of greater importance positive comments with lists of criticisms (such as June 2004 - The Higher Education Academy Generic Centre Enhancing student learning through effective formative feedback 11
14. 6. Encourages positive motivational feedback comments alone improved students’ Strategies that increase the quality of beliefs and self-esteem subsequent interest in learning and performance feedback drawn from research when compared with controlled situations where include: How can we make assessment a positive marks alone or feedback and marks were given. (1) making sure that feedback is learning experience for students? A key feature of Butler argued that students paid less attention to provided in relation to pre-defined the model of feedback (figure 1) presented in this criteria but paying particular attention the comments when given marks and paper is the importance attached to motivational to the number of criteria; consequently did not try to use the comments to (2) providing feedback soon after a beliefs and self-esteem. In the model, students make improvements. submission; construct their own motivation based on their (3) providing corrective advice, not appraisal of the teaching, learning and Butler (1987) has also argued that grading just information on strengths/ assessment context. This influences the goals student performance has less effect than weaknesses; (4) limiting the amount of feedback that students set (personal and academic) as well feedback comments because it leads students to so that it is used; as their commitment to these goals. However, compare themselves against others (ego- (5) prioritising areas for research has shown that external feedback can involvement) rather than to focus on the improvement; have a positive or negative effect on motivational difficulties in the task and on making efforts to (6) providing online tests so that feedback can be accessed anytime, beliefs and on self-esteem. It influences how improve (task-involvement). Feedback given as any place and as many times as students feel about themselves which, in turn, grades has also been shown to have especially students wish; affects what and how they learn. negative effects on the self-esteem of low ability (7) focusing on students with students (Craven, et al., 1991). greatest difficulties. Many studies have shown that, contrary to expectation, frequent high stakes assessment Dweck (2000) has interpreted some of these (where marks or grades are given) can lower the findings in terms of a developmental model that motivation to learn (Harlen and Crick, 2003). differentiates students into those who believe that Such assessments encourage students to focus ability is fixed and that there is a limit to what they on performance goals (passing the test) rather can achieve (the ‘entity view’) and those that than learning goals (Elliott and Dweck, 1988). In believe that their ability is malleable and depends one study, Butler (1988) demonstrated that on the effort that is input into a task (the 12 Enhancing student learning through effective formative feedback The Higher Education Academy Generic Centre - June 2004
15. ‘incremental view’). These views affect how 7. Provides information to teachers students respond to learning difficulties. Those that can be used to help shape the with an entity view (fixed) interpret failure as a The implication of these studies for teaching reflection of their low ability and are likely to give teaching practice is that motivation and self-esteem are more likely to be enhanced Good feedback practice is not only about up whereas those with an incremental view when a course has many low-stakes tasks providing good information to the students about (malleable) interpret this as a challenge or an with feedback geared to providing learning – it is also about providing good obstacle to be overcome. information about progress and achievement rather than high stakes information to teachers. As Yorke notes: These motivational beliefs, however, are not summative assessment tasks where The act of assessing has an effect on the immutable. In part, they depend on how teachers information is only about success or failure assessor as well as the student. or about how students compare with peers. Assessors learn about the extent to which provide feedback. Praising effort and strategic behaviours and focusing students on learning they [students] have developed expertise Other strategies that would help encourage and can tailor their teaching accordingly goals leads to higher achievement than praising high levels of motivation to succeed (Yorke, 2003, p482). ability or intelligence which can result in a include: (1) providing marks on written work only In order to produce feedback that is relevant and learned-helplessness orientation. In summary, after students have responded to feedback ‘feedback which draws attention away from the informative teachers themselves need good data comments; task and towards self-esteem can have a (2) allocating time for students to re-write about how students are progressing. They also negative effect on attitudes and performance’ selected pieces of work – this would help need to be involved in reviewing and reflecting on change students’ expectations about this data and in taking action to help close the (Black and Wiliam, 1998, p23). purpose; learning gap. (3) automated testing with feedback; (4) drafts and resubmissions. In the conceptual model (figure 1) information about students is provided when the learning outcomes are translated into public performances. Teachers generate this public information about students through a variety of methods – by setting assessment tasks and in June 2004 - The Higher Education Academy Generic Centre Enhancing student learning through effective formative feedback 13
16. class, through questioning of students and through observation. Such information helps teachers uncover student difficulties with subject A variety of strategies are available to matter (conceptual misunderstandings, for teachers to help generate and collate example) and difficulties with study methods quality information about student learning and help them decide how to use it. For while carrying out assessment tasks. example: (1) one-minute papers where students Frequent assessment tasks, especially diagnostic carry out a small assessment task and tests, can help teachers generate cumulative hand this in anonymously at the end of a information about students’ levels of class, such as... understanding and skill so that they can adapt What was the main point of this lecture? What question remains outstanding for you their teaching accordingly. This is one of the key at the end of this teaching session?; ideas behind the work of Angelo and Cross (2) having students request the feedback (1990) in the United States. They have shown they would like when they make an how teachers can gain regular feedback assignment submission; (3) having students identify where they are information about student learning within large having difficulties when they hand in classes by using short test-feedback cycles. assessed work; These strategies benefit both the student and the (4) asking students in groups to identify ‘a teacher (Steadman, 1998) and they can be question worth asking’, based on prior study, that they would like to explore for a adapted to any classroom situation or discipline. short time at the beginning of the next Moreover, implementation allows teachers and tutorial; students to share, on a regular basis their (5) quick evaluation strategies at key points conceptions about both the goals and processes in teaching. of learning (Stefani and Nicol, 1997). 14 Enhancing student learning through effective formative feedback The Higher Education Academy Generic Centre - June 2004
17. 3 The case studies The website http://www.ltsn.ac.uk/genericcentre/ senlef contains all of the case studies collected as part of this project, some 42 in total. This publication contains a small selection of case studies to illustrate the type and range included and the sort of information contained within each case study. The eight case studies selected cover a variety of institutional types and a range of disciplines. In total, they offer examples of all the seven principles being applied in practice. June 2004 - The Higher Education Academy Generic Centre Enhancing student learning through effective formative feedback 15
18. Case study 1 – Feedback as deviations from a ‘sound standard’ Abstract Discipline/course/subject area: Before each assessment, students are provided with ‘sound standards’ – descriptors against which to Personal Development Planning, Level 2 measure their work – and exemplars of work that goes beyond the standard expected and work that falls below. Staff explain where the exemplars deviate from the ‘sound standard’ to help students understand Institution: University of the Highlands and assessment criteria and formatively self-assess their own work. Subsequently, students get feedback on Islands Millennium Institute (UHIMI) and how their own work deviates form the ‘sound standard’ in the same way and can move on to self-assess Heriot-Watt University. their own work using the process. Start date: Term 2, 2002 Description of implementation Impact: The practice was introduced: within a course unit/module. What was the rationale for introducing the practice? The practice has been adopted by: other A previous study showed that students did not understand assessment criteria and wanted to know (and institutions. use) the reasoning behind judgements. Number of students affected: around 50 How was the practice implemented? in 2001-2. Not used by John Cowan this The feedback process works as follows: year but taken up by Dr Elisabet Weedon and Professor Ray McAleese (see below). 1. Select 4–6 headings under which work should be judged. 2. Describe for each heading what ‘sound standard’ work (valued at 55% if 40% is a pass and 70% a Contact: Professor John Cowan, distinction) would look like. Avoid value words like ‘adequate’ and ‘sound’ – rather, describe LEARN Unit, Perth College, Crieff Road, ‘adequacy’ and ‘soundness’. Perth PH1 2NX 3. At the outset, give students ‘sound standard’ descriptors and two sample pieces of work – one John.Cowan@hw.ac.uk better than the ‘sound standard’, one poorer. Explain the deviations from the descriptor that would raise the rating of one, and lower that of the other. Others involved: Dr Elisabet Weedon, Social Science, University of the Highlands 4. Provide feedback on students’ submitted work similarly – thus assuming it meets the ‘sound and Islands Millennium Institute (UHIMI) standard’ descriptors. and Professor Ray McAleese, Combined 5. Encourage student transition to self-assessment based on staff descriptors. Studies, Heriot-Watt University. 6. Encourage student transition to self-assessment based on headings and descriptors formulated by students themselves, in accordance with module goals. 16 Enhancing student learning through effective formative feedback The Higher Education Academy Generic Centre - June 2004
19. Case study 1 What resources were needed? Perceived benefits Very little additional resource is needed to implement this practice. For students… • The making of judgements is much Enablers that help/helped the practice to work more transparent and informative. • Improved performance. • Getting advice on where to enhance the process – asking students individually to help make the • Knowing what is expected of them and mechanism even more effective. seeing how to get there. • I didn’t ask permission to use the process – which created no problems since the students found it • The process makes the formative helpful from the outset. assessment more focused. For teaching/support staff… Points of advice • Our framework for resolving differences is more explicit and • Detach it from marking – to be more qualitative/formative feedback. objective. • Suspend disbelief: try it once, on coursework. I offer that encouragement with caution, as I’m not an evangelist – just an improver of my own practices and my students’ learning experiences. Issues/challenges Possible improvements/enhancements (suggested by the case study provider) For students… • They are confused initially by the ‘criteria’ – until you call them the ‘headings’ under which judgements are External commentary (related to the feedback principles) made. • It takes a while before some students This case demonstrates clearly what good performance is (goals, criteria and expected standards) – appreciate that less than the ‘sound Principle 3 – as well as facilitating the development of self-assessment in learning (P1) (enabling the standard’ doesn’t mean a fail. students to develop the capacity to self-regulate performance) and closing the gap between current and For teaching/support staff… desired learning outcome (P4). • Eschewing subjectivity: being explicit Possible improvements could include introducing appropriate coaching or training in devising criteria and about criteria and standards. in the practice of self-assessment. • Throwing aside habits familiar to us since we ourselves were learners. June 2004 - The Higher Education Academy Generic Centre Enhancing student learning through effective formative feedback 17
20. Case study 2 – Combining formative and summative assessment on a continuous basis Abstract Discipline/course/subject Students have the opportunity to sit a sequence of formative tests to prepare them for both the content and area: Nursing and Midwifery, format of exam situations. This builds students’ confidence, allows for ongoing practice and timely feedback DipHE/BSc Life Sciences and raises confidence levels. Institution: Bell College, Hamilton Description of implementation In what context does the feedback practice happen? Start date: 1998 We are involved in the teaching of the Life Sciences module on the Nursing and Midwifery course. Impact: The practice was introduced: within a course What was the rationale for introducing the practice? unit/module The practice was adopted Essentially, prior experience of students’ expectations. They want to know where they did well and/or went by: the department wrong. Rightly so! Number of students affected: around 550 How was the practice implemented? For a number of years we have been assessing the students by giving a series of tests. There are three class Contact: Jim Dick, Biological tests scheduled at various times during the duration of the module. The tests, ‘smallish bite-sized exams’, and Chemical Sciences, Bell contain the following types of questions: short answer, multiple choice, and labelling diagrams, and are carried College, Almada Street, out under exam conditions (see appendix, available on www.ltsn.ac.uk/genericcentre/senlef). It is expected Hamilton, ML3 0JB, that students complete the tests in around half an hour. Each test carries 25 marks. To pass the module the J.Dick@bell.ac.uk student must attempt all three tests and achieve a mark of ten or above in each test. The final grade awarded is not graded but simply credited as an S (satisfactory) or U (unsatisfactory). Others involved: Hugh Watson, John Larcombe, After each test the papers are marked and quickly returned to students during tutorial/lab sessions where Andrew MacKenzie – correct answers are given and topics causing concern, or where students have performed poorly, are Biological and Chemical explored. Second diet (re-sits) follow a week or two after the feedback sessions. The advantages of these Sciences and Health Studies. feedback sessions are enormous for students and staff alike and are summarised below. 18 Enhancing student learning through effective formative feedback The Higher Education Academy Generic Centre - June 2004
21. Case study 2 If the first attempt at a test is failed (less than 10/25) or missed, a second Possible improvements/enhancements (suggested by the case attempt (with different questions of course) will be offered to the student study provider) within the duration of the module. If a third attempt is required students must contact a Life Sciences lecturer for one-to-one feedback and to Not applicable arrange a third attempt. What resources were needed? External commentary (related to the feedback principles) Time is needed to carry out feedback sessions, and be prepared to adjust and alter teaching methods and/or vocabulary to make things clearer for This case study illustrates the use of feedback to: students. In addition, assessments need to be modified in light of findings. a) clarify what good performance is – goals, criteria and expected standards – (P3); and b) respond sensitively to learners’ needs in terms of timing, quantity, Enablers that help/helped the practice to work quality and individual differences and thus deliver high quality information to students about their learning (P5). • Having adequate staffing. Possible improvements could include automating and delivering the tests • Keeping groups small during feedback sessions. online so they can be easily accessed, taken anytime, any place and as many times as the students wish. Automation also has the advantages of: • Refinement of test questions resulting from a greater understanding by staff of student vocabulary and interpretation of notes, etc. (i) using a diverse range of question types which may be more interesting and motivating to the students and, (ii) reducing assessment workload (in terms of marking) for the teachers. Points of advice • Ensure anonymity for students during feedback sessions, such as marks written inside test papers and not on covers. • Possibly works well in field of Biological Sciences where many answers are ‘facts’– they are either right or wrong and all students sit same test paper. For obvious reasons, it could be a nightmare if applied in areas such as psychology or sociology where individual feedback could be required for each student! June 2004 - The Higher Education Academy Generic Centre Enhancing student learning through effective formative feedback 19
22. Case study 2 Perceived benefits Issues/challenges For students… For students… • Students now receive ongoing feedback on performance – this Some students appear anxious and open to scrutiny from their peer group if clearly encourages most of them. they think they have not performed well in a test. Given student numbers • Where a re-sit is required they have the opportunity to see where involved it is impossible to give feedback on a one-to-one basis in private. they went wrong and to discuss it with a member of staff prior to re-sit. For teaching/support staff… • Students can question why some of their answers were • The process can be time consuming and occasionally confrontational if unacceptable and this helps to identify areas of difficulty. a student is not happy with their results. • Feedback of this nature opens up the assessment process for students. They can see that there is no mystery surrounding • Occasionally students expect similar practices in other subject areas. assessment. Sometimes this expectation can cause a bit of friction between staff! • Students can see assessment criteria clearly and are aware that no preferential treatment is given to any student. • Conflict between ‘systems’ operated at college level and module level. For example, according to college regulations, exam scripts should not For teaching/support staff… show the name of the student but their matriculation number to ensure • There is constant quality assurance of test questions and marking anonymity and impartiality of marking. Given we need a rapid turn schemes from a student’s point of view (often more valid than round of scripts (often the day after the test) it is impossible to operate lecturers/examiners/moderators because we are so familiar with such a system based on matriculation numbers. Students are asked to the subject that there are times we assume too much put their names on the script to allow us to record marks and sort out comprehension and prior knowledge). scripts into the appropriate lab groups thus allowing us to return the • As a follow on from the above, wording of questions/diagrams marked scripts quickly. Too much bureaucracy would make the used are constantly being modified to ‘appeal’ to the student – not process impossible. Impartiality of marking is met by the means the lecturer. (Often we assume a greater comprehension of fine already described above. points of grammar, or breadth of vocabulary, than is realistic for the average student.) • Lack of time! • Students and staff build up a rapport that improves communication in all situations. (For example, they will ask questions in lectures!) 20 Enhancing student learning through effective formative feedback The Higher Education Academy Generic Centre - June 2004
23. Case study 3 – Feedback in interactive lectures using an electronic voting system Abstract Discipline/course/subject An electronic voting system/personal response system (PRS) allows a whole class to contribute an anonymous area: Various vote to any multiple choice question (MCQ) the lecturer offers, with immediate feedback of the aggregated class responses (how many voted for each alternative answer). This can be used in any way expressible by MCQs, all Institution: University of of which increase interactivity in lectures for all audience sizes. Feedback to the lecturer is as important as to the Glasgow students, and can be used to adapt the session on the spot and on the fly to the needs of that audience. Impact: The practice was Description of implementation introduced: across a faculty/ In what context does the feedback practice happen? school/group of departments. In lectures: we have used it in class sizes from 15 to 300, in first year and fourth year classes, in departments The practice was adopted by: across the university from philosophy to biology, psychology to computing science. the department, other What was the rationale for introducing the practice? departments in the institution and in other institutions. The biggest weakness of typical teaching at this university, relative to Laurillard’s theoretical model, is the emphasis on lectures where there is a paucity of to and fro interaction between learners and teachers. This Number of students affected: technology addresses this weak point in a generic way that can in principle help in every subject. Hard to say – growing all the How was the practice implemented? time! We obtained funding and purchased enough equipment for our two largest lecture theatres simultaneously, and Contact: Steve Draper, Room thus could offer a mobile service so that users would not have to change their teaching rooms. This follows the 524, Department of important approach of subordinating technology to the pedagogical aims. Advertising to all university staff Psychology, 58 Hillhead Street, recruited some lecturers who immediately could imagine a beneficial application. We supplied both equipment Glasgow, G12 8QB and technical assistance (in setting up the equipment on the day, and operating it if requested), so as to free s.draper@psy.gla.ac.uk lecturers to concentrate on managing the occasion and obtaining the desired pedagogic benefits. Pedagogical www.psy.gla.ac.uk/~steve/ suggestions about ways of using it are available on our extensive web pages, but usually client lecturers had a specific idea about how to use it when they approached us rather than seeking oral consultation about Others involved: Many! – see pedagogical methods (as opposed to technical and practical details) beyond our written material. web pages www.psy.gla.ac.uk/ What resources were needed? ~steve/ilig/ • The voting equipment: we spent £17,500 for enough equipment to cover our two largest lecture theatres simultaneously (650 students at once). • Lecturers have to design the questions, and adapt their lectures to use them. At its easiest, you can add a few self-assessment questions in a few minutes’ work to an existing lecture. Designing brainteasers, June 2004 - The Higher Education Academy Generic Centre Enhancing student learning through effective formative feedback 21
+1-315-636-4485
+91 98151-74050 
