Student learning through effective formative feedback

Contributed by:
Steve
This publication includes a small selection of the case studies, the theoretical model and the seven principles, and the workshop plans for using the materials. The project team explored feedback issues with higher education institutions (HEIs) across Scotland.
1. The Higher Education Academy
Enhancing student learning through
effective formative feedback
Charles Juwah, Debra Macfarlane-Dick, Bob Matthew,
David Nicol, David Ross and Brenda Smith
2. About the authors
Charles Juwah is Senior Educational Bob Matthew, a civil engineer by discipline, is David Ross is a chemist by discipline and has
Development Officer, and Deputy Head, at the now director of the Teaching and Learning been at Abertay for nineteen years, the last nine
Centre for the Enhancement of Learning and Service at the University of Glasgow having of which as Director of the Centre for the
Teaching (CELT), The Robert Gordon University, previously taught civil engineering in Ireland and Enhancement in Learning and Teaching. An
Aberdeen. His responsibilities include academic England. In his present post he has responsibility experienced quality assurance assessor, he has
and professional development, quality for supporting the enhancement of learning, been involved in the strategic development of the
enhancement and educational research. His teaching and assessment within the university. new enhancement-led model of quality
current research interest focuses on the He is also associate dean (research) within the assurance in Scotland, and was also Convenor
promotion and enhancement of the quality of faculty of education at Glasgow. He is currently of the Universities Scotland Educational
learning, online learning, assessment and the chair of Universities Scotland Educational Development Sub-Committee from 2001-2003.
feedback practices, postgraduate research Development Committee and a member of He is currently Chairman of the Universities
supervision and the use of portfolios and Universities Scotland Learning and Teaching Scotland/QAA Joint Working Group on
personal development planning in enhancing committee. In his spare time he is passionate Implementing PDP (ELF Project). His research
learning and skills development. about growing rare varieties of potatoes, interests are in Change Management in Quality
Japanese maples, and climbing Scottish Enhancement, Enhanced Learning Frameworks
Debra Macfarlane-Dick, the Student Enhanced mountains. and Student Assessment Feedback.
Learning through Effective Feedback (SENLEF)
Project Officer, is an educational developer at the David Nicol is an educational consultant within Brenda Smith is Head of the LTSN Generic
University of Glasgow. She joined the Teaching the Centre for Academic Practice, University of Centre (now Higher Education Academy) which
and Learning Service as a University Teacher in Strathclyde. He works with academic is dedicated to developing enhanced teaching,
2002 having previously worked in training and departments/faculties on educational assessment and learning practice in the UK HE
development. In 2003, she moved into her improvement projects in teaching, learning and sector. This involves working with 24 Subject
current role as Employability Development assessment in both online and face-to-face Centres, and working at strategic level with key
Adviser in the Careers Service at Glasgow. In her environments. Current projects include change agents in universities and other key
current role she has responsibility for supporting supporting the implementation of an institution- national network groups such as faculty
the implementation of the University’s wide virtual learning environment (VLE), developers, quality agencies and the UK funding
employability strategy by working collaboratively evaluating a JISC/NSF-funded digital libraries in bodies. She has acted as a consultant and has
with staff, students and employers to encourage the classroom initiative, and work on strategic facilitated workshops in many different countries
and develop initiatives and learning materials to change in relation to e-learning. Recent including South Africa, the Philippines, Colombia,
enhance student employability. Outside work, research publications have focused on the social Azerbaijan, Lithuania, the West Indies and
Debra enjoys art (as an art historian by dimensions of e-learning, learning objects, Malaysia. She has written widely on learning and
discipline), keeping tropical fish and gardening. electronic classroom feedback systems, shared teaching, including work on Research, Teaching
workspaces in learning and on risk and cost- and Learning, Assessment, Resource Based
benefit analysis in relation to e-learning. Learning and Lecturing to Large Groups.
3. Contents
1. Introduction 2
2. Rethinking formative assessment in HE: a theoretical model and
seven principles of good feedback practice 3
3. Case studies 15
Case study 1 – Feedback as deviations from a ‘sound standard’ 16
Case study 2 – Combining formative and summative assessment on 18
a continuous basis
Case study 3 – Feedback in interactive lectures using an electronic 21
voting system
Case study 4 – Enhancing effectiveness and efficiency in student 23
feedback
Case study 5 – Using immediate feedback in class: the New 25
Approaches to Teaching and Learning in Engineering (NATALIE)
Project
Case study 6 – Self and peer-assessment of written work in English 28
Literature
Case study 7 – Portfolios and feedback: enhancing self-assessment 31
Case study 8 – Work to win: formative feedback on demand 33
4. Plans for possible workshops 36
5. References 38
6. Acknowledgements 41
June 2004 - The Higher Education Academy Generic Centre Enhancing student learning through effective formative feedback 1
4. 1 Introduction
Student Enhanced Learning through Effective This publication includes a small selection of the The project team hopes that these might be
Feedback (SENLEF) was a project funded by the case studies, the theoretical model and the seven useful to both educational developers and
LTSN Generic Centre (now Higher Education principles, and the workshop plans for using the academics when they are attempting to enhance
Academy) to develop a resource for practitioners materials. their practice in the area of giving learners
wishing to improve their feedback practice to effective feedback.
It is worthwhile reiterating the seven principles of
students or get some new ideas on how to
good feedback practice that we have identified.
enhance their current practice.
These are listed below.
The idea for the project came from the
Universities Scotland Educational Development 1. Facilitates the development of self-
Committee. The project team explored feedback assessment (reflection) in learning.
issues with higher education institutions (HEIs)
2. Encourages teacher and peer dialogue
across Scotland. The outcomes from the project around learning.
are available on the Higher Education Academy
3. Helps clarify what good performance is
Generic Centre web site at
(goals, criteria, expected standards).
and include the following: 4. Provides opportunities to close the gap
between current and desired performance.
• A series of case studies 5. Delivers high quality information to
students about their learning.
• A theoretical model
6. Encourages positive motivational beliefs
• Seven principles for good effective practice and self-esteem.
• A questionnaire for others to contribute 7. Provides information to teachers that can
further case studies be used to help shape the teaching.
• Workshop plans for using the materials. These principles are explained in greater detail in
the next section.
2 Enhancing student learning through effective formative feedback The Higher Education Academy Generic Centre - June 2004
5. 2 Rethinking formative assessment in HE: aprinciples
theoretical model and seven
of good feedback practice
Dr David Nicol, University of Strathclyde not the sole province of the teacher. Peers often self-esteem. A number of writers have argued
Debra Macfarlane-Dick, University of Glasgow provide feedback – for example in group-work that feedback is under-conceptualised in the
contexts – and students generate their own theoretical literature in HE and elsewhere, and
This section explores how higher education
feedback while engaging in and producing that this makes it difficult to design effective
institutions might use assessment more
academic work (see below). Formative feedback practices or to evaluate their
effectively to promote student learning.
assessment also provides information to teachers effectiveness (Yorke, 2003; Sadler, 1998). While
Assessment provides a framework for sharing
about where students are experiencing difficulties there has been a move over the last decade to
educational objectives with students and for
and where to focus their teaching efforts. conceptualise ‘learning’ from a constructivist
charting their progress. However, it can generate
perspective (Laurillard, 2002, for example),
feedback information that can be used by This section summarises the research on
approaches to feedback have, until recently,
students to enhance learning and achievement. formative assessment and feedback. It includes
remained obstinately focused on simple
This feedback information can also help teachers the following:
‘transmission’ perspectives. Teachers ‘transmit’
realign their teaching in response to learners’
• A conceptual model of the formative feedback messages to students about strengths
needs. When assessment serves these purposes
assessment/ feedback cycle and weaknesses in their work assuming that
it is called ‘formative assessment’. It is argued
these messages are easily decoded and turned
that formative assessment should be an integral • Seven principles of good feedback
practice: these are drawn from the model into action. In contrast, in this paper, students are
part of teaching and learning in HE and that
and a review of the research literature assumed to construct actively their own
‘feedback’ and ‘feed-forward’ should be
understanding of feedback messages from tutors.
systematically embedded in curriculum practices. • Some examples of good practice
Moreover, these messages are assumed to be
strategies related to each principle.
Formative assessment aids learning by complex and difficult to decipher (Higgins, Hartley
generating feedback information that is of benefit There are two central arguments within this and Skelton, 2001; Ivanic, Clark and
to students and to teachers. Feedback on section (i) that formative assessment and Rimmershaw, 2000).
performance, in class or on assignments, enables feedback should be used to empower students as
The conceptual model and the seven principles
students to restructure their understanding/skills self-regulated learners and (ii) that more
presented in this paper are intended as tools that
and build more powerful ideas and capabilities. recognition should be given to the role of
teachers might use to analyse and improve their
However, the provision of feedback information is feedback on learners’ motivational beliefs and
own formative assessment and feedback
practices.
June 2004 - The Higher Education Academy Generic Centre Enhancing student learning through effective formative feedback 3
6. A conceptual model teachers give students feedback information on possess some of the same evaluative skills as
(b) – that is, how their performance compares to their teacher. For many writers, this observation
In a review article, Black and Wiliam (1998) drew
the standard – but that this feedback often falls has led to the conclusion that as well as focusing
together over 250 studies of formative
short of what is actually necessary to help on the quality of the feedback messages,
assessment with feedback carried out since 1988
students close the gap. For example, such teachers should focus their efforts on
spanning all educational sectors. The studies
information might be difficult to understand (such strengthening the skills of self-assessment in
that formed part of their meta-analysis were
as a comment that ‘this essay is not sufficiently their students (Yorke, 2003; Boud, 2000).
ecologically valid in that they were drawn from
analytical’) and especially if the learning goal (a)
real teaching situations. Black and Wiliam’s Figure 1 presents a conceptual model of
has not been fully assimilated in the first place.
analysis of these studies showed that feedback formative assessment and feedback that
Black and Wiliam (1998) further elaborate on this
resulted in positive benefits on learning and synthesises current thinking by key researchers
communication issue when they discuss the links
achievement across all content areas, knowledge into this topic (Sadler, 1983, 1989; Black and
between the way a feedback message is received
and skill types and levels of education. One of Wiliam, 1998; Yorke, 2003; Torrance and Pryor,
and what students do with that message.
the most influential papers underpinning the 1998). The figure is based on a model of
Black and Wiliam review, and the writings of other ...those factors which influence the feedback and self-regulated learning originally
researchers, is that by Sadler (1989). Sadler reception of a [feedback] message and the
published by Butler and Winne (1995). A key
personal decision about how to
identified three conditions necessary for students respond…[include]….beliefs about the feature in the model that differentiates it from
to benefit from feedback. The student must: goals of learning, about one’s capacity to commonplace understandings of feedback is that
respond, about the risks involved in the student is assumed to occupy a central and
• Possess a concept of the goal/standard or responding in various ways and about what
active role in all feedback processes. They are
reference level being aimed for learning should be like (p21).
always actively involved in monitoring and
• Compare the actual (or current) level of Any model of feedback must take account of the regulating their own performance both in terms of
performance with that goal or standard way students make sense of, and use, feedback their goals and in terms of the strategies being
• Engage in appropriate action which leads information. More importantly, however, is used to reach those goals.
to some closure of the gap. Sadler’s argument that for students to be able to
In the model, an academic task set by the
compare actual performance with a standard, and
Sadler argued that in many educational settings teacher (in class or set as an assignment) is the
take action to close the gap, they must already
starting point for the feedback cycle.
4 Enhancing student learning through effective formative feedback The Higher Education Academy Generic Centre - June 2004
7. Engagement with the task requires that students
draw on prior knowledge and motivational beliefs
and construct a personal interpretation of the
requirements and properties of the task. Based
on this internal conception, they formulate their
own task goals (which may be different from
those of the teacher) and engage in actions to
achieve these goals by applying tactics and
strategies that generate outcomes. Monitoring
these interactions with the task and the outcomes
that are being cumulatively produced, generates
internal feedback.
This feedback is derived from a comparison of
current progress against internal goals or
standards – gaps are identified (between
progress and goals) and further actions are taken
to close these gaps (Sadler, 1989). This self- and the path of learning (Butler and Winne, feedback information from external sources. The
generated feedback information might lead to a 1995). teacher’s feedback response (based on their
re-interpretation of the task or to the adjustment monitoring and assessment of student
In the model, external feedback to the student
of internal goals or of tactics and strategies. performance) must be interpreted and
might be provided by teachers, peers or others
Students might even revise their domain internalised by the student before it can influence
(placement supervisor, for example). However,
knowledge or beliefs which, in turn, would subsequent action (Ivanic, Clark and
students are always actively engaged in feedback
influence subsequent processes of self- Rimmershaw, 2000). This has important
processes. First, they generate aspects of their
regulation. If external feedback is provided, this implications for feedback processes in HE. If
own feedback as they monitor performance and
additional information might augment, concur or students are always involved in monitoring and
identify and make sense of gaps while carrying
conflict with the student’s interpretation of the task assessing their own work, then rather than just
out tasks. Second, they interpret and filter
June 2004 - The Higher Education Academy Generic Centre Enhancing student learning through effective formative feedback 5
8. thinking of ways of enhancing the teacher’s ability 7. Provides information to teachers that can ability to self-assess and self-correct.
be used to help shape the teaching.
to deliver high quality feedback we should be
In the conceptual model, the student or learner is
devising ways of building upon this capacity for The following sections provide the rationale for
always engaged in monitoring gaps between
self-regulation (Yorke, 2003). each principle in terms of the conceptual model
internally set task and personal goals and the
and the associated research literature. Brief
outcomes that are being progressively produced.
Seven principles of good feedback examples of how these principles might be
This monitoring is a by-product of purposeful
practice applied are also suggested.
engagement in a task. However, in order to build
on this process, and the student’s capacity for
From the conceptual model and the research
literature on formative assessment it is possible
1. Facilitates the development of self-regulation, teachers should create more
to identify some broad principles of good
self-assessment in learning formal and structured opportunities for self-
monitoring and the judging of progression to
feedback practice. A provisional list might include Over the last decade there has been an
goals. Self-assessment tasks are a good way of
the following seven. increasing interest in strategies that encourage
doing this, as are activities that encourage
1. Facilitates the development of self- students to take a more active role in the
reflection on both the processes and the products
assessment (reflection) in learning. management of their own learning (see Nicol,
of learning.
1997). Black and Wiliam (1998) make the
2. Encourages teacher and peer dialogue
around learning. argument that ‘a student who automatically Research shows that direct involvement by
follows the diagnostic prescription of a teacher students in assessing their own work, and
3. Helps clarify what good performance is
without understanding of its purpose will not learn’ frequent opportunities to reflect on goals,
(goals, criteria, standards expected).
(p54) while Sadler (1989) argues that the purpose strategies and outcomes are highly effective in
4. Provides opportunities to close the gap of formative assessment should be to equip enhancing learning and achievement (McDonald
between current and desired performance.
students gradually with the evaluative skills that and Boud, 2003). Moreover, if the skills of self-
5. Delivers high quality information to their teachers’ possess. These writers are assessment are developed progressively over the
students about their learning. concerned that an over-emphasis on teacher course of an undergraduate degree this would
6. Encourages positive motivational beliefs assessment might increase students’ support a model of higher education where
and self-esteem. dependency on others rather than develop their students are prepared for lifelong learning (Boud,
2000).
6 Enhancing student learning through effective formative feedback The Higher Education Academy Generic Centre - June 2004
9. An important aspect of self-assessment involves 2. Encourages teacher and peer student must play in constructing meaning from
helping students both to identify standards/criteria dialogue around learning feedback messages.
that apply to their work and to make judgements
While research shows that teachers have a One way of increasing the effectiveness of
about how their work relates to these standards
central role in helping a develop student’s own external feedback and the likelihood that the
(Boud, 1986).
capacity for self-assessment in learning, external information provided is understood is to
feedback from other sources (such as tutors or conceptualise feedback more as a dialogue
peers) is also crucial. Feedback from tutors and rather than as information transmission.
Examples of structured reflection
peers provides additional information that helps Feedback as dialogue means that the student not
and/or self-assessment are varied
and might include students: challenge students to reassess their knowledge only receives initial feedback information but also
(1) requesting the kinds of feedback and beliefs. Teacher feedback also serves as an has the opportunity to engage the teacher in
they would like when they hand in authoritative external reference point against discussion about that feedback. This is shown in
work;
which students can evaluate, and self-correct the conceptual model by the two-way arrows that
(2) identifying the strengths and
weaknesses in their own work in their progress and their own internal goals. link external processes to those internal to the
relation to criteria or standards student. The idea that feedback encourages
before handing it in for teacher In the conceptual model (figure 1), for external
dialogue is considered good practice by many
feedback; feedback to be effective it must be understood
writers on assessment. For example, Freeman
(3) reflecting on their achievements and internalised by the student before it can be
and selecting work in order to and Lewis (1998) argue that the teacher ‘should
used productively. Yet in the research literature
compile a portfolio; try to stimulate a response and a continuing
(4) setting achievement milestones (Chanock, 2000; Hyland, 2000) there is a great
dialogue – whether this be on the topics that
for a task and reflecting back on deal of evidence that students do not understand
formed the basis of the assignment or aspects of
progress and forward to the next the feedback given by tutors (for instance, ‘this
stage of action; students’ performance or the feedback itself’
report is not logically structured’) and are
(5) having students give feedback on (p51). Discussions with the teacher help students
each other’s work (peer feedback) therefore not able to take action to close the gap
to develop their understanding of expectations
also helps support the development (that is, he or she may not know what to do to
and standards, to check out and correct
of self-assessment skills (for make the report more ‘logical in structure’).
example, Gibbs, 1999). misunderstandings and to get an immediate
External feedback as a transmission process
response to difficulties.
involving ‘telling’ ignores the active role the
June 2004 - The Higher Education Academy Generic Centre Enhancing student learning through effective formative feedback 7
10. Unfortunately, with large class sizes it can be work in relation to standards) which can be
3. Helps clarify what good
difficult for the teacher to engage in dialogue with transferred to the assessment of their own work
performance is
students. Nonetheless, there are ways that (‘I didn’t do that either’, for example). Fourthly,
teachers might increase feedback dialogue even peer discussion can be motivational in that it Students can only achieve a learning goal if they
in these situations. For example, by reporting encourages students to persist and gives a understand that goal, assume some ownership of
feedback in class and structuring break out yardstick to measure their own performance it, and can assess progress (Sadler, 1989; Black
discussions of feedback or by using classroom against (see Nicol and Boyle, 2003). Finally, it is and Wiliam, 1998). In the model (figure 1),
technologies that collate student responses in sometimes easier for students to accept critiques understanding the goal means that there must be
class and then feed the results back visually as a of their work from peers rather than tutors. a reasonable degree of overlap between the task
histogram. This feedback can act as a trigger for goal set by the student and the goal originally set
teacher-managed discussion (for example, Nicol Good examples of feedback dialogue in class by the teacher. However, there is considerable
and Boyle, 2003). include: research evidence to suggest that there are often
(1) providing feedback using one-minute mismatches between tutors’ and students’
Another source of external feedback are the papers (Angelo and Cross, 1990); conceptions of goals and of assessment
students themselves. Peer dialogue is beneficial (2) reviewing feedback in tutorials where
students are asked to read the feedback standards and criteria.
to student learning in a variety of ways. First,
comments they have been given and discuss
students who have just learned something are Hounsell (1997) has shown that tutors and
these with peers – they might also be asked
often better able than teachers to explain it to to suggest strategies to improve performance students often have quite different conceptions
their classmates in a language and in a way that next time; about the goals and criteria for essays in
is accessible. Second, peer discussion exposes (3) asking students to find one or two undergraduate courses in history and psychology
examples of feedback comments that they
students to alternative perspectives on problems and that poor essay performance is correlated
found useful and to explain how they helped.
and to alternative tactics and strategies. with the degree of mismatch. In a similar vein,
Other ways of using feedback dialogue in a
Alternative perspectives enable students to revise planned way, for assignments, might involve: Norton (1990) has shown that when students
or reject their initial hypothesis and construct new (1) having students give each other were asked to rank specific assessment criteria
knowledge and meaning through negotiation. descriptive feedback on their work in relation for an essay task they produced quite different
to published criteria before submission; rankings from those of their teachers. Weak and
Thirdly, by commenting on the work of peers,
(2) group projects.
students develop objectivity of judgement (about incorrect conceptions of goals not only influence
8 Enhancing student learning through effective formative feedback The Higher Education Academy Generic Centre - June 2004
11. what students do but also the value of feedback Hence there is a need for strategies that 4. Provides opportunities to close the
information. If students do not share (at least in complement written materials and simple verbal gap
part) their tutor’s conceptions of assessment explanations. An approach that has proved
According to Yorke (2003) two questions might be
goals (criteria/standards) then the feedback particularly powerful in clarifying goals and
asked regarding external feedback. First, is the
information they receive is unlikely to ‘connect’ standards has been to provide students with
feedback of the best quality and second, does it
(Hounsell, 1997). In this case, it will be difficult ‘exemplars’ of performance (Orsmond, Merry and
lead to changes in student behaviour? Many
for students to evaluate gaps between required Reiling, 2002) alongside other resources.
researchers have focused on the first question
and actual performance. Exemplars are effective because they define an
but the second is equally important. External
objective and valid standard against which
One way of clarifying task requirements (goals/ feedback provides an opportunity to close the gap
students can compare their work.
criteria/standards) is to provide students with in the learning process between the current
written documents embodying descriptive learning achievements of the student and the
Strategies that have proved effective in
statements that externalise assessment goals clarifying criteria, standards and goals goals set by the teacher. If feedback information
and the standards that define different levels of therefore include: is not turned into action soon after it is produced
achievement. However, many studies have (1) providing better definitions of requirements then this is a missed opportunity. As Boud notes:
using carefully constructed criteria sheets and
shown that it is difficult to make explicit
performance level definitions; The only way to tell if learning results from
assessment criteria and standards through (2) providing students with exemplar feedback is for students to make some
written documentation or through verbal assignments with attached feedback; kind of response to complete the feedback
descriptions in class (Rust, Price and O’Donovan, (3) increasing discussion and reflection about loop (Sadler, 1989). This is one of the
criteria and standards in class; most often forgotten aspects of formative
2003). Most criteria for complex tasks are difficult
(4) involving students in assessment exercises assessment. Unless students are able to
to articulate; they are often ‘tacit’ and where they mark or comment on other use the feedback to produce improved
unarticulated in the mind of the teacher. As Yorke students’ work in relation to defined criteria work, through for example, re-doing the
notes: and standards; same assignment, neither they nor those
(5) workshops where students in collaboration giving the feedback will know that it has
Statements of expected standards, with their teacher devise their own assessment been effective (Boud, 2000, p158).
curriculum objectives or learning outcomes criteria for a piece of work;
are generally insufficient to convey the (6) combinations of the above five have proved In the conceptual model (figure 1), Boud’s
richness of meaning that is wrapped up in particularly effective. arguments about closing the gap can be viewed
them (Yorke, 2003, p480).
June 2004 - The Higher Education Academy Generic Centre Enhancing student learning through effective formative feedback 9
12. in two ways. First, closing the gap is about gap, especially in the case of planned
supporting students while engaged in the act of assignments. Invariably they move on to the next
production of a piece of work. Second, it is about assessment task soon after feedback is received.
providing opportunities to repeat the same ‘task- While not all work can be resubmitted, many Specific strategies to help students use
external feedback to close the gap are:
performance-feedback cycle’ by, for example, writers argue that resubmissions should play a (1) to increase the number of opportunities
allowing resubmission. External feedback more prominent role in learning (Boud, 2000). In for resubmission;
should support both processes: it should help addition, the external feedback provided to (2) for teachers to model the strategies that
students to recognise the next steps in learning students often focuses on identifying specific might be used to close a performance gap
in class (for example, model how to
and how to take them both during production and errors rather than providing constructive advice structure an essay when given a new
for the next assignment. about how performance relates to standards and question);
about how to make improvements in subsequent (3) teachers might also write down some
Supporting the act of production requires the ‘action points’ alongside the normal
tasks; and even when corrective guidance about
generation of concurrent or intrinsic feedback that feedback they provide. This would identify
how to improve is given, students often do not for students what they should do next time
students can interact with while engaged in an
fully understand it or know how to turn it into to improve their performance;
assessment task. This feedback would normally
action. (4) a more effective strategy might be to
be built into the task (a group task with peer involve students in identifying their own
interaction is an example here) or the task might action points in class based on the
be broken down into components each feedback they have just received. This
would integrate the process into the
associated with its own feedback. Many forms of
teaching and learning situation and involve
electronic feedback can be automatically the students more actively in the
generated to support task engagement (multiple generation and planned use of feedback.
choice, FAQs). Providing feedback at sub-task
level is not significantly different from other forms
of feedback described in this paper.
In HE, most students have little opportunity to use
directly the feedback they receive to close the
10 Enhancing student learning through effective formative feedback The Higher Education Academy Generic Centre - June 2004
13. 5. Delivers high quality information to to academic learning but that are more abstract ‘this essay was well-structured . . . However . . .’)
students about their learning and difficult to define (strength of argument, for arguing instead that descriptive information about
example). performance in relation to defined assessment
Another finding from the research is that a great
criteria is better received by students and is more
deal of external feedback given to students is not Students might also receive too much feedback,
likely to be acted upon.
of good quality: it may be delayed, not relevant or making it difficult to decide what to act on. In the
informative, or overwhelming in quantity, and so literature on essay assessment, researchers It has become common practice in recent years
on. Good quality external feedback is defined as have tried to formulate guidelines regarding the to provide feedback sheets with assessment
information that helps students trouble-shoot their quantity and tone of feedback comments. For criteria as a way of informing students about task
own performance and take action to close the example, Lunsford (1997) has advocated requirements and of providing consistent
gap between intent and effect. In the model providing only three well thought out feedback feedback in relation to expected goals. However,
(figure 1) processes internal to the student comments per essay. Moreover, these the construction of such feedback sheets does
(shown by the dotted line) are strongly influenced comments should indicate to the student how the not always encourage students to engage with a
by contextual factors in the environment over reader experienced the essay as it was read – task in a way desired by teachers. Sadler (1983)
which the teacher has considerable control. The ‘playing back’ to the students how the essay has argued that the use of such criteria sheets
teacher sets the task, assesses performance and worked – rather than offering judgemental often has unwanted effects. For example, if there
provides feedback. Research shows that in each comments. Such comments help the student to are a large number of criteria (12–20) they may
of these areas there is considerable scope for understand the difference between his or her convey a conception of an assessment task (an
improvement. intentions and the effects. Comments should essay, for instance) as a list of things to be done
always be written in a non-authoritative tone and (‘ticked off’) rather than a holistic process –
Feedback needs to be relevant to the task in
where possible, they should offer corrective something involving the production of a coherent
hand and to student needs. Despite this,
advice (both about the writing process as well as argument supported by evidence. So as well as
research shows that feedback information is often
about content) instead of just information about being responsive to student needs, teachers
about strengths and weaknesses in handed-in
strengths and weaknesses. should also consider whether the instruments
work or about aspects of performance that are
they use to deliver feedback are commensurate
easy to identify (such as spelling mistakes) rather Other researchers have argued against following
with the expected goals and task requirements.
than about aspects that are of greater importance positive comments with lists of criticisms (such as
June 2004 - The Higher Education Academy Generic Centre Enhancing student learning through effective formative feedback 11
14. 6. Encourages positive motivational feedback comments alone improved students’
Strategies that increase the quality of beliefs and self-esteem subsequent interest in learning and performance
feedback drawn from research when compared with controlled situations where
include: How can we make assessment a positive
marks alone or feedback and marks were given.
(1) making sure that feedback is learning experience for students? A key feature of
Butler argued that students paid less attention to
provided in relation to pre-defined the model of feedback (figure 1) presented in this
criteria but paying particular attention the comments when given marks and
paper is the importance attached to motivational
to the number of criteria; consequently did not try to use the comments to
(2) providing feedback soon after a beliefs and self-esteem. In the model, students
make improvements.
submission; construct their own motivation based on their
(3) providing corrective advice, not appraisal of the teaching, learning and Butler (1987) has also argued that grading
just information on strengths/
assessment context. This influences the goals student performance has less effect than
weaknesses;
(4) limiting the amount of feedback that students set (personal and academic) as well feedback comments because it leads students to
so that it is used; as their commitment to these goals. However, compare themselves against others (ego-
(5) prioritising areas for research has shown that external feedback can involvement) rather than to focus on the
improvement;
have a positive or negative effect on motivational difficulties in the task and on making efforts to
(6) providing online tests so that
feedback can be accessed anytime, beliefs and on self-esteem. It influences how improve (task-involvement). Feedback given as
any place and as many times as students feel about themselves which, in turn, grades has also been shown to have especially
students wish; affects what and how they learn. negative effects on the self-esteem of low ability
(7) focusing on students with
students (Craven, et al., 1991).
greatest difficulties. Many studies have shown that, contrary to
expectation, frequent high stakes assessment Dweck (2000) has interpreted some of these
(where marks or grades are given) can lower the findings in terms of a developmental model that
motivation to learn (Harlen and Crick, 2003). differentiates students into those who believe that
Such assessments encourage students to focus ability is fixed and that there is a limit to what they
on performance goals (passing the test) rather can achieve (the ‘entity view’) and those that
than learning goals (Elliott and Dweck, 1988). In believe that their ability is malleable and depends
one study, Butler (1988) demonstrated that on the effort that is input into a task (the
12 Enhancing student learning through effective formative feedback The Higher Education Academy Generic Centre - June 2004
15. ‘incremental view’). These views affect how 7. Provides information to teachers
students respond to learning difficulties. Those that can be used to help shape the
with an entity view (fixed) interpret failure as a The implication of these studies for teaching
reflection of their low ability and are likely to give teaching practice is that motivation and
self-esteem are more likely to be enhanced Good feedback practice is not only about
up whereas those with an incremental view
when a course has many low-stakes tasks providing good information to the students about
(malleable) interpret this as a challenge or an with feedback geared to providing learning – it is also about providing good
obstacle to be overcome. information about progress and
achievement rather than high stakes information to teachers. As Yorke notes:
These motivational beliefs, however, are not summative assessment tasks where The act of assessing has an effect on the
immutable. In part, they depend on how teachers information is only about success or failure assessor as well as the student.
or about how students compare with peers. Assessors learn about the extent to which
provide feedback. Praising effort and strategic
behaviours and focusing students on learning they [students] have developed expertise
Other strategies that would help encourage and can tailor their teaching accordingly
goals leads to higher achievement than praising high levels of motivation to succeed (Yorke, 2003, p482).
ability or intelligence which can result in a include:
(1) providing marks on written work only In order to produce feedback that is relevant and
learned-helplessness orientation. In summary,
after students have responded to feedback
‘feedback which draws attention away from the informative teachers themselves need good data
comments;
task and towards self-esteem can have a (2) allocating time for students to re-write about how students are progressing. They also
negative effect on attitudes and performance’ selected pieces of work – this would help need to be involved in reviewing and reflecting on
change students’ expectations about this data and in taking action to help close the
(Black and Wiliam, 1998, p23).
purpose;
learning gap.
(3) automated testing with feedback;
(4) drafts and resubmissions.
In the conceptual model (figure 1) information
about students is provided when the learning
outcomes are translated into public
performances. Teachers generate this public
information about students through a variety of
methods – by setting assessment tasks and in
June 2004 - The Higher Education Academy Generic Centre Enhancing student learning through effective formative feedback 13
16. class, through questioning of students and
through observation. Such information helps
teachers uncover student difficulties with subject A variety of strategies are available to
matter (conceptual misunderstandings, for teachers to help generate and collate
example) and difficulties with study methods quality information about student learning
and help them decide how to use it. For
while carrying out assessment tasks. example:
(1) one-minute papers where students
Frequent assessment tasks, especially diagnostic
carry out a small assessment task and
tests, can help teachers generate cumulative hand this in anonymously at the end of a
information about students’ levels of class, such as...
understanding and skill so that they can adapt What was the main point of this lecture?
What question remains outstanding for you
their teaching accordingly. This is one of the key
at the end of this teaching session?;
ideas behind the work of Angelo and Cross (2) having students request the feedback
(1990) in the United States. They have shown they would like when they make an
how teachers can gain regular feedback assignment submission;
(3) having students identify where they are
information about student learning within large
having difficulties when they hand in
classes by using short test-feedback cycles. assessed work;
These strategies benefit both the student and the (4) asking students in groups to identify ‘a
teacher (Steadman, 1998) and they can be question worth asking’, based on prior
study, that they would like to explore for a
adapted to any classroom situation or discipline.
short time at the beginning of the next
Moreover, implementation allows teachers and tutorial;
students to share, on a regular basis their (5) quick evaluation strategies at key points
conceptions about both the goals and processes in teaching.
of learning (Stefani and Nicol, 1997).
14 Enhancing student learning through effective formative feedback The Higher Education Academy Generic Centre - June 2004
17. 3 The case studies
The website http://www.ltsn.ac.uk/genericcentre/
senlef contains all of the case studies collected
as part of this project, some 42 in total. This
publication contains a small selection of case
studies to illustrate the type and range included
and the sort of information contained within each
case study.
The eight case studies selected cover a variety of
institutional types and a range of disciplines. In
total, they offer examples of all the seven
principles being applied in practice.
June 2004 - The Higher Education Academy Generic Centre Enhancing student learning through effective formative feedback 15
18. Case study 1 – Feedback as deviations from a ‘sound standard’
Abstract
Discipline/course/subject area:
Before each assessment, students are provided with ‘sound standards’ – descriptors against which to
Personal Development Planning, Level 2
measure their work – and exemplars of work that goes beyond the standard expected and work that falls
below. Staff explain where the exemplars deviate from the ‘sound standard’ to help students understand
Institution: University of the Highlands and
assessment criteria and formatively self-assess their own work. Subsequently, students get feedback on
Islands Millennium Institute (UHIMI) and
how their own work deviates form the ‘sound standard’ in the same way and can move on to self-assess
Heriot-Watt University.
their own work using the process.
Start date: Term 2, 2002
Description of implementation
Impact: The practice was introduced:
within a course unit/module. What was the rationale for introducing the practice?
The practice has been adopted by: other
A previous study showed that students did not understand assessment criteria and wanted to know (and
institutions.
use) the reasoning behind judgements.
Number of students affected: around 50 How was the practice implemented?
in 2001-2. Not used by John Cowan this
The feedback process works as follows:
year but taken up by Dr Elisabet Weedon
and Professor Ray McAleese (see below). 1. Select 4–6 headings under which work should be judged.
2. Describe for each heading what ‘sound standard’ work (valued at 55% if 40% is a pass and 70% a
Contact: Professor John Cowan, distinction) would look like. Avoid value words like ‘adequate’ and ‘sound’ – rather, describe
LEARN Unit, Perth College, Crieff Road, ‘adequacy’ and ‘soundness’.
Perth PH1 2NX 3. At the outset, give students ‘sound standard’ descriptors and two sample pieces of work – one
[email protected] better than the ‘sound standard’, one poorer. Explain the deviations from the descriptor that would
raise the rating of one, and lower that of the other.
Others involved: Dr Elisabet Weedon,
Social Science, University of the Highlands 4. Provide feedback on students’ submitted work similarly – thus assuming it meets the ‘sound
and Islands Millennium Institute (UHIMI) standard’ descriptors.
and Professor Ray McAleese, Combined 5. Encourage student transition to self-assessment based on staff descriptors.
Studies, Heriot-Watt University. 6. Encourage student transition to self-assessment based on headings and descriptors formulated by
students themselves, in accordance with module goals.
16 Enhancing student learning through effective formative feedback The Higher Education Academy Generic Centre - June 2004
19. Case study 1
What resources were needed?
Perceived benefits
Very little additional resource is needed to implement this practice.
For students…
• The making of judgements is much
Enablers that help/helped the practice to work more transparent and informative.
• Improved performance.
• Getting advice on where to enhance the process – asking students individually to help make the • Knowing what is expected of them and
mechanism even more effective. seeing how to get there.
• I didn’t ask permission to use the process – which created no problems since the students found it • The process makes the formative
helpful from the outset. assessment more focused.
For teaching/support staff…
Points of advice • Our framework for resolving
differences is more explicit and
• Detach it from marking – to be more qualitative/formative feedback. objective.
• Suspend disbelief: try it once, on coursework. I offer that encouragement with caution, as I’m not an
evangelist – just an improver of my own practices and my students’ learning experiences.
Issues/challenges
Possible improvements/enhancements (suggested by the case study provider) For students…
• They are confused initially by the
‘criteria’ – until you call them the
‘headings’ under which judgements are
External commentary (related to the feedback principles) made.
• It takes a while before some students
This case demonstrates clearly what good performance is (goals, criteria and expected standards) – appreciate that less than the ‘sound
Principle 3 – as well as facilitating the development of self-assessment in learning (P1) (enabling the standard’ doesn’t mean a fail.
students to develop the capacity to self-regulate performance) and closing the gap between current and
For teaching/support staff…
desired learning outcome (P4).
• Eschewing subjectivity: being explicit
Possible improvements could include introducing appropriate coaching or training in devising criteria and about criteria and standards.
in the practice of self-assessment. • Throwing aside habits familiar to us
since we ourselves were learners.
June 2004 - The Higher Education Academy Generic Centre Enhancing student learning through effective formative feedback 17
20. Case study 2 – Combining formative and summative assessment on a continuous basis
Abstract
Discipline/course/subject
Students have the opportunity to sit a sequence of formative tests to prepare them for both the content and
area: Nursing and Midwifery,
format of exam situations. This builds students’ confidence, allows for ongoing practice and timely feedback
DipHE/BSc Life Sciences
and raises confidence levels.
Institution: Bell College,
Hamilton Description of implementation
In what context does the feedback practice happen?
Start date: 1998
We are involved in the teaching of the Life Sciences module on the Nursing and Midwifery course.
Impact: The practice was
introduced: within a course What was the rationale for introducing the practice?
unit/module
The practice was adopted Essentially, prior experience of students’ expectations. They want to know where they did well and/or went
by: the department wrong. Rightly so!
Number of students
affected: around 550 How was the practice implemented?
For a number of years we have been assessing the students by giving a series of tests. There are three class
Contact: Jim Dick, Biological tests scheduled at various times during the duration of the module. The tests, ‘smallish bite-sized exams’,
and Chemical Sciences, Bell contain the following types of questions: short answer, multiple choice, and labelling diagrams, and are carried
College, Almada Street, out under exam conditions (see appendix, available on www.ltsn.ac.uk/genericcentre/senlef). It is expected
Hamilton, ML3 0JB, that students complete the tests in around half an hour. Each test carries 25 marks. To pass the module the
[email protected] student must attempt all three tests and achieve a mark of ten or above in each test. The final grade awarded
is not graded but simply credited as an S (satisfactory) or U (unsatisfactory).
Others involved: Hugh
Watson, John Larcombe, After each test the papers are marked and quickly returned to students during tutorial/lab sessions where
Andrew MacKenzie – correct answers are given and topics causing concern, or where students have performed poorly, are
Biological and Chemical explored. Second diet (re-sits) follow a week or two after the feedback sessions. The advantages of these
Sciences and Health Studies. feedback sessions are enormous for students and staff alike and are summarised below.
18 Enhancing student learning through effective formative feedback The Higher Education Academy Generic Centre - June 2004
21. Case study 2
If the first attempt at a test is failed (less than 10/25) or missed, a second Possible improvements/enhancements (suggested by the case
attempt (with different questions of course) will be offered to the student study provider)
within the duration of the module. If a third attempt is required students
must contact a Life Sciences lecturer for one-to-one feedback and to Not applicable
arrange a third attempt.
What resources were needed? External commentary (related to the feedback principles)
Time is needed to carry out feedback sessions, and be prepared to adjust
and alter teaching methods and/or vocabulary to make things clearer for This case study illustrates the use of feedback to:
students. In addition, assessments need to be modified in light of findings. a) clarify what good performance is – goals, criteria and expected
standards – (P3); and
b) respond sensitively to learners’ needs in terms of timing, quantity,
Enablers that help/helped the practice to work quality and individual differences and thus deliver high quality
information to students about their learning (P5).
• Having adequate staffing.
Possible improvements could include automating and delivering the tests
• Keeping groups small during feedback sessions. online so they can be easily accessed, taken anytime, any place and as
many times as the students wish. Automation also has the advantages of:
• Refinement of test questions resulting from a greater understanding
by staff of student vocabulary and interpretation of notes, etc. (i) using a diverse range of question types which may be more interesting
and motivating to the students and,
(ii) reducing assessment workload (in terms of marking) for the teachers.
Points of advice
• Ensure anonymity for students during feedback sessions, such as
marks written inside test papers and not on covers.
• Possibly works well in field of Biological Sciences where many
answers are ‘facts’– they are either right or wrong and all students
sit same test paper. For obvious reasons, it could be a nightmare if
applied in areas such as psychology or sociology where individual
feedback could be required for each student!
June 2004 - The Higher Education Academy Generic Centre Enhancing student learning through effective formative feedback 19
22. Case study 2
Perceived benefits Issues/challenges
For students… For students…
• Students now receive ongoing feedback on performance – this Some students appear anxious and open to scrutiny from their peer group if
clearly encourages most of them. they think they have not performed well in a test. Given student numbers
• Where a re-sit is required they have the opportunity to see where involved it is impossible to give feedback on a one-to-one basis in private.
they went wrong and to discuss it with a member of staff prior to
re-sit. For teaching/support staff…
• Students can question why some of their answers were • The process can be time consuming and occasionally confrontational if
unacceptable and this helps to identify areas of difficulty. a student is not happy with their results.
• Feedback of this nature opens up the assessment process for
students. They can see that there is no mystery surrounding • Occasionally students expect similar practices in other subject areas.
assessment. Sometimes this expectation can cause a bit of friction between staff!
• Students can see assessment criteria clearly and are aware that
no preferential treatment is given to any student. • Conflict between ‘systems’ operated at college level and module level.
For example, according to college regulations, exam scripts should not
For teaching/support staff… show the name of the student but their matriculation number to ensure
• There is constant quality assurance of test questions and marking anonymity and impartiality of marking. Given we need a rapid turn
schemes from a student’s point of view (often more valid than round of scripts (often the day after the test) it is impossible to operate
lecturers/examiners/moderators because we are so familiar with such a system based on matriculation numbers. Students are asked to
the subject that there are times we assume too much put their names on the script to allow us to record marks and sort out
comprehension and prior knowledge). scripts into the appropriate lab groups thus allowing us to return the
• As a follow on from the above, wording of questions/diagrams marked scripts quickly. Too much bureaucracy would make the
used are constantly being modified to ‘appeal’ to the student – not process impossible. Impartiality of marking is met by the means
the lecturer. (Often we assume a greater comprehension of fine already described above.
points of grammar, or breadth of vocabulary, than is realistic for the
average student.) • Lack of time!
• Students and staff build up a rapport that improves communication
in all situations. (For example, they will ask questions in lectures!)
20 Enhancing student learning through effective formative feedback The Higher Education Academy Generic Centre - June 2004
23. Case study 3 – Feedback in interactive lectures using an electronic voting system
Abstract
Discipline/course/subject An electronic voting system/personal response system (PRS) allows a whole class to contribute an anonymous
area: Various vote to any multiple choice question (MCQ) the lecturer offers, with immediate feedback of the aggregated class
responses (how many voted for each alternative answer). This can be used in any way expressible by MCQs, all
Institution: University of of which increase interactivity in lectures for all audience sizes. Feedback to the lecturer is as important as to the
Glasgow students, and can be used to adapt the session on the spot and on the fly to the needs of that audience.
Impact: The practice was Description of implementation
introduced: across a faculty/ In what context does the feedback practice happen?
school/group of departments.
In lectures: we have used it in class sizes from 15 to 300, in first year and fourth year classes, in departments
The practice was adopted by: across the university from philosophy to biology, psychology to computing science.
the department, other What was the rationale for introducing the practice?
departments in the institution
and in other institutions. The biggest weakness of typical teaching at this university, relative to Laurillard’s theoretical model, is the
emphasis on lectures where there is a paucity of to and fro interaction between learners and teachers. This
Number of students affected: technology addresses this weak point in a generic way that can in principle help in every subject.
Hard to say – growing all the How was the practice implemented?
time!
We obtained funding and purchased enough equipment for our two largest lecture theatres simultaneously, and
Contact: Steve Draper, Room thus could offer a mobile service so that users would not have to change their teaching rooms. This follows the
524, Department of important approach of subordinating technology to the pedagogical aims. Advertising to all university staff
Psychology, 58 Hillhead Street, recruited some lecturers who immediately could imagine a beneficial application. We supplied both equipment
Glasgow, G12 8QB and technical assistance (in setting up the equipment on the day, and operating it if requested), so as to free
[email protected] lecturers to concentrate on managing the occasion and obtaining the desired pedagogic benefits. Pedagogical
www.psy.gla.ac.uk/~steve/ suggestions about ways of using it are available on our extensive web pages, but usually client lecturers had a
specific idea about how to use it when they approached us rather than seeking oral consultation about
Others involved: Many! – see pedagogical methods (as opposed to technical and practical details) beyond our written material.
web pages www.psy.gla.ac.uk/ What resources were needed?
~steve/ilig/ • The voting equipment: we spent £17,500 for enough equipment to cover our two largest lecture theatres
simultaneously (650 students at once).
• Lecturers have to design the questions, and adapt their lectures to use them. At its easiest, you can add a
few self-assessment questions in a few minutes’ work to an existing lecture. Designing brainteasers,
June 2004 - The Higher Education Academy Generic Centre Enhancing student learning through effective formative feedback 21