Letters

Letters

The line between 'good' and 'over'-preparation

A recent undercover investigation for BBC Radio Five criticised a former examiner for providing private courses to language teachers on how to improve the GCSE oral exam grades of their least able students. The course provider advocated teaching scripted phrases so students who did not understand questions could pick up some marks. The acting Chief Executive of Ofqual, Isabel Nisbet, and Schools Minister, Jim Knight, stated this training of candidates was unacceptable, but Nisbet said there is tension and uncertainty between good preparation and over-preparation.

Given the high-stakes nature of public exams and the continuity required in exam format, question style and topics covered, it is inevitable that, to some degree, teachers will 'teach to the test' and seek every possible advantage for students. The 2005 Education and Skills White Paper highlighted that teaching to the test and formulaic responses were issues for GCE exams. In response, awarding bodies have developed new-style GCEs for fi rst teaching in 2009 that are intended to stretch and challenge students and provide increased opportunity for the assessment of higher order skills.

A QCA-funded study on Stretch and Challenge recently explored current teaching practices at GCE and the potential for backwash on the teaching and learning of the new assessments. Teachers and students described using intense exam preparation strategies, which were often engaging,innovative and models of good practice. Although most believed the educational aims of the new GCEs to be sound, it was recognised that the new assessments may change the way that teachers and students prepare for exams and that this could be quite daunting. As one research participant put it, 'No one wants to take any risks with their exam results'. AQA is planning to use the outcomes of this research to inform the development of support initiatives for teachers and students.

 

Suzanne Chamberlain, Anthony Daly and Michelle Meadows, AQA, and Jo-Anne Baird, The University of Bristol

 

REVIEW

 

Reflection and application

Unlocking assessment
Edited by: Sue Swaffield
Publisher: David Fulton
Price: £19.99
Ratings (out of five): usefulness (4), relevance (4), readability (5)

 

The most recent title in Routledge's Unlocking series presents a collection of essays on the purpose and application of assessment. This volume brings together an impressive array of authors including Margaret Carr, who co-directed New Zealand's famous 'Te Whariki' early years curriculum, Mary-Jane Drummond and the mathematician Jeremy Hodgen of King's College London.

There are three aims: to discuss the nature and purpose of assessment, to refl ect upon the value judgements involved in how we decide what we want to assess, and to look at how educational theory can inform our assessment practice. Unlocking Assessment provides a valuable overview of the issues underpinning educational assessment today and tentatively looks at what will undoubtedly be some of the issues for tomorrow; namely, trusting teachers' judgements and understanding how technology will support learning and assessment. The essays afford many clear pauses for refl ection on a wide variety of current issues while remaining firmly child-centred and rigorous in their demands.

As a secondary school teacher working primarily with students with special educational needs, I found this book discussed a number of refreshing views, for example the idea that assessment must enhance children's sense of themselves as capable learners and should refl ect the holistic way in which they learn.

The essays are written by professionals with years of experience in assessment and as teachers. The rich style is nonetheless accessible and I thoroughly enjoyed this book.

Martin Edmonds is a secondary school teacher working primarily with SEN and disaffected students

 

Read all about it ! At Make the Grade, we are keen to involve CIEA members and to air their opinions. If you would like to review the latest professional publications, please email your contact details to makethegrade@redactive.co.uk

 

Make the Grade welcomes letters.
Please include your address, job title and telephone number. We regret that we are unable to print letters from sources we cannot verify. By email: makethegrade@redactive.co.uk By post:
The Editor
Make the Grade
Redactive Media Group
17 Britton Street
London EC1M 5TP

 

British assessment research under scrutiny

Head to head:

Jo-Anne Baird

Let's fly the flag

There is an acute gap between innovative and pedestrian research, according to the latest report from the education sub-panel of the Research Assessment Exercise (RAE).
Its 2008 report said there was a lack of assessment research presented under the RAE, given the importance of this topic for the country's education system. Much high-quality research is applied, not theoretical, and is published in the grey literature, if at all. The 2008 RAE criteria, upon which researchers and their institutions were judged and fi nanced, valued peerreviewed publications in high-impact international academic journals. Its rankings ranged from 'unclassifi ed' ("quality that falls below the standard of nationally recognised work. Or work that does not meet the published defi nition  of research for the purposes of this assessment") through to 'four star' ("quality that is world-leading in terms of originality, signifi cance and rigour").

But we also produce high-quality, policy-relevant research that would be acclaimed nationally, rather than internationally, for which the RAE would award 'one star' ("quality that is recognised nationally in terms of originality, signifi cance and rigour").

Due to differences in educational practice, lots of research conducted in Britain might not be readily translated for an international audience. The fact that we assess using extended answer formats as the norm, and assume you need teaching experience to mark examinations, are an anathema in other cultural contexts. This may well be the reason that the RAE Education panel awarded fewer 'four stars' than other subject panels. (This has resulted in HEFCE awarding universities fi ve per cent less funding for education in this RAE.)

Structural factors also affect the international reach of British assessment research. Sixty per cent of education
research funding in universities comes from Government sources and this figure is likely to be higher in assessment research. Government-funded, policy and practice-relevant research tends to be short-term, as the policy context moves rapidly. Policy-makers want answers to new questions 'yesterday' so they can move on apace with their reforms. The result of this can be, at best, high-quality, highly specialised research that is not always well-linked with the wider literature. This has informed critical decisions in recent examination developments, such as the grading of the Diplomas, or the introduction of single level tests. But extra work is required to embed this research in the academic literature and link it to theory, or make explicit how it advances knowledge in a way that is not pedestrian. The Government also owns the intellectual property and might not agree to publication of politically sensitive findings.

We should recognise that a lot of British assessment research is valuable

Although rarely explicitly recognised as such, awarding bodies are another major assessment research funder - employing teams of researchers. Again, only some is published, as their main aim is to improve the quality of educational assessments, rather than to disseminate their fi ndings. Awarding bodies provide a valuable public service in this regard, without which, lots of policy decisions would have to be taken without evidence.

So, does the future look bright for high-quality educational research? Not necessarily. Assessment research is a niche area and it is hard to fi ll job vacancies. Routes for developing a research career in this fi eld are fewer now. Teachers are taught very little assessment theory and practice.

The CIEA's efforts to galvanize Masters degrees in assessment in universities should help. I am not advocating complacency but we should recognise that a lot of British educational assessment research is valuable - both internationally and nationally.

 

About the author
Jo-Anne Baird is reader in Educational Assessment at the University of Bristol

 

Jannette Elwood

Could do better?

 

Jannette Elwood asks if research in educational assessment needs to improve

The educational assessment systems across the UK are in a state of flux. There are significant local changes in relation to 7-14 year olds across Northern Ireland, England, Wales and Scotland as well as changes occurring across qualifi cation provision for 14-19 year olds. Such developments in assessment systems and structures, as well key education policy changes both nationally (such as Diplomas, removal of national curriculum testing at KS3 and formative assessment programmes) and internationally, have brought research on assessment and testing once more into key policy arenas, but also brought into focus debates about its general quality.

Research in education has been criticised as being at best variable in quality, and at worst not worth the money spent on it by government and other significant funders. The criticisms focused
on the quality of certain approaches to qualitative research and the increasing lack of quantitative research in the field.

These criticisms have been keenly rejected by the educational research community and the panel of users of educational research in this last research assessment exercise (RAE 2008), outlined the signifi cant contribution that educational research makes to policy and practice. However, there has been some acknowledgement that research into core areas of education such as assessment (at all levels, classroom learning and effective teaching in subject disciplines) may be less robust than in previous years. Indeed the RAE 2008 Sub-Panel report for Education noted with surprise the extent that assessment was 'under-represented in relation to [its] key importance to national education systems' (RAE, 2008, p2). It was suggested that some of this under-representation may refl ect patterns of available expertise within the area of assessment and testing more generally within university education departments. The quality of some of the research in assessment and other key areas submitted for review suffered through being too closely tied to shifting government agency priorities.

We see that expertise resides with researchers working across university departments, in practitioner organisations, (such as the examination boards) research organisations (e.g. NFER) and Government agencies. This situation has resulted in a fragmented range of assessment issues being investigated and a relatively low number of researchers working and gaining expertise in the area. Government agencies who require technical research and evaluation work in assessment to inform policy development are more frequently seeking expertise they need from organisations outside the UK.

This is an academic field lacking in coherence, with researchers isolated in their own area of work

The consequence of this is an academic field lacking in coherence, with researchers isolated in their own area  of work. Different research perspectives on the same assessment issues are rarely exposed or explored in depth between experienced and developing researchers from different theoretical traditions and disciplines in the fi eld. Fragmentation, lack of coherence and lack of developing expertise may well lead to a situation where assessment research will increasingly be carried out by non-experts who see its signifi cance to the policy arena yet lack the substantive background to understand the complexity of assessment practice.

The challenge for assessment researchers is to acknowledge these weaknesses and improve what we do.
We must establish coherence in the fi eld between researchers and practitioners and promote structured careers for new researchers to gain expertise in the area so that all is not lost on a scale that affects the quality of researchers and research in assessment.

 

About the author
Professor Jannette Elwood is director of research clusters: contexts of teaching, learning and assessment at Queen's University Belfast and deputy chair of the Research Committee at AQA