http://www.ahrc.ac.uk/What-We-Do/Build-the-evidence-base/Impact-examples/Documents/Understanding-Your-Project-A-Guide-to-Self-Evaluation.pdf
Information
Grants for the arts – self-evaluation
This
information sheet introduces the idea of self-evaluation for artists and arts
organisations. It provides a brief definition of evaluation, explains why we
think it is important for everyone and suggests some approaches to
self-evaluation. Finally, it lists resources to help with self-evaluation, many
of which are downloadable from the web.
Contents
Evaluation
involves gathering evidence before, during and after a project and using it to
make judgements about what happened. The evidence should prove what happened
and why, and what effect it had. The evidence can also help you to improve what
you are doing during the project and what you do next time (Woolf, 2004).
Evaluation is a valuable tool for learning and involves
critical analysis of your activities. Artists all make evaluative judgements
about their work and evaluation makes the ‘reflective practice of creative work
explicit and conscious’ (Moriarty 2002).
There are clear benefits to you if you evaluate your work.
•
Evaluation helps with planning, as it makes you think about what you’re
aiming to do, how you will do it and how you will know if you’ve succeeded
•
Ongoing feedback keeps you on track and helps to avoid disasters
•
Evaluation helps you to adapt/change as you go along
•
Evaluation is a good way of dealing with ‘quality assurance’ – you’re
keeping an eye on things to make sure quality is maintained
•
Evaluation helps prove the value of what you are doing
•
Evaluation records your contribution to the field you are working in
•
Your evaluation can help others working in the same field
•
Information you collect can also be used for reporting back to those
with an interest in the project (eg participants, funders) and telling others
about what you’ve done
•
The evidence you collect can support future funding applications
You can
put evaluation to work for you or your organisation. Moriarty (2002) is
unequivocal: ‘self-evaluation is a vital part of the discipline imposed in
undertaking creative work’.
We want you to share the results of
your self-evaluations with us. They help us build a clearer picture of current
arts practice and give us a better understanding of arts audiences and
participants. The evidence from your self-evaluations can help us be
better-informed and more effective advocates for the arts. This evidence also
provides us with information for reporting to Government about how we are
spending public money.
We also use your self-evaluations to feed into the
evaluation of our programmes of work – e.g. decibel, Grants for the
arts. Our programme evaluation shows what has been achieved overall as a result
of our funding and helps us to make decisions about what to do next and how. It
also helps us to improve our funding programmes over time. But it would not be
practical to evaluate individual projects, as it would take too much
time and too many resources. So as well as encouraging all those receiving
grants from us to self-evaluate, we also ask them to provide some basic
feedback through forms like the Grants for the arts activity report form. This
ensures that we have information about all the projects to feed into our
programme evaluation.
Rigorous evaluation of our work
enables us to accumulate a collective body of evidence which contributes to the
‘collective practice wisdom of the sector’, building a record of our ‘history
and achievement’ (Arts Victoria, 2002).
If, as artists and arts organisations, you evaluate your
work and share the results, it helps others to contextualise their work,
develop new ideas and learn from what you did. For this reason, being honest
about the results of your evaluation is essential. The participants and funders
of your work, and other artists and arts organisations, need to trust your
evaluation: it should be honest and explain problems and things which weren’t
as successful, as well as showing what the project achieved.
Evaluation takes time and
resources, and you cannot evaluate all projects in the same way and in the same
depth. Evaluation can be more or less formal, and more or less detailed, to
suit your purpose. Evaluating process is as important as evaluating product:
your evaluation should explore the quality and impact of both.
The way you approach evaluation
will depend on the type of project or activity you are doing, and whether you
are an individual or an organisation. For example:
• if you are an artist working on the research and development
of an artistic idea or a new way of working, your evaluation may involve just
you. It could involve others too, such as artists who are working with you on
the project or people outside the project who are giving you feedback on your
work. You will be making artistic judgements about process, materials, form and
content. You will also be making judgements about the results of what you did
and what you have produced
•
if you are an arts organisation developing
an artistic idea, your evaluation will possibly include artists, directors and
other project staff. It will involve artistic judgements and perhaps an
assessment of the impact on your organisation and its audience
•
if you are an arts organisation running a
participatory project, your evaluation will include project staff, including
artists and those participating in the project, whether they are young people, schoolteachers, adults or other
individuals or groups
• if
you are working in partnership with other people, they should always
have the chance to say how they feel about a project through the evaluation. This
applies to everyone, from funders through to participants in a community
project.
You should start thinking about
evaluation at the planning stage of a project. The main focus of your
evaluation will be the planned outcomes (your aims, objectives and targets) and
whether they were achieved. You should also build in checks to ensure that
unplanned outcomes are not missed.
You can organise an evaluation
yourself or you can ask someone else to do it for you. Either way you need to
include it in your budget, as whoever does the evaluation will need to be paid
for their time.
The things you need to sort out
when planning an evaluation include the following.
1.
What kinds of information or evidence you are
going to include in your evaluation – e.g. what people say, what they have done
(process and finished work), what you have done, how an audience has responded.
2.
What questions you are going to ask.
3.
How you plan to answer those questions – this is
really about what sort of information you need to answer the questions and how
you will collect it. Do you need numbers (e.g. 50 people attended 20 workshops)
or information with more depth? Useful evaluation usually combines both types
of information.
4.
When you should collect the information. As a
minimum you need to collect information at the end of your project but if you
can ask questions at the beginning of the project, you will have a ‘before’
picture against which you can look at the ‘after’ picture to assess change.
5.
How you will collect the information. You may
already have some of the information you need, perhaps from previous
evaluations or findings from market research. There are many different ways of
collecting information. e.g. – keeping a register, asking people in a
questionnaire, asking them to video their thoughts about a project, keeping a
diary, taking photographs, etc.
6.
How you are going to make sense of the
information you have collected.
7.
How you are going to present the results of the
evaluation.
8.
Who you are going
to share it with and how.
Moriarty (2002) reminds us that ‘self-evaluation is hard
work and time-consuming. The reward is that it can give us the ability to do
things beyond the best of our present available knowledge’.
We particularly recommend Partnerships
for Learning by Felicity Woolf, which has been revised this year (Arts
Council England, 2004). It will be useful to anyone who organises, funds,
delivers or takes part in participatory arts projects, although its specific
focus is education.
Partnerships for Learning divides
evaluation into five stages: planning, collecting evidence, assembling and
interpreting, reflecting and moving forward, and reporting and sharing. Each
section includes focus questions, a discussion of key issues, examples
illustrating key concepts and a summary checklist. The revised
Partnerships
for learning can be downloaded from the Arts Council website,
www.artscouncil.org.uk, or a hard
copy can be purchased from Marston Book Services Ltd, PO Box 269, Abingdon,
Oxon, OX14 4YN.Telephone: 01235 465500. Fax: 01235 465555.
1.
Scottish Arts Council Evaluation Toolkit (2003).
An interactive, online resource (e-tool) which develops and expands on the
approach to evaluation introduced in Partnerships for Learning. The
e-tool offers ideas and information about conducting each of the five stages of
evaluation, including planning and collecting, interpreting and reporting on
information. The toolkit also shows the theory being put into practice by
following a case-study project through each of stage of the evaluation. The
e-tool is currently being tested and you are invited to give feedback on how useful
you find it.
2.
Ellis, J (2004).
Practical monitoring and
evaluation: a guide for voluntary organisations. London: Charities’
Evaluation Service.
A
comprehensive guide to monitoring and evaluation aimed at small and medium
sized voluntary organisations. You can buy the full
guide
(Basic, Advanced and Toolkit) or the shorter basic set (Basic and Toolkit) from
Charities Evaluation Services, 4 Coldbath Square, London EC1R 5HL. The website
has other downloadable guides to self-evaluation, including Cupitt, S and
Ellis, J (2003), Your project and its outcomes. London: Charities
Evaluation Services and Community Fund. First steps to monitoring and
evaluation is also useful. Go to www.ces-vol.org.uk
and click ‘about monitoring & evaluation’.
3.
Ball, L (2004). The
artist’s development toolkit. An interactive resource, at www.itool.co.uk/Interactive/artdev/1005020101.php
This toolkit provides self-reflective material for artists and for art and
design students. It aims to help users to review their achievements and explore
ways in which they can develop themselves and their practice.
4.
Moriarty, G
(2002). Sharing practice: a guide to self-evaluation in the context of
social exclusion. London: Arts Council of England. This guide focuses on
processes which can be used by arts organisations, individual practitioners and
participants to reflect on and develop their practice. It also discusses why
self-evaluation is important for all artists and arts organisations. Available
at www.newaudiences.org.uk , News,
July 2003.
5.
Arts Victoria
(2002). Evaluating Community Arts and Community Well Being. Available at
www.arts.vic.gov.au , ‘publications’.
This guide presents a complete approach to evaluating community arts work,
using a framework based on process, impact and outcomes. It has practical
guidance on planning, conducting and presenting an evaluation. There are lots
of sample tools, such as sample evaluation indicators, a focus group outline, a
sample participant questionnaire and project journal guidelines.
6.
Voluntary Arts
Network (2003). Monitoring & evaluating arts events: why bother? A
helpful guide to evaluation which offers a manageable approach to evaluation
for community and voluntary arts organisations. Practical and straightforward,
the guide includes some less conventional suggestions for collecting
information from project participants, including young people. Sample
questionnaires and other data collection tools are also included. To download
from the Voluntary Arts Network website go to www.vaw.org.uk
and click ‘publications’.
7.
Walker et al
(2000). Prove it: measuring the effect of neighbourhood renewal on local
people. London: Groundwork, New Economic Foundation and Barclays PLC. This
guide describes how to measure the effect of community projects on local
people, on the relationships between them and on their quality of life.
Explains how to involve local people in setting targets and gathering
information. Can be downloaded at www.neweconomics.org/gen/newways_proveit.aspx
9.
Swords, M (2002) Built-in,
not bolt-on: engaging young people in evaluation and consultation. London:
New Opportunities Fund. This is a report which discusses effective ways of
consulting with young people. It has practical suggestions of how to engage
young people in evaluation. Can be downloaded from www.nof.org.uk – follow links to ‘research and
evaluation’, and then ‘publications’.
10. New Opportunities Fund (2002). Working in
partnership: a sourcebook, This book offers detailed and practical guidance
on working in partnership across many different sectors and includes a section
on self-evaluation. Can be downloaded from www.nof.org.uk
– follow links to research and evaluation, and then publications.
Arts Council England
Grants for the arts
The Hive, 49 Lever Street
Manchester M1 1FN
Phone: 0845 300 6200
Textphone: 020 7973 6564
© Arts Council England June 2011
