Evaluating Societal Impact

A rough how-to guide

This guide offers more practical information related to societal impact evaluation. It should be seen as an emergent, ever evolving, collaborative document – the intention is to incorporate further evaluation methods and examples. So, if you develop or discover a useful technique for evaluation or have questions about the current content do let us know.

It draws upon a range of sources, including the experience of ESI and literature on societal impact evaluation methods, to provide a brief introduction to a range of evaluation methodologies and examples of how they can be used throughout the life of your project. For each piece of content we indicate the purpose (why), target audience (who), resource type (what) and what point in the research cycle (when).

Why

General tips&tricks/ Grant/ Insight/ Collaboration/ Formal evaluation (e.g. SEP)/ Annual interview

Who

Policy/ Support/ Research/ Education

What

Impact evaluation and activity/ Impact environment and capacity/ Impact strategy and ambition

When

Plan/ Do/ Describe/ Share

 

 

Why

How to get started

Who

Academics

What

Tips

When

Plan

Before setting out and undertaking impact related activities it is important to consider the aims of the project (i.e. what you want to achieve). As you will be unable to tell whether your project was a success unless you have decided what it was supposed to accomplish. Also good to be aware of the difference between outcomes, outputs and activities: outcomes are the changes you hope to bring about as a result of what you’re doing; activities are the things you do in order to make these changes happen and outputs are the things you can measure when it comes to the results of your activities. A typical outcome would be “to increase awareness in Rotterdam of EUR’s work in sustainability”. This would lead to activities with outputs such as “to host a series of public events that effectively increase the awareness of those attending”. Outputs and outcomes should be as specific and detailed as possible, as they will form the basis for any evaluation undertaken. The next things to consider are: what evidence you need to collect to know you have met your outputs (i.e. indicators), the focus of the evaluation (i.e. what and who with) and when the evaluation will take place (i.e. pre and post project, during the life of the project).

To help establish an impact proposition, you can think of possible activities on a canvas such as this:

1. Impact proposition

2. Target groups

3. Key partners

4. Key activities

What can your research add for others?

Who can benefit fom your impact proposition?

Who do you need to reach your target group?

What will you and your key partners do to realise your value proposition?

 

 

 

 

5. Channels

6. Quantified reach

7. Key resources

8. Target group relations

What channels will you use to reach your partners and target group? (e.g. media, personal contacts, etc)

How can you quantify your reach with your target group?

What people or external resources are you dependent on to realise your impact proposition?

How will you keep in touch with your target group?

 

 

 

 

9. Cost stucture

10. Revenue

What costs are associated with your impact proposition?

Is there potential for financial revenue?

 

 

The answers to these questions will shape what method(s) you decide are appropriate for the evaluation of your project. The more you think about the answers to these questions the more you will be drawn towards particular methods of evaluation. Furthermore, the clearer you are in your outcomes and outputs, the more you will be able to use monitoring and evaluation to judge your success.

The table below summarises some of the most common evaluation techniques, so you can think through and choose the most appropriate techniques to evaluate your project. You shouldn’t restrict yourself to this list – if your thinking about your aims has given you ideas for new and better ways to measure success, don’t be afraid to use them!

Whatever evaluation methods you decide to use, it is good practice to tell people what information you are collecting, why you are collecting it, and what you will do with it (i.e. the purpose and uses). The approach to evaluation should be transparent and it is important to make it clear to all parties involved how much (or how little) the information you gather will inform future work.

Why

Ensure you are aware of the diversity of impacts you can plan for.

Who

Policy/Support/Research/Education

What

Impact strategy and ambition

When

Plan

We distinguish 9 types of impact corresponding to the broad scope: commercial impact, policy impact, social impact, health impact, cultural impact, environmental impact, technological impact, legal impact and international impact.

  1. ECONOMIC IMPACT Driving economic growth, generating new products and services and creating jobs.
  2. POLICY IMPACT Informing, influencing and improving decision-making by government and public bodies, NGOs and in the private sector. Increasing the efficiency and/or quality of public services, directing investment to priority areas and raising business productivity.
  3. SOCIAL IMPACT Informing public debate, stimulating public interest, improving welfare, equality and inclusion, and improving quality of life and opportunities.
  4. HEALTH IMPACT Creating new drugs and treatments and developing new therapies. Improving education and training, public awareness, and access to health care provision, as well as policy, legislation, standards or guidelines.
  5. CULTURAL IMPACT Enhancing and preserving our cultural heritage, producing cultural artefacts, creating, inspiring and supporting new forms of expression, and enhancing our understanding of minority groups and communities.
  6. ENVIRONMENTAL IMPACT Delivering energy savings and reduced emissions, improving management and conservation of natural resources, stimulating public awareness and influencing policy, improving business and public service operations, and environmental risk management.
  7. TECHNOLOGICAL IMPACT Developing new and improving existing technologies.
  8. LEGAL IMPACT Improving law enforcement methods, effecting legislative change and improving legal practice and access to justice.
  9. GLOBAL IMPACT Delivering positive impacts from our research overseas including collaborating with partners in other countries.

Why

To develop understanding societal impact

Who

Policy/Support/Research/Education

What

Impact environment and capacity

When

Share

To work with an embedded and positive impact culture as EUR community, it is imperative that we develop a shared understanding of what societal impact entails. As a first step in collectively defining our impact identity, the Evaluating Societal Impact (ESI) team and Strategy Office invite you to join a series of dialogue sessions.

We want you to connect to colleagues and share insights about creating societal impact in your daily work, so our shared understanding of societal impact pays tribute to the diversity within the EUR.  At the end of a dialogue session you... ​

  • Know what we consider good practices that capture the heart of our impact mission.​
  • Are aware how EUR colleagues (like you) work on societal impact, and of how you can help each other.​
  • Worked on finding connections and new possibilities for societal impact. ​
  • Know how to contribute to EUR's societal impact mission (and what you might need to support your work on impact).​

We challenge you to ask yourself: ‘What is societal impact for me?’, ‘How do I work on impact?’ and ‘What skills and people-based factors do I have or would I like to have to achieve impact?’. We encourage asking these questions and keeping the conversation going in your own network or on a theme that connects to your own work. That is why we invite you to organize your own Impact Dialogue. Re-watch the Impact Dialogue Kick-off here.

Need more information and/or download our Impact Dialogue Starter kit? Browse the information page here or contact Lotte Houtepen

Compare @count study programme

  • @title

    • Duration: @duration
Compare study programmes