Non-academic impact is the demonstrable contribution that excellent research makes to society and the economy and its benefits to individuals, organisations and/or nations. This is not to be confused with academic impact which is the demonstrable contribution that excellent research makes in shifting understanding and advancing method, theory and application across and within disciplines.
During the REF assessment, many of you will be aware that we have adhere to quite a strict notion of impact which is: an effect on, change or benefit to the economy, society, culture, public policy or services, health, the environment or quality of life, beyond academia.
Impact is typically assessed by its significance and reach:
- Significance: how meaningful, valuable or beneficial is your work to those involved
- Reach: how far-reaching is your work, either geographically or in terms of the diversity of your beneficiaries?
Being able to evidence impact comes down to being able to articulate the causal link between your research and the resulting change - and to keep a good record of this - monitoring your progress and evaluating your outcomes is crucial. In order to create impact, it's important to know who your audience is - identify who you want to talk to and why. There are many ways to start a conversation: the key ingredient to impact is engagement - knowledge exchange and dissemination can be achieved in many forms, such as public engagement , working with industry or third and public sectors, with social enterprises and policymakers.
Impact planning toolkit
By identifying the potential impact of your research, you can start planning and prioritising knowledge exchange, public engagement or dissemination activities to inform and guide your project. This toolkit has been adapted to help researchers understand their potential impact, consider new and existing stakeholders and get the most out of their engagement activities.
- What impact could come from my research?
- Who else is interested in my research?
- How can I engage stakeholders with my research?
- Important things to consider when considering your impact agenda
- What can or should I be measuring?
- What support do I look for?
- Evidencing impact: best practice
- Glossary of terms
Non-academic impact relates to uptake of research outside of the academic environment. If you're fortunate, the potential impact of research can be clearly identified but very often, you will need to consider the wider context or value of the research or think creatively about the stakeholders that could help your research to have impact. Impact very rarely occurs spontaneously, in isolation or in a linear fashion.
The type of impact you can have will vary depending on your research area but can include:
- cultural, e.g. changing opinions
- economic, e.g. job or wealth creation
- environmental, e.g. lowering carbon dioxide emissions
- educational, e.g. improved engagement of hard to reach groups with a discipline
- health and wellbeing related, e.g. improved diagnosis or patient outcome
- social welfare and public services, e.g. improved efficiency
- public policy and legislation, e.g. influencing a change in law
- operational and organisational change, e.g. improved manufacturing process
- practitioner/professional services improvement, e.g. a change in services
- societal, e.g. changes in awareness and understanding
- technological, e.g. wider reach of research
To understand the different potential stakeholders affected by your research, consider the potential changes that could result from your research.
- What do you want to achieve?
- What is the current context?
- What changes could happen through different stakeholders being aware of or using your research?
- How can you make contact with these stakeholders - are you working with sensitive material or vulnerable groups, are there ethical considerable to take into account?
Being able to identify your stakeholders and beneficiaries can help you to target your plans effectively:
- Stakeholders are organisations, groups or individuals who are affected by or can affect a decision, action or issue related to your research
- Beneficiaries are groups or individuals, either at a local, national or global scale, who ultimately are affected, influenced or experience an improvement from the research with or without direct contact
You can start to identify your stakeholders and beneficiaries by thinking about:
- Who else has a common goal or mutual interest?
- Can you identify specific groups you would like to work with from the broad categories in the above stakeholder map?
- Why would your research be important to each group?
- How do the different groups interact?
- Who else influences the different groups?
Prioritisation of stakeholders and beneficiaries
- It isn't always effective or possible to interact with all the identified stakeholders. You can determine who will be best to interact with by considering:
- How likely are the groups identified to be affected by the research?
- How interested are the different groups likely to be in your research?
- What capacity do they have to use your research?
This will allow you to target your activities and consider potential risks to the impact.
There are many different ways to engage stakeholders and beneficiaries with your research. Once you have identified the stakeholders you want to engage, you can tailor your activities accordingly and develop communication strategies that provide effective interactions across the board.
There are many types of engagement activity. You should select the type that best suits your research and the stakeholder's needs.
Things to consider when planning activities
- What activities are you already conducting?
- Is there an intermediary, such as a knowledge exchange professional, external expert or artist you should work with?
- Do you need a communication strategy to raise awareness of your research to new groups or the public? What is the best platform for this?
- What is involved in implementing your activity?
- Are there events or platforms run by the University that you can use?
- Which external engagement events or projects should you participate in to help achieve your impact and create your knowledge network?
Creating and nurturing impact can take time. You should decide whether it is you or a member of your team that is best placed to oversee these activities.
Some of the possible risks with stakeholder engagement are:
- The stakeholder agenda is already developed without considering research evidence
- The stakeholder has concerns over confidentiality
- The research may challenge views of the groups and thus not be taken up
- Stakeholders and beneficiaries cannot use research communication material
- The research doesn't address the question in the manner required by the stakeholder group
- The stakeholder lacks the capacity to engage with the research or implement the findings
- The stakeholder timeframe differs to your project timeframe
- A key contact leaves the organisation and the relationships with the stakeholder groups breaks down
It is useful to consider how to manage expectations and when to engage with the groups identified. Early engagement with stakeholders is often beneficial to establish their needs, identify how best to engage them with the research, build flexibility into engagement plans and manage concerns.
Some questions to consider:
- Will you engage at the beginning, during or at the end of the project?
- Will the stakeholder's requirements shape the research project?
- Will you keep them updated throughout the project and if so, how?
- How will you keep in touch after the project is completed?
It can be easier to gather evidence of your activities as they occur rather than trying to find it months later, so it's important to be prepared. It is important to collect evidence to help demonstrate a clear link from your research to the impact, this will guide your future research strategy. When planning your activities, you should think about what you want to achieve from each activity (your outcomes) - this will help determine what you should measure. Where possible you should collect qualitative and quantitative information.
Collecting information while conducting the activities can help to indicate where impact may be achieved at a later stage as well. For example, by keeping a record of attendees at an event you can see a link to your research if one of the attendees then uses your research to influence a policy.
- What can be measured?
- What is the baseline? It is important to understand the current situation so as to identify the influence, effects or changes that have taken place.
- Who has been engaged? Meeting agendas, Eventbrite invitations, attendee lists, demographics, numbers of attendees.
- How did they react to the research? Feedback from attendees, secondary reach from attendees passing on information.
- What online activity has there been? Retweets, web hits, downloads, media coverage. Altmetrics may help with this.
- Did a collaboration project achieve its goal? End of project reports, press releases.
Measurements collected by others
You should discuss the need to collect evidence impact of research with your stakeholders at an early stage so that they are aware of your requirements and the reasons behind it.
- What do your stakeholders already measure?
- What performance measurements would your stakeholders be happy to share with you?
- Are there indicators collected by local, national or global bodies?
Recording evidence. Evidence of impact can be collated and stored in Pure .
If you want to discuss getting started with knowledge exchange or public engagement, contact the Impact team.
- Do you or your team need training?
- Does the University already have a relationship with potential partner? BDOs, stakeholder engagement
- Do you need advice on any of your activities? PERU, Impact team, RDAs, Commercialisation
- Do you want help managing your relationships? Stakeholder engagement teams
- Do you need advice on confidentiality? Link to governance team
There is no 'one size fits all' approach to evidencing impact, the methods you use will be dependent on both the research and impact that has been (or is hoped to be) achieved.
Collecting evidence:
When: Evidencing impact not only an end of project activity. The research use should be reviewed regularly to capture interim and unforeseen impact.
At the start of the research project or partnership gather information on the current external situation. This baseline will allow you to identify the change due to the research over time.What: Impact is often described in terms of reach and significance.
Reach - the spread or breadth of influence or effect on the relevant constituencies. Reach is not assessed in purely geographic terms, nor in terms of absolute numbers of beneficiaries, but rather based on the spread or breadth to which the potential constituencies have been affected.
Significance - the intensity of the influence or effect.
Simply put, this is the breadth of potential beneficiaries that have been affected and how much they benefited or a situation changed.
Some impacts may be demonstrated through a single piece of information, however more often a credible series of independently verifiable qualitative and quantitative indicators or proxy measures need to be brought together to demonstrate impact.
Quantitative and Qualitative evidence:
Quantitative evidence, e.g. attendance data, patient numbers, sales, statistics or organisation uptake, should be collected to describe the reach.Qualitative evidence provides the understanding as to the context of the issue and significance of the impact and can include written reports, testimonials and case studies.
Where to keep evidence
You should have an easily accessible place to store the evidence of your impact.
Types of Evidence
Types of information
What can it provide?
How to get it
Information available in the public domain e.g. online, in the news or in reports.
This information can help demonstrate change - it helps to have independent sources of information on the previous situation or scale of the issue from website or reports from external stakeholders.
There are different exploratory approaches that can be used to find information using internet searches
If you believe that your research will be featured in the media or through partner press releases or on their websites you could set up an Altmetrics or Google to capture that information.
The UK Office of National Statistics has a searchable database of national and local statistics which can help describe the context of the issue.
If your potential impact is international then searching the UN Global Issues , relevant Charities or Government web pages may provide information on the scale of the issue.
Information on research use from partners
This information can provide evidence of process or organisational changes. It may also be the only way to obtain confidential or commercially sensitive data.
It is always best to discuss impact at the start of the research collaboration, that way you will know what your partner wants to achieve using the research and how they will demonstrate it. At that point you can discuss what evidence your partner would be happy to share with you.
Even if your project has started or ended, you can approach your partners to see if they are happy to tell you about the benefit they realised from using the research or being a part of the collaboration. They may be able to provide evidence of this.
Event attendance and feedback
The attendees information can provide a quantitative and qualitative baseline of your potential beneficiaries and the long term change in their perceptions and behaviour.
The public engagement team have developed resources to help evaluate events
If your research was used by a partner organisation in an event or if you took part in an event that was hosted externally you could ask for the feedback they have collected.
If you have presented at a workshop/focus group or event, ask the organiser to include some of your questions in their feedback form.
Follow up feedback
Impact may take time to demonstrate so being able to contact participants 6 months or a year later is useful way to find out if any has occurred.
The public engagement team have developed resources to help evaluate events
Awards
The award can either be for your research contribution to the impact or an assessment by research users of the impact e.g. product to performance.
Keep all records of short listing or awards through organisation press releases, media coverage and correspondence.
Independent reviews
Reviews by individuals who are external to the university but considered lay experts can provide an assessment of the significance the research contribution of the activity.
Examples of Independent review used to demonstrate significance in REF2014 Impact case studies.
Media & News article
News and media articles can demonstrate dissemination and can be used alongside other indication of the reach of the potential audiences.
Weblinks, screen shot and dates of publication or broadcast all need to be captured.
Websites such as Statista or BARB can provide readership or viewership numbers.
Downloads/viewing statistics and Social media comments
This can be used to show the effectiveness of your engagement and a pathway to the impact you are claiming. This information depends on where share your research e.g. web pages or social media
Google analytics is a tool which lets you track user information on websites.
Tracking your research or project web pages can provide numbers on
- How many people have visited the web page
If you're receiving a lot of comments on social media, you may not want to record all of them. You may just want to record a selected few that demonstrate engagement or the public debate that has been effected. Site such as Wakelet can be used for this.
- The average time spent on the web page
Weblinks can change so you should keep the link and the date it was taken along with a screen shot of the evidence you are citing.
- Who is accessing the web page
Twitter can provide information on the location and demographics of individuals and track downloads and retweets.
Altmetrics can be used to discover who is talking about your research in the public domain. it covers a wide range of online sources including news outlets and policy documents. However, it may give an incomplete picture as the link to the journal article or DOI number of the research paper must be used directly in the mention. Therefore, some discussions about the research paper could be missed.
Weblinks to independent sites and visitor traffic can be measured
Testimonials
Testimonials can be both qualitative containing evidence of effect or influence of your research and who it influenced and quantitative providing dates, economic figures or numbers of appearances/attendees.
Reach out to individuals who are qualified (e.g. senior figures) to evidence the impact the research has made. The most effective way to do start this is via introduction email, explain the purpose of the request, where the email evidence can be saved and referenced at a later date.
Testimonials should come from an independent, well respected figure who directly mentions the research work and how it has affected them.
Being explicit in asking what evidence you would like them to provide will help you to gather evidence which clearly supports what the impact of the research has been. For example:
Where possible, the testimonial statement should provide quantitative examples of the impact, describing how the research has led to the impact.
· Exactly how has the research has influenced their organisation
· Outline of how guidelines/standards were adopted in practice resulting in efficiencies/benefits.
· What can be done now that could not have been done before
Correspondence
This can provide qualitative evidence of who your research influenced and the significance of this effect.
You should keep emails or correspondence you receive on your impact activities. You will need to be able to review the emails quickly for relevance to the impact you are claiming.
This could be an email from an organisation requesting you attend a meeting or talk to a group. The email will contain details of the organisation and event and also potentially why they want your expertise or perspective.
You can do this by making a note of who this individual is and the organisation they belong to.
It could be from individual or organisation that your research has impacted (directly or indirectly).
You can create a timeline of your activities in relation to the impact from invites to talks or meetings.
It might be in response to a keeping in touch email you sent out to partners that has provided information of the impact your research has had for them.
Create a file for relevant emails e.g. 'impact emails' or similar to be accessed at a later date.
Term
Definition
Arts-based knowledge exchange
A means of conveying research using the creativity of artists or the creative sector.
Collaborative research project
A set piece of research work with an existing or prospective external organisation or stakeholder.
Commercialisation
Market research, patents, license agreements, IP and spin-out companies.
Community of interest collaboration
A social learning process whereby a group of people with a common interest come together to share, develop and advance a knowledge base.
Educational material/outreach
Educational gatherings for the purpose of conveying evidence and knowledge.
Events
Workshops, interactive small group events, public lectures, debates, practitioner conferences.
Film and audio
Films or animations that explain and explore your research. These can be used to provide background information on complex research.
Media
Use of television, radio or print to raise awareness of the research or attract media participation e.g. public/practitioner awareness campaigns.
Networking
Interactions with external groups to advance your profile or meet collaborators.
Opinion leader/champion
External supporters that are well connected, credible and persuasive. They are considered knowledgeable, trustworthy, accessible and have a willingness to share knowledge with the community.
Patient-mediated intervention
Any intervention aimed at changing the performance of healthcare providers using patient-focused interventions to improve clinical practice.
Policy brief
A document providing a rationale to choose a particular policy or intervention.
Practice tools/decision aids
Tools which make the application of research user friendly.
Practitioner training
Facilitated training provided to practitioner groups in order to change attitudes, knowledge or practice behaviour.
Press release
A written or recorded communication directed at members of the media for the purposes of announcing something with news value.
Proof of concept project
Development of commercial ideas to the point at which they can engage with external collaborators and gain further investment.
Public engagement
Engaging the public with your research through university, faculty or departmental platforms. Support available from the Public Engagement Team.
Secondments
Transfer of people in and out of the University to enhance the application of research knowledge.
Social media
Using technology to share or co-create knowledge, for example via Twitter, websites, blogs, surveys.
Stakeholder interaction
Increasing stakeholder participation through steering groups, workshops, sandpits etc.
Working with a knowledge exchange professional
Faculty specialists who can link researchers with external partners.