A Framework for User Engagement
We present a simple framework for analysts and their organisations when thinking about and planning user engagement activities. It is informed by the relevant practices of the Code of Practice for Statistics (see principles V1: Relevance to users and T4: Transparent processes and management) and builds on practice areas that we identified in our review of the Defra group’s engagement approach.
The framework highlights six practice areas. As producers of statistics you need to promote your statistics, understand the need for information, listen to what users have to say and act on the feedback. And to be effective across the organisation, user engagement needs to be planned and collaborative.
The framework is not so much a cycle, as all of the practice areas should be considered throughout your work and across the work of the organisation. Having said that, for analysts it might be helpful to begin with what you do already and think how you can raise awareness about your statistics. Use your statistics as the starting point for the conversation and have an ongoing dialogue with your users and stakeholders.
For the organisation, again start with the purpose and mission:
- How do you make a difference to society?
- How do you deliver your services and who do you work with?
- How can you work with other partners and the public to better understand their needs and deliver effectively?
- How do your statistics help inform your service?
- What information do you need to be successful?
To answer these questions and meet your organisational goals effectively, organisations need to have a culture of engagement, listening to their stakeholders, partners and staff. Statisticians and other analysts can make a valuable input to this conversation.
The six practice areas
The practice areas below are not mutually exclusive: for user engagement to be effective, each area needs to be considered and addressed alongside the others. For instance, being transparent about developments to the statistics (Promote) is vital for maintaining an ongoing dialogue (Listen).
Plan
- Be strategic in your approach to engagement
- Be open about your commitment to engagement
- Be flexible in ways of engaging
- Be proportionate in investing resources
Promote
- Encourage use of the statistics and raise awareness
- Be transparent and actively inform users about changes and developments
- Disseminate and present to interested networks
- Support use for a wide range of users
Listen
- Establish an ongoing dialogue to understand users’ needs
- Make yourself available to users
- Ask users for connections to other potential users
- Allow users to plan or volunteer for engagement opportunities
Understand
- Identify the types of users and potential users
- Know what users need the statistics for
- Learn about the wider social, economic and political context
- Be aware of gaps in data and insight identified by users
Act
- Proactively involve users in the production and development of the statistics
- Be transparent about development and action plans based on user views
- Explain where user needs cannot be met
- Review outcomes and effectiveness of engagement activities
Collaborate
- Identify and join up existing activities across analytical teams, departments and themes
- Partner with other professions and functions, including policy makers and communications teams
How to approach user engagement
The goal is to engage effectively by:
Be proactive in involving users and other stakeholders by seeking their views rather than waiting for those with an interest in the statistics to come forward. Look to anticipate how their needs are likely to evolve over time. Explore a range of ways of involving users in the production and development of their statistics. This can help ensure you hear from a mix of voices. Otherwise, there is a risk that certain voices dominate and that the feedback collected does not reflect wider user needs. Consider how you might target different user groups – communications routes to reach expert and non-expert users may be different, so tailor your communications and engagement to different audiences.
Listen to the suggestions from users about how to engage – they can steer you to the preferred means for providing their input. Map or understand the range of users you engage with and identify possible gaps. Review the range of users you connect with and keep challenging to broaden your contacts. Consider who might have a potential interest in your statistics and look out for new uses of your statistics.
A proportionate approach to understanding use, based on the public profile of and interest in the statistics, may be needed. Not all sets of statistics, particularly those with a specialist audience, have a large or meaningful broader user base to access. You can show your understanding of use by regularly publishing information about the types of users and uses of the statistics. You may need to balance the needs of different types of users – being open about the trade-offs and the rationale behind them is essential. It may be the case that you cannot satisfy some users’ needs immediately but can put in place a programme of development to deliver the improvements. A strategy for improvement can ensure that resources are used effectively, as well as help make clear your commitment to deliver statistics that meet users.
It is important to be open and transparent in your engagement approach. Be open to hear about the issues of concern – don’t presume to know the areas of greatest concern to users. Consider the broader social, economic and political contexts for the statistics. Allow users to plan or volunteer for engagement opportunities. Be open to challenge, be aware of your own potential unconscious bias. Often producer teams are good at telling users about changes to the statistics after they have happened but may be less forthcoming about upcoming developments to the statistics – instead, anticipate the need to alert and inform users and if possible think about how they can be involved in the changes.
Being transparent about the outcomes of individual user engagement activities, and making it clear publicly how and when you have taken on board feedback, can mean that users can see the impact of their contributions and understand the rationale for your decisions. Be open where user needs cannot be met, by giving feedback about the reasons for the decisions made and the constraints. Publish the findings and actions from user surveys, even if feedback is gathered on ongoing basis.
Where possible work with other teams to engage with users and share learning and insight. This may reduce the burden on individual producer teams and maximise the impact of user engagement. It is helpful to regularly review the outcome and effectiveness of user engagement activities, to ensure that user engagement is, and continues to, support the value and quality of all statistical outputs. Where necessary, adjust your approach to engagement to allow you to better promote your statistics and capture the views of a range of users as you develop your statistics.
It can also be invaluable to have specialist user researchers involved in leading engagement activities and gathering evidence about information needs and gaps. Find out about and get involved in public engagement activities in other parts of your organisation and collaborate with other statistics producers who share similar users and stakeholders. Build relationships and connections across functions in your organisation, with policy makers, operational staff and other analysts – they will also be gathering insight from stakeholders. Your combined efforts will be more productive and fruitful as you learn from each other.
Case studies and Useful Resources
Here are some examples of good practice highlighted in our review of Defra group’s approach to user engagement. You can find more case studies on user engagement on the online Code of Practice for Statistics. Further examples of good practice are given on the GSS user engagement web page.
The Natural England analytical team responsible for the People & Nature Survey for England has run several user workshops and a user conference to promote the statistics and get user input on the development of the statistics. These were attended by many users from inside and outside government. The events gave the team the opportunity to tell users directly about the latest published statistics, and it encouraged interaction between different types of users.
The team continues to update users and other stakeholders about the release of statistics and development of the survey through a regular newsletter. The team is also supporting use of the statistics in specific ways, for example, by running training sessions for Natural England staff. Most attendees are familiar with the statistics but have never done their own analysis of the raw data. The training session gives them the confidence to undertake their own analysis, which increases use and understanding of the data within the department.
To gather user views on the content, format and release of Defra’s Food Statistics Pocketbook, the statistics team launched a short, continuous survey through Google Forms. Almost 100 people completed the survey over an 18-month period and the team continues to receive responses. The team said it was a helpful exercise as users confirmed they were happy with the changes to the format, and the team made additional improvements to the output based on the feedback.
Defra’s air quality statistics team launched a similar continuous survey to gather views on the emissions of air pollutants statistics, and other teams told us they have also considered this approach. There are advantages to such surveys – they are easy to manage and less time-consuming for users to complete – although they may provide less insight about user needs than a detailed survey or meetings with users and stakeholders.
Defra has published a User Engagement Policy Statement, which contains a clear summary of who its users are, what users want, and how it engages with users. The key objectives are “to ensure that our statistics are of value to our users, informing debate and having an impact on decision-making” and it includes a number of commitments that draw on the practices of the Code.
The statement acts as a reference point for both statistics teams (it provides guidance) and for users (it is a public statement of ambition). It was developed collaboratively by the Head of Profession support staff and Defra’s User Engagement Working Group, a central team that takes forward specific user engagement initiatives. Other government departments and bodies, such as Ofsted, Ofqual and NHS England (PDF), have produced similar statements or strategies.
Defra has an internal implementation plan for the policy, to ensure that teams are working towards a shared standard of engagement. It has several aims, including identifying and categorising the range of users and user groups, determining usage trends, and developing insight into user needs, and for each aim there are a number of actions and measures of success.
Here are some additional resources which may be useful when planning your user engagement strategy.
OSR review User engagement in the Defra Group
OSR blog Collaboration is key
Government Statistical Service (GSS) User engagement website – head here for updates on the new user engagement strategy for statistics and tips on a wide variety of ways of engaging users, including:
- Brainstorm
- Build user personas
- Use what you already have
- Web analytics
- Social media
- Newsletters
- Consultations and surveys
- User events
- User groups
- User research
- Online forums
Join the conversation with other statisticians and analysts engaging users by using the Slack #userengagement channel. It is a great an opportunity to network, ask questions and share good practice with the GSS Best Practice and Impact team (BPI) and other analysts.
Download this Guidance as a PDF