MEASURE Evaluation (Li, Makulec, Nutley); Palladium (Li, Nutley); John Snow, Inc. (Makulec)
"Using a design framework enabled us to engage stakeholders in creative facilitation approaches centered on understanding data users and their needs, empathizing with the challenges they face, and brainstorming and designing tangible solutions."
This report describes the design process, activities, and outcomes (prototypes) from a user-centred design activity to strengthen use of data within HIV programmes. It presents the background and approach, findings, and recommendations and reflections from the experience of using a user-centred design approach to understand the experiences of data users, identify current barriers facing health information systems (HIS) users, and develop creative solutions for tackling these issues. MEASURE Evaluation (funded by the United States Agency for International Development [USAID] and the United States President's Emergency Plan for AIDS Relief [PEPFAR]) worked with the South African design firm Matchboxology (MBX) to adapt and implement a user-centred design method to explore the human factors that impact the many different types of stakeholders and individuals who are involved in the production and use of data. The design approach engaged data users directly in identifying barriers to data use and prototyping innovative solutions that could be tested and adopted.
As MEASURE Evaluation explains, data-informed decision making is the outcome of complex system dynamics in which technical, organisational, and behavioural factors interact to create specific facilitators or barriers to data use. Countries are working to improve HIS to produce higher-quality and more timely data. Significant attention has been paid to designing technological tools to make data more usable. For example, tools such as dashboards facilitate the availability of interpretable data and can have a role in encouraging effective use of data for decision making. However, the mere existence of these data is not enough to guarantee their use in decision making for programme planning and policy development. Understanding the motivations, needs, internal decision-making processes, pain points, and experiences of those who produce and use data - including health workers, managers, and policymakers - as they interact with others and complete work tasks is described in the document as critical to developing innovative solutions to support the use of data for programmatic decision making at the district level.
Design applied to data use involves these core principles:
- Empathy: the ability to understand the challenges that others experience. Design looks to uncover the unique experiences of data users, rather than grouping experiences by job title or role. Understanding individual experiences in how key stakeholders use or don't use information, perceptions around work priorities, and motivating factors (e.g., recognition, job satisfaction, and positive or negative feedback from superiors) can help address barriers to data use. Bringing together people with varied backgrounds and multiple perspectives in the design process creates opportunities to hear from HIS staff with different roles and experiences, fostering deeper empathy.
- Prototype: developing context-specific solutions. Through workshop activities, participants work with one another to develop innovative, pragmatic solutions that respond to their real-life experiences and understanding of their barriers and pain points in using data for decision making. Using a design process results in recommendations for products or tools that are appropriate specifically to the user.
- Co-design: building ownership through the design process. Data users take an active role in creating custom-tailored solutions to the challenges they perceive as most critical. Facilitating the process using design techniques and approaches that promote critical thinking through workshop activities helps build a culture of data use by encouraging the participants to discuss openly and honestly the barriers to data use that arise for them at work. This engagement can increase the likelihood that users will carry design solutions forward and nurture effective and sustainable data use interventions.
The design activity detailed in the report followed an iterative process, from cultivating empathy with data users to developing prototypes to address critical barriers to data use. Four districts were selected for the activity: two in Tanzania and two in South Africa. Participants were data users, such as programme managers and clinicians, and data producers, such as information or monitoring and evaluation (M&E) officers, working in the field of HIV at the district and provincial levels. In each district, MEASURE and MBX collaborated with the Ministry of Health to conduct immersion interviews, facilitate two design workshops, and develop a set of actionable prototypes for testing. One of the characteristics of these design activities that is distinct from those of traditional training and engagement sessions is the creative facilitation approach - using semi-structured activities that give participants an opportunity to put themselves in the minds of others and develop thoughtful, creative solutions to challenging problems. The activities conducted during interviews and workshops are summarised in the report. For example, to explore the differing experiences of data users, participants created personas to represent different types of data users. Participants were asked to think beyond job titles and to instead discuss the personalities, attitudes and motivators, challenges and barriers, and responsibilities of various data users and producers in their contexts.
Many themes and key insights around barriers to data use emerged from the immersion interviews. These were described as "truths" about the experience of using data for decision making, and were designed to represent a range of experiences. Ultimately, the four districts in South Africa and Tanzania chose the same definition of success: a user-centred HIS that provides "clear, easily understandable, and user-friendly information that empowers every data user to make informed and effective program decision making." Each district identified key barriers to data use and created a set of prototypes to overcome the barriers and strengthen the use of HIV data at the district level. For example, three of the prototypes in South Africa focused on expanding or embellishing the existing Data Use Champions initiative implemented through MEASURE Evaluation. Provincial and district leaders were tasked with supporting local adoption and use of the new ideas. In each country, local MEASURE staff were active partners and received recommendations for potential inclusion in future workplans.
When tasked with developing innovative solutions to address barriers to data use, participants created interventions ranging from simple, easily implemented solutions (e.g., half-day role swaps to foster empathy among staff or leveraging of closed user groups on existing communication platforms) to high-investment technology solutions (e.g., creation of mobile applications or portals). Most of the prototypes provided solutions to such problems as a lack of communication and feedback regarding data and performance, an inability to access easily interpretable data at lower levels, and a dearth of support for champions of data use.
Many ideas emerged from the workshops and completed prototype development that could be taken forward by local government or MEASURE Evaluation teams in Tanzania and South Africa. MBX, as the design partner on the activity, provided a summary of recommendations for next steps that would be feasible within specific periods (see the report). In addition advice for future design approaches is provided, and includes these selected communication-related recommendations:
- When planning, engage local individuals or teams to identify districts and make local connections and create short summaries that can be used to explain the activity to potential participants.
- Invest in strong facilitators who can manage a design process, understand the issue, and encourage participants to think critically and in different ways about the identified issues.
- Encourage active participation by local experts and staff who understand the data use context to help probe and encourage participants to uncover new insights and solutions.
- Use highly interactive activities and creative facilitation, such as the use of music, to keep participants deeply engaged throughout the workshop.
- When testing ideas, provide detailed feedback decks and immediately share facilitation materials with local stakeholders in order to build on the momentum for action generated during the workshops.
- Identify highly motivated people (i.e., data use champions) to move prototypes forward.
- Plan feedback mechanisms to follow up with people such as data use champions and local data use advisors to provide additional context and updates on next steps.
- Devise a long-term evaluation strategy (if resources allow) to assess both the effect of the design process and the impact of the prototypes taken forward, using a mixed-methods approach.
MEASURE Evaluation states that feedback from participants highlighted the value of the workshops in creating an open space for them to identify challenges, develop shared understanding, and generate their own ideas and solutions. Participants said they appreciated the participatory approach used, and valued the opportunity to think critically and share experiences across sub-districts. Creating space to connect for participants representing different roles, departments, and divisions added value, and some participants noted that the informal sharing of ideas was a highlight. At the end of the workshop, some participants accepted responsibility to move some of the prototypes forward. And, through the Global Fund to Fight AIDS, Tuberculosis and Malaria, John Snow, Inc. is adapting and applying a design approach to data use challenges in West Africa to further test this approach to improve the use of data for decision making.
"Five Lessons Learned from Applying Design Thinking to Data Visualization", by Amanda Makulec and Barb Knittel, ICT Works, October 9 2017; and MEASURE Evaluation website - both accessed on October 11 2017. Image credit: Matchboxology