Evaluating Your Codesign

• How will you decide how well your codesign worked?
• Do you want to do a structured evaluation?
• Which aspects of your codesign do you want to evaluate? For example, do you want to evaluate how well your codesign functioned and felt for participants (process evaluation), or how well it worked to change behavior (outcome evaluation), or both?
• What kinds of resources (time, funds) do you have for evaluation?
• How would an evaluation be used by yourselves or others? How does your answer to this question help you decide what kind of evaluation to do?

ENSPIRE did a detailed process evaluation of our codesign

Our evaluation assessed a range of questions relating to our codesign methods and process. We gathered data from co-facilitator meetings and debriefing sessions as well as codesigner surveys and interviews.

How well did our codesign process work?

Data source: Weekly meetings where co-facilitator pairs shared their observations with other co-facilitators

Questions included:

• What worked well during this codesign session?
• What were the biggest challenges during this codesign session?
• Were there any contextual or interpersonal factors that appeared to affect participant engagement or group dynamics?
• Did we do all activities as planned? If not, what did we change and why?
• Do we plan to do anything differently for our next session? What will we change and why?
• If anyone missed the session this week, what was the reason for their absence (if known)?
• Is there anything else that we would like to share regarding this codesign session or the whole codesign process so far?
   

How did co-facilitators feel about their own involvement?

Data source: A debriefing session with all community co-facilitators soon after the end of codesign

Questions included:

• What went well when implementing codesign?
• What was challenging?
• What interest would you have had in participating in more planning and design work?
• How well do you think the concept of having a staff and community co-facilitator pair worked?
• Overall, how would you describe your experience?
• What were your thoughts about doing this in teams organized by racial/ethnic identity? What worked well? What was problematic?
• In terms of timing, what did you think about being brought in very close to the launch of codesign sessions?
• How did the logistics work for you? How was scheduling? How did the technology work for you?
• How well did the orientation and training sessions prior to codesign work?
• How well did the meeting structure work for planning and debriefing?
• How would you like to be involved moving forward?
   

What were codesigners' experiences with codesign?

Data source: Short online surveys of codesigners after sessions 5, 7, and 10

The surveys used a 5-point Likert scale (ranging from 1 = strongly disagree to 5 = strongly agree).
The following statements were used to solicit feedback on group norms (for example, showing respect for others), content, and overall experience:

• I felt like my ideas were respected by my group.
• I enjoyed participating in the discussions.
• My facilitators made the sessions fun.
• I think the content shared during these sessions was useful to the goal of codesign.
• I’m proud of what we accomplished during these sessions.

We also asked open-ended questions so codesigners could provide additional feedback:

• What went well in the session?
• What would you change about the session?
• Other comments?

Data source: Individual interviews conducted with codesigners 3 months after codesign

Each interview was conducted by a staff facilitator who had the same racial/ethnic identity as the codesigner but who had not facilitated their codesign team. Interview topics included:

• Decision to join codesign
• Experience as a codesigner
• Perception of own racial/ethnic identity, especially as it related to organization of teams
• Perception of and satisfaction with final codesign products
• Codesign engagement, including facilitation and activities
• Effectiveness of codesign support (food, incentives, devices)

Some highlights from our evaluation

Our evaluation showed that co-facilitators were successful leading codesign partly because they trusted participants to take the design lead, ensured balanced participation, and created a comfortable environment for sharing, where all opinions were respected. Co-facilitators also provided guidance to stay on task, organized the sessions, answered questions in real time or later, provided clarification, and took good notes for future reference.

Michelle Man et al. “Implementing and evaluating co-design: A step-by-step toolkit,” New Philanthropy Capital, npproduction.wpenginepowered.com/wp-content/uploads/2019/07/Co-design-guidance-July-2019.pdf, accessed October 29, 2024.