Impact Assessment

Organizational Mycology supports the efforts of open science by creating adaptable and open methods for evaluating and assessing projects. We deliver useful and impactful feedback that can be used for the long term: in community development efforts, in leadership decisions and strategy, and in practical ways like grant reporting.

We often start with a qualitative set of intercepts to develop quantitative instruments that enhance evaluation and inquiry more effectively. We interview stakeholders and ask questions that probe and investigate participant and community experiences. This data identifies common patterns that could be monitored more closely in qualitative and quantitative ways. 

Overall, we work to:   

• Set up research objectives and structures for assessing the program. 
• Build capacity for the organization to own the assessment process through co-developing research instruments to support the program's needs more sustainably.  
• Advise and support team members as they learn these new skills and capabilities so that they can independently continue deploying these research instruments.

Across all activities, we strive to equip groups with the tools needed to self-assess their programs in a way that is reproducible and amenable to long-term comparisons and meets the needs of program leadership, the wider community, and funders.  

We typically work in three phases. 

Phase 1 - Discovery Phase  

In the discovery phase, we discuss the program's goals and the team’s vision. This stage level-sets the program’s goals so that we can incorporate them into the evaluation process, provide meaningful data for impact reporting, and make it easy for the community to convey these insights to funders. Discovery is an essential, all-hands phase because each program and organization differs in its intended impact, meaning that assessments must be tailored to these intentions and desired outcomes.

Using these inputs, we refine a research plan to ensure that the research maps onto the needs of the program and the group’s future goals in funding for the program. 

Phase 2 - Instrument Development

We set up all engagements to encourage the groups we work with to quickly and reproducibly collect evaluation data to use however needed (e.g., final reports, grant applications, blog posts, program improvement). We see this as an iterative process to understand progress toward program goals, how to develop outcomes and metrics that are trackable, and ways to  develop tools to efficiently evaluate outcomes. We combine this with ongoing support for staff as the instruments are deployed and data is analyzed. We strive to consider appropriate automation where possible and useful (e.g., in engagement tracking or citation tracking), while ensuring that the appropriate stakeholders remain in the loop and data are collected and used responsibly and effectively.

Phase 3 - Instrument Deployment 

During the instrument deployment phase, we will work to advise and develop team skills so that the tools we create are targeted to the community's needs. While the instruments are deployed and data is returned, we offer ongoing support to make sense of the data and to equip the team with skills and tooling for conducting this research in future cohorts.