Back in June, a few of our evaluators headed down to Atlanta for Summer Evaluation Institute 2018. The purpose of attending the conference was threefold:
1. to build a strong foundation in the methods and profession of evaluation
2. to learn about new methods or frameworks that would be applicable to the many different projects CCNY works on
3. to network with other evaluators in the field and build professional connections within AEA (American Evaluation Association).
It was important to build a baseline understanding/ language in evaluation as many in the department come from different disciplines/ professional backgrounds. It also provided the opportunity for them to attend more advanced workshops that aligned more closely with their own interests and skills development paths. They attended sessions focused on racial equity lens, economic evaluation, GIS, ethnography and evaluation, innovative reporting, causal knowledge mapping, cognitive interviewing, and strengthening non-profits’ evaluation capacity (full workshop schedule available here).
Potential barriers to successful evaluation include data fluency as well as the culture around evaluation. There is no single preferred methodology or framework; it is about using the right methodology for the right data and program and type of question one is asking. Therefore, it is crucial to be aware of and able to apply multiple methodologies based on the questions that are meant to be answered through evaluation. Here at CCNY, we use a general framework of Utilization Focused Evaluation, which is all about including the client in the evaluation process to create a product that is usable.
We asked Molly, Caitlin, and Emily for their thoughts on the efficacy and content of these workshops:
Molly: The two most useful seminars I attended were about improving organization and visualizations of evaluation reports and incorporating ethnographic methods in evaluation work. The first provided me with a whole new framework for engaging clients and stakeholders in determining how they would like to use the results of an evaluation and then provided tools for us to improve reporting. The second provided me with a context for incorporating my experience and passion for qualitative research as part of more traditional evaluation projects. I am already using tools and new methodologies as part of projects, which is helping me to improve the process of working with clients, approaches to engaging stakeholders throughout the evaluation process, and creating deliverables that are more meaningful and useful for the community. My mom even bought me the presenter’s book for my birthday!
Caitlin: There were two seminars that stood out: adding costs to your evaluations (e.g. cost-benefit analysis) and strategies to developing evaluation plans. Lately, it has seemed that many potential clients are wanting to analyze the economic impact, as well as program outcomes, so the cost seminar help to expand my toolkit to perform economic evaluations. Also, the evaluation planning seminar allowed me to network with other evaluators with best practices around the discovery phase of an evaluation, which is crucial because this will guide the rest of the evaluation and I have been using these techniques with my new projects already. I would recommend this conference to others, especially those new to the field and I would definitely go back.
Emily: The two sessions that stood out the most were the innovative reporting session as well as the cognitive interviewing session. The session on innovative reporting garnered discussion on different reporting methods to try an incorporate (e.g., games, brochures, audio recording, etc.). The cognitive interviewing session was insightful as to how questions may be interpreted differently than how they are intended. Information from this session will be beneficial in gaining information from clients, focus group facilitation, as well as survey development. I was able to take the resources/tool provided and improve on methods for internal reports. Additionally, I was able to network with other evaluators who were working on similar projects. We were able to discuss barriers and brainstorm ideas to overcome those barriers.
They also went to happy hour with colleagues hailing from DC, Atlanta, and Toronto at the Sun Dial, which boasts some of the best views of the city.
Overall, CCNY’s attendees are in consensus: a trip well worth making and to look forward to in the future!