Evaluation of collaboration efforts is important to assess what has been accomplished, as well as serve as a guide to how to change the collaboration mid-course, should that be necessary. Sometimes collaborations are funded efforts that require evaluation in order to qualify for funding or refunding; or they are model projects that need to be measured in order to be replicated. Plans for the evaluation should be made early in the life of the collaboration. Considerations about the evaluation process include the following:
What criteria will be used to evaluate the collaboration and its activities?
Who will conduct the evaluation?
At what points will it the evaluation take place?
Who will be involved in evaluating the collaboration? (members, the community, funders, etc.)
What processes should the evaluation utilize? (focus groups, surveys, interviews, participant observation)
How will the collaboration use the evaluation information? Will evaluation findings be used to redirect collaboration work?
Who should receive and review the evaluation findings?
A. PARAMETERS FOR EVALUATION
Ideally, parameters for evaluation will be created during the FORMATION stage of development, so that the collaboration can use them to guide its progress. During this initial phase, the collaboration members should decide on key elements for monitoring the collaboration, such as the following:
1. the work of the collaboration
· problems and needs to be addressed by the collaboration
· the specific goals and objectives for the collaboration
· the activities and steps to accomplish the goals and objectives
· the time period during which specific things are to be accomplished, or deliverables produced
· the process by which things are to operate and get done
· the assignments and roles that collaboration partners agree to assume
· an understanding of what will constitute success and what specific outcomes are to be achieved
2. the process of collaborating
· structure and operating procedures
· methods of communication and inclusion of members
· methods of accountability to the membership base and the larger community
· methods of resolving conflict
· methods of leadership and membership development
· methods of decision-making
3. collaboration development and image
· resources needed and plans to acquire them
· developmental milestones for each phase: formation, implementation, maintenance, termination
· efforts to keep the public informed about the collaboration
· identification of key targets or opinion leaders who need to recognize the collaboration
Once these decisions have been made by the collaboration, an evaluation team can be assembled, and the evaluation process and instruments created. In addition to examining 1) the work of the collaboration, 2) the process of collaborating, and 3) collaboration development and image, the evaluation may also want to examine:
4. the context for collaboration - initially and at later points
· economic and political conditions: initial, and any changes over time
· issues giving rise to the collaboration and their relevance later in the process
· community or constituency relationships
· power dynamics within the collaboration and between the collaboration and its targets
5. the collaboration's impact on external goals
· measures of impact on key problems or issues
· community awareness of collaboration impact
· effectiveness or appropriateness of specific tactics and strategies
· target's recognition of and responsiveness to collaboration effort
Some or all of the above parameters should be assessed continuously. However, at least annually, a formal evaluation is needed to provide useful feedback to the collaboration, indicate accomplishments, and areas for further improvement. It could be a time for celebration and reflection, taking both the credit and the blame.
B. MONITORING AND COMPLIANCE
Collaboration evaluation will be facilitated by ongoing data collection, documentation and record-keeping. These materials can serve to chronicle collaboration development, processes and strategies used, and participants involved. At a minimum, collaborations should keep track of the following items:
operating procedures or by-laws (and their use / relevance)
memoranda of understanding about key leadership roles and division of labor
collaboration agreements or subcontracts for each participant, particularly if funded
membership agreements that delineate tasks for members and expectations of the collaboration
coalition database including a mailing list
job descriptions and qualifications for staff and leadership
chronological log of phone calls, faxes, mailings, and meetings
evaluation materials including member surveys, content analysis of meetings, etc.
minutes, agendas and sign-in sheets from each meeting, at each level, including issues discussed, decided, followed-up, or postponed, and next steps
records of negotiating sessions with external targets, or meetings with outsiders regarding the collaboration's business - record of decisions and commitments made
all correspondence to/ from the collaboration to members, funders, public, targets
funding proposals and budgets
publicity and media coverage of collaboration work
Building the collective history and memory of a collaboration requires ensuring that old and new participants have access to shared information. Records and data on the collaboration should be stored centrally, in a secure system maintained by staff and / or leaders. It should be easy to locate records on the collaboration should they be needed by members or funders. As a standard procedure, staff should provide collaboration leaders with activity and funding reports, as well as copies of relevant correspondence, evaluation materials, and certain key documents. Because collaborations operate on the basis of mutual accountability, the responsibility for maintaining accurate records and files is a critical task. Using written evaluations and reports can facilitate collaboration growth and the mutual respect of the membership.