Joint evaluation of AFSC’s projects ‘BADER: Youth Civic Engagement in Jordan’ and ‘Palestinian Youth: Together for Change (PYTC 3)’
1. Introduction
American Friends Service Committee (AFSC) is a US-based Quaker organization devoted to service, development, and peace programs throughout the world. It works with people and partners worldwide, of all faiths and backgrounds, to meet urgent community needs, challenge injustice, and build peace.
Through its regional office in Jordan, AFSC implements programs in the Middle East. AFSC’s projects ‘BADER: Youth Civic Engagement in Jordan’ and ‘Palestinian Youth: Together for Change (PYTC)’, both with their current implementation period from 01/2020 to 12/2022, are supported by Brot für die Welt, the development and relief agency of the Protestant Churches in Germany.
This joint evaluation is to assess both projects individually, while on a synthesis level generating cross-project learnings and providing an assessment of the organisational and project management set-up of the projects.
The evaluation will be commissioned by AFSC Philadelphia and will be accompanied by AFSC and Brot für die Welt jointly. The evaluation is expected to take place from April to June 2022.
2. Scope, objectives and users of the evaluation
The joint evaluation is to provide an independent assessment of the projects, on a project specific and on a synthesis level, as well as assessing the organizational and project management setup of the projects between the different units and offices of AFSC and Brot für die Welt.
The objective of this joint evaluation is to generate insights that enable project specific but also cross-project learning and exchange. Furthermore, the evaluation is to yield recommendations for the improvement of the organisational and project management set-up of the projects between the different units and offices of AFSC and Brot für die Welt.
Specific objectives are:
- The underlying results logic of the projects is reconstructed and assessed.
The projects have been assessed along the OECD/ DAC criteria. Within this assessment:
Hindering and helpful factors as well as unintended positive and/or negative results have been captured.
Good practices and lessons learned from the projects are highlighted.
The effectiveness of the projects’ organizational management, partnerships, and project management set-up is reviewed.
Recommendations for future programming, organizational management, partnerships, and organizational cooperation.
The evaluation will be primarily used by AFSC Jordan and Palestine and its projects’ partners and youth participants. Further users are AFSC Philadelphia and Brot für die Welt.
3. Evaluation design and methodology
The principles and standards of OECD DAC for a participatory, credible, gender-sensitive and fair evaluation shall be observed. Both quantitative and qualitative methods shall be used. Data collection shall promote self-reflection amongst the target groups. The chosen methods shall be inclusive and respect the social and cultural context of the target groups. In the development of the evaluation design and the choice of methods, correct research ethics need to be applied. The evaluation should be guided by the “do-no-harm” principle. The anonymity and confidentiality of individual informants should be protected. In addition, the evaluator(s) will keep confidentiality of the evaluation's information, data, and findings. The evaluation will have to triangulate information. The evaluator(s) will have the opportunity to dissociate themselves from particular judgments and recommendations. They should acknowledge any unresolved differences of opinion within the team in the report.
The methodical design is to be suggested and described in the technical proposal and refined in the inception report by the evaluator(s). The documentation of the methodical approach is a fundamental component of the evaluation report.
4. Process, reporting and timetable of the evaluation
The exact number of working days depends on the proposed methodology and should be outlined in the proposal in line with the suggested methodology.
4.1 Reporting/ Products of the evaluation
Products to be produced during the evaluation:
- Inception Report, including detailed methodology, evaluation matrix showing with which methods the questions will be assessed, data collection tools and proposed timeline, in English.
- Discussion meeting on inception report.
- Agenda for fieldwork.
- Debriefing presentation on preliminary findings for each project in English.
- Draft final report in English.
- Presentation of preliminary results and recommendations to AFSC and Brot für die Welt in English and gathering of feedback.
- Final report in both English and Arabic.
4.2. Evaluation design/inception report
The draft and final version of the inception report have to be provided in English.
a) The consultant(s) will submit an inception report of approx. 5-8 pages without the annexes
b) The inception report shall contain:
- key data of the evaluation (name, number, duration of the projects to be evaluated, title of the evaluation, commissioner of the evaluation, contractor of the evaluation, date of the report)
- a detailed description of the evaluation design and methodological approach, including the selection of samples, strategies for analyses and collecting data, hypothesis on outcomes and impacts, an analysis of risks, mitigation measures and limitations
- an evaluation matrix detailing the methodological operationalization of the evaluation objectives and questions (annex)
- a detailed actual timeline and field work agenda (annex)
- data collection tools to be used (annex).
c) The inception report will be presented and discussed in a meeting of the evaluator(s) with AFSC and Brot für die Welt.
4.3 Final evaluation report
The draft and final evaluation version of the evaluation report have to be provided in English. Additionally, an Arabic translation of the final report is to be submitted after approval of the final evaluation report in English.
The final report (60 pages max + annex) shall have an Executive Summary of 5 pages maximum. It is expected to contain a section with the assessment of the project along the OECD DAC criteria and related questions for each project as well as a section of evaluation findings on an overarching, synthesis level. As a minimum, it will include the following contents:
· Key data of the evaluation.
· Executive summary: about 5 pages.
· Introduction: the purpose of the evaluation, evaluation scope, and key questions. Short description of the evaluated subject and relevant frame conditions as well as the reconstructed results logic of the projects
· Evaluation design/methodology
· Project specific key results/findings: regarding the questions pointed out in the TOR and structured according to the OECD DAC evaluation criteria for each of the two projects.
· Results and findings on a synthesis level.
· Conclusions based on evidence and analysis.
· Project specific and overarching actionable and realistic recommendations regarding future steps/activities/follow-up. · Lessons learned (generalizations of conclusions applicable for broader use).
· Annexes (ToR, list of persons/organizations consulted, literature and documentation consulted etc.).
The draft evaluation report, specifically findings and recommendations will be presented to AFSC and Brot für die Welt and oral as well as written feedback provided will be integrated into the final report. The opportunity for comment on the draft evaluation report is intended to clarify any issues that staff and partners feel may be unclear or misunderstood and for quality assurance. Any issues on which staff, partners, and evaluators disagree may be dealt with by including an Appendix for staff and partners' comments.
5. Responsibilities and duties
The selected evaluator(s) will enter into two contracts with AFSC, one under each of the projects. However, the evaluation is to be understood and carried out as one integrated evaluation assignment. Please note that the evaluator or evaluation team is expected to travel to both project locations to conduct data collection (Jordan and Palestine). AFSC will manage the evaluation as it will be the organization entering into a contract with the selected evaluator(s). Focal points for the evaluator(s) will be AFSC Jordan and AFSC Palestine Program Managers. Comments to the products of the evaluation will be provided by the project partners, AFSC and Brot für die Welt. The AFSC Regional Director assisted by the Program Managers is responsible for approving the inception report and providing comments on the draft Evaluation report in collaboration with relevant AFSC units and Brot für die Welt. AFSC will:
- Facilitate the necessary documentation.
- Assist in contacting individuals and groups to conduct in-depth interviews, focus groups, etc.
- Provide and cover the cost of venues for focus group discussions (if necessary).
- Book accommodation where required.
The evaluator(s) will:
- Finalize the arrangement of the agenda for interviews, etc.
- Arrange and cover the cost of transportation to the project locations
- Arrange and cover the cost for translation.
6. Profile of the consultant(s)
- Sound experience in carrying out evaluations in the context of development cooperation, using a mixed methods approach and triangulation.
- Deep understanding and knowledge of the Jordanian and Palestine context and youth situation.
- Deep understanding of current youth projects carried out in Jordan and Palestine/ Israel.
- Previous experience of work with a faith-based organization would be an asset.
- Ability to read and write and communicate in English and Arabic.
How to apply:
Submission of evaluation offers
FOR SUBMISSION AND FULL TERMS OF REFERENCE WITH MORE DETAILED INFORMATION ABOUT THIS CONSULTANCY PLEASE REFER TO https://www.afsc.org/sites/default/files/documents/TOR_Joint_Evaluation_AFSC_Jordan_Palestine_final.pdf
Qualified candidates should submit offers to bolimat@afsc.org, dhammoudeh@afsc.org and evaluation_ausschreibung@ewde.de by COB March 30th, 2022, in electronic format in English language. A
The submission of offers from qualified female evaluators is welcomed.
The commissioning party reserves the right to conduct telephone or personal interviews in order to reach a decision. Furthermore, the right to award the contracts at the time of the submission of the offer is reserved. Only complete offers will be considered.
