Turkey: End of project Evaluation

1. Summary

Project Number:

207834

Country:

Syria

Area of implementation

Azaz, Aleppo Governorate

Project title:

Building the Resilience of Women and Children through Child Protection and GBV Prevention in Northern Aleppo

Duration:

12 months

Start Date:

January 2017

End Date:

December 2018

Primary Sector of Intervention:

Protection

Type of Implementation:

Direct Implementation by World Vision Turkey

Evaluation requirements:

Summative Final Evaluation

Donor:

Irish Aid

Budget:

USD 8,000,000

Evaluation Methodology:

Quantitative: Household survey

Qualitative: Key informant interviews (KIIs), focus group discussions (FGDs)

Desk review of secondary data and sources

Evaluation Users:

WVI, Irish Aid, WV Ireland

Evaluation Dates:

November 2018- December 2018

Anticipated draft evaluation report release date:

December 6, 2018

2. Background Information

Project Background

In Syria, there are 13.1 million people in need of humanitarian assistance, including 5.3 million children under the age of 17.^[1] March 2018 marked the start of the 8th year of the humanitarian crisis in Syria and, despite reductions in the level of hostilities in some parts of the country, violence has remained or even increased. Internally Displaced People (IDPs) figures have risen to 6.6 million, in addition to 2.3 million people living in hard-to-reach, militarily encircled locations.^[2] Displacements trends point to increased IDPs moving to NWS as nearly 95% of the 1.5 million most recent IDPs have sought refuge in the northern parts of the country, including the 50,000 IDPs from the eastern Ghouta conflict^[3] and 10,064 individuals, including women and children, who were evacuated from South West Syria to Idleb and other areas in North West Syria. These new IDPs arriving join the almost one million longer-term IDPs already residing there. Between the existing IDP populations, the host community and new arrivals, the resources of humanitarian organizations are overstretched.^[4] The conflict remains the principal cause for the widespread humanitarian needs, decimating infrastructure, disrupting public services, and displacing households.

In Syria, the magnitude of need is great in the North, where conflict is high. The NWS governorates of Idleb, Aleppo and Hama are a volatile area with large IDP movements. The effect of these two dynamics is not only furthering the loss of life and human suffering, but also straining civil infrastructure. The urgency of the current moment could not be understated. In NWS, operational space is winnowing as the Syrian Regime and Turkey continue to make territorial gains. With this comes the collateral need for life-saving services, which can rapidly respond to the conflict and the ensuing IDPs. As conflict lines are shifting, IDP populations are compressing, becoming denser in fewer locations which further stresses and overwhelms the local capacity to respond.

The crisis has created disproportional and unrestrained attacks that inflict extensive hardship and destruction upon the civilian population. Various forms of violence permeate daily life, particularly affecting women and children, creating a huge need for psychosocial support.^[5] The conflict has also had a devastating economic impact and has contributed to worsening living conditions across the country. This has resulted in a lack of purchasing power, limited employment opportunities and widespread displacement, which has resulted in a number of negative coping strategies, such as early marriage and child labor. According to the 2018 Syria Humanitarian Needs Overview (HNO), violence against children constituted one of the key protection concerns: child labour was reported as occurring in 82% of surveyed communities; out of desperation, girls and boys of all ages are reported to be performing work beyond their mental, social or physical capacities which may deny them their basic rights. According to the HNO, 85% of sub-districts report early marriage as a major concern caused by attempts to relieve the economic burden on families, or as a perverse ‘protection strategy’ to prevent girls being forcibly taken by fighters. This was followed by domestic violence (71%) & sexual violence (52%). While women & girls (especially in female-headed HHs) are disproportionately affected by all forms of GBV, boys are also at risk.

The combined effect on the Syrian population of ongoing hostilities, physical and economic insecurity, have all become powerful stress factors in increasing the population’s need in psychological assistance. The HNO 2018 noted that there is also a dire need to increase availability of mental health services in Syria. Local NGOs have expressed the urgent need for psychosocial support for survivors of GBV, indicating the need for taking more precautions to ensure a safe environment for IDP women. Survivors of physical and emotional abuse are not only in imminent danger of sexual exploitation, but are also in urgent need of medical and psychosocial assistance resulting from such exploitation. A survivor-centred approach to GBV response and prevention remains one of the main priorities for humanitarian agencies.

As IDPs create greater strain services have become overstretched, damaged/inoperable, or unable to keep pace with the magnitude of need. The needs of conflict-affected people, especially mobile IDPs, are urgent yet broad. When displaced, households often lose everything, are traumatized, and are displaced to foreign areas, which are themselves under conflict. For the displaced, needs span across sectors and carry with the added challenge of meeting them immediately while mobile.

To address these needs, the ‘Building the Resilience of Women and Children through Child Protection and GBV Prevention in Northern Aleppo’ aims to strengthen and continue its women and child protection services through protection activities, inclusive of psychosocial support and access to safe spaces for conflict-affected women, girls and boys children, along with awareness session with men and community members and informal leaders. The proposed project also includes a community-based protection component to raise awareness from within the community in the long term.The proposed activities are in line with the HRP proposed response, namely the provision of specialized protection services, including counselling or individual PSS services, referrals to other partners for other life-saving services (e.g. health).

Project’s Beneficiaries and Geographical Coverage

WV proposes under this grant to continue and improve its existing activities in seven Child-Friendly Spaces (CFS) and seven Women and Girls Safe Spaces (WGSS) in the camps of Bab Alsalama, Bab Alnoor, Shamareen and Alharameen camps and WV is the only provider of GBV and CP services there. The population of target camps is currently estimated at 22,980 direct beneficiaries (1950 men, 9590 women, 5720 boys and 5720 girls) in A’zaz district, Aleppo governorate, Syria with and average household size of 6 people. The mobile services are expected to reach an additional 10,000 more beneficiaries in the immediate vicinity.

Achieved Beneficiaries Numbers

Beneficiaries’ Category

Year 2017

Year 2018

Total

Children 3-13

4464

7880

12344

Youth 13-15

197

422

619

Youth 16-22

180

350

530

Women

512

977

1489

Men

4421

9212

13633

Totals

9774

18841

28615

Proposed Intervention

1.Child Protection Response

The proposed activities will also respond to child protection concerns in the area in the following model:

· Provision of awareness raising and parenting skills sessions and events

Trained staff will facilitate awareness raising sessions with parents and caregivers within marquees of collective shelter and our child-friendly spaces to talk about child protection in emergencies topics, like psychosocial distress, early childhood development, unaccompanied and separated children, child labour and parenting skills among others. These sessions will be used to inform the community about our child protection centers and services; and this will also ensure outreaching the most vulnerable children in the area.

· Provision of targeted, age-appropriate psychosocial support, structured play and learning activities for IDP children

The seven CFS’s will provide structured play and PSS activities. Activities are designed to work with the following age groups separately: (4-7, 8-11, 12-14). There are dedicated animators who ensure that children with special needs are included in the activities.

· Establishing a child protection committee

Drawing on a recent successful experience whereby WV created a protection-focused camp committee and in coordination with the camp management, WV will facilitate the creation for a child protection committee in one of the target camps. Those who actively participate in the child-protection awareness raising sessions and show concern and enthusiasm for children issues in the camps will be approached to become members in a child protection committee. The committee will receive focused trainings in child’s rights, early childhood development, child protection in emergencies and the duties of parents, caregivers and communities towards children. The committee will assist in identifying and referring cases of distressed, abused or exploited children. They will provide information about available services, disseminate information and messages to children and caregivers, and they will receive guidance on how advocate for increased child protection measures at the camp level.

2.GBV Response

The GBV response will take place in the following model:

· Increased awareness and action on GBV prevention, mitigation and response among women, men, girls and boys

Trained staff will facilitate awareness raising sessions with women and adolescent girls within marquees of collective shelter and the WGSS to talk about gender-related and GBV topics particularly women economic empowerment, (sexual) exploitation, domestic violence, child marriages, and among others. Such sessions are used as entry points to build trust and interest leading to outreaching at-risk or survivors of GBV. Visiting our centers, those at risk or survivors of GBV will meet experienced facilitators trained in receiving and responding to GBV disclosures.

· General and targeted, age-appropriate, culturally safe GBV psychosocial support (PSS) activities

In five existing women and girls’ safe spaces, WV will provide comprehensive PSS services which includes age-appropriate activities, community-based support, the promotion of positive coping mechanisms, resilience and self-esteem as well as targeted focused PSS and counselling. To avoid the socially alienating and stigmatizing associations with the word “psychological”, this component will be presented to the community as “social activities.” A culturally safe environment for recreational and collaborative activities that seek to restore normalcy (for example artistic and handcraft works, discussions about gender awareness, gender roles, women and girls’ new roles and successful coping examples from the community or similar contexts like Bosnia, Palestine, etc) will be provided. Such activities will be conducted by trained female facilitators. Staff in Turkey supervising these activities are currently engaged in a capacity building initiative about provision of PSS to GBV survivors at the GBV Sub cluster level in Gaziantep. The activities will seek constructing meaning from women and girls’ shared suffering and finding positive, adaptive strategies to cope with their situation, ensuring that women experiencing high-levels of psychosocial distress and/or abuse get one-on-one counselling.

· Utilizable life skills and vocational courses

In line with the recommendations of the UNHCR study on the MHPSS of Syrians[6], interventions aimed at improving livelihoods may significantly contribute to improving the mental health of IDPs, perhaps more than any psychological intervention. This proposed intervention will focus on providing pragmatic vocational courses coupled with a contextualized life skill supplement. Under this grant, WV will continue its community-supported vocational trainings which are embroidery, sewing, hairdressing, wool Knitting, handicraft and recycling.

· Improved GBV multi-sectorial coordination and referrals

WV is already part of a service mapping conducted by GBV Sub cluster. Through designated field focal points, WV will use this platform to ensure that women and girls at risk and GBV survivors are more aware of referral mechanisms and referred to an appropriate quality services.

· Day Care services

As the project will be targeting women from all stages in life, day care services will be provided by and coordinated with child protection (mobile) teams as many women might be unable to attend due to a lack of alternative childcare arrangements. We have received complaints from women that their husbands would only allow them to visit women centers if they take their children with them. Hence, integrating child protection activities with women protection is expected to improve women attendance numbers and will also positively impact the socialization and normalization of children of distressed mothers.

Objectives and Outputs

Outcomes

• Girls and Boys affected by conflict have improved access to psychosocial rehabilitation services and safe spaces for learning and play

• Children (13-15) and youth (16-22) have opportunities for growth and development, enabling them to make safe career choices

• Parents/ Caregivers have increased awareness and action on Child Protection issues and services available

• Increased awareness and action on GBV prevention, mitigation and response among women, men, girls and boys

• Women and girls at risk and GBV survivors are more aware of referral mechanisms and referred to an appropriate quality service provider

Outputs:

• Child Friendly Spaces established and operational

• Psychosocial support mechanisms and services strengthened

• Children (13-15) and youth (16-22) have opportunities for growth and development, enabling them to make safe career choices

• Increased Life Skills among targeted children (13-15) and youth (16-22)

• Increased awareness raising of child protection in emergencies

• Increased community awareness on GBV through campaigns/ informational sessions

• Women's Support Groups formed and functioning

• Women and girls at risk and GBV survivors are more aware of referral mechanisms and referred to an appropriate quality service provider

• Women's Support Groups trained on referral pathways

• Women's centers are established/ operational

• Vocational Education and Training at Women and Girls Centers

• Provide women who completed IGA activities with protection kits

• Increased linkages with local civil society actors (NGOs, Local councils, camp management entities)

• Development of agreements between WV and local civil society

Indicators to be Evaluated:

· Proportion of girls and boys able to express themselves with confidence and participate actively in discussion Disaggregated by gender

· Children report an increase in positive benefits and changes in their lives Disaggregated by gender

· Proportion of conflict affected girls and boys who are positively engaged and demonstrate improved emotional wellbeing Disaggregated by gender

· Percentage and description of youth groups engaged in skill based community initiatives Disaggregated by age and gender

· Percentage of women who can cite at least three key messages from GBV campaigns Disaggregated by gender

· Percentage of women and men with improved knowledge, attitudes and practices towards mitigating/reporting actions on GBV Disaggregated by Gender

3. Evaluation Target Audiences

The evaluation will serve as a learning resource providing evidence for planning, programming and replication of similar WASH interventions for the following local and international stakeholders:

• Donor: The evaluation will be submitted to the Irish Aid to highlight the effectiveness and impact of the implemented project. The report will show to what extent the objective and the respective sub-objectives are met. It offers the opportunity for WVI to be accountable towards the its donors as the process will be documented in details and information will be generated based on the perception of the community.

• WV Ireland: As the Support office is in direct contact with the Irish Aid, the evaluation report will also be presented to WV Ireland to show the effectiveness and impact of the implemented project and highlight any challenges faced at the level of implementation. The Support Office will be also engaged in the evaluation process by providing feedback and approval on the ToR as well as the evaluation report.

• WVI Syria Office: The data collected from the evaluation will inform WVI on the successes and challenges faced throughout the program implementation. It will also highlight the levels of sustainability and the asses the relationship with main partners. The evaluation will further contribute to generating lessons learned and recommendations that should guide future similar interventions.

• Main partners and collaborators who will first be involved in the evaluation process and provide valuable input throughout the process about their collaboration with WVI and their experience in the project as a whole.
  The finding will also be useful for them as they provide information about the effectiveness and impact of the intervention in their community and recommendations that might be relevant to consider in the coming phase, after the project has ended.

• The evaluation will also help ensuring accountability for all beneficiaries and community members in the areas of operations.

4. Evaluation Purpose and Objectives

The evaluation will be based on the four specific evaluation criteria used by World Vision’s Humanitarian and Emergency Affairs (HEA) which is also based on OECD-DAC criteria: relevance, timeliness, efficiency and effectiveness/impact. The four criteria, as they relate to achieving the outcomes and goal of the project, will also serve as the main framework for presenting the evaluation findings. The proposed evaluation criteria are:

· Relevance- How well did the project correspond to local needs and priorities? How did WV target the most vulnerable populations, and what were the challenges of reaching vulnerable, affected populations? Did the project address priority problems faced by the target areas and communities and was the project consistent with policies of both donors and recipient governments (or agencies)?

· Timeliness- To what degree did the project implement timely Response activities? Did the project respond in a timely manner to the affected community’s needs? What affected timely delivery of assistance (including coordination issues), and what measures were taken by WV to address the issues?

· Efficiency- To what degree did the project provide efficient support project implementation? Were inputs (staff, time, money, equipment, relief goods) procured and/or used in the best possible way to achieve outputs and could implementation have improved? Which main factors influenced the efficiency of implementation such as supply chains, and how did this affect the delivery of aid?

· Effectiveness/Impact- To what extent did the project activities meet their intended purpose? Were outcomes achieved? What is the evidence of both the impacts of the Response as well as the unintended positive and negatives effects on the affected population since the conflict? What real difference has the activity made to the beneficiaries? What has happened as the result of the project.

Developing the analytical framework, i.e. refining and operationalizing the questions posted above, will be done in close cooperation between World Vision and the consultant.

The evaluation will be used to:

· Support accountability of the implementing partners vis-à-vis the donor on the outcomes of the Action and the usage of the funds provided through the provision of the evaluation report.

· Support learning between the implementing partners on the management of their cooperation through the process of the evaluation and the discussion of the evaluation results.

5. Evaluation Methodology

The evaluation methodology is informed by the M&E plan specified during program design and it is aligned with WV standards for ensuring good quality evaluation process. The evaluation methods should be a combination of quantitative and qualitative methods involving the relevant project stakeholders, and beneficiaries at the community level. Data collection methods may include but are not limited to secondary data information (project documents will be provided by WVI), surveys with target population in villages, Key Informant Interviews (KII) and focus group discussions (FGD) with main stakeholders.

Primary data collection is expected to comply with recognized ethical principles. Data are expected to be collected on smartphones or tablets, using Open Data Kit (ODK), Kobo collect, SMAP or similar applications, and processed electronically. The consultant will provide the cleaned primary data set in Excel to World Vision and grants World Vision rights of storage and further usage. The consultant is expected to:

· Provide a time schedule, a breakdown by location and a breakdown of resources needed for primary data collection in the locations covered under this project;

· Hire, train and supervise enumerators, which must include both men and women, and care for their provisions and security;

· Obtain the necessary permissions for gathering primary data from the relevant authorities.

For quantitative methodology specifically, sample size would be calculated based on industry standards with a confidence level of 95% and a margin of error of 5%, statistical significance will be set at 0.05 and confidence intervals should be calculated and added. With a total beneficiaries’ number of 28615 beneficiaries, the sample size is determined to be 380 respondents, plus with 10% oversample, would yield to 418 respondents. The overall sample is then divided proportionally per type of intervention leading to 197 beneficiaries under CFS component and 221beneficiaries under GBV component.

Sampling under each component is then divided proportionally to the operation areas. Finally, randomly selected beneficiaries will be chosen in each area from the beneficiaries’ lists.

The consultant is expected to develop the appropriate methodology and tools that would best capture the objectives of the evaluation.

6. Roles and Responsibility

Team Members and Roles

Role

Evaluation Phase

Responsibilities and tasks

Consultant

Planning

· Develop and submit the evaluation methodology (sample selection, quantitative & qualitative methodology)

· Draft evaluation matrix

· Conduct desk review of key documents in the documentation briefing book

· Finalize question matrix by gathering information needs and feedback from relevant stakeholders and prioritize evaluation objectives

· Develop data collection tools based on the tools used during the initial baseline

· Gather feedback on tools from evaluation advisors and conduct pilot testing of the tools where relevant

Data Collection & Analysis

· Recruit and train data collectors if relevant for both quantitative and qualitative data collections

· Conduct quantitative and qualitative data collection (supported with data collectors where relevant)

· Supervise and assure the quality of quantitative data collection

· Supervise data entry clerk

· Set up a plan for quantitative data analysis, conduct in depth data analysis and interpretation of results

· Provide weekly feedback to WVI team

Reporting, Validation & Follow up

· Draft the Evaluation report

· Prepare power point presentations of the main findings

· Finalize the Evaluation report based on feedback from WVI team

· Submit a 2 pager summary fact sheet (in English and Arabic) to be disseminated to relevant stakeholders.

Data Collectors

· Carry out quantitative and/or qualitative data collection as relevant

· Report to evaluation consultant

MEAL Manager

Planning

· Develop evaluation TOR

· Develop evaluation work plan

· Follow-up on recruitment of evaluation consultant

· Develop documentation briefing book

· Provide feedback on evaluation methodology and matrix

· Approve the methodology and ensure its quality

Data Collection & Analysis

· Provide feedback on data collection tools

· Follow up with the consultant on data collection and analysis, ensure quality of data collection

· Provide feedback on data analysis and ensure the quality of interpretation

Reporting, Validation & Follow up

· Provide feedback on draft evaluation report

· Share internally the draft version of the report and follow up on the feedback received

· Co-facilitate stakeholder interpretation workshop with consultant

· Review and share final version of the report

Project Manager & team

Planning

· Provide input on TOR and research questions

· Provide documents for literature review

Data Collection & Analysis

· Provide feedback on data collection tools

· Coordinate with consultant in relation to data collection logistics

Reporting, Validation & Follow up

· Provide feedback on draft evaluation report and recommendations

7. Logistics

The recruited consultant will be in charge of finalizing logistical issues related to the transportation to the different areas where data collection will be completed.

8. Products

The expected deliverables throughout the process of evaluation include:

1. Evaluation Methodology (including Evaluation Matrix)

2. Budget Breakdown

3. Validated and tested data collection tools

4. Thematic in-depth analysis of qualitative data

5. Data sets (software outputs)

6. Notes and transcription of KII and FGDs

7. Tabulation of results including descriptive analysis, associations and comparison with baseline data

8. Evaluation Report (based on Template provided by WVI)

9. 2 pager fact sheet summary (in English and Arabic)

All collected data is legally owned by WVI and the consultant is expected to hand over all data sets and notes of the interviews or FGD to the organization. The Consultant shall maintain in confidence and protect all information provided to him/her by WVI, its employees, and beneficiaries. The consultant may only disclose the extent necessary to perform the evaluation.

9. Qualifications of the consultant

The consultant (team) must have proven expertise and experience in social research, protection in humanitarian emergency responses, and be able to implement the research in Syria following the required procedures and in the required languages.

Proof of these is to be provided by submitting, together with the application:

· An overview of relevant works

· Working samples

· Contact details for references

· The proposed research team´s CVs.

Requirements in detail:

· Have excellent team leading skills, with the ability to work with a varied team.

· Have expertise and experience in social science research, including mixed methods and notably quantitative and qualitative primary data collection methods

· Have expertise and experience in research, monitoring and evaluation in humanitarian emergency response contexts, preferably related to the Syria crisis, as well as related technical standards

· Have a good understanding of the Syria crisis

· Have permission to work, travel and undertake primary data collection in the project target locations and be able to do so in the language of the primary target groups (Arabic)

· Have proven experience in conducting participatory qualitative and quantitative evaluation studies with superior analytical skills

· Have a good knowledge of data collection software (such as Kobo, SMAP) and statistical packages (such as SPSS)

· Have excellent report writing skills

· Have excellent written and spoken English and Arabic

· Hold a minimum educational qualification equivalent to a master´s degree in a relevant field.

Application

The consultant should submit a proposal comprising the following:

· A technical proposal which indicates how the consultant is going to undertake the activities highlighted in the ToR, indicative budget with initial indication of the enumerators and timeframe required

· A capacity statement detailing the consultant´s ability to deliver a quality evaluation report within the given timeframe, including an overview of relevant work and technical experience

· At least 2 samples of previous relevant works undertaken, including at least 1 evaluation report that was 100% led by the lead consultant

· Contact details for 2 references for similar assignments done not more than 2 years ago

· CVs of the key personnel on the evaluation team

· A financial budget with explanations about the line items

· Any appendices the consultant sees as relevant to the application.

Applications should be submitted electronically to gaziantep_procurement@wvi.org by 21th November 2018.

Child protection & data confidentiality

The external evaluator, along with all enumerators, will be required to sign and follow World Vision International´s child protection standards and protocols of behavior, which will be provided to the research team selected.

All primary data collected by this evaluation process is to remain confidential and is not to be shared with third parties.

10. Budget

The consultations fees available for this end of program evaluation will include transportation, logistics and accommodation fees identified by the consultant, as well as recruitment of data collectors/data entry clerks and data collection, data entry, translation fees and report writing. The accommodation fees will include venue rental, meals and coffee breaks for FGDs. A budget breakdown needs to be submitted by the consultant.

^[1] Humanitarian Needs Overview 2018

^[2] Moumtzis, P. (April 2018) Syria Humanitarian Operations. OCHA Presentation.

^[3] UNHCR: Syria Flash update on recent events, 5 April 2018

^[4] OCHA Situation report, 29 March 2018

^[5] Humanitarian Needs Overview 2018

[6] Culture, Context and the Mental Health and Psychosocial Wellbeing of Syrians, UNHCR, p39. (Available online at http://www.unhcr.org/55f6b90f9.pdf )

HOW TO APPLY:

World Vision is one of the world’s leading humanitarian development and advocacy organization and operates in nearly 100 countries worldwide.

World Vision (WV) is inviting potential consultants, which are all strong and highly regarded within the industry to participate in and submit the following as a response for End of project Evaluation :

1- Full technical proposal, including company profile and /or CV's

2- Full financial proposal based on deliverable, with breakdown on every applicable Cost.

Interested consultants should submit their proposalsto the following email: Gaziantep_Procurement@wvi.org as per below instructions:

Subject of the email must be exactly in the following format:

RFT1552-HPP END OF PROJECT EVALUATION

Deadline for submitting the proposal: Monday, 26th November 2018.