MENOA Annual Narrative Report: 01.01.2019 - 31.12.2019 N TOS-EUTF-NOA-REG-03-01

In 2019, the MENOA team took over data entry on EUTF common output indicators from DG NEAR
and IPs, at DG NEAR’s request. This work included:

Disaggregation definitions for EUTF common output indicators. In 04 2018, the MENOA
team had developed a spreadsheet of proposed shared technical definitions of
disaggregation by age, gender, beneficiary type and service type for the 41 EUTF common
indicators for Altai and as ameans of developing shared disaggregation definitions to support
the indicators applying to alltthree EUTF windows. These were shared and further developed
with Altai in November and December 2018. In January 2019, following several requests for
additional meetings and discussion, the MENOA team refined the definitions again and
shared them with Altai in an attempt to finalize the disaggregation and to share with DG
DEVCO and DG NEAR. Due to a complex travel schedule and other priorities, Altai was not
able to meet and discuss the proposed definitions in this period. MENOA then shared the
proposed definitions and our approach to aggregation and disaggregation with DG NEAR on
13/02/19. DG NEAR agreed that the shared definitions for age, gender and beneficiary type
were suitable and final. Shared definitions for service types were more challenging due to
the different activities, objectives, priorities, and ways of working in each window. On
11/04/19, DG NEAR decided that we will not pursue a shared definition of service type further
so that data collection could proceed in the NOA window based on a shared definition of age,
gender, and beneficiary type only.

AKVO template and data entry. At the end of 2018, DG NEAR asked that MENOA take over
data entry into AKVO from them and from IPs to ensure more consistent data in the system.
To support this, the MENOA team developed a custom template that included all of the basic
project information AKVO requires for each EUTF project, for example, IP name, title, project
abstract, strategic objectives, chosen common output indicators, among others. In January
2019, the team then used DOAs and LFs to populate this template for each IP and sent it to
them for verification. Once the IP verified and/or corrected the information, the MENOA
team entered it into AKVO throughout Q1 2019. After Q1 and as new projects came on board
or existing projects updated their common output indicators, the MENOA team continued to
update AKVO with new projects and new indicators. DG NEAR could then review the
information periodically and publish it at their discretion. Some key information was
excluded from the template (contractual, financial and location) to keep this information
private. DG NEAR added this information directly into AKVO, reviewed data entered be
MENOA, and could then publish it.

Initially, this entry of basic information did not include actual values for the indicators
themselves, since IPs had not yet reported values. Instead and as noted, it documented basic
project information required by AKVO and the common output indicators each IP had chosen
for cross-window reporting on their project. Throughout 2019, as IPs provided data on these
output indicators either in their own format or using their customized DCT (the agreed,
official format for reporting the data before AKVO entry), the MENOA team entered the data
into AKVO. Once entered, DG NEAR was able to review and publish the data, again at their
discretion. The aforementioned AKVO fact sheet was designed to provide IPs with an easy
resource explaining the data reporting process to AKVO.

Technical memos on common output indicator definitions. In late September, DG NEAR
requested technical review on changes proposed by DG DEVCO and Altai to the long-
established definitions of the 41 common output indicators (COls) used by all three windows
of the EUTF (sometimes called common or cross-window indicators). DG DEVCO and Altai
originally developed these definitions and asked that DG NEAR employ them for NOA IPs in

11

MENOA Annual Narrative Report: 01.01.2019 - 31.12.2019 N T05-EUTF-NOA-REG-03-01

spring 2018. As developed with DG NEAR, the MENOA team worked with IPs to help each
one identify 1-3 of their extant output indicators that could also serve as COls for reporting
to DG NEAR. This was because the vast majority of IPs already had output indicators that
could be aligned with the COls. It was also the case that adding additional requirements after
so many contracts were already signed and projects were in the field was not feasible for IPs.
MENOA thus worked with IPs to align output indicators and built them into IPs log frames
and for reporting purposes, into their DCTs over the course of 2018 and 2019.

DG NEAR contacted MENOA with the information that DG DEVCO and Altai were now
developing an MLS for Sahel and Lake Chad and wanted to introduce and change definitions
of especially some of the EUTF Objectives 2 and 3 COls to accommodate the nature of the
work in this window.

At the same time, however, DG NEAR explained that any revisions to existing definitions or
entirely new definitions could not be adopted for NOA. This was because although the Sahel
and Lake Chad window had only begun to set up an MLS, NOA already had over 40 contracts
with finalised log frames (including common output indicators) in the field and reporting
against their indicators.

The MENOA team then reviewed and produced a detailed, technical memo on the proposed
changes with a focus on their fit with NOA and with EUTF priorities more broadly. The MENOA
team then attended a technical meeting to discuss the proposed changes in Brussels and to
offer feedback and comment in person (03/10/19).

VI. EUTF NOA Monitoring Reports: Data Analysis and Reporting
This work contributed to

Result 1.1.4 Data for the defined indicators are collected, processed and reported regularly
and delivered

Output 1.1.4.8 Analytic reports (regional level) and
Output 1.1.4.9 Analytic reports (national level).

In 2019, the MENOA project and DG NEAR developed a timeline and format for the EUTF NOA
Monitoring Reports. The purpose of these reports is to provide DG NEAR, the full Commission, EU
Member States, EUDs, IPs and the public with up-to-date information on the progress of EUTF
implementation in NOA and to monitor results and learning, including progress on outputs and
immediate outcomes.

In Q2 2019, the MENOA team developed an analytic approach, analytic tools, data cleaning
and verification protocols and a format for NOA Monitoring Reports (NMR1). Each IP reports
to DG NEAR against their individual project log frame that, in the majority of cases, has been
aligned with the EUTF NOA objectives. As such, the team needed to develop tools that would
support assessment of progress at different levels, capturing and integrating quantitative and
qualitative data provided by the IPs and multiple international sources to contextualize
factors affecting progress by country and region. The team thus developed and refined
criteria and analytic tools to analyse the broader country and regional context, progress at
the individual contract/project level, and at the country and regional levels, as well as a
detailed quantitative tool to capture the allocation and distribution of funds by country,
partners, and strategic objectives to drive and produce a portfolio analysis of the EUTF NOA
contracts.

12

MENOA Annual Narrative Report: 01.01.2019 - 31.12.2019 N T05-EUTF-NOA-REG-03-01

 

» Following receipt of over 30 separate IP progress reports beginning on 05/04/10, the MENOA
team employed the analytic approach and tools to assess the progress of each
contracts’/projects’ implementation, outputs and immediate outcomes (where reported). In
May, DG NEAR sent another tranche of approximately 15 progress reports to analyse and
aggregate, followed by several other progress reports in small groups through late June.
These analyses were all conducted and formulated into findings about the entire NOA
portfolio, and at the country and regional levels and used to produce the first EUTF NOA
Monitoring Report in June 2019.

« In October and November, DG NEAR shared 45 new IP monitoring reports for analysis and
inclusion in the second EUTF NOA Monitoring Report (first draft November 2019). Both
reports include a set of portfolio, country-level and regional findings and initial lessons
learned.

»  Drafts of both reports were closely reviewed and commented on by DG NEAR, EUDs, and IPs.
The final version of the first report was posted to the EUTF website; the second report will
be finalized and posted in Q1 2020.

VII. EUTF NOA Monitoring Presentations
The presentations described below contributed to

+» Result 1.1.4 Data for the defined indicators are collected, processed and reported regularly
and delivered part of

» Output 2.1.1.6 Ad-hoc in-person mtgs with DG NEAR.

At DG NEAR’s request, the MENOA team developed multiple versions of detailed presentations that
describe and explain the status of Implementation and results from the investments of the EUTF in
NOA. The presentations drew on NOA Monitoring Reports 1 and 2 to share key findings with the
EUTF Board, the EUTF NOA Operational Committee and the EUDs. They focused on progress made
on implementation of the EUTF NOA strategy overall as well as on specific results at the project,
country and regional levels as of June 2019 for the first report and as of November 2019 for the
second report. As acomplement to the written reports, the presentations aimed to highlight for the
EUTF Board, Operational Committee and EUDs the most up-to-date progress on implementation
and results, as well as cross-cutting findings regarding the entire portfolio of contracts in NOA. DG
NEAR received a separate, professionally formatted presentation and detailed talking points prior
to each meeting and MENOA team members travelled to Brussels in June 2019 to present the
findings to the Board; in July 2019 to present to the Operational Committee; and again in December
first to present to the Operational Committee and then to share and discuss findings with the EUDs
two days later.

Vi. Project Management and Staffing

Activities focused on project management and staffing helped contribute to

» Result 2.1.1 EU headquarters, EU delegations and project implementers have increased
knowledge and skills in the area of project design, monitoring and evaluation and

» Output 2.1.1.5 Regularly scheduled and Ad hoc phone mtgs with DG NEAR
Project Management

To support on-going project oversight; delivery of technical support to IPs, EUDs, and DG NEAR;
and high-quality reporting, key project management activities included:

13

MENOA Annual Narrative Report: 01.01.2019 - 31.12.2019 N T05-EUTF-NOA-REG-03-01

Weekly MENOA team meetings to review work plan progress, determine leads of key
project tasks, identify challenges and gaps, and brainstorm solutions and means of ensuring
continued support and reporting to IPs, EUDs, and DG NEAR

Monthly and then weekly check-in meetings with DG NEAR to develop project
deliverables, plan for upcoming missions and reports, address new needs as they arise and
discuss ways to provide needed technical support to all project partners

Proposed regular meetings with EUDs. In January, DG NEAR requested that MENOA
increase communications with EUDs to support up-to-date information sharing on the
‘state-of-play’ with each IP in the relevant country. MENOA reached out to each of the four
relevant EUDs and requested a monthly meeting in February 2019 and offered to produce a
regular written update. In practice, EUDs preferred not to hold monthly check in meetings
or receive more reports. They instead preferred that we maintain our MENOA practice of
putting them in copy on all email communications with IPs and including them in some
phone meetings with IPs working in the relevant country as a means of keeping them up-to-
date.

Continuous support to IPs via the “M&E helpdesk” where IPs, EUDs and DG NEAR can
request technical support and ask questions regarding Action Fiche, log frames, DOAs,
DCTs, analysis and reporting or any other topics related to the EUTF NOAMLS that they
might want to discuss

Quarterly reports following the calendar year to keep DG NEAR up-to-date on the progress
of project activities and results

An annual, in-person internal team training on the MLS, with a specific focus in 2019 on
DCTs, data collection and cleaning, aggregation and analysis and providing input on tracking
progress at the strategic level via the eMGPP project

Adoption at the February 2019 PSC of an updated MENOA work plan and log frame as well
as Annual Report 2018, followed later in the year by budget revisions (to support additional
staff; see Staffing below). The original work plan was structured around primarily ad hoc
activities. The updated versions reflect the specific tasks, subtasks and timelines as aligned
with planned results (documented in the updated MENOA project log frame) needed to
support the development and use ofthe EUTF NOA MLS

Project Steering Committee meeting, Brussels, Belgium. On 1 February 2019, MENOA
condurted a Project Steering Committee meeting with | from DG NEAR and
from ICMPD. The focus of the meeting was
the review of 2018 activities in the annual report, the current ‘state-of-play’, the updated
workplan and budget alignment to reflect the updated structure, activities and staffing of the
MENOA project and planned activities for 2019. The MENOA team also participated in akkick-
off meeting for the eMGPP project following the Steering Committee meeting to introduce
and discuss the planned objectives and activities of eMGPP. This meeting included from DG
NEAR end from ICMPD == |

2019 Project Steering Committee meeting, Brussels, Belgium. On 19 November 2019, the
MENOA and eMGPP teams conducted a joint Project Steering Committee meeting with |
from DG NEAR and
from ICMPD. The purpose of this meeting was to review 2019

14

MENOA Annual Narrative Report: 01.01.2019 - 31.12.2019 N T05-EUTF-NOA-REG-03-01

activities on both projects and look ahead to planned activities for 2020 in relation to the
conclusion of the EUTF in 2020.

Staffing

To support additional capacity on the MENOA team and to meet the demand for log frame
alignments, updates and DCTs; IP’s and EUD’s needs for technical assistance, support and tools;
as well as analysis and reporting, the team conducted two rounds of vacancy announcements,
interviews, hiring and on-boarding of the following new team members:

A new MEL Officer (P3) to oversee and coordinate the project work plan; ensure the timely
delivery of planned deliverables; respond to requests for ad hoc meetings and deliverables
and contribute to data collection, analysis and reporting (October 2019)

A new Associate Project Officer, Monitoring and Evaluation (P1) who is fluent in French,
English and Arabic, to support project administration and activities (December 2019)

This very considerable investment of time resulted in the hiring of two very strong new team
members to support the MENOA project. Unfortunately, in November the MEL Officer had to
resign suddenly due to health reasons. A new TOR was immediately posted and a new round of
applications collected, with plans to hire anew MEL Officer in Q1 2020. For 2020, MENOA will
also place a resource person in the ICMPD Brussels office to increase availability to attend ad hoc
in-person meetings upon demand.

IX. Additional/Ad hoc Activities

DG NEAR’s Contract Planning and Monitoring tool. Beginning in April 2019, the MENOA
team reviewed and updated DG NEAR’s tool with information on the status of each EUTF
NOA project’s log frame, DCT and AKVO as well as descriptive information on project status.
The MENOA team completes this approximately twice per month and sends itto DG NEAR to
keep them up-to-date on each project. DG NEAR also updates it with new contracts, dates
and other information to help ensure MENOA is aware of the status of new and existing
contracts.

EUTF Mid-term Evaluation. At the request of the external evaluator, the MENOA team met
with the evaluation team several times throughout 2019 to provide information on the
development, structure, implementation and uses of the EUTF NOA MLS. This included
explanations of the log frame alignment processes, DCT development and use, technical
assistance to IPs, data collection and analysis for the NOA Monitoring Reports, among other
topics. In March 2019, DG NEAR also asked that the team review the evaluator’s mapping
report, which the team completed in a very short two-day turnaround as requested.

MENOA ROM. In November 2019, DG NEAR informed MENOA that there would be a ROM
mission to assess MENOA’s work. To support this effort, multiple members of the MENOA
team and the broader ICMPD team provided interviews to the ROM lead, as well as extensive
documentation and explanation of project process and deliverables.

15

MENOA Annual Narrative Report

01.01.2019

Challenges and Solutions

- 31.12.2019

N TO5-EUTF-NOA-REG-03-01

During 2019, the MENOA team encountered, and developed solutions to, the following project
challenges:

(ee 1 SJa7=12H5 Solutions

1.

Insufficient project team capacity to
meet the daily demand for technical
support from project partners

High demand from IPs for additional
support on log frame alignment and
data reporting coupled with
unfamiliarity with MEL and how an MLS
works, non-responsiveness to follow up,
and/or non-incorporation of technical
input and suggestions on log frames

Inconsistent receipt of IP progress
narrative reports and data for analysis
in NOA Monitoring Reports and
requests for inclusion of new reports
piecemeal/after cut off dates

Inconsistent information flows and
communication between DG NEAR and
MENOA regarding feedback on
deliverables, IP projects’ contracting
status, log frame finalisation or
approval (with repercussions for DCTs),
new project needs, and ad hoc requests

Added an MEL Officer to increase project oversight and
implementation capacity and replaced an outgoing
Project Officer with a new, very strong and highly
skilled one

Continued to offer daily support to IPs and to initiate
additional follow as a means of extending the strong,
positive working relationships already established and
to encourage increased communication flow.

Developed a new approach to and resources for in-
person MLS workshops that include review of what an
MLS and how it works to deepen IP understanding of
its purpose and uses for not only monitoring but for
strengthening implementation and results.

Produced a new workflow document to supplement
the published 2017 guidelines and provide IPs with a
short, step-by-step diagram of the alignment and
reporting processes as well as roles and responsibilities
of the IP, EUD, DG NEAR and the MENOA team in these
processes

Agreed with DG NEAR that they would share progress
reports to MENOA on a rolling basis rather than
delivering a large tranche at once to support regular
analysis and efficient development of NOA Monitoring
Reports.

Set stricter, forward looking deadlines for receipt of IP
progress reports and policy of including any received
after deadline in the next report

Implemented exchange of Contract Planning and
Monitoring table to provide DG NEAR with detailed
information on the status of each project twice/month
on both the MENOA and DG NEAR side.

Continued to hold regular, monthly phone calls plus
often weekly ad hoc calls between the MENOA team
lead and DG NEAR Programme Officer to increase
communication and provide clear, up-to-date
information on IPs and other project developments as
well as to request feedback on delivered products.

Increased touch point meetings to weekly with new
Programme Officer after arrival in September 2019.

MENOA Annual Narrative Report: 01.01.2019 - 31.12.2019 N T05-EUTF-NOA-REG-03-01

Cooperation with Other Actors

 

As in 2018, the MENOA team continued to cooperate with the following actors (in addition to DG
NEAR) during 2019:

DG DEVCO to review revisions to common output indicators and definitions for the Sahel and
Lake Chad region and to review the mid-term evaluation Mapping report

EU Delegations in Egypt, Tunisia, Libya and Morocco to align IP projects with EUTF NOA
strategic and specific objectives as well as review, refine and/or develop IP project log frames
and DOAs with a focus on ensuring each IP developed a clear, feasible and relevant set of
project outputs, outcomes, and indicators

EU TF NOA IPs to provide in-person, written and phone-based support on alignment of their
project log frames with the EUTF NOA strategic and specific objectives; data collection,
reporting and DCT usage; and other ad hoc requests for technical support

Altai MLS support team to EUTF Horn of Africa and Sahel and Lake Chad windows to support
their data validation mission to Libya, provide Libya log frames and DOAs, review proposed
revisions t0 common output indicator definitions and disaggregations for Sahel and Lake
Chad and to provide input on their lessons learned exercise for DG DEVCO

MENOA ROM team to provide them with multiple interviews and documents and answer
detailed questions on EUTF MLS processes and supports

Planned Activities 2020

 

In 2020, the MENOA project plans to develop and implement the following key activities:

Align or revise alignment of new or existing NOA Action Fiches and IP project log frames and
DOAs with the EUTF NOA strategic and specific objectives and support refinement of output,
immediate outcome and indicator sets to ensure they are relevant, feasible and reportable.

Develop additional customized DCTs for new or existing projects based on their log frames and
to review, clean and ensure consistency of data reported.

Validate and enter into AKVO updated common output indicator data from NOA projects to be
reviewed and published by DG NEAR.

Refine and re-share the log frame alignment and data reporting workflow document to support
and deepen IP and EUD understanding of these key MLS processes and benefits of project
monitoring in their own work.

Conduct in-person MLS workshops for IPs working in Tunisia, Libya, Morocco and regional
projects (in French for Tunisia and Morocco) also to deepen understanding of the EUTF NOA
MLS and to support data reporting to DG NEAR for analysis and inclusion in NOA Monitoring
Reports. If desired, MENOA can also provide a fifth workshop for IPs working in Egypt.

Implement EUTF NOA resource webpage developed in 2019 to be linked to the EUTF webpage,
consolidating all MLS NOA tools and resources in an easily accessible site for IPs and EUDs to
use.

Provide support to the eMGPP project, especially development of aggregated monitoring data,
original survey data, and third-party migration data (via the data missions and inventory) to
assess progress on EUTF NOA strategic and specific objectives.

37

MENOA Annual Narrative Report: 01.01.2019 — 31.12.2019 N T05-EUTF-NOA-REG-03-01

Convene monthly meetings with DG NEAR to ensure smooth and consistent project
communication and development of project deliverables.

Report progress on individual project log frames, DCTs and AKVO and overall progress to DG
NEAR using the Contract Planning and Monitoring tool and receive updates on contracting from
DG NEAR using the same tool.

Continue weekly internal MENOA team meetings to ensure smooth and consistent project
implementation and internal coordination meetings with the eMGPP team to support their data
collection and analysis.

Produce qualitative data analysis using a qualitative data analysis software (QDAS)-based
database to organize, code and assess IP narrative progress reports and to contextualize findings
as a key input for NOA Monitoring Reports.

Conduct quantitative data analysis (aggregation, frequencies and cross tabs) of indicator data as
provided by IPs and using the EUTF NOA database developed for the EUTF NOA as another key
input for NOA Monitoring Reports.

Develop NOA Monitoring Report 3 and 4 (June 2020 and at DG NEAR’s discretion, December
2020) based on qualitative and quantitative data analyses and cross-cutting themes and
findings, including a EUTF NOA portfolio analysis, country reports, regional report and lessons
learned at each level in a highly professional format.

Develop and present to the EUTF NOA Operational Committee highly professional slides decks
reporting results and learning from NOA Monitoring (June 2020 and at DG NEAR’s discretion,
December 2020).

Produce a MENOA lessons learned exercise focused on lessons related to how to develop and
implement an MLS for monitoring and reporting progress on a large-scale regional migration
initiative.

18