ROM BACKGROUND CONCLUSION SHEET (ONGOING) ROM ID
Project Title
Country Project Task Manager
290-717
Increased Food Security and Sustainable Livelihoods for Poor and Marginalized Households in Weste
Expert(s)
ROM field visit dates
South Sudan GIRLANDO Paolo
start:
RALL Martin
end:
29 Apr 2013
1. RELEVANCE
10 May 2013
The extent to which the objectives of the intervention are still consistent with beneficiaries’ needs and partners' and donor's policies. 1.1 Does the intervention presently respond to the needs of the target groups? a. Were there any changes in the situation of the target groups and the context which have, or will, influence the relevance of the operation for target groups? b. Have the activities of other actors such as government and donors changed the needs and priorities of the target groups? c. From the target groups' perspective, what is the level of priority of the needs the operation is addressing?
A
B
C
D
The project remains highly relevant to the needs of the target communities in both counties. In Jur River County increased staple crop production (acreage expansion and higher production targets per unit) will reduce food insecurity, particularly for highly food insecure and vulnerable groups. In Wau Town and in Jur River County, households will benefit from the introduction of kitchen gardens and poultry rearing, improving their well-being as well as increasing their income. The aspect of utilization of food has been explicitly added in the project to consolidate the effects of increased agricultural productivity and marketing of food crops on the nutritional status.
There have not been any changes in the situation of the target communities. According to the results of the baseline survey carried out during the inception phase at the start of the project, only 2% of the farmers owned a plough, 67% owned a hoe, 20% a rake, 3% a wheel barrow, 48% an axe and 3% a cart. Only 15.7% of the households had a vegetable garden and only 11.5% had access to water. Of those who had access, only 40% had access to perennial water source and 53% had it within 0.5 km. Most households were severely food insecure: 83.7% had no food in stock in the past one month before the survey; 82% slept hungry; and 80% spent a whole day without food. Only 20.3% reported that they rarely had times when there was no food to be eaten. The average number of months where there was not enough food (hunger gap) is 3. While the target group has numerous high priority needs, such as water, education and health, food security remains their key priority.
Page 1 of 18
1.2 Does the operation presently support the policy (or its development) of the partner government and is it in line with existing policy? a. Have there been any changes in Partner government policy which have had, or will have, an impact on the relevance of the operation? b. Is the operation supporting the development or improvement of a sector policy?
A
B
C
D
Government policy is based on the 2011-2013 South Sudan Development Plan (SSDP), which is an interim Poverty Reduction Strategy Paper. The SSDP is endorsed at ministerial level and is a precursor to a five year plan (2014-2018) currently being developed. The SSDP identifies priorities for Government and donor spending across four pillars focusing on stability and service delivery, namely; governance, social and human development, economic growth (rural development) and conflict prevention and security. This project supports the rural development pillar, which is based on supporting smallholder agriculture.
The State Government has developed a 5 year strategic plan that defines strategies broadly on the basis of identified needs, ranging from infrastructure to marketing system development. These have been incorporated into the project design; for instance, the needs assessment showed that the capacity of the Ministry of Agriculture is limited and the provision of (extension) services to farmers is almost non-existent. The current action has developed a specific strategy to improve extension services to farmers, working closely together with the State Ministry, and including one of their staff members in the project formulation workshop in January 2012. There have not been any changes in policy.
The project provides an opportunity to develop effective and sustainable models for improving agricultural production by smallholder farmers.
1.3 Is the operation in line with EC development policy and strategies? a. Is the operation in line with the latest EU development cooperation policy? b. Is it aligned with EU policy for the specific sector in the country/region? c. Does the operation respect the EU's international commitments such as the Paris Declaration and follow-up? d. Is the operation embedded in and supporting policy dialogue which the EUD/HQ is engaged in?
Note: a = very good; b = good; c = problems; d = serious deficiencies.
A
B
C
D
The EU's strategy in South Sudan is to support the implementation of the SSDP, which is seen as a sound foundation for programming, and to this end a Single Country Strategy (Response Strategy) for South Sudan 2011-2013 was developed. Provided the Government's five-year plan for 2014-2018 builds on the SSDP, the EU Single Country Strategy Paper (SCSP) will simply be updated and no new strategy produced. The SCSP contains Joint Programming Sectors which respond to specific priorities identified under the SSDP pillars and are aligned with the responsible ministries to facilitate partner country ownership. This project supports the Joint Programming Sector: Natural Resources (Food Security). The intervention supports general EU policy on combatting poverty and achieving the MDGs. It is not very relevant to the Paris Declaration as it is implemented by an international NGO in the context of a still very weak and under resourced government. Policy dialogue is thus also very incipient.
Overall conclusion - Relevance
A Page 2 of 18
2. QUALITY OF DESIGN
The internal coherence and validity of the intervention logic, its formalization in a logframe (or other format) and the implementation arrangements. 2.1 Does the present intervention logic still hold true and is it clear and coherent? a. Does a logframe exist and what is its quality? b. Is the operation's underlying intervention logic coherent, clear and realistic? c. Is the approach adopted in the design taking sufficiently into account previous experience and state of the art knowledge in similar interventions? d. Are the resources, capacity and timeframe adequate to achieve the project purpose? e. Does the intervention logic explicitly mention risks and assumptions and are they specific, up to date and holding true? Are risk management arrangements in place? f. To which degree does the design foresee sufficient flexibility for adaptation to a changing context? g. Are the indicators SMART?
A
B
C
D
The logframe (LF) of the project is generally of a high quality, reflecting well the intervention logic. In order to achieve the Overall Objective (OO) of reducing malnutrition, two Specific Objectives (SOs) have been defined to address two of the main causes of malnutrition, namely food insecurity at household level and weak institutions. Four Results (Rs) have been defined which represent the components of the project's strategy, namely: increased production of staple crops, increased diversification of crops, improved marketing system and increased capacity of local institutions. Appropriate activities have been designed to achieve the Rs, as well as activities for the Inception Phase. The design of the project reflects a sound understanding by the ICCO-led consortium of both this kind of rural development project and the local context, based on their previous experience in the area and the lessons learned.
The resources allocated to the project, including staff, appear to be adequate. The time available (3 years) is also likely to be sufficient, although ideally this should include three growing seasons (April to November). Unfortunately, due to the timing of project approval and contract signing, the first season was missed and the project started in July. The project requested a derogation to start in April but due to a miscommunication, approval was not received. Should funding allow, a no cost extension may be advisable to allow the project to support a third cropping season (or at least the start of it) in 2015, in order to achieve the full potential impact. Alternatively, the contractual start date could be changed to coincide with the actual start date.
Although the consortium includes small local NGOs and the University (CUoSS-FAES: Catholic University of South Sudan, Faculty of Agriculture and Environmental Sciences), which could weaken its capacity, there is a strong capacity building component and the staff of the NGOs are experienced. Besides, the use of only local staff (and none of the ICCO or DORCAS expat staff costs at Country Office level are covered) results in a cost effective human resources component (only 22% of the total budget). The risk analysis of the project is mostly appropriate and reflected in the assumptions in the LF. Some of these, however, are not really external risks but existing cultural factors that the project has to deal with, such as gender inequality and dependency on aid handouts. The project appears to be aware of external risks and manages those related to security, in particular, on an ongoing basis. The inclusion by the EU Delegation (EUDel) of an Inception Phase (an excellent innovation) provides an early opportunity for the project to adapt to any changes identified in the baseline study. However, no significant changes were recommended, rather some fine tuning of various aspects. Flexibility is being maintained with regard to aspects such as cost recovery strategy.
The baseline study made it possible to quantify more accurately the targets of the OVIs, and a comprehensive process was subsequently undertaken, with the aid of external expertise, to set up an M&E system in a fully participatory way, and improve the OVIs. They are thus SMART, although those for R4 (capacity building) do not actually measure increased capacity.
Page 3 of 18
2.2. Do the implementation arrangements take into account the capacity of the partners, and is the design fully supported by them? a. Are the timescale and activities realistic with regard to the stakeholders' capacities, organizational structure and implementation arrangements? b. Have the relevant stakeholders been actively involved, as a driving force, in the design process? c. Do all relevant stakeholders, especially the target group, understand and agree on the intervention logic?
A
B
C
D
The key stakeholders or partners of the project include the consortium partners (ICCO, DORCAS, CAD, ECSCARD), the additional partner CUoSS-FAES, the State Ministry of Agriculture, Forestry and Irrigation (SMoAF&I), local government at county, payam and boma levels, local NGOs and community/village leaders. Although relatively complex, the implementation arrangements, roles and responsibilities seem to be clearly understood by all. With appropriate capacity building, as planned, it is likely that all stakeholders will be able to carry out their assigned activities. All have participated fully in the original design and in the inception phase, including in the baseline study (e.g. CUoSS-FAES students and SMoAF&I officials). This is a notable feature of this project. The intervention logic was thoroughly workshopped with all stakeholders and is thus clearly understood and supported.
d. Are the roles and responsibilities of all partners clearly defined and understood by all concerned? e. Does the operation foresee adequate capacity development support?
2.3 Is the current design sufficiently taking cross-cutting issues into account? a. Have the relevant cross-cutting issues (environment, gender, human rights and governance, donor coordination or others) been adequately mainstreamed in the design?
Note: a = very good; b = good; c = problems; d = serious deficiencies.
A
B
C
D
The cross cutting issues have been reasonably well incorporated. Only environmentally sound agricultural practices will be promoted. Targets have been set for women's representation on committees and as a percentage of lead farmers and peer trainers of farmers. Beneficiary targeting prioritises the poorest farmers. Governance and transparency are promoted through the project steering committee which is chaired by the SMoAF&I and co-chaired by a Payam Administrator. Donor coordination is effected through existing forums and committees at State level.
Overall conclusion - Quality of design
Overall conclusion - Relevance and quality of design
B A
Page 4 of 18
3. EFFICIENCY
A measure of how economically (in terms of quality, quantity and time) resources/inputs are converted to outputs. 3.1 How well is the availability and use of inputs and resources managed? a. To what degree are inputs and resources provided/available on time from all parties involved to implement activities? b. To what degree are inputs available at planned costs (or lower)? c. Are staffing arrangements proving adequate? d. Are inputs monitored regularly, and by whom, to encourage cost-effective implementation of activities? e. Are operation resources managed well and in a transparent and accountable manner? f. Is the current budget break-down conducive to the implementation of the operation? g. Are all contractual procedures clearly understood and do they facilitate the implementation of the operation?
b. Are funds spent in line with the implementation of activities? If not, why? c. Is there a need to change any of the planned activities? If so, how well have these changes been managed? d. How well are activities monitored? Is monitoring used to take corrective action? e. How well does the operation coordinate with other, similar interventions (if any) for synergy and in order to avoid overlaps? f. Is a logframe (or an equivalent tool) actively used as management tool? If not, why? g. Is a work plan/implementation schedule available and actively used by project management?
B
C
D
Aside from the misunderstanding over the start date, from the actual start inputs have been provided on time. This was in spite of delays in the transfer of funds from ICCO in the Netherlands to the local partners, due to the international banking embargo placed on the country, and some procurement requiring importation and approval of exemption of payment of import duties (e.g. vehicles) has only just been completed due to delays in approval by government. However, the partners were able to compensate by advancing their own funds and using their own vehicles. In fact, the project was able to get going remarkably quickly with additional staff recruited within one month and local procurement and Inception Phase activities completed in the first quarter (counting from the actual start date in July). By the second quarter, field activities were well under way. Generally inputs have been procured at the budgeted cost, however both the baseline survey and M&E consultancy cost considerably more than budgeted (but was money very well spent). Some senior staff costs appear to be much higher than budgeted. Staffing arrangements are generally adequate, although the CUoSS-FAES staff are new to development work and require more mentorship and support to carry out their activities more efficiently and timeously; this (relatively small) component is well behind schedule.
Funds are managed by each partner of the consortium on an imprest basis, in accordance with MoUs signed with ICCO. Staff have all been trained in EU procedures and financial management in general and seem to be performing well. Most organisations use accounting software. Each organisation produces a monthly financial statement and quarterly reports are submitted to ICCO which consolidates information for the overall financial statement and report. The latest available consolidated report (to 6 April 2013) looks acceptable, without serious anomalies, although clarity is needed on whether the project should be charging for staff and operating costs incurred prior to the actual start date in July. The budget is sound, in general, but has some defects that should be corrected to take into account actual costs and improved estimates, e.g. some staff costs, consultant costs and ploughs (the budgeted cost of ploughs is only about half of what they are likely to cost, resulting in a shortfall of over 5.000 Euros for this item).
3.2 How well are the activities implemented? a. To what extent are activities implemented as planned/scheduled? If there are delays, have the reasons been identified and remedial action been taken to get the operation back on track?
A
A
B
C
D
Most activities have been implemented as scheduled, or with minor delays that can be corrected in the short term. The team has been efficient in identifying the reasons for delays and addressing them. For example, delays in implementation by CUoSS-FAES were resolved by giving more responsibility to the FAES in Wau, rather than dealing with the head office in Juba, and then providing them with more hands-on assistance. By Result (R), progress can be summarised as follows: R1 (staple crops): On track - farmer groups formed, training started, ploughs being procured, seeds procured locally and ready for distribution, cassava stalks being sourced. R2 (diversification): Vegetable demo gardens (5) developed and training well advanced. Chickens distributed. R3 (marketing): Planned for later this year. R4 (capacity building): Several courses run already; needs analysis of government being done; delay in CUoSS-FAES component, but catching up and delay not critical.
As at 6 April (12 months from contractual start date and after 9 months of actual implementation, i.e. 25% of time elapsed) expenditure was at 26%.
Some minor modifications were made to three activities as a result of the recommendations of the baseline study.
A comprehensive M&E system was designed in a fully participatory way, with the help of a consultant, and staff were trained on its application. A start has been made to using it and populating the data base. Each partner reports on a quarterly basis to ICCO who compile consolidated narrative and financial reports. ICCO has been reporting quarterly to the EU up to now, but this has now been reduced to 6 monthly. Progress is reviewed in consortium team meetings and in the meetings of the project steering committee (PSC). Coordination with other interventions is effected through NGO coordination meetings which include State quarterly partner coordination meetings, County NGO monthly coordination meetings, Food Security & Livelihood NGOs monthly partners coordinating meetings, consortium partners coordination meetings.
The project plans and monitors activities as per the project LF which is the main tool used for this purpose.
Page 5 of 18
3.3 How well are the outputs achieved? a. Are the outputs delivered as planned and in a coherent manner e.g. logical sequence? b. What is the quality of the outputs? Are they likely to lead to the intended outcomes? c. Have the outputs been produced/ delivered in a cost-efficient manner? d. Are the outputs accessible to the target group? e. Are they correctly reflected through indicators?
A
B
C
D
The project has performed well in terms of delivering its planned outputs in the planned sequence, the key ones being: baseline study, revised contract documents for Inception Rider, M&E system, training courses, functioning steering committee, mobilised farmers and stakeholders, vegetable training gardens, procurement of inputs, and staple crop demonstration plots prepared.
The quality of the above outputs is very good and without doubt they will contribute to the Results. They are correctly reflected in the OVIs. In relation to cost effectiveness, as noted above, staff costs are relatively low, compared to similar projects, but due to the number of partners, combined office costs are quite high. However, implementation appears to be as cost effective as practically possible. For example, CAD has built a simple staff compound out of local materials in Kangi, due to the cost of traveling from their home base in Aweil to the project area. All outputs are accessible to the beneficiary farmers.
Page 6 of 18
3.4. How well are the Partners involved and contributing? a. Do the inter-institutional structures (e.g. steering committee, monitoring and reporting system, etc.) facilitate efficient implementation? b. Is there good communication between partner government, EU, project management and other stakeholders? c. If necessary, are specific arrangements (e.g. Memoranda of Understanding, etc.) in place to promote active stakeholder involvement?
Note: a = very good; b = good; c = problems; d = serious deficiencies.
A
B
C
D
This is one of the several outstanding aspects of this project. Through involving all stakeholders from the start in the design phase, through the inception phase and baseline study and now on the PSC, while involving government even in some of the activities, inter-institutional coordination is excellent and facilitates implementation considerably.
Coordination and communication with government is excellent, both through the PSC (chaired and deputy chaired by government) and bilateral contacts at different levels. It is equally good with the EUDel, mainly through the FSTP TA. The PSC and MoUs between partners are important tools that promote involvement and ownership.
Overall conclusion - Efficiency
B
Page 7 of 18
4. EFFECTIVENESS The extent to which the intervention's objectives (on outcome and project purpose level) are, or are expected to be, achieved. 4.1. How well is the operation achieving its expected outcomes? a. Have the expected outcomes been achieved to date? b. What is the quality of the outcomes? c. How do target groups assess their usefulness? d. Do all target groups (and everybody in the target group) benefit from the operation as expected?
A
B
C
D
Progress in achieving the Results can be summarised as follows, in terms of achieving the targets in the OVIs: R1 (staple crops): OVIs relate to increased area cultivated and yields achieved, which can only be measured after first cropping season. Based on the activities implemented to date the prospects appear to be reasonable. R2 (diversification): Also too early for any impact yet, but prospects are positive. R3 (marketing): Planned for later this year. R4 (capacity building): Several courses have been given already and progress is on track. Better OVIs are needed to measure actual capacity built. The target groups are positive about progress so far and are optimistic about improving their livelihoods.
e. Are there any factors which prevent target groups from benefitting?
Page 8 of 18
4.2. As presently implemented, what is the likelihood that the project purpose will be achieved? a. To what extent has the project purpose been achieved so far? Is this measurable through the indicators or is there other evidence for this? b. Given the achievement and quality of outcomes so far, what can be said about the likelihood of achieving the project purpose within the timeline of the operation?
A
B
C
Progress in achieving SO 1 (improved food security and nutrition) can only be judged after the 2013 harvest towards the end of the year. Progress with SO 2 (capacity building of institutions) is on track and some impact can already be observed in relation to the CUoSS-FAES. CAD field staff seem to be weak and a specially targeted intervention may be necessary.
The prospects of achieving both SOs appear to be good, judging from the quality of activities and products so far and on the soundness of project management. The assumptions are still holding and there have not been any significant impacts from external factors, either on the project or on the target households.
c. To what extent has the operation adapted to changing external conditions (risks and assumptions) in order to ensure the achievement of the outcomes and the project purpose? d. Are there any unexpected, negative effects on the target group which have occurred or are likely to occur due to the operation? Did project management take remedial action against these? e. Are there any unexpected positive effects on the target group which have occurred or are likely to occur?
Note: a = very good; b = good; c = problems; d = serious deficiencies.
D
Overall conclusion - Effectiveness
B
Page 9 of 18
5. IMPACT TO DATE Likelihood of positive and negative, medium to long-term effects of an intervention, both direct and indirect, intended and unintended. 5.1. What are the operation’s direct impact prospects (i.e. contribution at the level of overall objective)? a. Are there any changes on the level of the Overall Objective which can be observed (through indicators) so far? Can the operation be assessed as having contributed to these changes? b. Given the progress so far, what direct impacts appear likely by the end of the operation?
A
B
C
D
It is still too early (before the first cropping season has even started) for the project to have made any impact at the level of the OO (reducing malnutrition). However, on the same basis that a positive prediction was made, above, about the prospects of achieving the SOs, the same applies to the OO. Of course, external factors such as the amount of rainfall received, the security situation, etc, will have a major bearing, but it is not possible to venture any predictions. The project has the potential to contribute to demonstrating an effective and sustainable model for increasing agricultural production and food security, provided the lessons learned are properly documented.
c. Are any external factors likely to jeopardize the operation’s direct impact? d. Does the operation contribute to the development or improvement of related policies?
5.2 To what extent does/will the operation have any indirect (positive/ negative) impact? a. Is there any unplanned positive impact on the final beneficiaries? b. Are there any observable or expected spill-over effects? Are there any indications that elements/aspects of the operation will be rolled out to or taken up by other parties?
A
B
C
D
It is too early for any unplanned impacts to be observed, or any spill-over effects. Likewise, it would be mere conjecture to talk of long term effects at this stage. So far donor coordination is facilitating synergies, although there is some frustration that the FAO continues to offer free food handouts in areas where the project is trying to discourage this approach.
c. What are the negative consequences, if any, of the operation on the target group and others? Did the operation take timely measures to mitigate negative impact? d. What are the likely environmental, social, cultural, gender and economic long term effects? e. Do donor coherence, complementarity and coordination encourage synergies and/or improve the potential impact of the operation?
Note: a = very good; b = good; c = problems; d = serious deficiencies.
Overall conclusion - Impact to date
B Page 10 of 18
6. SUSTAINABILITY TO DATE Likelihood of the continuation of benefits of an intervention after its completion. 6.1 What is the financial/economic viability of the continuation of benefits after the end of the operation? a. Is there a viable financial sustainability plan in place and is it being implemented? i.e. if the benefits have to be supported after the operation’s end, will funds be available? If so, by whom? By the partner government/project authority? Or is continued donor support required? If so, is it likely to be available? b. If there are costs for continued access to the benefits, are target groups in a position to assume their share after the completion of the operation? c. Are there any external factors that might jeopardize the sustainability of benefits, and if so, have appropriate measures been taken to forestall this?
A
B
C
D
The project's financial sustainability strategy is based essentially on strengthening self reliance at household level through increased agricultural production and marketing, supported indirectly by strengthened local institutions for sustainability. After the project ends (and even before that) farmers will need to be able to afford to replace tools and ploughs, and acquire seeds, either through saving enough of each harvest, or being able to supplement what they save by buying from neighbours and/or local markets. It is a sound strategy, provided the project achieves its objectives, although clearly (as the targets indicate) the project can only expect to create sustainable food security amongst a portion of the target group, not all (the target is a 50% increase). The external factors most likely to undermine sustainability are beyond the control of the project, i.e. climate, security and macro economic factors.
The technologies being introduced by the project (animal traction, improved cropping practices, cassava, vegetables, etc) are all very simple and are affordable, in principle. A major focus is on enabling farmers to produce their own seeds, the main input. There is also a component of support to the SMoAF&I to start developing an extension service so that technical support to farmers can be sustained.
The project is very well aware of the issue of sustainability and the need for a sound economic phase-out strategy, and seems to be doing all that is practically possible to address the issue.
d. Are the target groups and/or relevant authorities/institutions able to afford the maintenance or replacement of the technologies/services/outputs introduced by the operation? e. Is the financial/economic dimension of the phasing out strategy being adequately addressed and implemented as far as necessary to date?
6.2. What is the level of ownership of the operation by the target group and relevant stakeholders? a. Is an exit strategy integrated in the design and has the implementation been managed accordingly? b. Is there any evidence of further commitment of the relevant stakeholders? c. Is operation implementation demanddriven or is there simply passive buy-in from target groups? d. Do the target groups plan to continue assuming their role in ensuring continued outputs and outcomes? If so, are they likely to materialize? e. To what extent have they been actively involved in the implementation and steering process?
A
B
C
D
As already noted, one of the notable features of the project is the high level of ownership that it has managed to generate amongst both farmers and government institutions. This has been achieved both as a result of the design of the project, as noted in Chapter 1, as well as the attitude and approach taken by the implementing consortium. For example, the composition and leadership of the PSC, and the level of interaction of the project with government at all levels, show a clear effort to ensure government ownership. The involvement of the CUoSS-FAES to develop a sustainable local research capacity, with some related extension services such as a testing laboratory, is another example. Although the SMoAFI does not have the resources at present to establish an extension service, there is a reasonable chance of it starting to do so within the life of the project, and certainly it intends to do so as soon as possible. Buy-in from target groups (to the extent that it is possible to determine during a brief visit) appears to be good; farmer groups are well aware of their responsibilities and are clear about what benefit they are likely to obtain, which suggests they are likely to sustain their improved production. The groups include local traditional leaders and are led by farmers elected by their peers on the basis of their leadership qualities and skill as farmers.
As noted above, the PSC is driven mainly by government (all levels are represented) and other stakeholders, not by the project. They participated in the design, make input into planning and comment on results. The project does not present formal reports with financial information to the PSC, however.
f. How far is the operation embedded in the local structures of the target group (possibly different from institutional structures)?
Page 11 of 18
6.3. To what degree does the policy environment support the operation? a. Is the national, local, sector and budgetary policy environment an enabling factor for the continuation of benefits? What specific support is being provided? b. Do changes in policies and priorities affect the potential sustainability of the benefits? If applicable, has the operation adapted to ensure long-term support?
A
B
C
D
Although the project is fully aligned with both National and State policies and development priorities, as noted in Chapter 1, due to the low level of development of the new government, exacerbated by the ongoing austerity measures introduced after oil production was halted, there is no budgetary support for the kinds of activities being undertaken by the project. Crucially, there is as yet no functioning extension service or any other kind of government support for subsistence and smallholder agriculture.
However, the future prospects appear reasonable - there is political will and resources are likely to follow at some stage.
c. If relevant, is any public and private sector policy support likely to continue after the operation has ended?
6.4. To what extent does the operation contribute to partners' capacity development? a. Does the operation contribute to the development of partner's individual and organizational capacities for sustainable delivery of outputs and outcomes? b. How far is the operation embedded in institutional structures that are likely to function beyond the life of the operation?
A
B
C
The project has been designed with a significant capacity development component, from training of farmers and building group capacity, to strengthening the local NGOs members of the implementing consortium, to building the capacity of the FAES at the CUoSS, and assisting government to understand and carry out its planning, monitoring and support functions. In this way the project will be as embedded in local structures as it practically feasible. However, as the government institutions are still weak, the institutional sustainability strategy is actually based on self reliance at individual farmer level, aided by community support mechanisms. Eventually it is likely that government extension and other support functions will be developed.
c. Will an adequate level of qualified human and institutional resources be available in the future in order to continue delivering the operation's stream of benefits?
Note: a = very good; b = good; c = problems; d = serious deficiencies.
D
Overall conclusion - Sustainability to date
B Page 12 of 18
7. HORIZONTAL ISSUES 7.1 Quality Systems, Monitoring and Evaluation a) Were the QSG comments taken into consideration and included in the final design and applied during implementation?
Yes
No
N/A
b) Are the issues identified by ROM regarding design the same as those addressed in the QSG checklist?
Yes
No
N/A
c) Have previous evaluations or reviews (such as ROM, reviews by the EU operational manager) led to changes in the operation?
Yes
No
N/A
d) Is the available monitoring and reporting information on the operation's progress comprehensive and reliable in order to ensure the possibility to evaluate results and learn lessons?
Yes
No
N/A
Please comment on any of the questions / aspects above, qualitative data is very valuable: The project did not go through the QSG. No previous ROM or other evaluation exercise was undertaken. However, the EUDel provided input into the development of the Inception Rider after the baseline study was undertaken (some changes to activities and improvement of OVIs).
The M&E system is sound and a data base has been developed which is being populated. It includes impact related data and thus in principle will be able to support lessons learning.
7.2 Review of Technical Cooperation/Capacity Development Quality Criteria Adaptation to the context and existing capacity a ) Are there critical constraints in the context which are likely to prevent the CD support from achieving its objectives?
Yes
No
N/A
Yes
No
N/A
c) Do local partners effectively lead in the planning of CD support beyond formal endorsement?
Yes
No
N/A
d) Do local partners provide the inputs (human or physical) that would be required to enable the CD support to be effective?
Yes
No
N/A
b) Is the CD support adequate vis-Ă -vis the present capacity of the local partner?
Demand driven TC/CD and ownership
Page 13 of 18
Result oriented TC/CD e) Are the outputs or outcomes of the CD support clearly specified and still relevant (or adjusted to changes of context)?
Yes
No
N/A
f) Are they regularly monitored and/or assessed (e.g. through a joint performance dialogue or an annual reporting)?
Yes
No
N/A
g) Is the CD support taking into account CD interventions from other donors in the same sector?
Yes
No
N/A
h) Is there a donor coordination mechanism led by local partners and encompassing CD support?
Yes
No
N/A
i) Is CD support embedded in the broad institutional context of the local partners and have unnecessary parallel mechanisms been avoided?
Yes
No
N/A
j) Do contracted experts, project managers and NGO staff take instructions from the partner and not the EC? (while some form of reporting to the EC can still take place)
Yes
No
N/A
Harmonisation of TC/CD
Project Implementation Arrangement
Please comment on any of the questions / aspects above, qualitative data is very valuable: CD support is provided both through training of farmers and through the institutional capacity building activities of R4. As all project staff are nationals, and contracted experts are from neighbouring countries (baseline study, M&E system) there is no EU CD support as such. Through the PSC and regular liaison with institutional beneficiaries, these participate in identifying training needs and monitoring activities.
7.3. EC Visibility Does the operation contribute to promoting EC visibility (e.g. does it comply with the EC Guidelines)?
Yes
No
N/A
Please comment on any of the questions / aspects above, qualitative data is very valuable: After 9 months EU visibility is still low. No signboards have been erected, little branding is visible on equipment and products and communities are only vaguely aware of the EU. This is in spite of all project staff being trained on the EU's visibility guidelines.
Page 14 of 18
8. CROSS-CUTTING ISSUES 8.1. Have practical and strategic gender interests been adequately considered in the operation's strategy? If so, how and to what effect? If not, why not? If n/a, explain. Please consider the following aspects of gender mainstreaming: a. Has the operation been planned on the basis of a gender-differentiated beneficiaries’ analysis?
Yes
No
N/A
Gender is mentioned in the proposal, and targets have been set for women's representation on committees and as a percentage of lead farmers and peer trainers of farmers, but it cannot be said that there is a comprehensive strategy to challenge entrenched gender roles.
According to the OECD Gender Policy Marker, the project os rated as G-1.
b. To what extent will / could the gender sensitive approach lead to an improved impact of the operation? c. What is the likeliness of increased gender equality beyond the operation's end? d. According to the OECD Gender Policy Marker how would you classify this operation?
8.2. Is the operation respecting environmental needs? If so, how and to what effect? If not, why not? If n/a, explain. Please consider the following aspects of mainstreaming environmental aspects:
Yes
No
N/A
The project is promoting environmentally sound farming techniques, avoiding agro chemicals. It is unlikely to cause any environmental damage.
a. Have environmental constraints and opportunities been considered adequately in the operation's design? b. Are good environmental practices followed during implementation (in relation to use of water and energy and materials, production of wastes, etc.)? Does the operation respect traditional, successful environmental practices? c. What capacities exist (within the operation, among partners and the operation's context) to deal with critical risks that could affect the operation's effectiveness such as climate risks or risks of natural disasters (in the case of operations in sensitive geographical areas / natural disasters hotspots)? d. Has environmental damage been caused or likely to be caused by the operation? What kind of environmental impact mitigation measures have been taken? d. Is the achievement of project results and objectives likely to generate increased pressure on fragile ecosystems (natural forests, wetlands, coral reefs, mangroves) and scarce natural resources (e.g. surface and groundwater, timber, soil)?
Page 15 of 18
8.3. Has (good) governance been mainstreamed in the operation? If so, how? If not, why not? If n/a, explain. Please consider the following aspects of governance: a. Does it take into consideration the differential impact of poverty on disadvantaged groups? b. Is the operation designed in such a way that it takes into account potential conflict?
Yes
No
N/A
Good governance in relation to the beneficiaries has only been mainstreamed to the extent that the selection criteria for beneficiaries of the different activities are based on their poverty status. Governance at project coordination level is excellent; the PSC is driven mainly by government and plays a proactive role in steering the project, for example in defining criteria for selection of beneficiaries. The project reports to the PSC on progress, but this does not include any financial information. Internal reporting is of acceptable quality, including the financial data presented.
The only anti corruption measures are those linked to EU procurement procedures.
c. Is regular, transparent, financial reporting built into the operation? Are its results widely circulated and understandable? d. Are there effective anti-corruption monitoring tools in place?
8.4 Does the operation actively contribute to the promotion of Human Rights? If so, how? If not, why not? If n/a, explain. a. Has there been an analysis of “winners and losers” regarding possible “discrimination” of target groups by the operation?
Yes
No
N/A
The project does not have a specific human rights focus, except in relation to the selection of beneficiaries, as already described.
b. Will the operation help to ensure respect for any relevant human rights and not cause them to be reduced in any way? c. Do any interested parties and observers raise HR concerns?
Page 16 of 18
9. LIST OF PERSONS INTERVIEWED/DOCUMENTS ANALYSED Name / Position
Institution / other
Paolo Girlando, Task Manager
EU Delegation (Paolo.GIRLANDO@eeas.europa.eu / 0954356804)
Dominic James, Extension Worker
DORCAS (jamesindia@gmail.com / 0956128225)
Kalisto Inyani Adrawa, Programme Officer
ICCO (kalisto.inyani@icco-cooperation.org / 0955577004)
Jaap Vermeulen, Country Director
ICCO (Jaap.Vermeulen@icco-cooperation.org / 0977467252)
Mary Tabu, Extension Worker
DORCAS (tabu.mary@yahoo.com / 0955819499)
Ceasar Lupai, Training & HR Manager
DORCAS (hopeceasar@yahoo.co.uk / 0955200825)
Olum Augustine, FS Officer
DORCAS (olumaugustine@hotmail.com / 0956505436)
Moses Deng, Bishop & Chair of CARD
ECS-CARD (bishop@wau.angelican.org / 0955602769)
Andrew Aping, Project Officer
ECS-CARD (development@wau.angelican.org)
Pio Apai, Extension Worker
ECS-CARD (apaiuchali@yahoo.com)
Valentino Unango, Executive Director
Jur River County
Goit Dengoch, Chairperson of Council Daniel Akau, Dean
Solomon Ewot, Director of Faculty of Agriculture & ES
Marko Macker, Administrator
Peter Malong, Extension Worker Taban Richard, Logistics Officer
5 groups of farmers (3 for vegetables, 2 staple crops)
Jur River County
Catholic University of South Sudan, Wau Campus (danielakau@gmail.com
Catholic University of South Sudan, Wau Campus (ewotsolomon@yahoo
Kangi Payam (0956149086) CAD (0914607643) CAD (0956323467)
Wau and Jur River County
Documents Analysed
Contract with annexes (Description of the Action, budget, logframe, general conditions) Derogation to start before contract signing CRIS Fiche, 28/03/2013 Baseline Survey Report Inception Rider (revised Description of the Action, budget, logframe) Consolidated Quarterly Report Oct-Dec 2012 Quarterly Reports of implementing partners, Jan-Mar 2013 Draft Consolidated Quarterly Financial Report, Jan-Mar 2013 EU Single Country Strategy (Response Strategy) for South Sudan 2011-2013 FSTP TA Field Visit Report, Dec 2012
Page 17 of 18
OVERVIEW OF SUB-CRITERIA GRADES Sub-criteria 1.1 Relevance for target groups 1.2 Relevance for partner 1.3 Relevance for EU 2.1 Intervention logic
Grade A A B B
2.2 Partners and Design 2.3 Cross-cutting issues 3.1 Inputs 3.2 Activities 3.3 Outputs 3.4 Partners and Implementation 4.1 Outcomes 4.2 Project Purpose 5.1 Direct impact 5.2 Indirect impact 6.1 Financial sustainability 6.2 Ownership 6.3 Policy support 6.4 Capacity Development 7.1 a) QSG comments 7.1 b) QSG and ROM on design 7.1 c) Evaluations and reviews 7.1 d) Progress information 7.2 a) TC/CD - constraints 7.2 b) TC/CD – capacity 7.2 c) TC/CD – partner lead 7.2 d) TC/CD – partner input 7.2 e) TC/CD – specified results 7.2 f) TC/CD – monitoring 7.2 g) TC/CD – other donors' intervention 7.2 h) TC/CD - donor coordination 7.2 i) TC/CD - embedded 7.2 j) TC/CD - staff instructions 7.3 EC visibility 8.1 Gender 8.2 Environment
A B B B B A B B B B C A B A N/A N/A Yes Yes No Yes Yes Yes N/A N/A Yes Yes Yes No No Yes Yes
8.3 Good Governance 8.4 Human rights
Yes N/A
Page 18 of 18