Services on Demand
Article
Indicators
Related links
-
Cited by Google
-
Similars in Google
Share
African Evaluation Journal
On-line version ISSN 2306-5133
Print version ISSN 2310-4988
AEJ vol.10 n.1 Cape Town 2022
http://dx.doi.org/10.4102/aej.v10i1.608
ORIGINAL RESEARCH
Transforming voluntary organisations for professional evaluation into central pillars of national evaluation systems
Tebogo FishI; Ayabulela DlakavuI, II; Matodzi AmisiI; Steven MasvaureI
ICentre for Learning on Evaluation and Results - Anglophone Africa (CLEAR-AA), Faculty of Commerce, Law and Management, University of the Witwatersrand, Johannesburg, South Africa
IIDepartment of Politics and International Relations, Faculty of Humanities, University of Johannesburg, Johannesburg, South Africa
ABSTRACT
BACKGROUND: Voluntary organisations for professional evaluation (VOPEs) have proliferated across the African continent, seeking to contribute to building effective national evaluation systems (NESs) that enhance development planning, implementation and appraisal.
OBJECTIVES: A study commissioned by the Centre for Learning on Evaluation and Results - Anglophone Africa (CLEAR-AA) and the African Evaluation Association (AfrEA) in 2021 highlights key strengths and challenges faced by African VOPEs in their quest to help strengthen NESs across the continent.
METHOD: The study was qualitative by design, employing document analysis, key informant interviews and focus group discussions with selected VOPEs across Africa.
RESULTS: While African VOPEs have proliferated in number, they take different forms, and their focus tends to be shaped by their respective national contexts. A key challenge for African VOPEs is internal human and financial capacities, while the external environment within which these national evaluation associations operate also presents risks such as co-optation and opportunities such as networking and resources.
CONCLUSION: This article argues for the need for African VOPEs to improve their internal financial and human capacity, thereby enabling these professional associations to endeavour to contribute to strengthening the supply and demand sides of NESs, as well as refining their advocacy to reflect Africa-centred evaluation approaches and criteria.
Keywords: VOPEs; Africa; capacity building; evaluation systems; capacity development.
Introduction
Since the Paris Declaration (PD) on Aid Effectiveness in 2005 and the Accra Agenda for Action (AAA) of 2008, there has been a shift in National Evaluation Capacity Development (NECD) towards building country-led and owned evaluation systems (Lomeña-Gelis 2013). In the PD, over 100 countries and international agencies had pledged to improve the effectiveness of development aid, in order to ensure better development outcomes. Country ownership, donor alignment with national development priorities, donor programme harmonisation, results-focused development planning and management, and mutual accountability were the five core principles of the PD (KPMG 2011). To operationalise the PD, government ministers of development planning and the heads of bilateral and multilateral development institutions adopted the AAA in September 2008 in Accra, Ghana. Through the AAA, represented governments and development agencies agreed to promote country ownership of development endeavours, building partnerships for development among governments, donors, the private sector and civil society and attaining intended development results that improve people's lives (African Development Bank 2008).
Since the endorsement of the above-mentioned declarations, some countries in Africa have made strides in developing National Evaluation Systems (NESs) through institutionalising evaluation within government (Goldman et al. 2018). The development of NESs is based on the premise that if African countries can monitor and evaluate their development programmes, they will be in a position to use evidence in development decision-making and achieve better development outcomes.
The development of NESs has been an agenda item for several international development organisations, specifically those that are responsible for funding development interventions. Several approaches have been used to develop NESs, and some of these approaches include strengthening individual monitoring and evaluation (M&E) skills and knowledge and strengthening institutional M&E architecture by developing M&E policies, strategies and frameworks (Fraser & Morkel 2020). Within these various approaches, there is also a realisation that professional organisations like voluntary organisations for professional evaluation (VOPEs) can play a significant role in NESs in Africa (Kosheleva & Segone 2013). This realisation led to the formation and further development of VOPEs with the key objective of making them a key partner in the development of NESs. Voluntary organisations for professional evaluation are regarded as pillars of NESs, as their primary objective is to professionalise evaluation practice and ensure an adequate supply of evaluators with the requisite skills and capacity to evaluate national development plans, policies, programmes and projects (Beney et al. 2015; Ngwabi, Mpyana & Mapatwana 2020). The need to strengthen VOPEs led the International Organisation for Cooperation in Evaluation (IOCE) and United Nations Children's Fund (UNICEF) to set up the EvalPartners Initiative in 2012 to support national VOPEs through peer-to-peer support and a toolkit that VOPEs can draw on to build their institutional capacity (Carter 2013). Voluntary organisations for professional evaluation in Africa have also mushroomed and joined the global campaign for institutionalising evaluation within processes of governance and development practice (Ngwabi et al. 2020).
This article examines the role that VOPEs in Africa are playing to support NESs, including contributing to the institutionalisation of evaluations within the public sector in Africa. The latter is routinely done through evaluation guidelines and related evaluation frameworks. The article further explores the form, functions and current state of VOPEs in Africa. It also examines what VOPEs have achieved so far and the challenges they face vis-à-vis supporting NESs. This article emanates from a discussion paper commissioned by the Centre of Learning on Evaluation and Results - Anglophone Africa (CLEAR-AA) and the Africa Evaluation Association (AfrEA) in 2021. The discussion paper documented the experiences of a select number of VOPEs in Africa.
Context, background and concepts
A VOPE is defined as an association of evaluation professionals whose mission is to serve as a forum that discusses and shares professional experiences and challenges of the evaluation profession (Ojha 2013). Through these deliberations, a VOPE seeks to advance the profession and develop professional evaluation standards (Kosheleva & Segone 2013:8). Similar to other professional associations, VOPEs are, therefore, self-organised communities of practice whose membership is open to both evaluation practitioners (evaluators) and commissioners of evaluation (Ojha 2013). The core objective of VOPEs is to develop evaluation as a distinct profession that induces favourable development outcomes. Voluntary organisations for professional evaluation use modes such as conferences, workshops, seminars and publications as platforms through which they share evaluation best practices and knowledge. The European Commission recognises VOPEs as central pillars of NECD interventions. The European Commission, through its EVALSED Evaluation Capacity Development Indicators, identifies VOPEs as communities of practice that should be established to engage in open and systematic dialogue with policymakers (European Union 2008:30).
The origins of national VOPEs can be traced back to the last quarter of the 20th century when the Evaluation Research Society (founded in the United States of America in 1978) and the Canadian Evaluation Society (established in 1981) came into existence. These two VOPEs are widely recognised as the first two national VOPEs (Kosheleva & Segone 2013:9). In 2003, the IOCE was established as a global umbrella association of regional and national VOPEs. The mandates of the IOCE include contributing to the building of evaluation capacity and leadership (particularly in the Global South), bridging the relationship between evaluation theory and practice, addressing challenges facing evaluation theory and practice and assisting the evaluation profession in being responsive to global challenges (Ojha 2013:1). The database of the IOCE recognises 154 national VOPEs globally (International Organization for Cooperation in Evaluation 2022). This is evidence of the proliferation of VOPEs in the 21st century, compared with 15 VOPEs worldwide in the 1990s (eds. Rugh & Segone 2013). This mushrooming of VOPEs in the new millennium is also indicative of the impact of the IOCE in identifying and nurturing VOPEs worldwide through its network.
Despite the proliferation of VOPEs worldwide, the IOCE Directory lists only three African VOPEs: the Africa Gender and Development Evaluators Network (AGDEN), the African Evaluation Association (AfrEA) and the South African Monitoring and Evaluation Association (SAMEA) (IOCE 2022). Given the IOCE's extensive network of regional and national VOPEs, it would be strategic for African VOPEs to integrate into such an extensive network of evaluation associations.
Conceptualising national evaluation systems and their core features
An NES is defined as the collection of M&E frameworks that guide the commissioning and undertaking of M&E and the individual and institutional M&E capacity to use M&E information to inform development planning, policymaking and decision-making (Chirau et al. 2020:2). It is, therefore, discernible that an NES exists when M&E, and use of evidence therefrom, has become institutionalised in an organisation or institution such as government. The United Nations (2012:7) affirmed the above definition by asserting that an NES ought to develop an equilibrium between the supply of M&E information (i.e. undertaking programme monitoring and evaluations) and the demand for M&E information by policymakers and other development decision-makers. Goldman (2018:2) identified six constitutive elements of a functional public sector NES:
the existence of an M&E policy that conceptualises monitoring and evaluation and assigns M&E roles to various public sector institutions
methodology whereby development indicators are identified, criteria for selecting government interventions (development programmes) to be evaluated and the identification of data collection methodologies to be used for performance monitoring and evaluations
defining the M&E roles of the various public sector institutions (government ministries, departments, agencies and offices)
individual and institutional M&E capacity in the public sector, including capacity building plans
integration of nongovernment institutions in the government-wide NES, including parliament, nongovernmental organisations and development partners (donors)
quality of M&E products (reports) and their utility in decision-making, policymaking and budgeting.
This conceptualisation of VOPEs and NESs is important given the article's primary focus on VOPEs and their efforts to strengthen NESs. The conceptualisation of an NES and its constituent elements is also important as it provides a conceptual framework against which to analyse VOPE evaluation capacity development efforts, as well as provides strategic recommendations about how African VOPEs can contribute to the building and/or strengthening of various elements of NESs across Africa.
Methodology
This article used a qualitative phenomenological study design. The initial phase of the study was a desktop review of the literature on VOPEs across Africa. The goal was to build a sufficient data set to look for emerging themes on VOPEs and to use the data set and the developed themes for further validation with key informant interviews and facilitated focus groups. The extensive literature review on VOPEs covered how evaluation associations have evolved over the years, the form they take and the challenges they face. In addition, a detailed analysis of the work of nine VOPEs in Africa was also performed. This included an analysis of information on their websites such as membership, activities, annual plans and constitutions, among others. The analysis was conducted on VOPEs in Kenya, Uganda, Tanzania, Senegal, Cameroon, Namibia, Zimbabwe, Zambia and Ghana. Information about Namibia's VOPE was sourced primarily through interviews because this particular evaluation association is in its formative stages. The VOPEs were purposefully selected and represent anglophone African VOPEs and two francophone African VOPEs.
Validation of the findings workshop (facilitated focus group) was held with 27 participants from different countries and organisations in November 2020. Fifteen of the participants represented VOPEs; this included representatives from the continental VOPE umbrella body, the AfrEA. Ten of the validation workshop participants were partner organisations, including the UNICEF regional and country offices, CLEAR-AA, CLEAR-FA, IOCE and the African Development Bank, while a further two participants were from government ministries and departments. The workshop was useful both for testing out the emerging findings and to close some of the gaps in the findings. After the virtual workshop, three follow-up interviews were held with three VOPEs as a follow-up on some of the information they had provided during the workshop. These interviews gave more insights into the functions of the individual VOPEs. Although three VOPEs are not representative of all VOPEs in Africa, the intention was not to be representative but to identify some of the innovative evaluative work that is happening and specific challenges that these VOPES might face because of their context. Moreover, the follow-up interviews with the three VOPEs served the purpose of triangulating the information already collected. The VOPEs and other institutions that informed the study on which this article's content is based are reflected in Table 1.
Limitations
The methods used in this study have their limitations. Most VOPEs do not update their websites, and some of the information was dated. This shortcoming has a material impact on the analysis; however, follow-ups were done with three VOPEs individually to get the latest information and updates. The three VOPEs were the Zimbabwe Evaluation Association (ZEA), the Namibia Evaluation Association (NEA) and the Senegal Evaluation Association (SenEval). In addition, data that were collected during the project have been triangulated with existing literature on VOPEs and follow-up interviews with the aforesaid three VOPEs, and there was a great deal of alignment between the different sources. This provides a level of reliability and validity to the data used in this study, despite its limitations. The findings presented are, therefore, valid and can be a basis for a conversation about ways to strengthen VOPE work in Africa.
Ethical considerations
Ethical clearance to conduct this study was obtained from the University of the Witwatersrand Human Research Ethics Committee (Non-Medical) (reference number: H20/05/21).
Findings
Voluntary organisations for professional evaluation in Africa: Normative design versus actual experiences
Voluntary organisations for professional evaluation in Africa take different organisational forms that are shaped by context, access to resources, organisational capacity and the nature of leadership. In addition, VOPEs are referred to by different names. Some incorporate themselves as evaluation societies, such as the Evaluation Society of Kenya (ESK), or forums, as in the case of the Ghana Monitoring and Evaluation Forum (GMEF). Other VOPEs use the concept of associations, such as the ZEA. Some VOPEs focus exclusively on evaluations, as is the case of SenEval, ZEA, NEA and ESK. This is interesting because the VOPEs referred to in this paragraph are situated in countries where there has not been significant progress on expanding the evaluation of public policies and programmes because the governments generally do not have a standing budget for the evaluation of programmes (CLEAR-AA 2019). Other VOPEs incorporate both monitoring and evaluation practice within their name, as is the case of the Zambia Monitoring and Evaluation Association (ZaMEA) and the GMEF.
Even where a VOPE's name suggests the exclusion of monitoring, this is often just in the name and not in practice. For example, the ESK's mission includes promoting professionalisation, capacity building, advocacy, utilisation and sound governance of monitoring, evaluation and research practice (CLEAR-AA & AfrEA 2021). In Tanzania, although the Tanzania Evaluation Association (TanEA)'s mission (articulated in its website and constitution) has been articulated as promoting and strengthening evaluation practices in the country through capacity building, advocacy, professional development and networking, in practice the VOPE has incorporated monitoring professionals and provides capacity-building activities targeting these monitoring professionals (CLEAR-AA & TanEA 2020).
Given their advanced years of existence, some VOPEs in the Global North play an integral role in sustaining NESs. For instance, the Canadian Evaluation Society (CES) runs a Professional Designation Program (PDP) that provides credentialing for evaluators (eds. Rugh & Segone 2013). Through the PDP, the CES supports the development of quality evaluators, thereby contributing to the supply of professional evaluators in the Canadian NES. What the analysis of African VOPEs suggests is that unlike evaluation-focused VOPEs in the Global North, monitoring practitioners are an important part of African VOPEs because performance monitoring is a more pronounced practice compared with evaluation (Porter & Goldman 2013). African VOPEs also recognise the symbiotic relationship between M&E and other related subjects like data management, data visualisation, research and policy analysis, and they have focused on a wide variety of activities related to these focus areas. A further paramount aim for African VOPEs is to promote the value and use of M&E evidence in policy development and implementation (CLEAR-AA & AfrEA 2021).
Membership
Voluntary organisations for professional evaluation membership are characterised by heterogeneity: commissioners and users of evaluation, students, researchers, performance monitoring practitioners, evaluators, data scientists, as well as evaluation trainers or capacity-building practitioners all constitute the membership of a typical African VOPE. Voluntary organisations for professional evaluation in Africa represent a range of development professions. The diversity of membership can be a strength that enables VOPEs to access a broad range of development stakeholders: government, civil society, development partners and higher education institutions (HEIs). However, our analysis found that most VOPEs are yet to fully utilise this heterogeneity of membership to promote the demand and use of evaluative evidence in policies and programme implementation within the wider NES where their membership is employed.
An important observation is that VOPEs are experiencing membership growth; however, despite the growth, there are indications that VOPEs in the continent are still having challenges in retaining members and getting members to fully participate in the execution or management of VOPE-related activities. Furthermore, the lack of resources is a perennial challenge for African VOPEs, which is not unique to VOPEs in Africa (CLEAR-AA & AfrEA 2021). As a result of limited resources, VOPEs like ZEA and TanEA still did not have the proper software to manage their membership database, although they are working on addressing this. SenEval was implementing a basic survey to gain a better understanding of the needs of the membership and improve VOPE offerings. Despite these challenges, there is a clear indication from the CLEAR-AA and AfrEA (2021) VOPEs discussion paper that in different African countries, there is a latent membership base that VOPEs can leverage with improved membership management strategies and improved value offering to members, especially emerging evaluators. As M&E gains traction in different public service institutions on the continent, demand for M&E is set to increase, and so will the number of practitioners in the field.
Voluntary organisations for professional evaluation's role in the national evaluation system
The United Nations Evaluation Group (2012:13) asserted that a strong civil society is an essential building block for a strong national monitoring and evaluation system. Voluntary organisations for professional evaluation are an important group within the civil society umbrella, actively building a culture of regular and systematic M&E practice and evidence use. The role of VOPEs within the wider NES is further explained in the ensuing subsections.
Building a community of practice
From the data collected, it is evident that VOPEs are creating platforms where evaluators can disseminate their evaluation findings, learn new evaluation techniques, think about the role of evaluation in society from different perspectives and promote professional and ethical standards, thereby contributing to the development of the evaluation profession. By participating in international networks and forums, VOPEs are maintaining a platform for discourses on the interface between development interventions and evaluation and engage in outreach and publicity to recruit new members and forge strategic partnerships (Holvoet & Dewachter 2013).
Strengthening members' capacity
A review of activities that VOPEs implement demonstrates that capacity building is an area VOPEs are most involved in and in which they have had relative success in implementing their goals. Voluntary organisations for professional evaluation implement a range of activities to strengthen members' technical capacity. This includes continuous professional development training on specific methods or tools, for example, Statistical Package for the Social Sciences (SPSS) training provided by ZEA to members or training on data collection, analysis and visualisation provided by GMEF. It also includes the development of educational programme curricula in cooperation with local and foreign educational institutions. For example, SenEval has been innovative in using international evaluators who are visiting Senegal to provide workshops or training at minimal to no cost to the association. Voluntary organisations for professional evaluation have also been active in hosting regular webinars on topical issues in M&E. Voluntary organisations for professional evaluation also reported peer-to-peer learning exchanges. The ZEA reported carrying out peer-to-peer exchanges with GMEF and ESK and organising some exchanges with ZaMEA. All selected VOPEs for this study, except for the one in Namibia that is still starting up, reported hosting regular conferences and workshops. The conferences or M&E weeks are important activities in the calendar of M&E practitioners in different countries. Some VOPEs have been able to secure the participation of governments and other key partners in their conferences and workshops. For example, the Government of Uganda is a key participant in the Uganda M&E week hosted by the Uganda Evaluation Association (UEA). Ghana Monitoring and Evaluation Forum regularly partners with the Government of Ghana in carrying out evaluation capacity-building activities. Even when the government is the main host, as was the case with the Kenya Evaluation Week in 2018, the VOPEs are collaborating with the government in the delivery of workshops.
Despite these initiatives, there is no record of any of the VOPEs empirically evaluating whether the activities they are implementing are effective and are reaching their members and how these are contributing to building national M&E capacity. This raises questions about the extent to which VOPEs understand the M&E capacity gaps they are addressing and how effective their interventions are. Nonetheless, capacity building remains an important area of work; most practitioners enter the M&E field without any formal training on M&E and from different academic disciplines. Considering that VOPE members also occupy different roles in their organisation and within the M&E field, it also makes sense that VOPEs need to implement a variety of activities. As most VOPEs do not know their exact baselines in terms of capacity gaps and needs, they are unable to know with certainty if what they are providing (accredited and otherwise) is what members need or what is needed in the country to progress the M&E agenda.
Strengthening national evaluation system
Voluntary organisations for professional evaluation such as TanEA, ZEA, UEA and GMEF identify promoting evidence-informed decision-making, development planning and policymaking as their objectives. In addition, these VOPEs have been working with their respective governments to strengthen M&E capacity. For example, ZEA signed a Memorandum of Understanding (MoU) with the Parliament of Zimbabwe to support parliamentarians to use M&E evidence in their oversight function. Although ZEA has struggled with implementing some of the activities in the MoU, this is an important initiative that indicates the demand that exists and a recognition of the value of M&E - and the VOPE - by parliament. In addition, ZEA has been working closely with the cabinet office. In 2019, TanEA in collaboration with CLEAR-AA conducted training on the importance of evaluation as an accountability tool for the Tanzanian parliamentarians who are members of the Tanzanian Chapter of the African Parliamentarians Network on Development Evaluation (APNODE). The Tanzania Evaluation Association also partnered with UNICEF to raise awareness on the need for parliament to demand evaluation reports finalised by government ministries, departments and agencies. Such evaluations can strengthen parliament's oversight vis-à-vis the executive. Access to M&E evidence has the potential to strengthen parliamentary portfolio committees' performance oversight function over respective government institutions over which they have oversight jurisdiction.
Voluntary organisations for professional evaluation have increasingly been supporting governments in their efforts to institutionalise the evaluation function. For example, the UEA works closely with the Ugandan government on evaluation capacity building to enable the practice of evaluation to flourish in the country (Goldman et al. 2018). Likewise, the Senegalese government is increasingly talking about evaluations and also invites members of SenEval to comment on government evaluation Terms of Reference (ToRs) as covered in the quote below:
'So we've done work with government advocating for evaluation; perhaps we should have done more, but we've had a few significant events …. The government is doing some strategic evaluations at the moment, they say … on universal healthcare, cash transfers and the national strategy for child protection, and … I have given comments on the terms of reference either directed to ministry when they ask me or through UNICEF….' (Workshop participant, male, representing the Senegalese Evaluation Association [SenEval])
Lomeña-Gelis (2013) also argued that it was SenEval's advocacy for the institutionalisation of evaluation that contributed to the March 2012 government decision to establish the Commission for the Evaluation and Monitoring of Public Policies and Programmes. This Commission is situated in the President's Office. Despite not being fully operational almost 10 years later, the Commission for the Evaluation and Monitoring of Public Policies and Programmes is a key step towards the institutionalisation of M&E within the Senegalese public sector.
The CLEAR-AA and AfrEA study (2021) revealed that the key functions of VOPEs are to advocate for both the building of evaluation capacity among its members (who are M&E professionals that supply and demand M&E information) and creating demand from the M&E system for M&E information through advocacy and dialogue with parliamentarians, policymakers and development planning and programming entities such as government, civil society organisations and donors. Voluntary organisations for professional evaluation should, therefore, be designed to help M&E systems to achieve an equilibrium between the supply of M&E evidence and the demand thereof from the users of M&E evidence.
The authors believe that VOPEs have made great progress in promoting evaluation; however, they are still finding it difficult to establish themselves and to achieve sufficient capacities to significantly contribute to and influence the way NESs are developing. Our analysis is supported by Karkara (2013), who argued that VOPEs require the capacity to develop effective policy advocacy strategies, including finding ways to mobilise resources for advocacy.
Enabling partnerships
Findings suggest that the degree to which a VOPE can make an impact on NESs is also determined by its relations with the other stakeholders in the NES. Governments, development partners and other VOPEs are the stakeholders with which VOPEs co-exist within the national NESs and thus with which they should build relations. Successful VOPEs are those that are able to build networks with the entire NES (including international organisations) and leverage these networks to advance their members' interests and the interests of the M&E profession. For example, the UEA works closely with the Directorate of M&E in the Prime Minister's Office, a key oversight entity within the Government of Uganda. The UEA's work with the government is strengthened by the deployment of government staff within the VOPE's management committee. Ghana Monitoring and Evaluation Forum has established relationships with the Ministry of M&E in Ghana. Likewise, TanEA is establishing relationships with the President's Office, Public Service Management and Good Governance (PO-PSMGG). Such linkages between VOPEs and the government provide gravitas and legitimacy for a national evaluation association, providing it with avenues to influence the government development agenda.
It, therefore, follows that partnerships with governments are key as they legitimise the existence and operations of VOPEs. In the African context, government-recognised VOPEs should form partnerships with public sector entities (ministries, departments, agencies, offices and commissions) to promote the undertaking, institutionalisation and use of information and evidence from M&E across government institutions and the NES more broadly. Through such partnerships, VOPEs participate in government-commissioned evaluations of development plans, policies, programmes and projects. Recognition by the government, therefore, enhances a VOPE's legitimacy and credibility within a national NES and the international M&E economy (CLEAR-AA & AfrEA 2021:23).
Partnerships with development partners, such as United Nations (UN) agencies, also provide a vital source of funding and technical support for VOPE activities, conferences and evaluation capacity-building interventions (CLEAR-AA & AfrEA 2021:31). Both UNICEF and the United Nations Development Programme (UNDP) have been instrumental in the founding and growth of several VOPEs. For example, the Nairobi M&E Network was initiated by the M&E officer of the UNICEF Regional Office for Eastern and Southern Africa in 1997. This group held about 50 meetings over 3 years. The meetings took place on UN premises, and participants were from a range of UN agencies and NGOs, the University of Nairobi and sometimes from various government departments and donors. Likewise, the Niger M&E Network (ReNSE, Réseau Nigérien de Suivi et Evaluation) was initiated by the UNICEF M&E officer based in Niger (Segone et al. 2006). Similar experiences are reported in Zimbabwe and Senegal, where UNICEF and UNDP have been critical in the earlier processes of establishing these two countries' evaluation associations.
The evolution of M&E in the continent shows a reliance on international donor funding for development initiatives, access to advanced M&E systems and internationally renowned M&E experts. International development partners wield immense influence within African countries' NESs (Cloete 2016). However, the power of international development agencies is increasingly being counterbalanced by a growing cohort of regional capacity-building agencies that have actively advocated for Afrocentric evaluation practice.
Regional NECD partners frequently mentioned by VOPEs include CLEAR-AA, CLEAR-FA, the African Capacity Building Foundation and the African Leadership Initiative. In Uganda and Ghana, Twende Mbele is also mentioned as an influential NECD collaborator. The role of these partners is slightly different from that of international development partners. The growing influence of regional partners is an important development in the continent, indicating growing M&E technical capacity and expertise in the continent. Although much of the funding for NECD activities originates from the Global North, the shift towards using Africa-based evaluation capacity development institutions can only be beneficial in the long run. It opens opportunities to construct M&E practice and capacity-building approaches that are responsive to the context of different countries and VOPEs.
Notwithstanding the benefits of networking and partnering with the other NES stakeholders, there is an inherent risk of VOPEs being co-opted by these financially endowed and more influential stakeholders. Stakeholders such as governments, regional and global evaluation formations such as EvalPartners present both opportunities and risks that VOPEs must navigate and manage to preserve their relevance, sustainability, impact and responsiveness to the unique national M&E systems and development contexts that vary from one African country to the next. Should a VOPE compromise its independence, its M&E advocacy, originality and relevance could be suppressed in favour of pursuing the agenda of the more powerful co-opting partner(s). Such an event is antithetical to a VOPE's standing objective of advocating for the mainstreaming of M&E practice that is relevant, appropriate and responsive to unique country contexts.
Peer-to-peer learning
Voluntary organisations for professional evaluation are not homogenous and have different areas where they excel. Furthermore, VOPEs are taking opportunities to learn from each other; for example, in 2017, ZEA sent a delegation to GMEF to learn from the work that GMEF was doing in the Ghanaian NES. This was followed by another ZEA exchange programme with ESK on membership management. The ZEA also reached out to the SAMEA to learn how the latter was managing members, the costs and the general functionality of the South African VOPE. The ZEA also participated in the 2019 SAMEA conference, and before the COVID pandemic, ZEA had extended an invitation to SAMEA leadership to participate in the 2020 Zimbabwe Evaluation Week. Although the Namibia Evaluation Association (NamEA) is still in its formative days, it has also expressed a willingness to learn from the more established national evaluation associations on the continent. The Southern African VOPEs have also formed a network with the chairpersons of SAMEA, NamEA and ZAMEA connecting to share experiences and resolve challenges they face. SenEval also mentioned the importance of learning from other VOPEs; they have been trying to establish formal partnerships with the American Evaluation Association (AEA) and others. The francophone VOPEs are also collaborating through the Francophone Network of Evaluation (RFE).
Although most of the peer-to-peer learning has occurred without external facilitation, some requires funding. United Nations Children's Fund, EvalPartners and AfrEA have been hesitant about funding peer-to-peer exchanges between VOPEs. More VOPE-to-VOPE interactions need to be facilitated to encourage sharing of experiences and mutual learning.
Strengthening the work of voluntary organisations for professional evaluation
A significant challenge facing VOPEs in Africa is resource limitation. Voluntary organisations for professional evaluation have limited sources of revenue, and this challenge is not unique to VOPEs in Africa. Across the globe, associations have relied on revenue from conferences and capacity-building workshops. Even then, the conferences and workshops can only be profitable when there is sponsorship from external partners to reduce the input costs of such events. The success of this conference and workshop model also relies on members being able to pay a fee to attend the events. The challenge is compounded by the formative stage of the M&E profession in Africa, whereby a significant number VOPE members are consultants without full-time employment or consistent work opportunities (CLEAR-AA & AfrEA 2021). Such a challenge is common in African countries whose economies are developing. Consequently, VOPEs in Africa cannot charge exorbitant fees for their activities, as this may make them inaccessible to the majority of the membership base. This exclusionary outcome is not desirable as the profession is still being nurtured. This is the complex challenge of revenue generation most African VOPEs face.
Membership fees are another source of income; however, they are not sufficient for making VOPEs financially sustainable entities (CLEAR-AA & AfrEA 2021:29). The VOPEs interviewed during the CLEAR-AA and AfrEA study (2021) have kept fees inexpensive so that costs are not a barrier for individuals and institutions to sign up. For example, ZEA charges $20 for individual members, $10 for students and $180 for institutions. With approximately 150 paying members, these fees are inadequate to cover operation costs - including paying for an administrator. Uganda Evaluation Association charges both annual subscription and one-time registration fees. For individuals, this amounts to $53.80 and an annual subscription of $26.90. For institutions, UEA charges $538.00 per organisation, with annual fees of $403.50. In SenEval, 100 members generally pay for their membership, and largely through membership fees, the VOPE can raise between $5000 and $6000, which has been enough to cover their minimum operational costs, including affiliating fees. However, this does leave the association with limited resources for other activities. Table 2 outlines disaggregated membership fees of select African VOPEs.
Limited resources mean that VOPEs often do not have the means to employ full-time staff to manage activities. This then limits the ability of VOPEs to fully implement their strategies and activities aimed at building the M&E capacity of members. Such strategies and activities include developing evaluation guidelines as a means of advancing the evaluation profession. Moreover, limited resources also hinder the capacity of VOPEs to carry out their evaluation advocacy mandate vis-à-vis the broader NES. Voluntary organisations for professional evaluation that find ways to fund full-time secretariats such as SenEval, UEA and ZEA (before UNICEF funding expired) find that they improve the coordination of their work, freeing the volunteers to focus on key NECD activities.
An added challenge is that only a few VOPE members tend to understand and promote the mandate of the VOPE. This means that VOPE members are not maximising their potential impact on a given country's M&E system. Furthermore, several study respondents indicated that the same VOPE members are often called upon to volunteer for different activities, serving on boards and concurrently implementing various NECD initiatives of the VOPE. This can lead to volunteer fatigue. One respondent suggested that this is partly because of the country's formative stage regarding M&E practice:
'In Namibia - the concept of M&E is emerging, and professionals are not a lot - hardly time to volunteer to do VOPE work. Most people are doing other things, such as consultancy.' (Workshop participant, female, representing the Namibia Evaluation Society)
It is unlikely that African VOPEs are going to get an injection of additional human and financial capacity. Voluntary organisations for professional evaluation, therefore, need to be innovative in how they counteract their current capacity limitations. SenEval's example is demonstrative of how VOPEs can reduce expectations, capitalise on partnerships and use legacy board members to implement the VOPE's evaluation agenda.
A related challenge is that VOPEs in Africa lack clear strategies on how to execute their mandates and mission (CLEAR-AA & AfrEA 2021:30). Voluntary organisations for professional evaluation are, therefore, unable to unify members around a common programme of action that is well articulated and codified in organisational documents. Without practical strategies and action plans, VOPEs cannot systematically operationalise their visions and objectives for growing M&E practice and entrenching it in development practice and governance processes such as development planning, policy and programme formulation, implementation, budgeting and general decision-making.
Discussion
The nascent literature and data on VOPEs suggest that these professional associations are prospective pillars of resilient NESs. National Evaluation Capacity Development interventions are increasingly recognising the value of VOPEs and their prospective impact on strengthening country M&E systems. The established assumption in NECD discourses is that a stronger NES that entrenches M&E practice organically results in greater availability and use of M&E evidence in developmental and governance processes of budgeting, agenda-setting, development planning and policy and programme formulation and implementation. Furthermore, entrenched M&E practice results in regular and systematic performance monitoring of development programmes and policies, as well as periodic evaluation of such development interventions to appraise their outcomes and impact. The use of monitoring and evaluation evidence is crucial in policy, programme development planning and budgeting cycles. The value of VOPEs to NECD interventions, therefore, lies in their ability to advocate for the strengthening of supply, demand and use of M&E information. The integration of M&E practice into the practices of budgeting, development planning, agenda setting, policy and programme formulation and implementation is, therefore, a strategic objective to which African VOPEs should work towards.
The findings of this study have shown that two critical components are important for VOPEs to be effective in contributing to NECD. The first component is strengthening their human and financial capacity, which will enable them to be sustainable, function better and respond to the needs of their members and the broader NES. To this end, this article recommends that VOPEs attract more evaluation professionals, which will in turn lead to greater membership fees, more financial resources and human resources and capacity. To attract a greater number of evaluation professionals and professionals in related fields (i.e. research and consultancy), VOPEs ought to position themselves as associations of choice for evaluation professionals. Being attractive to evaluation professionals requires VOPEs to be clear and strategic in terms of their vision, mission and strategic objectives for the broader NES within which they operate and that they seek to strengthen. Moreover, VOPEs should then make concerted efforts to operationalise such an organisational declaration of intent vis-à-vis the NES, which shall serve as a clarion call for evaluation professionals to join such a noble effort. By attracting a greater number of members (including pre-eminent evaluators), VOPEs will also be creating a favourable internal environment that allows the evaluation association to better articulate its vision, mission and objectives vis-à-vis the NES to external stakeholders of this evaluation ecosystem.
Voluntary organisations for professional evaluation should also commit to a common strategy of recruiting young and emerging evaluators by approaching upcoming evaluation practitioners at graduate and postgraduate levels of study in universities. To this end, VOPEs ought to consider stronger relationships with universities and other HEIs. In particular, VOPEs should reach out to university departments that offer M&E modules and qualifications and recruit students from such departments as interns, volunteers or learnership programme candidates. An important caveat for this strategy is for VOPEs to not engage in practices of exploitation. Should VOPEs be in a financial position to offer paid internships and/or learnerships, then they should offer such as the first course of action rather than going for the option of appointing unremunerated volunteers. Recruiting experienced and emerging evaluation professionals is one area VOPEs should constantly work towards, as they have not saturated the entire evaluation economy and its labour market.
As the article has established that membership fees are insufficient, the article recommends that VOPEs intensify their fundraising efforts vis-à-vis the broader NES stakeholders such as bilateral and multilateral donors, private entities and intergovernmental organisations (i.e. development partners). Voluntary organisations for professional evaluation ought to emphasise their vision, mission and strategic objectives vis-à-vis the NES and also emphasise the value of evaluation when approaching these potential donors, highlighting the importance of evaluation in enhancing development planning, policymaking, development programming, resourcing and implementation. Countries across Africa have a common implementation challenge, and by positioning evaluation as one of the potential remedies to this implementation malaise, VOPEs can mobilise donations.
Recommendations
Contributing towards strengthening the broader NES is the central theme of this article. The article recommends that African VOPEs strive to contribute to the six constitutive elements of an NES referred to in the earlier conceptualisation section of the article. To this end, the strategic recommendations of the article are as follows:
1. Firstly, VOPEs ought to cultivate stronger relations with their respective country governments and lobby for involvement in processes that lead to the formulation of M&E frameworks such as evaluation policies and the supplementary policy implementation frameworks such as national evaluation plans and national evaluation guidelines. It is only after forming good relations with governments that VOPEs stand a chance of being included in the development of such M&E frameworks. Upon inclusion into the process of developing these M&E frameworks, VOPEs ought to amplify their impact by advocating for the inclusion of high-priority areas such as decolonising evaluation, gender mainstreaming, social inclusion and youth empowerment within these M&E frameworks. These are topical developmental issues with which professions such as M&E should be engaged, and that can attract members to VOPEs. By so doing, VOPEs will be leveraging their participation to shine a spotlight on these critical priority areas that dominate current global development and evaluation discourse.
2. Secondly, when appraising the implementation or extent to which development interventions have achieved their objectives, VOPEs ought to raise awareness among their respective governments about the value of evaluative evidence in highlighting what has worked, what has not worked and why. Such evaluative information is vital for resolving programme implementation challenges, resource allocation for programmes and future development planning. This recommendation seeks to amplify the contribution of VOPEs in positively impacting government demand for evaluations. By raising awareness about the value of evaluative evidence in the policy cycle and decision-making, VOPEs can elevate evaluation to the centre of the governance and development agenda. It is through such a position that VOPEs will be able to play a central role in the NES, such as producing key evaluation frameworks that can help build an evaluative culture in public and non-public sectors in respective countries. This recommendation is informed by the reality whereby the practice of evaluation is intrinsically linked to public policy, development programmes and projects and the shift towards results-based budgeting. It is through its integration in these governance and development spaces that evaluation practice - and VOPEs - will gain hard currency as central pillars of NESs.
3. Thirdly, African VOPEs should contribute to and advance scholarship and the gradual transition towards Africa-centred evaluation practice that is responsive to African development values, aspirations and diverse African cultures and community needs. This can be collaboratively done with other NES stakeholders such as HEIs who produce and deliver monitoring and evaluation curricula to both current and future evaluation practitioners. As advocates of M&E practice, African VOPEs require a more nuanced and context-responsive approach when planning and executing evaluation capacity-building initiatives (i.e. M&E training workshops at national and regional VOPE conferences) across the continent. It is perhaps time to intensify discourse regarding the need for African VOPEs to advocate for Afrocentric development and M&E practice centred on the identification of African development values such as social justice and greater opportunities for the marginalised groups in society (i.e. women, youth, children, rural emerging farmers and people living with disabilities). Such Afrocentric M&E practice should focus on developing participatory and inclusive M&E data collection and analysis methodologies, in collaboration with HEIs offering M&E courses. Such methodologies should be linguistically inclusive and gender and culturally aware. In essence, African VOPEs must dedicate time and scholarship to developing participatory data collection and analysis methodologies such as participatory action research, ethnography and observation. Even when such seemingly inclusive methodologies are being developed, greater care must be taken by African VOPEs and their membership in ensuring that such methodologies empower the marginalised and do not expose them to heightened marginalisation, discrimination and abuse. One of the common developmental challenges in Africa is access to education for rural children and adolescents. Africa-centred evaluation methodologies ought to be able to appraise the extent to which development interventions targeted at these disadvantaged groups are ameliorating the lack of access to education. Such context-relevant evaluation data collection methodology will enable evaluations to make considered and appropriate recommendations on how development should augment access to early childhood development education, basic education and higher education to the historically marginalised rural youth and children in African societies. African VOPEs, therefore, need to understand and reflect on the development and M&E context in which they are situated. An African VOPE should, therefore, be constituted by Afrocentric M&E practitioners who collectively and tenaciously promote M&E practice that measures development from an African lens.
4. Fourthly, pursuing sustainability while managing the risk of co-optation by donors. As mentioned in the earlier section addressing VOPE human and financial resource constraints, VOPEs need to develop and strengthen relationships with other stakeholders. While such partnerships are important for resource mobilisation, African VOPEs must balance the risk of co-optation and compromising their authentic vision, mission and objectives vis-à-vis NESs within which they operate.
5. Fifthly, VOPEs also need to understand the capacity of their membership so that their capacity development initiatives are relevant and critical to improving the quality of monitoring and evaluative capacity.
Conclusion
Voluntary organisations for professional evaluation are emerging as vital NES stakeholders that ought to be targeted for technical capacity building as part of NECD interventions. In the long term, technically capacitated VOPEs have the potential to play a major role in helping to build an M&E culture that sustains NESs in many countries. The M&E profession and NECD fraternity have embraced VOPEs as foundational actors in the construction of strong NESs systems that are able to inform development planning, policy and programme formulation, implementation, budgeting and evidence-based decision-making. To effect a positive impact on national NESs and national development agendas, VOPEs need to build capable memberships, improve their financial sustainability and promote a paradigm, methodologies and ideology of evaluation rooted in Afrocentric conceptualisations of development and indicators thereof. Thus, African VOPEs ought to internalise the challenge of advocating for evaluation criteria that inquire about the degree to which development interventions address pressing socio-economic and political challenges facing African populations. This is important if African VOPEs are to transcend the blind imitation of the Western-centric VOPE model that is not necessarily cognisant of and responsive to the African context. A key risk to mitigate for VOPEs when embarking on fundraising is to balance the threat of co-optation by larger and financially endowed NECD partners such as governments, development partners and larger VOPEs.
Acknowledgements
The authors acknowledge the CLEAR-AA staff who contributed to the collection of data for the African VOPEs discussion paper. Special thank you to all sampled African VOPEs who participated in the study.
Competing interests
The authors declare that they have no financial or personal relationships that may have inappropriately influenced them in writing this article.
Authors' contributions
T.F. contributed to the original discussion paper curation and the development of this article. A.D. contributed to the development of the article. M.A. contributed to both the original discussion paper from which the article is written and the article itself. S.M. contributed to the conceptualisation of the VOPE discussion paper and the development of this article.
Funding information
This research received funding from CLEAR-AA and the African Evaluation Association (AfrEA).
Data availability
The data that support the findings of this study can by made available by the corresponding author, A.D., upon reasonable request.
Disclaimer
The views and opinions expressed in this article are those of the authors and do not necessarily reflect the official policy or position of any affiliated agency of the authors.
References
African Development Bank, 2008, Accra agenda for action, third high level forum on aid effectiveness, African Development Bank, Abidjan.
Beney, T., Mathe, J., Ntakumba, S., Basson, R., Naidu, V. & Leslie, M., 2015, 'A reflection on the partnership between government and South African monitoring and evaluation association', African Evaluation Journal 3(1), 1-6. https://doi.org/10.4102/aej.v3i1.164 [ Links ]
Carter, B., 2013, Evaluation capacity development, Governance and Social Development Resource Centre (GSDRC), Birmingham.
Chirau, T.J., Blaser-Mapitsa, C., Amisi, M., Masilela, B. & Dlakavu, A., 2020, 'A stakeholder view of the development of national evaluation systems in Africa', Africa Evaluation Journal 8(1), a504. https://doi.org/10.4102/aej.v8i1.504 [ Links ]
CLEAR-AA, 2019, Monitoring and evaluation systems in five African countries, University of Witwatersrand, Johannesburg.
CLEAR-AA & AfrEA, 2021, Professional associations in the M&E sector: Discussion paper, University of the Witwatersrand, Johannesburg.
CLEAR-AA & TanEA, 2020, Developing the Tanzania Evaluation Association visibility strategy: Report from the scoping study, University of the Witwatersrand, Johannesburg.
Cloete, F., 2016, 'Developing an Africa-rooted programme evaluation approach', African Journal of Public Affairs 9(4), 55-70. [ Links ]
European Union, 2008, Developing evaluation capacity: A study for the commission, European Commission, Brussels.
Fraser, D.I. & Morkel, C., 2020, 'State of monitoring and evaluation in anglophone Africa: Centre for learning on evaluation and results in anglophone Africa's reflections', African Evaluation Journal 8(1), 1-8. https://doi.org/10.4102/aej.v8i1.505 [ Links ]
Goldman, I., 2018, 'The emergence of government evaluation systems in Africa: The case of Benin, Uganda and South Africa', in Monica Lomeña-Gelis (ed.), eVALUation matters, second quarter 2018, pp. 1-12, Africa Development Bank, Abidjan.
Goldman, I., Byamugisha, A., Gounou, A., Smith, L.R., Ntakumba, S., Lubanga, T. et al., 2018, 'The emergence of government evaluation systems in Africa: The case of Benin, Uganda and South Africa', Africa Evaluation Journal 6(1), a253. https://doi.org/10.4102/aej.v6i1.253 [ Links ]
Holvoet, N. & Dewachter, S., 2013, 'Building national M&E systems in the context of changing aid modalities: The underexplored potential of national evaluation societies', Evaluation and Programme Planning 41, 47-57. https://doi.org/10.1016/j.evalprogplan.2013.06.001 [ Links ]
International Organization for Cooperation in Evaluation (IOCE), 2022, VOPE directory, viewed 23 May 2022, from https://ioce.net/vopes/vope-directory/.
Karkara, N., 2013, Advocating for Evaluation: A toolkit to develop advocacy strategies to strengthen an enabling environment for evaluation, United Nations Children's Fund, New York.
Kosheleva, N. & Segone, M., 2013, 'EvalPartners and the role of voluntary organisations for professional evaluation in the development of national evaluation capacity', in J. Rugh & M. Segone (eds.), Voluntary organizations for professional evaluation (VOPEs): Learning from Africa, Americas, Asia, Australasia, Europe and Middle East, pp. 8-12, United Nations Children's Fund (UNICEF), New York, NY.
KPMG, 2011, The Paris declaration on aid effectiveness and impact: What's next in Busan?, KPMG International, London.
Lomeña-Gelis, M., 2013, 'Evaluation development in Senegal', African Evaluation Journal 1(1), a23. https://doi.org/10.4102/aej.v1i1.23 [ Links ]
Ngwabi, N.T., Mpyana, O.G. & Mapatwana, A., 2020, 'Reflections from emerging evaluators in shaping voluntary organizations for professional evaluation capacity-building initiatives', African Evaluation Journal 8(1), 1-6. https://doi.org/10.4102/aej.v8i1.509 [ Links ]
Ojha, V., 2013, Role of CSOs and VOPEs in strengthening national evaluation capacity development, EvalPartners, New York, NY.
Porter, S. & Goldman, I., 2013, 'A growing demand for monitoring and evaluation in Africa', African Evaluation Journal 1(1), a25. https://doi.org/10.4102/aej.v1i1.25 [ Links ]
Rugh, J. & Segone, M. (eds.), 2013, Voluntary organizations for professional evaluation (VOPEs): Learning from Africa, Americas, Asia, Australasia, Europe and Middle East, United Nations Children's Fund (UNICEF), New York, NY.
Segone, M., Patel, M., Rouge, J.C. & Russon, C., 2006, 'National and regional evaluation organizations as a sustainable strategy for evaluation capacity development', in M. Segone & A. Ocampo (eds.), Creating and developing evaluation organizations: Lessons learned from Africa, Americas, Asia, Australasia and Europe, pp. 217-240, Desco, Lima.
United Nations, 2012, National evaluation capacity development: Practical tips on how to strengthen national evaluation systems, United Nations Children's Fund (UNICEF), New York, NY.
United Nations Evaluation Group, 2012, National evaluation capacity development: Practical tips on how to strengthen national evaluation systems, United Nations, New York, NY.
Correspondence:
Ayabulela Dlakavu
ayabulela.dlakavu@wits.ac.za
Received: 26 Jan. 2022
Accepted: 30 May 2022