GAMET HIV Monitoring & Evaluation Resource Library

The Importance of Managing for HIV Results

If you do not measure results, you can not tell success from failure
If you can not see success, you cannot reward it
If you can not reward success, you are probably rewarding failure
If you can not see success, you can not learn from it
If you can not recognize failure, you can not correct it
If you can demonstrate results, you can win public support
Adapted by Kusek and Rist (2004) from Osborne & Gaebler, 1992

Since the turn of the century, ‘managing for development results’ and ‘improving aid effectiveness’ have become the mantras of international development. Driven by international agreements, managing for HIV results have become a critical priority and an integral part of the HIV response for governments and their development partners alike. Examples of agreements that have contributed towards the focus on managing for HIV results and improving the effectiveness of HIV aid include:

Millennium Development Goals (2000)
UNGASS Declaration of Commitment to HIV (2001)
The Monterrey Consensus (2002)
Agreement on the Three Ones (2004)
The three Managing for Development Results roundtables (2002), (2004), (2007)
The Paris Declaration on Aid Effectiveness (2005)
The recommendations of the GTT to improve efforts to ‘make HIV money work’ (2005)
The agreement to work towards universal access for all persons who need it (2006)

NOTE: All the text for this Section of the Library can be downloaded from the Library - the main text, and a set of Annexes.

Click here to read more

Need for M&E systems
HIV M&E system challenges
Introduction to 12 components
Explanation of 12 components
Application of 12 components

The need for Monitoring and Evaluation systems to measure results

M&E are symbiotic processes designed to measure results. Within an HIV context, M&E is important in providing an understanding of where the HIV response is at any given point. For example, monitoring can provide an estimate of the proportion of the population who know their HIV status. Evaluation, on the other hand, indicates why targets (e.g. reduced number of new HIV infections) and outcomes (e.g. changes in high-risk sexual behaviour) are being, or not being, achieved (Kusek and Rist, 2004). Such data empowers HIV decision makers to better understand factors that contribute to good, or alternatively poor, performance (McKay, 2007). This will enable better decision-making and improved accountability for those involved in implementing HIV interventions.

Results-based M&E has become a new focus area within the broader context of traditional implementation M&E. Whereas implementation M&E is project-based and focuses on one intervention only, results-based M&E is designed to address “a range of interventions and strategies”. Results-based M&E takes a systemic view of the wider developmental environment and outcomes to which such interventions contribute. (Fukuda-Parr, Lopes, and Malik (2002), quoted by Kusek and Rist, 2004:17). More information about results-based M&E is available here, or via a download of all ten steps of setting up a results-based M&E system.

For these reasons, national, harmonized, results-based, and multisectoral HIV monitoring and evaluation systems (‘national HIV M&E systems’) are needed to improve the performance of the HIV response in all countries, few of which currently have such systems, due to the challenges involved in their establishment.

Challenges in setting up HIV M&E systems

As a professional field, M&E is comparatively new, as too are HIV M&E systems and – in terms of experience – the personnel and institutions that support these systems. The recent introduction of the field and the concept of drawing data from a single M&E system poses a challenge for development partners, as does the pressure to deliver HIV results from this.
The innovation of HIV M&E has led to a great demand for skilled professionals and capacity in building such systems. There is however a dearth of skilled M&E professionals, and a lack of harmonized training courses and technical advice.
Multisectoral demand for HIV M&E systems has led to the need to concurrently build such systems in all sectors, while a national system is being developed.
The decentralization of the HIV response, in line with local government reform, has led to demand for the decentralization of the HIV M&E system – requiring sub-national and national systems to be developed simultaneously.
The power inherent in making data widely available poses a political challenge and emphasises the need for greater accountability.
Prevailing misconceptions about the purpose of M&E and suspicions about its ‘policing’ function and lack of integration into traditional reporting systems must be addressed and debunked.

For more data about the challenges for HIV M&E systems, go here.

Addressing the challenges: Introducing the 12 components of a functional HIV M&E system

Addressing these challenges will take time and professional application. One part of the solution to the challenges is to create a standard or common definition of what constitutes a functional HIV M&E system.

The World Bank’s GAMET team used a systems thinking approach to support the strengthening of national HIV M&E systems. A ‘system’ is a group of interacting, interrelated, or interdependent elements forming a complex whole (American Heritage Dictionary, 2007). ‘Systems thinking’ is about gaining insights into the whole by understanding the linkages and interactions between the elements that comprise the whole system (Senge, 1990). It follows then that in a systems thinking approach, it is necessary to:

Identify the components of the system (understanding that they are interrelated), as a means to describe the system; and
Ensure that each component is functional, as a means to ensure that the system is functional.

This thinking and application has led to the World Bank identifying the desired components of a functional HIV M&E system and using these components as the basis of a methodology for building HIV M&E systems.

In 2007, development partners were tasked by the MERG to simplify the way in which M&E systems were being assessed. This led to a common definition of ‘a functional, national and multisectoral HIV M&E system’ being agreed to by MERG.

MERG agreed that an HIV M&E system will be defined by means of 12 components (the same components, with some minor modification, identified by the World Bank). This consensus is important in harmonizing HIV M&E systems building efforts and establishes an organizing framework for the development of HIV M&E systems. The 12 components are illustrated below as intersecting and interdependent parts of a larger whole (HIV MERG HIV, 2007b):

Organisational structures with HIV M&E
Human capacity for HIV M&E
HIV M&E partnerships
HIV M&E plan
Costed HIV M&E work plan
Advocacy, communications and culture for HIV M&E
Surveys and surveillance
Routine programme monitoring data
Supportive supervision and data auditing
HIV database HIV evaluation, research and learning
Data analysis, information production and
Using data for decision making

Source: HIV MERG TWG 2007b

Grouping the 12 components: The 12 components can then be sub-divided and arranged into three linked resource and activity rings (Source: HIV MERG HIV, 2007b):

The outer ring (green): This links 6 components related to people, partnerships and planning that support data production and data use – i.e. the enabling environment for HIV M&E to function
The middle ring (blue): This links 5 components related to data management processes – collection, capture, and verification of all types of HIV M&E data, according to all levels of the HIV M&E staircase (see staircase below)
The Inner ring (red): This involves analyzing data to create information, which is then disseminated to inform and empower decision-making at all levels.

You can download a summary PowerPoint presentation about the 12 components here.

Defining the 12 components

(Source: HIV MERG HIV, 2007b)

Interdependence: The graphics above illustrate the different components and rings as intersecting and interlocking parts of an integral whole, confirming the interdependence of the 12 individual components and the three rings.

Components versus implementation steps: These 12 components are not implementation steps and are not necessarily sequential. Different countries may focus on different components, as dictated by resource availability and need.

Applicable to M&E systems of different HIV epidemic states: The organizing framework of a national HIV M&E system is applicable in all HIV epidemic states, as the purpose of the third of the Three Ones (to generate appropriate data to inform the national HIV response) applies to all epidemic states. Therefore, irrespective of whether a country has a concentrated, generalized or mixed epidemic, the M&E system should contain some mix of the 12 components. Countries may prioritise different components at different points in time to suit their specific needs, or group components together as required. The most important point is that a national HIV M&E system achieves certain performance standards, irrespective of how these components have been arranged or grouped.

Component use by level and sector: Each of these 12 components could equally be applied to the national or sub-national (decentralized) HIV M&E system. When developing plans for strengthening the HIV/AIDS M&E system, stakeholders can introduce any of the 12 components at any level of the M&E system. In practice, however, improving system performance in one component or at one level may require concurrent intervention at another level. In other words, a systems thinking approach is needed when using these components to improve system performance. For example, health managers and workers, no matter how skilled, are unlikely to perform essential M&E functions without adequate supplies and equipment, proper motivation and support, and technical guidance.

The 12 components as an organizational framework

The 12 components constitute an organising framework for an HIV M&E system, and can therefore address the challenges of HIV M&E systems in the following ways (HIV MERG TWG, 2007b):

As an organizing framework for the staff, resources, support, and funding required for the HIV M&E system.
As the basis for conducting assessments on the national HIV M&E system in which the status of each component can be assessed.
As a way to develop joint M&E reports or updates on the status of an M&E system, in which each of the components can be reported on.
As a way to establish a clear division of labour at country level and a framework within which all partners can work together.
As a means to develop indicators to measure levels of M&E system operationalization, and the extent to which each component is operational within it.
As the basis for job descriptions and the capacity building of staff in the national HIV M&E unit, this should ensure that the staff members between them, have the capacity and competence to focus on all 12 components of the national HIV M&E system.
As the basis for a checklist of the kind of information needs to be addressed by the national HIV information system.