MONITORING AND EVALUATION
Toolkit: Scaling Up HIV-Related Legal Services
3 4
Whether services are responsive to clients, effective in resolving problems within a reasonable timeframe, supportive and non-judgemental. Whether there is a process understood by clients for lodging complaints if clients are dissatisfied with the legal service provided.
Most significant change evaluations The ‘most significant change’ method is an example of participatory monitoring and evaluation. The method does not require use of a fixed set of indicators. It encourages participants to share, discuss and document their stories of significant change that result from activities such as introducing legal advice services and human rights education to a community. The approach is to identify changes that are easy for everyone to understand, such as ‘changes in people’s lives’. These may be broad categories that are not precisely defined, such as performance indicators, but are deliberately left loose. Change stories are collected from people directly involved in the service, such as people living with HIV. The stories are collected by asking a simple question, such as: ‘During the last month, what was the most significant change that took place for people’s lives as a result of the legal service?’ Respondents are encouraged to report why they consider a particular change to be the most significant one. The stories are then analysed and the most significant accounts of change are selected. A group of participants is asked: ‘From among all these significant stories of changes in people’s lives, what do you think was the most significant change?’ Every time stories are selected, the criteria used to select them are recorded and fed back to stakeholders. Funders can be asked to select stories that best represent the sort of outcomes they wish to fund, and why. This information is fed back to legal service managers. Reasons why legal services may find this evaluation method useful include: 1 It is a participatory approach that requires no special professional skills. It is easy to communicate across cultures. There is no need to explain what an indicator is. Everyone can tell stories about events they think were important. 2 It encourages analysis as well as data collection. 3 It can be used to monitor and evaluate bottom-up initiatives that do not have predefined outcomes. See: Davies R, Hart J (2005). The ‘most significant change’ (MSC) technique, a guide to its use. Available at www.mande.co.uk/docs/MSCGuide.pdf. 40
Care needs to be taken in monitoring the quality of advice and representation because of the need to respect clients’ interests, including the desire of clients for information relating to them to be confidential. It may not be appropriate for third parties to observe, monitor and report how legal advice is provided unless client consent can be obtained or the information can be anonymized prior to review. Conducting voluntary surveys of client satisfaction after the conclusion of a case is a way to monitor the quality of advice and representation. Attention will need to be given to ensuring that information about a client’s case gathered in this way is only disclosed to others with client consent, consistent with the principle of client-centredness. HUMAN-RIGHTS-BASED MONITORING AND EVALUATION Human-rights-based monitoring and evaluation is an approach that uses a human rights lens to monitor the operation of legal services and the outcomes achieved for clients and the community. This can promote the human rights accountability of organizations providing HIV-related legal services. Monitoring and evaluation can be explicitly informed by a human rights approach by adopting a human rights impact assessment methodology. Obligations arising from international human rights law are legally binding on national governments. This includes all those employed within the public sector, such as lawyers working in state legal aid agencies. The conduct of state legal aid agencies and outcomes achieved can be monitored against standards and norms of international human rights laws, such as those related to non-discrimination and privacy. Within a human rights framework, lawyers working in the private sector and nongovernmental organizations are expected to carry out their activities with full regard for the fundamental