Gaining insight into impact is in many ways the next frontier in supply chain accountability. Companies working to promote responsible practices need ways to track the effects their programs are having, both to determine whether problems are in fact being solved (and resources well spent), and to identify ways to improve existing programming. With demands for more and better transparency from companies on their supply chain efforts, the need to demonstrate authentic impact grows. Evaluation of social impact can be difficult to do well, however, particularly where programs are multi-dimensional and ongoing, and attribution of observed changes to particular interventions is not straightforward.
As Verité has expanded its work with clients to develop long term, comprehensive programs to address serious social issues in sectors like agriculture, we have been experimenting with ways to supplement audits and other monitoring approaches with qualitative approaches that can be used to verify and deepen the insights coming from more standardized impact data.
Earlier this year, Verité conducted a pilot to test the viability of a participatory impact evaluation method called the “Most Significant Change” (MSC) technique, which involves the collection of first-person narratives of impact from the people most affected by the program in question. The MSC approach was first developed in the natural resource management field, and has recently been taken up by development industry practitioners for evaluation of complex rural development programs, particularly in agriculture. MSC evaluations involve asking people affected by a program to describe in their own words which changes resulting from the program have had the greatest impact on their lives, behaviors, and views. Their words are collected as brief narratives and recorded by neutral, trained “scribes,” and then validated and further analyzed by focus groups of interview participants and other stakeholders. Verité finds this approach particularly appealing as a way of putting more emphasis on the experiences and voices of workers, who are generally still not as central to most compliance programs as they should be.
Verité piloted this technique in collaboration with Philip Morris International (PMI) and two PMI suppliers in March in a small community in Malawi, where PMI’s Agricultural Labor Practices (ALP) program has been implemented intensively for several years. The scribes were recruited independently from a local language institute and trained by Verité, and interviews were conducted with farmers and farm workers in the local language (Chichewa). The participants were asked simply to reflect on the biggest impact – either positive or negative – that the ALP program has had on them. The farms were selected by the suppliers, however no one from the suppliers was present during the interviews, and responses were anonymized to protect the farmers’ and workers’ confidentiality.
The narratives generated through the exercise powerfully brought to life impacts of the ALP program in the voices of real people – at least within this select and admittedly unrepresentative group of farms. Pointing to the importance of the program’s emphasis on direct contracting between the supplier and the tobacco growers, one farmer said:
I managed to buy a piece of land. I have also bought four goats as a result of the contract farming. The goats help me when I need money because I sell some. They also help me because I can slaughter one and use it to pay people for weeding my tobacco fields.
Another farmer simply said,
Since I started following ALP, I have never experienced hunger in my house.
Narratives from workers were also revealing, sometimes pointing to fundamental cross-cutting improvements such as freedom from hunger or abuse, and other times reflecting the impact of specific elements of the ALP program such as ensuring a safe and fair working environment and adequate accommodation, or the requirement that workers be informed of their rights and conditions of employment before starting work. One worker said:
Before I came here I used to work long hours, without being given enough food, like a slave. Here I am given enough food, for example porridge, before I start working, after work I find water to wash myself and lunch plus dinner in the evening just like anyone else in the family. Because of the care I am given I can see a big change because I am now healthy and full of energy such that I work very hard and joyfully because I hardly fall sick…my employer trusts me because I don’t skip work due to illness.
Another worker said:
The most significant change is that at the moment, I get to know how much money I will get at the end of all the work before I actually start the work. Elsewhere, where I worked before, I would only know my pay on the day of payment, and there was no discussion about it, but at present, we discuss the payment before starting work.
Quotations like these are remarkable for the ways they reflect the interconnected and at times intimate nature of the impact of responsible supply chain programs, reframing impact in terms that are less about corporate compliance, and more about workers’ and farmers’ well-being and sense that their working conditions are fair and safe. In this small sample, the impacts of the program highlighted by participants were overwhelmingly positive, but the MSC technique can be used to identify negative impacts and unanticipated consequences of social programming as well. Verité anticipates that the MSC method will prove a valuable tool in our toolkit for evaluating the impact of our work with PMI and other companies that are implementing complex labor practice remediation programs.
For more information about the MSC approach or ALP program, please contact Elizabeth Garland.