Measuring outcomes of leadership programmes for grassroots leaders

Researchers, implementors and policy makers working in the development space recognise that grassroots leaders are potent forces for catalysing change at the community level. These leaders are deeply committed to building inclusive communities and work tirelessly to improve the quality of life of their constituencies. Recognising the catalytic role played by these leaders, several leadership programmes to strengthen knowledge, skills, capacities, capabilities, and networks of these leaders have emerged.

Within the ambit of policy, advocacy, and strategic communication interventions, support to grassroots leaders is made largely to advance an issue (e.g. to promote adoption of contraception among young and low parity couples) or to increase community self-determination (e.g. listen to and give voice to the needs and aspirations of women vis-à-vis sexual reproductive and maternal health).

How are these leadership programmes different? Developing leaders for social change goes much beyond the traditional focus of managerial leadership within a traditional structure - the focus is on fostering mindsets and an armoury of skills, tools and resources that help the leader grapple with and address complex and systemic issues Also, given the background, education, training and access to infrastructure and technology of these leaders, their needs and support required are very different from positional leaders (ORS Impact). This makes the contours of grassroots leadership development programmes unique.

Programmes have a three-pronged focus - individual leadership capacity, organisation capacity and community/system/issue level (McGonagill and Reinelt 2011, Sabatelli et al 2005, Hannum et al 2007). The former focuses on personal transformation and strengthening competencies of leadership, communication, collaboration and systems thinking, among others. While organisational capacity includes elements of leadership & governance, programme planning & implementation, and partnerships; mobilisation, civic participation and network building are components in focus at the community level.

What do we need to keep in mind? First, the primary focus of these programmes is to develop individual capacities; and the assumption and belief is that this will have trickle down enhancement effects at the organisation and community level. In reality, this may or may not pan out as envisaged - the pathways between individual change and organisational and community change are unpredictable and circuitous (to say the least). Second, while the transformative journey of the leader begins when he/she joins the programme, concomitant changes at the organisation and community level take time to deepen and develop. Thus, in order to understand the full impact of the programme, donors, implementing agencies and Monitoring, Evaluation and Learning (MEL) teams, need to invest, implement, and be connected with the programme (respectively) over a long-time horizon.

How does one think about measuring leadership capacities? A good starting point is to delineate a Theory of Change (ToC), bringing clarity on unit of change, timeframe helping evaluators focus on measurement that supports learning across projects. The next step is to outline specific evaluative questions at each of the three levels. For example, at the individual level, questions could include - "How has awareness and competencies changed? Which training elements are most associated with change?", while at the organisation level, our focus can be on understanding, "To what extent has the organisation's partnerships improved or the extent to which technology and media is being leveraged?". (Meehan, Reinelt and Leiderman 2015).

This must be followed by a conversation on 'time frames' in measurement. We need clarity on what types of changes we foresee at the individual, organisation, and community level and what is the timeline for the same. Given that the grants in our policy, advocacy and strategic communication portfolio span a two-year time-horizon, we are most likely to see short-term/immediate changes (within 12 months) at the individual level, and potentially some intermediate outcomes (12-24 months) at the organisation level. At the individual level, within a 12-month period, we potentially can see results stimulated by actions of the programme or its participants such as knowledge gained or collaborations between participants (these results are oft-referred to as 'episodic measures'). Between 12-24 months, we may see 'developmental measures' (such as sustained change in behaviour). Long-term, paradigm shifts in thinking and ways of working among leaders and changes in the community, or system are likely seen after 24 months (Grove et al 2005)

What should we measure and how? At the individual level, attention is devoted primarily to episodic measures of core leadership competencies such as advocacy skills, systems thinking & social analysis, communication, collaboration, problem solving and participatory methods as well as leaders' ability to recognise personal strengths and weaknesses. Secondarily, developmental measures of cross-cultural competence, risk taking, adaptation and reflection can be captured. A spectrum of methods is envisaged - multi-rater capacity assessments (360 degree) take the lead and are buttressed with participatory methods such as appreciative inquiry and most significant change, case studies and reflective participant questionnaires. A quick digression on capacity assessments. While our quick scan revealed abundant options (e.g. Concord Leadership Assessment Survey, Leadership Practices Inventory, Campbell Leadership Index, Community Leadership Development Measure) we realise that we would need to customise these tools, for both a grassroots leader and for an Indian context! At the organisation level, we can capture developmental measures in the intermediate time frame. These include changes in systems & structures, technology upgrades, use of media, learning & innovation, and ability to motivate & mobilise constituents. Methods include an amalgam of internal documents (e.g. budgets, reports), media tracking, organisational capacity assessment, key informant interviews and participatory methods.

In conclusion, over the last decade, there has been an explosion of grassroots leadership programmes with the aim of driving sustained transformative change. Measuring whether and how these programmes trigger change is indispensable - for strategic decision making, design, strategy, and deployment considerations. Measuring nebulous concepts such as capacities, capabilities and civic participation that unfold quietly is daunting. However, a long-term, embedded MEL approach, with a measurement architecture anchored in a ToC, with culturally appropriate tools and an amalgam of methods that monitor and evaluate change across a continuum can aid this process.