Agricultural Systems 155 (2017) 88–102

Contents lists available at ScienceDirect

Agricultural Systems
journal homepage: www.elsevier.com/locate/agsy

Towards a complexity-aware theory of change for participatory research MARK

programs working within agricultural innovation systems
Boru Douthwaitea,⁎,1, Elizabeth Hoffeckerb
a
WorldFish, PO Box 500 GPO, 10670 Penang, Malaysia
b
D-Lab, Massachusetts Institute of Technology (MIT), 265 Massachusetts Ave., Cambridge, MA 02139, United States

A R T I C L E I N F O A B S T R A C T

Keywords: Agricultural innovation systems (AIS) are increasingly recognized as complex adaptive systems in which
Theory of change interventions cannot be expected to create predictable, linear impacts. Nevertheless, the logic models and theory
Complexity of change (ToC) used by standard-setting international agricultural research agencies and donors assume that
Agricultural innovation systems agricultural research will create impact through a predictable linear adoption pathway which largely ignores the
Participatory action research
complexity dynamics of AIS, and which misses important alternate pathways through which agricultural
Empowerment
research can improve system performance and generate sustainable development impact. Despite a growing
Evaluation
body of literature calling for more dynamic, flexible and “complexity-aware” approaches to monitoring and
evaluation, few concrete examples exist of ToC that takes complexity dynamics within AIS into account, or
provide guidance on how such theories could be developed. This paper addresses this gap by presenting an
example of how an empirically-grounded, complexity-aware ToC can be developed and what such a model might
look like in the context of a particular type of program intervention. Two detailed case studies are presented
from an agricultural research program which was explicitly seeking to work in a “complexity-aware” way within
aquatic agricultural systems in Zambia and the Philippines. Through an analysis of the outcomes of these
interventions, the pathways through which they began to produce impacts, and the causal factors at play, we
derive a “complexity-aware” ToC to model how the cases worked. This middle-range model, as well as an
overarching model that we derive from it, offer an alternate narrative of how development change can be
produced in agricultural systems, one which aligns with insights from complexity science and which, we argue,
more closely represents the ways in which many research for development interventions work in practice. The
nested ToC offers a starting point for asking a different set of evaluation and research questions which may be
more relevant to participatory research efforts working from within a complexity-aware, agricultural innovation
systems perspective.

1. Introduction be predicted by understanding the behavior of discreet actors within

the system (Axelrod and Cohen, 2000; Axelrod and Cohen, 2000). In
Agricultural innovation systems are increasingly understood to be CAS, small initial changes in system conditions can create large and
complex adaptive systems, a type of complex system with specific unanticipated impacts throughout the system, even when system
characteristics that hold significant implications for interventions components are connected in ways that are causally deterministic
seeking to create “impact” within these systems. In complex adaptive (Miller and Page, 2007).
systems (CAS), a wide array of heterogeneous actors adapt their While complex adaptive systems do not readily lend themselves to
strategies and actions based on the actions of others and on changing control or management due to their unpredictable nature (Spielman
system conditions, while contributing to these changing conditions et al., 2009; Arkesteijn et al., 2015), they can be successfully intervened
through their evolving responses to them (Spielman et al., 2009; Klerkx into if the intervener has an understanding of the dynamics of CAS and
et al., 2010). As a result of the dynamic nature of these inter-connected how to harness these (Williams, 2011). Snowden (2010) proposes a
changes, CAS produce unpredictable yet recognizable patterns, such as strategy of seeing program intervention as catalytic probes that
co-evolution, path dependency and emergent properties, which cannot stimulate patterns of activity. Program staff then stabilize and amplify

⁎
Corresponding author.
E-mail address: [email protected] (B. Douthwaite).
1
Currently: Boru-Consult, Cushalogurt, Kilmeena, Westport, F28 W654, Ireland.

http://dx.doi.org/10.1016/j.agsy.2017.04.002
Received 24 November 2015; Received in revised form 5 April 2017; Accepted 6 April 2017
0308-521X/ © 2017 Elsevier Ltd. All rights reserved.
B. Douthwaite, E. Hoffecker Agricultural Systems 155 (2017) 88–102

Table 1
Comparison of the traditional approach to agricultural research for development with a recent complexity-aware one.
(Adapted from Klerkx et al. (2012) and Douthwaite (2016).)

Characteristics Linear approach to AR4D Complexity-aware approach to AR4D

Name “Transfer of technology” or “pipeline” “Agricultural innovation systems”

Era Central since 1960s to present From 2000s to present
Mental model and activities Supply technology to next user Co-develop innovation involving multi-actor processes and partnerships
Knowledge and disciplines Single discipline driven (mainly plant breeding) Transdisciplinary, holistic systems perspective
Drivers Supply-push from research Responsiveness to changing contexts, patterns of interaction
Source of innovation Scientists Multiple actors, innovation platforms
Role of farmers Adopters or laggards Partners, entrepreneurs, innovators exerting demands
Role of scientists Innovators Partners, one of many responding to demands
Key changes sought Benefits accruing from technology adoption Institutional change, increase in system capacity to innovate
Dynamic Research begins quickly according to a pre-defined Intervention begins by building relationships and trust through an open research
agenda agenda

beneficial patterns and dampen down and kill off negative ones. This is 2012; Schut et al., 2016). This narrative holds that agricultural
similar to the improvisational model of change management proposed researchers develop knowledge, technology, and processes to address
by Orlikowski and Hoffman (1997) in which planned change gives rise the problems of farmers and other agricultural system actors. These
to emergent change that then provides opportunity for further planned innovations are passed on to other organizations who are tasked with
change. promoting their adoption and use (Hellin et al., 2008). Impact for end-
A sub-set of agricultural research interventions over the past twenty users and for the system derives from the adoption, use, and scaling of
years have been designed by actors who are aware of the complex these improved technologies and ways of doing things, which can
nature of agricultural innovation systems (AIS). These interventions include new or improved methods (Ayele et al., 2012; Schut et al.,
have sought to harness the dynamics of complexity to catalyze system 2016; Gaunand et al., 2015; Joly et al. 2015; Wigboldus et al. 2016).
learning, innovation, and adaptive change within AIS. Examples of This model has several names in agriculture including the “pipeline”
these “complexity-aware” approaches to agricultural research include approach to innovation (Sumberg et al., 2003), the “central source of
Integrated Natural Resource Management in the 1990s (Campbell and innovation” model (Biggs, 1990) and the “transfer of technology” or
Sayer, 2003), Learning and Action Research in the 2000s (Probst and “diffusion of innovation” approach (Klerkx et al., 2012). In industry, the
Hagmann, 2003), and Adaptive Collaborative Approaches (Ojha and model is called the “delivery” mode or “over-the-wall” approach
Hall, 2013). Such approaches cast extension agents and researchers in (Leonard-Barton, 1995). We call this conventional model the “adoption
the role of “innovation brokers” (Klerkx et al., 2012), and facilitators of impact pathway” where “impact pathway” refers to a causal chain of
multi-stakeholder innovation processes (Dugan et al., 2013; inputs, processes and outcomes that lead to impact.2
Kraaijvanger et al., 2016). Klerkx et al. (2012) provides a summary of In the past five years, several studies have sought to better under-
the evolution of systems and complexity-aware approaches based on a stand and describe how agricultural research efforts create societal
literature review. Table 1, adapted from that paper, compares the impact, focusing on uncovering diverse impact pathways and on
traditional linear approach to technology development and transfer understanding aspects of the research process which themselves con-
with a complexity-aware one to illustrate the dimensions of difference tribute to producing and sustaining impact over time (Gaunand et al.,
between the two approaches. 2015; Schut et al., 2014). These studies have highlighted the impor-
While much has been written on the need for systems approaches tance of process-related factors, such as the quality and duration of
when intervening into complex natural, social, and/or economic research partnerships, the nature of roles and relationships between
systems, less has been said about the outcomes that result from using researchers and stakeholders, and the type of research strategies used in
these approaches. There is, however, a small body of empirical work particular contexts as important determinants of impact (Joly, et al.
which is starting to show that these approaches generate benefits that 2015; Schut et al., 2016). However, the insights and findings emerging
contribute to the ability of local systems to evolve in ways that from this work have not yet been incorporated into usable, alternative
contribute to inclusive and sustainable development. Complexity-aware theories of change (ToC) which could guide the program planning and
research interventions into AIS can build multiple types of social capital evaluation work of major actors in international agricultural research.
(Uphoff and Wijayaratna, 2000), increase system actors' skills and Despite significant criticism from within the literature, the long-
confidence in systematic experimentation, and lead to the development established adoption impact pathway therefore remains the dominant
of new practices and technologies as well as the application of existing overarching change narrative for major international funders of re-
agricultural knowledge and technology to new local contexts (Ayele search and innovation related to global development (Dalrymple 2008;
et al., 2012; Sterk et al., 2013; Hounkonnou et al., 2016; Kraaijvanger Renkow & Byerlee, 2010). It is also the dominant change narrative for
et al., 2016). There is also evidence that these approaches improve the agenda-setting institutions for international agricultural research such
functioning of local and regional institutions (Hounkonnou et al., 2016) as the Food and Agriculture Organization (FAO) and the CGIAR3
as well as the linkages and relationships between key system actors
(Douthwaite et al., 2015). The benefits of these outcomes can be
significant for rural smallholders and other system stakeholders: Uphoff 2
An impact pathway is a more descriptive synonym for “theory of change” (ToCo)
and Wijayaratna (2000) found that investment in farmer-led irrigation (Douthwaite et al., 2003), which describes how and why a program works (Weiss, 1995).
groups built specific forms of cognitive and structural social capital that ToC is useful to guide implementation and as the basis of theory-driven evaluations
allowed farmers to significantly increased agricultural productivity in (Douthwaite et al., 2003; Stame, 2004).
3
The CGIAR is a worldwide partnership addressing agricultural research for develop-
the face of sudden and severe water scarcity over thousands of hectares ment carried out by 15 research centers. The CGIAR's vision is a world free of poverty,
in the Gal Oya area of Sri Lanka. hunger and environmental degradation (CGIAR, 2016). As of 2014, the CGIAR employed
Despite evidence that complexity-aware approaches can produce more than 8500 researchers and support staff worldwide, with an annual budget of US
valuable results, the dominant narrative about how agricultural re- $800 million (Agropolis International, 2015). While CGIAR funds represent a small
proportion of the total global funds invested in agricultural research in developing
search creates impact, particularly in the context of developing
countries, the CGIAR influences how this investment is conceptualized, implemented and
economies, remains complexity-blind (Ekboir, 2003; Klerkx et al., evaluated

89
B. Douthwaite, E. Hoffecker Agricultural Systems 155 (2017) 88–102

(formerly known as the Consultative Group on International Agricul- research outputs as well as research processes led to outcomes in the
tural Research). The adoption impact pathway has the advantage of case of a program for which research process and empowerment were
being familiar, simple, and offering a plausible path to touching the seen as important pathways to impact from the outset. We conducted a
lives of large numbers of people. The extent and benefits of adoption detailed analysis of two cases of successful program implementation,
and resulting returns on investment can be calculated and claims can be examining how research process and output led to early outcomes and
made for the impacts of specific technologies and practices that have impacts and describing the causal dynamics at play as identified by key
achieved wide-scale use, such as cell phones, mobile money, or stakeholders and researchers (both internal and external) who were
improved seed varieties. involved in each of the cases. Drawing from our understanding of
While this impact pathway applies to research carried out within complexity science, realist evaluation, and reflexive monitoring and
existing innovation trajectories (Ekboir, 2003), for example plant evaluation, we develop a timeline and causal narrative for each
breeding and maintaining the yield potential of modern crop varieties, intervention and construct from these a middle-range ToC which
the overwhelming focus on it has obscured other ways in which describes the key features and dynamics present in both cases. As a
agricultural research and innovation efforts are producing development middle-range theory, this model seeks to describe the key dynamics of
impact. Complexity-aware programs are particularly disadvantaged by the cases at a level of abstraction which might allow the model to
the adoption impact pathway narrative because they are not attempting capture essential features of other complexity-aware interventions into
to manage towards predicable outcomes within existing innovation agricultural innovation systems. We also develop an overarching ToC,
trajectories, but rather to provoke and then harness beneficial system under which the middle-range theory is nested, that identifies self-
interactions and dynamics (Douthwaite et al., 2003; Arkesteijn et al., reinforcing feedback loops that are possible when programs pursue
2015; Ton et al., 2014) in the process of catalyzing and supporting new both technology development and empowerment pathways.
ones. These programs therefore cannot easily forecast their impacts ex
ante and may also produce unexpected impacts which are not included 2. Theoretical foundations
in the adoption impact pathway and which can therefore remain
invisible to evaluators, donors, and organizational decision-makers. In order to develop an empirically-based, complexity-aware ToC, we
Complexity-aware programs, therefore, face challenges in communicat- draw on several different bodies of theory to develop the conceptual
ing their impact to donors, particularly in the absence of causal models framework underlying our approach. Several core concepts informing
that more accurately describe how these programs work and what our approach come from Realist Evaluation, starting with the idea that
results they produce. the degree to which programs bring about change depends on how
In this paper, we focus on the case of one program which found people interpret and use what programs provide. Programs trigger
itself facing the challenge of communicating its way of working and underlying causal mechanisms, often rooted in the cognitive processes
research outcomes: the CGIAR Research Program on Aquatic going on inside people's heads, which are influenced by context and
Agricultural Systems (AAS), for which the first author worked as a history (Pawson and Tilley, 1997; Westhorp, 2014). According to Weiss
research theme leader for four years. As we will illustrate, AAS was (1997) mechanisms are the responses that program activities generate.
operating in a complexity-aware way, yet four years into its intended In this paper, we develop what Pawson (2013) calls a middle-range
12-year lifespan, the program was closed by the CGIAR because the theory that can abstract across cases to identify the common mechan-
impacts it was producing were not judged to be significant in terms of isms at work. Middle-range theory is useful because it can guide new
their ability to contribute to the CGIAR's overarching results frame- projects in developing their context-specific or particular ToC, and
work, based on the adoption impact pathway (CGIAR Consortium, provide a framework for accumulating learning (Pawson, 2013).
2015). Important outcomes emerging from the program, such as the According to Pawson and Tilley (1997, p.123–4):
program's contribution to building capacity to innovate in the geo-
“The basic idea of middle-range theory is that the propositions do
graphic areas in which it worked, were not seen or valued for their
not have to be developed de novo on the basis of local wisdom in
ability to contribute to the CGIAR's overall impact goal to bring 30
each investigation. Rather they are likely to have a common thread
million people out of poverty by 2024 (CGIAR, 2016).
running through them traceable to [the] more abstract analytic
The story of this program and its closure highlights a gap in the
frameworks …”.
existing literature and in practice which this paper addresses: the lack
of concrete, empirically-based theories of change that are consistent In developing an approach to creating a complexity-aware ToC, we
with a complexity perspective and that demonstrate how programs also draw on the idea that useful ToC should be nested (Mayne, 2015)
engaging with complexity produce development outcomes and impacts. such that a program or research system will have an overarching ToC,
There is a growing literature calling for complexity-aware evaluation of describing its high-level causal assumptions, under which more detailed
programs that intervene in complex systems (e.g. Douthwaite et al., and grounded ToC is developed for individual projects, or elements of
2003; Stame, 2004; Rogers, 2008; van Mierlo et al., 2010; Patton, 2011; them. Nesting helps prevent ToC from becoming overly complicated
Britt and Patsalides, 2013; Arkesteijn et al., 2015), but these authors such that the diagrams no longer readily communicate their causal
stop short of developing ToC that could be used or tested in these logic.
evaluation processes. Similarly, a number of authors within the We take as given that, even in complex systems, change happens
agricultural systems and evaluation literature have argued for and through relatively stable patterns of activities that emerge and die away
proposed frameworks to inform the implementation and evaluation of over time. These patterns have been called technology trajectories
complexity-aware interventions (e.g., Pretty and Chambers, 1993; Hall (Ekboir, 2003), innovation trajectories (Douthwaite and Gummert,
et al., 2003; Kristjanson et al., 2009; Nederlof et al., 2007), but these 2010) outcome trajectories (Paz-Ybarnegaray and Douthwaite, 2016)
frameworks have remained largely normative and have not been and beneficial coherence within attractors (Snowden, 2010). An
translated into empirically-grounded models with clear implications empirically-based ToC should give a sense of recurring patterns of
for practice. We have found one paper to date (Douthwaite et al., 2003) behavior that programs may have catalyzed or contributed to catalyz-
that has proposed a theory of change (ToC) to guide implementation ing, along with other factors. Linked to this idea is Scriven's (1976)
and evaluation of projects that develop embodied technologies in observation that successful programs have a distinctive modus oper-
complex systems. However, this ToC does not model how the agricul- andi–at some level of abstraction, they trigger similar mechanisms
tural research process builds the capacity of the people and institutions across the places in which they work, even if those sites differ to some
who take part, and how that capacity fosters innovation. extent in context and history. The final element informing our
In this paper, we develop a non-linear ToC that models how conceptual framework is the finding from Senge's (1990) systems

90
B. Douthwaite, E. Hoffecker Agricultural Systems 155 (2017) 88–102

dynamics work that complex processes have self-reinforcing and research interventions bring about change within complex agricultural
dampening processes that interact with each other. Self-reinforcing innovation systems, we needed cases which exemplified a complexity-
processes provide for leverage, or in other words, for relatively small aware, participatory approach. These cases met that criteria and were
interventions to have a large impact. Complexity-aware ToC should therefore selected for further development and analysis for this paper.
model for both self-reinforcing and dampening processes.
3.3. Data sources and analysis
3. Methodology
The original four cases included in the AAS program report were
3.1. Overview selected and developed using case study methodology (Yin, 1989) as
described in more detail in the report (Douthwaite et al., 2015). Each
In this paper, we present two cases from the AAS research portfolio case had two hub-level authors who developed a timeline of key events
which program staff highlighted as cases that were starting to produce and processes in the case and a narrative to describe causal links
strong outcomes at the time the program was closed. Through in-depth between them. This timeline and narrative was developed from their
case histories, we present the details of each case and then examine own direct experience as participants in the work and from a range of
these to discern the dynamics and causal mechanisms operating in each source materials, including program monitoring and evaluation data,
case: what outcomes were emerging, what impact pathways these were existing research and program reports, and staff reflection during after-
leading to, and what key factors were combining to create these action reviews. Drafts produced by hub-level authors were reviewed
outcomes, from the perspectives of program staff, researchers, and and interrogated by an international AAS research team member (a co-
evaluators engaged in a six-month-long process of documenting and author on this paper), resulting in several rounds of clarification,
assessing each case. We use these findings to inductively develop a additional data collection, and verification at the local level, until the
middle-range ToC to describe the emerging outcome trajectory com- case histories were deemed to be sufficiently documented, triangulated,
mon to both cases. We also develop a higher-level model with broader and verified.
applicability in which the first model is nested. We offer both models For this paper, we re-analyzed the two selected case histories to
and their accompanying narratives as alternate ToC – alternative understand what outcomes were achieved and how, with a particular
narratives for how agricultural research can create impact within rural focus on understanding the dynamics of causality present in the cases.
agricultural innovation systems. The two models allow us to see We cross-checked and supplemented the initial case material with data
different types of outcomes, a different impact pathway, and important from a separate Outcome Evidencing process conducting by AAS staff
causal connections and mechanisms which we are blind to when between March 2014–2015, subsequently published as a methods note
viewing complexity-aware programs through the lens of the conven- in the American Journal of Evaluation (Paz-Ybarnegaray and
tional adoption impact pathway. We conclude by drawing out the Douthwaite, 2016). Outcome Evidencing involved identifying, cluster-
implications of complexity-aware ToC for generating evaluation and ing, and verifying outcomes and impact pathways for each of the hubs,
research questions that are more useful and relevant to programs conducted with participation from hub-level staff, local stakeholders,
seeking to harness the dynamics of complexity within agricultural international research staff from AAS, and independent evaluators. The
innovation systems. Outcome Evidencing process resulted in hub-level outcome evidencing
reports for both countries (Paz-Ybarnegaray, 2014; Chisonga et al.,
3.2. Selecting the cases 2014). Revisiting the cases with new data, using a researcher external
to the AAS program who was attentive to potential biases, helped
The two cases presented in this paper were developed by AAS strengthen the internal validity of the cases. We then used the cases to
program staff to document key areas of program learning and results develop a middle-level ToC describing how RinD inputs and other
following a program review conducted in January 2015 (Douthwaite factors contributed to observed outcomes. We also built on existing
et al., 2015). The review brought together staff from each of AAS's five synthesis from the final chapter of the program report (Douthwaite
regional hubs to engage in cross-hub learning regarding emerging et al., 2015).
program outcomes. The review identified several cross-cutting themes,
including community engagement, partnerships and “inclusive 3.4. Background on the cases
science”. This refers to instances in which researchers and scientists
found themselves working with farmers and local stakeholders in a way The CGIAR Research Program on Aquatic Agricultural Systems
that was different from business as usual and which involved shifting (AAS) was one of fifteen research programs implemented by the
from top down and transactional relationships towards engaged and CGIAR, and was launched in 2011 with the aim of reducing poverty
more equal partnership. This term was agreed upon by participants in and improving food security for small-scale fishers and farmers
the workshop as one that captured their shared experience of using dependent on agricultural systems (AAS, 2011). AAS established
AAS' approach of “research in development” (described below) as operations in locations bounded by an important aquatic agricultural
compared with standard approaches to agricultural research for devel- system, which the program referred to as “hubs.” These were strategic
opment (R4D). “locations within key aquatic agricultural systems where innovation
Five inclusive science cases were identified by consensus in the and learning can bring about development outcomes” (AAS, 2013, p.
January 2015 program review by hub teams of two to five people, 5). AAS hubs were set up in five locations: Zambia, Bangladesh,
including the hub leader. The choice was later verified with the full hub Cambodia, the Philippines and Solomon Islands.
teams, respectively. From five inclusive science cases developed by AAS AAS set out to achieve its goal by developing a complexity-aware
staff, four were selected for development and publication in a chapter in research approach called “research in development” or RinD for short
the program report Research in development: Learning from the CGIAR (Dugan et al., 2013). The program coined the term to signal its
Research Program on Aquatic and Agricultural Systems (Douthwaite et al., intention to carry out research in support of—and embedded with-
2015). Other chapters of the report covered other cross-cutting themes. in—on-going development processes, in contrast to research “for”
Following publication of the report, we selected two of the four development, in which researchers produce outputs from outside a
“inclusive science” cases as the source material for this paper, choosing system to be adopted by users within that. The RinD approach involved
those we independently assessed as best exemplifying the participatory an engagement process in which the program facilitated stakeholders
research approach at the heart of AAS' Research in Development (RinD) both at the hub-level (regional) and local community level to articulate
methodology. In seeking to understand and model how participatory their respective priorities and visions. Program staff then brokered

91
B. Douthwaite, E. Hoffecker Agricultural Systems 155 (2017) 88–102

agreement on specific issues that villagers and hub-level stakeholders Zambia as it is in neighboring countries as well. It is estimated that
were motivated to work on as a way of developing relevant technol- nearly one third of the total biomass of fish harvested in Zambia is lost
ogies as well as building linkages and capacity for local development. due to various types of post-harvest loss and mismanagement (Béné,
The RinD approach is based on and uses techniques from partici- 2011), reducing the amount of fish available to consumers and
patory action research (PAR). PAR is a participatory process of inquiry contributing to food insecurity. Furthermore, when fish reaching the
that uses iterations of acting and reflecting to answer questions about market has been degraded in quality, it reduces the prices customers are
real life concerns to improve the wellbeing of those engaged (Reason willing to pay and represents significant lost income to fishing families,
and Bradbury, 2001; Apgar and Douthwaite, 2013). The process is processors, and traders.
dynamic and continuous, enabling feedback in real time, unlike most
research endeavors that present findings after the fact. The participa- 4.1.1. Challenges in the agricultural system
tory and action-oriented focus is assumed to build ownership of the These challenges, which are common throughout Zambia, are
process by the participants, who learn through their own experiences prevalent in the Barotse Floodplain, an area located in the upper
and are able to change their own lives and social worlds (Apgar and Zambezi river where AAS established its program hub in 2011. This
Douthwaite, 2013). Participants volunteer to engage based on interest area contains a fishery of around 80 species and employs an estimated
and motivation, rather than on the basis of a standard menu or set of 70,000 people (AAS, 2013). Due to the lack of cold chains, fresh fish is
criteria. The RinD approach used by AAS is described in detail processed in the Barotse using sun-drying and smoking, both of which
Douthwaite et al. (2015). produce brittle fish that are easily damaged in packing and transport,
AAS was an outlier within the portfolio of 15 CGIAR Research while being susceptible to loss from insects, which lay their eggs in the
Programs in adopting PAR as a central research methodology and using fish while it is drying. Insects and rodents also eat dried fish in storage
it to determine research priorities, rather than predetermining these and to prevent this, some processors use toxic chemicals to protect the
before work started. As the program began implementation, the fish and prolong its life.
differences became clearer and concerns began to be raised at the In addition to the substantial challenges related to post-harvest
CGIAR system level as to whether this was the sort of research the spoilage and inadequate processing methods, a value chain needs
CGIAR should carry out (pers. comm. Wayne Powell, CGIAR Chief assessment conducted by AAS in 2013 identified falling fish catches
Scientific Officer, October 2014). In 2015, after steep funding cuts, the in the Barotse Floodplain as another issue placing pressure on the fish
CGIAR decided to close two CGIAR Research Program, one of which value chain (Longley et al., 2016). The study identified, overfishing, a
was AAS. The reason given for this decision was poor performance failure to respect a fishing ban during fish breeding season, and the use
against three criteria: a bibliometric analysis of science quality; a of fishing nets (mosquito nets) with illegally small mesh size the main
performance rating and anticipated performance (CGIAR, 2016). The reasons given.
latter received a failing grade, and was based on two inputs, one of
which was a review by the CGIAR's Independent Science and Partner- 4.1.2. The research engagement process
ship Council (ISPC) of the AAS proposal to extend its work for eighteen The AAS team began its engagement in the Barotse floodplain with a
months from mid-2014 (ISPC, 2014). The ISPC critiqued AAS for being scoping phase that identified a compelling development challenge
an experiment in development, for being unclear about its technologies facing the floodplain and stakeholders with responsibility and interest
and for using a theory-based evaluation approach that did not require to tackle it (AAS, 2012). This so-called “hub development challenge”
the use of control groups (ISPC, 2014). The critique can be understood was “to make effective use of the seasonal flooding and natural
as the ISPC evaluating AAS against the mainstream view of a CRP – a resources in the Barotse floodplain system through more productive
program that develops technologies for which treatment effects can be and diversified aquatic agricultural management practices and tech-
assessed by giving some villages the treatment and others not, and nologies that improve the lives and livelihoods of the poor”
measuring the difference. AAS had failed to communicate its complex- (Douthwaite et al., 2015, p. 63). There followed a stakeholder
ity-aware approach and results in a way that decision-makers in CGIAR consultation workshop in June 2012, in which identified stakeholders
found convincing. from the floodplain, including community, government, research, NGO
The program closed in early 2016 after four years of a planned and private sector representatives, identified opportunities to tackle the
twelve-year lifespan. This created an arbitrary end-point in each of the challenge. Subsequently, AAS staff conducted community-level engage-
cases, as the respective innovation processes were still mid-course at the ment and visioning as well as village-level action planning in 10
time. For this reason, in each case we report on outcomes and emerging communities between August and September 2012, leading to a
impact pathways that were able to be documented by AAS program program design workshop in October 2012, facilitated by the first
staff before the closure of the program; however, these are inherently author. The purpose of this workshop was to identify how hub-level
early-stage. stakeholders could best support priorities that had been established by
community members while also meeting opportunities identified dur-
ing the stakeholder consultation workshop. One agreement was for AAS
4. The Cases to set up a fish value chain initiative, which included a value chain
needs assessment study conducted from May–August 2013.
4.1. Case 1: improving post-harvest fish processing in Zambia4 During this same period, AAS set up a fish value chain working
group to guide analysis of the study results and inform next steps. This
As a landlocked country, Zambia obtains its fish products from working group had 30 members and included participants from the
inland waters including lakes, rivers, and fish ponds, which are traditional local authority (Barotse Royal Establishment), the
typically distant to major markets for fish. With lack of ice in fishing Government of Zambia, NGOs, market development organizations, fish
boats, long travel times to market under hot sun with no refrigeration, traders, and providers of services and inputs to the value chain.
and handling practices which expose fish to insects, rodents, and Following completion of the value chain needs assessment, a partici-
contamination, spoilage of fish is a common and costly problem in patory planning workshop was conducted in September 2013. This
workshop included members of the value chain working group as well
4
as community members from the fishing camps that had been surveyed
A previous version of this case was written by Conrad Muyaule and Catherine Longley
(WorldFish program staff), and published by Douthwaite et al. (2015) in the AAS Working
and from the 10 AAS focal villages. During the workshop, participants
Paper Research in Development: Learning from the CGIAR Research Program on Aquatic formed into three interest groups based on the top three priorities that
Agricultural Systems. emerged during the workshop: 1) fisheries co-management; 2) coop-

92
B. Douthwaite, E. Hoffecker Agricultural Systems 155 (2017) 88–102

eratives, associations and access to finance; and 3) postharvest proces- appropriate salt levels in the fish.
sing (Douthwaite et al., 2015, p. 64). These activities were carried out jointly by fishers, processor-
Following the workshop, the fish value chain working group traders, and researcher members of the group, under the supervision
members met and agreed to form themselves into an innovation of the Department of Fisheries members. In this process, the trainers
platform – that is to form and motivate a number of interest groups discovered that a small percentage of community members already
that would periodically share the results from their respective PAR knew how to salt fish through their interactions with Congolese buyers.
processes. The PAR group members invited them to join the training efforts and
AAS hired a value chain coordinator to set up the platform and to help teach others in the community. By April 2015, the salted fish PAR
convene regular joint reflection and planning meetings. The members group had grown to 42 members, including 22 members of fishing
of the three interest groups were then invited to submit proposals to communities and all group members knew how to salt fish and how to
AAS regarding how they wanted to pursue their interests as part of the sell it. Furthermore, the traders in the group guaranteed that they
innovation platform. In October 2013, the members of the postharvest would buy any fish that group members salted. AAS program staff
processing interest group submitted a proposal to AAS to work on fish anticipated that in mid-July 2015 the PAR group members would start
salting as a potential approach to reduce post-harvest loss. The group salting their fish; however, a particularly low fish catch led to a delay in
included processor-traders and trainers from the Department of implementing this plan. Members of the PAR group attributed the low
Fisheries who had some existing experience in processing and trading fish catch to overfishing in general, and failure to implement a fishing
salted fish, which was uncommon at the time in Zambia but produced ban during previous breeding seasons in particular (see Fig. 1).
in DR Congo and Angola where there is an established market. During a
proposal development workshop facilitated by AAS, the Principal 4.1.3. Outcomes and emerging impact pathways
Fisheries Officer in the Department of Fisheries provided training to Through an outcome evidencing process conducted between July
group members on drying and handling salted fish, and the group and November 2014, AAS program staff identified four emerging
decided to rename itself the Salted Fish Participatory Action Research impact pathways by which the program was contributing to change,
(PAR) Group. one of which was improved fisheries management. None of the path-
By this point, the group had grown to include 20 members from a ways had been specifically anticipated by program staff from the start;
diverse range of stakeholder groups. These included 12 fish processor- all had emerged from dynamics put in place through the RinD process.
traders, three representatives from the Department of Fisheries, a Through the processes of documenting, outcome evidencing, and
nutritionist from the Ministry of Agriculture, one staff member from writing up the salted-fish case, the following outcomes were identified
Caritas-Mongu (a local NGO and AAS partner), two representatives of as emerging from the work to improve post-harvest fish processing in
the Barotse Royal Establishment, and one representative from Nono Zambia.
Enterprise, a private cold storage company. The group was convened
and facilitated by the WorldFish AAS value chain coordinator and met 1. The identification of a locally-adapted fish processing method
quarterly to develop the approach to salting fish. The group's first steps
were to try different fish salting and drying methods. Based on these The development of a locally acceptable and replicable method of
initial experiments, they decided to recommend “one part salt to three processing fish was one of the objectives of the research process, and by
parts fish” and the use of a slanted drying rack and the removal of gills mid-2015 this objective had been accomplished. By engaging directly
as the preferred method. In March 2014, AAS convened the first fish with fishers, processors, traders, and members of the Department of
value chain innovation platform meeting and invited the salted fish Fisheries who had prior experience in fish processing methods, the
PAR group to present their work to date. salted fish PAR group was able to relatively quickly identify an
During subsequent discussions, and based on the suggestion of AAS alternative to the traditional methods of sun drying and smoking and
staff, PAR group members agreed to introduce fish salting into the AAS develop this method to the point that it could be used to produce dried
focal communities and work towards developing a market for the fish that would be desirable to local consumers. The development of a
product. However, it was noted that the safety of salt levels in the fish method of processing fish with salt contributes to an impact pathway of
had not yet been verified, the market for salt fish still needed to be reducing post-harvest losses in the fish value chain, one of the priority
identified and the profitability needed to be determined. As a first step, goals established by the project through consultation with stakeholders.
in July 2014, some fish traders from the group took the initiative to
display their salted fish at the Provincial agricultural show. 2. The development of a value chain for salted fish
Unexpectedly, all the fish was purchased and some customers subse-
quently went to the traders' association store to find more. Since salted In addition to developing a technical solution to the problem of
fish in Zambia is associated with Congolese and Angolan traders and post-harvest spoilage of fish, the salted PAR group assembled the basic
since this was the salted PAR group's first experience demonstrating components of a value chain for salted fish, including producers who
locally salted fish, they were surprised by the strong local demand for knew how to use the method, traders who were aware of the product
the product. and interested to buy it, and end-consumers who knew how to cook
Following this experience and receiving a training from AAS in with the salted fish and were willing to buy and consume this product.
participatory action research (PAR) cycle (plan, act, observe, reflect), This value chain development occurred alongside the research process
the salted fish group launched a PAR process involving four fishing and as a direct result of the engagement of various stakeholders, such as
communities in the AAS focal area. The group formulated a set of fishers, processors, traders, and consumers, in the PAR process. The
research questions, starting with an overarching question regarding the development of a local value chain for salted fish contributes to an
viability of salting fish as a means to deal with fish that would impact pathway of increased earnings for fishers and processors (as a
otherwise have to be sold very cheaply or go to waste. Within that, result of selling higher-quality fish) and improved food security for
the group specified more detailed questions covering four areas: the community members as more fish is able to make it to market
profitability of producing and selling salt fish; the optimum storage and undamaged.
transport conditions; the demand and supply of salt fish; and how well
the recommended fish salting and de-salting method would work. 3. The creation of a new multi-stakeholder platform capable of
Starting in October 2014, group members began conducting activities facilitating innovation processes
in the focal communities, including how to salt and de-salt fish, cooking
demonstrations on how to use salted fish, and taste testing to determine Another outcome area which formed part of the program's strategy

93
B. Douthwaite, E. Hoffecker Agricultural Systems 155 (2017) 88–102

Fig. 1. Timeline of key events (events in black, significance in grey).

was the successful creation of an innovation platform (multi-stake- importance of the fishing ban and their stronger working relationship
holder group) that was capable of engaging in and leading local with department staff, trader-processors were able to advocate for the
innovation processes related to the fish value chain. The fish value fishing ban to their peers in fishing communities and work towards
chain innovation platform supported several multi-stakeholder groups building a greater understanding at the village level regarding the
at the community level, including the salted fish PAR group. The importance of sustainable fisheries co-management. Despite being
platform provided a neutral and “safe space” within which stakeholders unanticipated, these outcomes was highlighted by program staff as
could build relationships and engage in joint work, directly enabling the most significant because it contributes to the outcome trajectory of
and contributing to the outcomes below. improved fisheries management identified by outcome evidencing as a
way the program is starting to address the ambitious hub development
4. The improvement of relationships among stakeholders in the challenge agreed by stakeholders at the beginning of the project.
aquatic system
4.2. Case 2: rehabilitating abaca in the Philippines5
An outcome of the research process which was not anticipated by
AAS program staff was an improvement in the relationships between The Philippines is the world's largest producer of abaca, a relative of
two groups of stakeholders in the aquatic system, namely the staff from the banana plant used primarily to produce cordage, pulp, fiber, and
the Department of Fisheries and trader-processors of fish. Prior to paper. Over 1.5 million Filipinos depend on the abaca industry for their
working together in the PAR group, trader-processors and fishers livelihoods (PhilFIDA, 2013), but in the 1990s an abaca bunchy top
viewed the department staff as “persecutors” due to their efforts to virus (ABTV) epidemic decimated production in many of the prime
enforce seasonal fishing bans during breeding season, a key component abaca-producing provinces. Some of the hardest-hit provinces were
of sustainable fisheries management. Through their close interactions within the AAS hub, located in the Visayas-Mindanao (VisMin) region, a
during the PAR process as well as innovation platform meetings, fisher- marine triangle in central Philippines. Southern Leyte, one of eight AAS
processors started to understand the importance of sustainable fishing focal areas within the hub, harvested just 954 metric tons of abaca in
practices and the relationship between these stakeholders improved 2013 compared to 8491 metric tons harvested in 2005 (PhilFIDA, 2013)
considerably, to the point that trader-processors in the PAR group and in 2014 was 12th out of 15 major abaca production areas, down
started to persuade their peers that the fisheries staff were “not the from 2nd before the infestation. At current prices, a ton of abaca is
enemy, but rather a user-friendly service providing guidance and worth US $1100, so this decreased production represented a major
education on how to conserve fisheries” (Douthwaite et al. 2015 p. income loss in a region where many farmers subsist on $1.50 daily
66). This growing mutual understanding led directly to the outcome income.
described below, which was also unanticipated by AAS staff but proved
to be one of the most important impact pathways emerging from the
project. 4.2.1. Challenges in the agricultural system
Support from the Philippines government for tackling ABTV during
5. Increased consensus around the need for sustainable fisheries co- the 2000s was directed to programs that focused on eradicating infected
management plants. However, these programs lacked community support because
farmers wanted to continue to grow abaca, not have it removed from
As a result of increased trust and improved working relationships their farms. In many communities in Southern Leyte, basic commu-
developed through their interactions in the PAR group and the nication between farmers and the government had broken down due to
innovation platform, trader-processors went together with staff from
the Department of Fisheries to fishing communities to explain the 5
A previous version of this case was written up by Lily Ann Lando and Maripaz Perez
importance of allowing time for the fish to breed and to publicize the (WorldFish program staff), and published by Douthwaite et al. (2015) in the AAS Working
reasons for the fishing ban. With their newfound understanding of the Paper Research in Development: Learning from the CGIAR Research Program on Aquatic
Agricultural Systems.

94
B. Douthwaite, E. Hoffecker Agricultural Systems 155 (2017) 88–102

a misunderstanding resulting from the fact that the local word for as one of the principles of AAS's RinD approach was to motivate
“medicine” was the same as “chemical,” which the technicians used to farmers' willingness to invest their own resources in their action plans.
describe herbicide. Farmers expected that their abaca plants would be Farmers' organizations in both barangays created a committee on abaca
treated with medicine and recover. Instead, technicians sprayed them and together with researchers developed an action plan for community-
with herbicide and killed them, along with all uninfected plants as well. based abaca rehabilitation.
This led to erosion in trust and poor implementation of replanting In May 2014, AAS staff monitoring the project found that none of
programs. Farmers were angry with technicians, and the technicians the farmers had acted on their action plans to eradicate infected plants
became afraid to go back into the communities. As a result, by 2013 and plant the hybrid varieties. Through speaking with the farmers, staff
many farmers in Southern Leyte were not practicing eradication learned that there were several reasons for this. Farmers had misunder-
voluntarily or regularly; nevertheless, they were still looking to the stood the eradication protocol; they thought that they had to kill
government to “do something” about the epidemic. everything on their plots, which included coconut and karlang (a local
Government institutions such as Philippine Council for Agriculture, variety of taro). Karlang was thought by the researchers to be an
Aquatic and Natural Resources Research and Development (PCAARRD), alternate host to the aphids that carry ABTV, but farmers were
Philippine Fiber Industry Development Authority (PhilFIDA), and unwilling to eradicate it as it was their main alternate cash crop to
National Abaca Research Center (NARC) had been working on abaca abaca. Furthermore, farmers were unhappy with the hybrid abaca
since the 1990s but were not working together or coordinating; instead, varieties provided and wanted seedlings of their traditional varieties, as
they saw each other as competitors for scarce funding and were jealous they felt that these provided better fiber quality as well as more fiber
of their mandates. For their part, university researchers were focused on than the hybrids.
their respective R & D agendas, and tended to view farmers as a source To address this impasse, researchers from AAS and VSU organized
of sample materials for disease management and for breeding work, focus group discussions with the farmers in May 2014. During these
using their fields for multilocation trials of varieties. Some research meetings, they assessed farmers' existing knowledge of ABTV through a
institutions held field days to show the farmers progress of their pre-test, clarified the eradication protocol, and agreed to investigate
research work, but not to obtain feedback on whether the research whether the aphids found on karlang were the specific vector for ABT. If
was relevant or useful to the farmers in the first place. not, then karlang would not have to be eradicated as part of the
Adding to the challenges, abaca farms in Southern Leyte were protocol. For their part, farmers started discussing the inclusion of the
mostly located in marginal areas not easily reached by extension agents neighboring barangays of Javier and Maria Plana in the abaca work.
and researchers, yet if ABTV eradication is not performed correctly and Since Javier is situated between Maac and Mahayahay, the farmers said
sustained, the virus can easily spread since abaca reproduces through that it should be included because any crop protection practices they
suckers, which carry the disease. All of the native abaca varieties implement will be useless if Javier plantations remain diseased. Also,
favored by farmers at the time were susceptible to ABTV and hybrid the Mahayahay farmers shared that most of them had their abaca
varieties were not yet available for general release. Finally, since the plantations in Maria Plana and so it would be logical to include Maria
aphid which spreads ABTV subsists on other crops commonly found in Plana in the program. They then took on the responsibility of talking to
the barangays (the local word for communities), rehabilitating abaca farmers in these two other barangays.
requires collective action from farmers; they must all agree to eradicate While the farmer's suggestion demonstrated an increased under-
crops that the aphids feed on in order to successfully control the spread standing of the epidemiology of ABTV, it also raised a new challenge:
of the virus. there was not enough tissue cultured planting material available for
farmers in all four communities, due to the laboratory process required
4.2.2. The research engagement process to produce it. After a series of conversations, researchers and farmers
AAS program engagement began in the Philippines in similar negotiated a seedling distribution system that could address this supply
manner to Zambia, based on tackling a hub development challenge bottleneck. A first tranche of farmers would receive 50 seedlings each,
identified during a scoping phase and agreed at a stakeholder consulta- with the agreement that they would repay the planting material in
tion workshop. During the first half of 2013, AAS carried out commu- 4–5 months when their seedlings produced suckers. Each mother plant
nity visioning, needs identification, and action planning in eight focal produces 3–6 suckers in that period, and each farmer agreed to repay
barangays within the VisMin hub region, selected on the basis of with two suckers, giving 100 suckers back which could then be given to
poverty and representativeness criteria. Two of the barangays in Sogod, two other farmers to plant, until all members of the abaca farmers
Southern Leyte, identified the rehabilitation of abaca as their main committee had received 50 seedlings each.
dream and priority; villages expressed the view that there would be no The engagement leading to this agreement proved to be a turning
more poor people if abaca was “given back to them” (Douthwaite et al. point in the relationship between researchers and farmers in this case.
2015, p. 59). In response, AAS commissioned the National Abaca Farmers started to ask the researchers about conducting research on
Research Center (NARC), part of the Visayas State University (VSU) their own questions related to abaca and whether they could adjust the
to conduct a rapid appraisal of the feasibility of abaca rehabilitation in experimental protocols. One farmer suggested that he wished to
this area (Tabada et al., 2013). The survey, completed in November conduct comparisons between his tissue cultured material and those
2013, found that the two barangays, Maac and Mahayahay, were losing growing naturally on his land which had been certified virus-free by
USD 2 million per year as a result of the fall in abaca production from NARC. Another farmer asked to change the research protocol by
pre-infestation levels of 1700 ha per year to just 250 ha per year in planting his abaca on the flat land closer to his house, which was
2013. This represented a major drop in income for the local economy, easier for him to access than the hills where it is usually grown. He
given that about 6 of every 10 people in the barangays were living offered that he could then compare the performance of his plants on the
below the poverty threshold (Tabada et al., 2013). flatland with his neighbor's plants on sloping land. AAS staff facilitated
The study found that it was possible to restore abaca in Sogod, but an agreement that both farmers and researchers would take actions
only with the strict implementation of eradication protocols, including based on each other's preferences and priorities, and agreed to meet
eradicating alternate hosts to ABTV, and the use of resistant varieties. quarterly. Some farmers decided to meet monthly as well, without AAS
Based on this finding, the twenty or so farmers who had participated in facilitation, to compare their data and share ideas, while VSU-NARC
the feasibility study in the two communities agreed to implement the hosted 10 farmers from each barangay to visit their abaca hybrid
recommended protocols and asked for planting material and financial research plots and attend a forum on abaca production technologies.
support from AAS to do so. AAS researchers agreed to provide planting During this time, AAS staff realized that it was important to form a
material in the form of tissue-cultured hybrid seedlings, but no money, multi-stakeholder coalition around the abaca work. Building on a

95
B. Douthwaite, E. Hoffecker Agricultural Systems 155 (2017) 88–102

Fig. 2. Timeline of key events (events in black, significance in grey).

previous but now defunct coalition called ADMART (Abaca Disease organized a formal launch. In keeping with local tradition, members
Management and Research Team), AAS convened an initial meeting organized a motorcade with a banner showing the logos of the member
with stakeholders from academia, research organizations, national and agencies and the tagline: Kauban ta sa Coalition Abaca (we are part of/
regional agencies in July 2014. These included VSU-NARC, Southern we support the Abaca Coalition). Community representatives from
Luzon State University, Sogod and Southern Leyte local government Mahayaha hired a van, the Maac farmers brought their motorcycles,
units, and regional line agencies including the Department of Science and the representatives from Javier and Maria Plana rode in the official
and Technology, Region 8 (DOST8), the Department of Agriculture agency vehicles. An additional 4000 seedlings were distributed to
Region 8 (DA8), and the Philippine Fiber Industry Development farmers at this event, including farmers from Javier and Maria Plana.
Authority (PhilFIDA). These groups were brought to the table through Later that month, PCAARRD delivered a check for the first tranche of
the convening power of the AAS country program leader, who had the budget for the Science and Technology Community-based farm in
strong personal connections through her previous position in the Sogod, amounting to over 2 million PhP (US$ 42,500). PhilFIDA also
Department of Science and Technology, and the director of NARC, made a commitment to provide 5000 additional tissue-cultured seed-
Dr. Gaspasin, who had taught most of the people in the room at some lings through June of 2015 and potentially another 5000 through
point. Stakeholders present at the meeting agreed that a coalition December, representing a contribution of PhP 250,000 (US$ 5300).
should be reconvened to enable the various agencies to work together. As of the end of February 2015, farmers were conducting farmer-led
Building on this agreement, AAS organized an abaca stakeholder field trials of tissue cultured native varieties, continuing to participate
consultation workshop in September 2014, with an even larger range of in the seedling distribution system, and developing a strategic commu-
stakeholders. Agencies presented their work on abaca to each other, and nication campaign for abaca rehabilitation, which included materials to
engaged in an exercise to describe future scenarios for the abaca industry popularize and spread the eradication protocol throughout the com-
in the Philippines. During these conversations, the agencies decided to munities. In addition, farmers agreed to serve as resource people in a
formalize a new coalition to replace ADMART. In a departure from the radio communications campaign which was planned to be organized by
norm, they decided to begin working together immediately using their university-based researchers. The next steps that were being planned by
current programs and budgets, rather than waiting to obtain a common AAS program staff at the time the program was closed included
new source of funding. PCAARRD agreed to include Sogod in its target bringing farmers and processors/end-users of abaca together to start
sites for abaca research and to set up a community-based science and working on the development of an inclusive value-chain for abaca (see
technology farm, while PhilFIDA agreed to collaborate with DOST 8 to Fig. 2).
channel the distribution of tissue-cultured planting material to Sogod to
support the seedling distribution scheme. 4.2.3. Outcomes and emerging impact pathways
Following this meeting, VSU-NARC and DOST8 contributed 4000 Outcomes related to this case were mapped, described, and verified
tissue-cultured seedlings and PhilFIDA provided another 1500 seedlings by AAS staff through an outcome evidencing process conducting
to the distribution program, which provided a first group of 71 farmers between March and October 2014 and resulting in the publication of
from Maac and Mahayahay barangays with the agreed-upon 50 an Outcome Evidencing Report for the VisMin Hub published by
seedlings each. Soon after, a second tranche of 51 farmers from Maac WorldFish (Paz-Ybarnegaray, 2014). Through an analysis of this report
and 5 farmers from Mahayahay were also able to receive 50 seedlings and the details of the case, we have identified five outcome areas below,
each. Two farmers from Javier and three from Maria Plana received 250 which contribute to two major impact pathways. The first impact
seedlings total and by October 2014 this initial group of farmers had pathway is around the successful rehabilitation of abaca, a major source
planted their disease-resistant abaca. By January, farmers in this first of income and livelihood for the region, and the second involves the
group reported that their plants had produced suckers and started strengthening of the capacity of local system stakeholders to take
sharing their data and results with AAS researchers. effective joint action towards the realization of common objectives and
On February 2, 2015, the new multi-stakeholder Abaca Coalition local development priorities.

96
B. Douthwaite, E. Hoffecker Agricultural Systems 155 (2017) 88–102

1. Farmers in four communities working to rehabilitate abaca with PhilFIDA in Southern Leyte, and PhilFIDA saw NARC as a
competitor. A representative of PhilFIDA recalled that “we were
Prior to the participatory research effort, farmers were not imple- isolated from the other groups, particularly NARC. We had no commu-
menting eradication protocols, yet nevertheless dreamed of restoring nication, no exchange of ideas, and we were not aware of their research
their abaca production. By early 2015, over 200 farmers from the four outputs” (Paz-Ybarnegaray, 2014, p. 5). The Department of Science and
barangays had implemented the eradication protocol, cleared their land Technology (DOST) was not working on abaca at all, and VSU-NARC
of infected plants, and received and planted virus-free tissue-cultured was not working in Sogod, although the town was only 2 h away by bus.
seedlings. Farmers were becoming more open to the use of hybrid Through their engagements in the abaca rehabilitation process,
seedlings and were demonstrating initiative in terms of persuading their stakeholders experienced a shift and improvement in the dynamics of
neighbors to join the eradication protocol, proposing their own research their relationships, from suspicion to greater mutual appreciation and
questions, leading field trials of native varieties, organizing their own from a competitive stance to a climate in which partners were
meetings to share results and contributing to communication cam- voluntarily contributing funds from their own budgets to support a
paigns. These outcomes resulted not only from farmers' increased common agenda. These strengthened relationships and improved
knowledge regarding eradication and rehabilitation protocols, but also linkages were identified by AAS program staff as a key factor
from their hands-on engagement in the research process, their en- contributing to the impact pathways of successful abaca rehabilitation,
hanced research skills, and –crucially–their newfound motivation to a key factor in the formation of local stakeholder groups capable of
engage in and enlist others in the rehabilitation work, kindled by a sustaining this work as well as progress towards other local develop-
growing realization that they possessed the ability collectively to ment objectives.
achieve the goals they had set out for themselves.
5. The creation of stakeholders' groups capable of mobilizing collective
2. Increased supply and access to disease-free seedlings action

One of the bottlenecks previously preventing farmers from engaging As of February 2015, three new stakeholder groups had been
in abaca rehabilitation was lack of access to disease-free plant material. created through the research process, each of which was contributing
Several factors came together to enable stakeholders to overcome this in significant ways to the abaca rehabilitation effort. The first two
bottleneck. Agencies that had previously not been working together groups were abaca committees established within the Farmers'
joined forces through the Abaca Coalition to contribute from their own Associations of Maac and Mahayahay barangays. The Maac Abaca
budgets and make tissue-cultured seedlings available to the commu- Committee contained 85 members, the Mahayahay Abaca Committee
nities. Farmers and researchers, through ongoing interaction, conversa- contained another 50 members, with a research specialist with the
tions, and negotiations, developed a creative distribution strategy, DOST 8 (which previously had not engaged in abaca work) leading the
which facilitated the rapid multiplication of the stock of disease-free coordination between groups. The third new group was the Abaca
plant material. This increased access to virus-free abaca enabled Coalition, which had grown to include representatives from 16 national
farmers in the initial two barangays to implement their action plans and local-level with an interest in abaca, including three universities
to rehabilitate their farms, while also making it possible for them to engaged in abaca research, national government agencies, regional line
enlist farms in the neighboring two barangays in the rehabilitation agencies, local government units, media, the local chamber of com-
effort, increasing the chances of sustained success for the eradication merce and industry representatives from the private sector.
and rehabilitation effort. This platform facilitated linkages between various stakeholders,
assisted in building visibility and community buy-in for the abaca
3. Farmers and researchers conducting joint research and development rehabilitation work, and—crucial to the success of the project—enabled
an agreement among stakeholders to provide access to the tissue-
While the relationship between farmers and researchers started with cultured plant material needed in order to implement the rehabilitation
misunderstandings, over the course of the project, researchers were plan. This contributed both to the impact pathway of abaca rehabilita-
able to build trust with farmer and farmers were able to influence the tion as well as a new and unanticipated area of impact emerging from
research process to align it better with their priorities. Initially, this case: namely, the capacity of system stakeholders to mobilize
researchers wanted farmers to eliminate karlang and farmers wanted existing resources and take effective joint action to achieve common
to rehabilitate abaca using their preferred native varieties, Inosa and development objectives.
Laylay, which were susceptible to ABTV. Through ongoing dialog, by
the end of the project, researchers and farmers had agreed to trial 5. Cross-Case findings
disease-free tissue cultured materials of both native and hybrid varieties
to test whether this reduces the chance of disease spread and how the In order to develop a middle-range ToC which can describe how the
fiber quality of the hybrids compares to that of native species. Farmers RinD approach worked in both cases, we must identify the common
were actively involved in the research process, proposing research outcomes to which the program contributed, the program inputs that
questions and serving as partners in the research process through were provided, and the causal mechanisms which were triggered and/
tracking their results and sharing them with other farmers and or harnessed by this program activity.
researchers. Engaging farmers as co-researchers built the capacity of
local system actors to innovate, both in terms of developing suitable 5.1. Outcomes
disease-free plant material and creating a distribution system.
In both cases, the AAS staff implemented the same programmatic
4. New and improved relationships between system stakeholders approach – the RinD approach. Table 2 summarizes the outcomes
identified in both cases as well as the common mechanisms which
In addition to overcoming technical challenges related to the correct contributed to bringing about these outcomes.
implementation of eradication protocols and the supply of disease-free In both cases, RinD led to motivation and agreement between
seedlings, the research process built new linkages between actors who stakeholder groups to work on a common issue. Joint technical work
were not previously working with each other, while improving relation- and capacity-building on this issue led to improvements in technical
ships between stakeholders who had a history of prior engagement. options to address the challenge and in the value chains needed to put
Before the launch of the Abaca Coalition, PCAARRD was not working these options into use. This work contributed to improving relation-

97
B. Douthwaite, E. Hoffecker Agricultural Systems 155 (2017) 88–102

Table 2
Comparing case outcomes revealing common mechanisms.

Case 1: improving post-harvest fish processing in Case 2: rehabilitating abaca in the Common mechanisms
Zambia Philippines

The development of a locally-adapted fish Farmers in four communities working to - Identification of an existing, commonly-experienced need triggers
processing method rehabilitate abaca engagement and action;
- Participatory research, joint technical work and capacity-building lead to
identification of suitable technical solutions
The development of a value chain for salted fish Increased supply and access to disease-free - Early, tangible results build motivation among stakeholders to engage
seedlings with (and stay engaged with) the process
- Engagement leads to collaboration among key actors who previously
were not working together
The creation of a new multi-stakeholder platform Farmers and researchers conducting joint - AAS uses convening power to bring key stakeholders together for initial
capable of facilitating innovation processes research and development meetings and participatory workshops
- Stakeholders realize areas of shared interest and propose ways to
continue working together.
The improvement of relationships among New and improved relationships between - Frequency and quality of engagement builds mutual understanding and
stakeholders in the aquatic system system stakeholders trust;
- Tangible results of collaboration reinforce the benefits of working
together.
Increased consensus around the need for sustainable The creation of stakeholders' groups - AAS' limited role (of process facilitation) leaves space for local actors to
fisheries co-management capable of mobilizing collective action take ownership of the process and generate next steps and new
initiatives;
- Progress on technical challenges inspires confidence to address
additional issues.

ships among system stakeholders, and these improved relationships in through a process designed to identify an area of common interest and
turn played a critical role in facilitating next steps in the work. As concern, shared by at least two different stakeholder groups in terms of
collaboration continued, stakeholders built social capital and a set of function (e.g. researchers and farmers). Identifying this area of shared
process skills that allowed them to take effective collective action and interest took several months but proved key to building trust and
pursue other mutually-beneficial local development objectives. motivation, given the work received little external funding beyond
process facilitation.
5.2. Inputs
3. Facilitation of engagement between existing stakeholders and
In addition to outcomes, another essential component of a theory of linkages to new stakeholders
change are the inputs provided by a program or intervention.
Consistent with the conceptual framework used in this paper, we take The facilitation activities provided by program staff included
as given that programs do not bring about change directly through their facilitation of workshops, after action reviews and other events which
inputs and key activities, but rather through the ways in which program brought key stakeholders together. The facilitation role played by staff
participants and stakeholders interpret and use what the program also included facilitation of community-level PAR processes. The
provides (Pawson, 2013). In order to construct a ToC to model these quality of facilitation in terms of the frequency of meetings and the
cases, we must therefore identify what was provided by AAS (through successful management of conflict was an important factor in main-
the RinD approach) which was used by participants and stakeholders in taining participants' motivation to remain engaged for sufficient periods
ways that contributed to achieving the outcomes described above. By of time to enable the identification of areas of mutual common interest
comparing the two cases as well as re-analyzing the synthesis chapter of among at least two key stakeholder groups.
the program report (Douthwaite et al., 2015, pp. 81–87) we can identify
six key input that were used by program participants and stakeholders 4. “Safe space” for stakeholders to build trust and develop working
in the process of achieving key outcomes. relationships

1. A process to engage stakeholders in developing a joint vision of Facilitation processes in which program staff played the role of an
success “honest broker” helped groups and platforms to become ‘safe spaces,’
where different stakeholders could tackle common problems and build
In both cases the program engaged with stakeholders to identify and stronger working relationships. Stakeholders and program participants
agree on a pressing development challenge facing a major aquatic were able to use the trust and social capital built in this way to explore
agricultural system in the system. Staff used this to identify a set of focal initially agreed-upon research questions and to identify new and more
communities in which it then worked with groups to identify their complex ones, e.g. moving from how to salt fish to how to co-manage
vision and steps to achieve it. Community priorities were communi- fish stocks more sustainably.
cated back to hub-level stakeholders in process to agree the main
initiatives upon which the program would work, e.g. improving the fish 5. Opportunities to “learn by doing” supported by coaching
value chain in Zambia and rehabilitating abaca in the Philippines. This
process took at least eight months in each hub but enabled the Participants found opportunities to learn new technical and soft
identification of attractors relevant to multiple groups of stakeholders. skills through the process of carrying out research together such as
farmers learning how to carry out field trials, and group members
2. A process to identify an issue of common interest learning to take on more responsibility. Coaching played an important
part of this “learning by doing” with researchers coaching farmers in
In each case, the initial motivation of stakeholders was kindled research methods and AAS facilitators coaching group leaders in the

98
B. Douthwaite, E. Hoffecker Agricultural Systems 155 (2017) 88–102

Fig. 3. Causal model (ToC) of how the RinD approach worked.

skills and principles required to carry out quality PAR. Complex solutions include the development of new ways of working
and new institutional arrangements (e.g. new groups and platforms).
6. Knowledge inputs with high relevance to local stakeholders There are a number of self-reinforcing loops shown in the model
which seek to depict dynamics of learning and adaptive change present
The program also provided the inputs normally expected of a in the cases. As groups of farmers and other local stakeholders built
research initiative, such as needs assessment surveys, feasibility studies, their capacities for research, technical R & D, and effective joint action,
information to answer key questions of local stakeholders and access to they began to identify technical solutions to the challenges facing them
new technology. New knowledge and technology was tested in the joint (box 11), for example finding a way of salting fish or establishing
field trials and subsequently adopted by farmers if successful. whether abaca varieties were disease resistant, as researchers claimed.
Researchers used survey and study data to pursue their research and Later, as the groups strengthened, they began to see they could tackle
publication plans. deeper and more complex challenges (12), for example tackling over-
fishing in Zambia or improving farmer – researcher – system stake-
5.3. A Theory of Change (ToC) to describe how RinD worked holder interaction in the Philippines.
These solutions produced benefit streams that motivated further
The common outcomes and inputs described above allow us to efforts to improve and find new solutions, which in turn continued to
develop a middle-range theory of change (ToC) to describe how the build the capacity of participants and the groups (11). This capacity,
RinD approach worked, shown in Fig. 3. Like most ToC, this is which can be understood as the capacity for the local system to
comprised of a chain of inputs, outputs, and outcomes which are innovate, included increases for both individuals and groups in:
connected by arrows to suggest causal connections, which can be tested
empirically through evaluations and research. This model derives its • New technical skills, e.g. how to carry out experiments and analyze
hypotheses regarding causal connections from evidence at the program the results;
level (gathered through the outcome evidencing process described • Self- and collective-efficacy;
earlier), and like most standard ToC, uses this evidence to create a • Ability to assess options and identify key system challenges;
simplified causal narrative that can graphically depict a working theory • Ability to go through iterative visioning, planning and reflective
regarding how a program creates impact. However, unlike standard learning cycles;
ToC, the model we present follows the format of causal loop diagrams • Capacity to link to other actors and to use linkages strategically in
(Team TIP, 2011) rather than the more familiar, linear “if/then” support of plans; and
formulation of conventional ToC. Causal loop diagrams allow a non- • Enhanced capacity for effective collective action.
linear depiction of causality (e.g. allows for self-reinforcing and
dampening loops) and can incorporate insights from complexity science Like all ToC, this one contains several assumed “contextual factors”
and systems dynamics referenced previously. that are key elements of the model. The first is that a development
The model (Fig. 3) depicts key initial program activities (boxes 1 challenge exists which is relevant to all stakeholders and around which
and 2), and the resulting initial outcomes of these activities (boxes 3 they can find common purpose. The second is that participants are
and 4) which lead to increases in the motivation of system stakeholders willing and able to work collectively in groups towards that common
to engage in the subsequent PAR process. The motivation of system purpose. In some highly-intervened areas, for example parts of
actors, combines with a set of inputs from AAS in the form of Bangladesh, farmers have become fatigued by a continual cycle of
facilitation (6), technical skill training (7), hands-on experience and projects wanting to facilitate groups and participatory approaches
exposure to a set of soft skills such improved leadership (8), links to (Conway and Mustelin, 2014). The third assumption, which is perhaps
other actors and the opportunities that this affords (9), and recognition the most important, is that actors facilitating the engagement process
of their work (10) enable the PAR process to function (box 5). This are able to do so in a way that is perceived as “neutral” and trustworthy
deeper and more intensive level of program activity leads to the by participants, and for a long enough period of time to allow groups to
generation and use of solutions to technical (11) and complex (12) find their own momentum and begin to drive the process themselves. In
challenges as increased capacity for local development (13) grows. the two cases presented in this paper, AAS staff highlighted that one of

99
B. Douthwaite, E. Hoffecker Agricultural Systems 155 (2017) 88–102

the ingredients of success was the willingness of staff to spend extended innovate contributes to better and/or faster rates of rural innovation
periods of time (up to a month) in the focal communities and maintain (3) thus increasing the benefits from adoption of research technologies
the engagement process through regular meetings, workshops, and (4) and from other innovation processes addressing other issues. A self-
reflection sessions. This provided the time required for groups to begin reinforcing loop exists in which increased rate of innovation leads back
to define their own work, fueled by mutual interest and a growing sense to the development of new ideas, knowledge and/or technology, that in
of collective efficacy. turn builds capacity to innovate, and so on. A dampening loop exists if
the adoption of new technology leads to the loss of diversity and/or
6. Discussion capacity (e.g. knowledge, skills, relationships) from which future
innovation trajectories could emerge, such as the loss of local plant
The ToC presented above differs from the conventional pipeline ToC varieties, local knowledge and expertise, or low external input farming
in agricultural research in important ways, including its starting practices.
assumptions, its component parts, and the story of causation linking This model shifts the focus of evaluation from questions related to a
them together. More importantly however, it also leads to a different set specific technical solution and the extent and impact of its adoption
of questions that evaluators, researchers, and decision-makers can ask (although these questions remain) towards questions related to the
when seeking to understand if a complexity-aware intervention is quality and effectiveness of the innovation process and the resulting
producing “impact.” In the dominant “pipeline” ToC, development system capacities that have been developed. Since this alternate route
impact is understood to be achieved through the sourcing, develop- to impact involves leaving stakeholders and program participants in the
ment, testing, adoption, and widespread dissemination of particular system better able to tackle both technical and complex challenges
innovations and technologies; the impact on people's lives is understood relevant to them, an evaluator would look for outcomes related to
to result primarily from the use of these “breakthrough development enhanced technical and adaptive capacities, as well as evidence of
innovations” which include devices, (e.g. solar lanterns); technologies increased capacity for local development. Depending on the context,
and platforms (e.g. mobile money) and new practices (e.g. conditional this might include questions related to the formation and functioning of
cash transfers) (USAID, 2015). Evaluations based on the pipeline TOC groups and platforms designed to address local challenges and the
put the emphasis on assessing first the adequacy (suitability, scalability, effectiveness with which they are doing so, the nature and results of
novelty, etc.) of the “solution” generated by the research process, experimentation taking place, organizational leadership and effective-
second and perhaps most importantly, the number of “beneficiaries” ness, and quality as well as extent of inter-system and intra-system
who have adopted the solution, and finally the livelihood or wellbeing linkages.
impacts to those beneficiaries of the new technology, innovation, or The evaluator may look for evidence of self-maintaining groups and
research output. entrepreneurs starting to drive local change processes, as well as
In contrast, the complexity-aware TOC we have developed shows adoption and spread of solutions to technical and complex issues
that impact is achieved through building the capacity of the rural developed by participating and spin-off groups and individuals. He or
innovation system to innovate, in part through the development of she will want to see the extent to which individuals and organizations
technical solutions, but also through the development and strengthen- are joining the groups and platforms and investing in their own
ing of key types of infrastructure and capacities (such as new platforms, resources in them. In terms of initiative outputs, the evaluator would
networks, skills, and ways of working). Hence, we can model agricul- assess their merit less in terms of how they are judged by research peers
tural research for development as achieving impact through two (although within the CGIAR this remains important), and more on how
interdependent impact pathways: the technology adoption pathway, they are being interpreted and used by intended and unintended users.
described above, and an empowerment (or capacity) pathway (see A complexity-aware ToC, therefore, can help a complexity-aware
Fig. 4). In this overarching causal model under which our middle-range program to be evaluated against the types of results it is intending and
ToC sits, carrying out research to tackle technical issues (1), as part of designed to achieve. This can make the difference as to whether a
the adoption pathway, is also an important way to build system program continues to be funded, or not. A complexity-aware ToC can
capacity to innovate, if carried out collaboratively (2) and thus is a also be useful for staff working within such programs, contributing to
contributor to the empowerment pathway. Increased capacity to developing program-level monitoring, evaluation, and learning frame-

Fig. 4. An overarching causal model showing agricultural research leads to impact through the action of two complementary impact pathways.

100
B. Douthwaite, E. Hoffecker Agricultural Systems 155 (2017) 88–102

works that more accurately reflect how programs work as well as the aware in its design. This ToC offers a starting point for testing a
types of outcomes and impacts they typically produce. Given increasing different set of hypotheses than those which are embedded within the
calls for “reflexive M & E” and embedded, real-time learning and common, linear impact adoption pathway. It also suggests a different
reflection within complex system-change initiatives (van Mierlo et al., direction and logic for how ToC might be constructed for complexity-
2010), complexity-aware ToC can offer frameworks to guide these aware programs moving forward. When impact pathways can be
efforts which may be more relevant and useful to program staff than modeled, their various causal connections and hypotheses can be
existing linear logic models. clearly stated and empirically tested, which can inform the develop-
Researchers interested in how agricultural research works, or not, to ment of models that more accurately describe reality. We therefore see
bring about change can also benefit from working with ToC that seek to the ToC presented in this paper not as a definitive causal model of how
more accurately model the dynamics of change in complex agricultural the two AAS projects that were profiled worked, but rather as an
innovation systems. In the same way that complexity-aware ToC lead to example of how such models can be constructed which can lead to
a different set of evaluation questions from conventional logical subsequent refinement, empirical testing, and improvement over time.
frameworks (logframes), they also generate a different and potentially Our hope is that this provides a pathway which stimulates the creation
fruitful set of both descriptive and causal research questions related to of better-fitting and more useful models that illustrate and commu-
understanding the dynamics of specific change processes involving nicate how complexity-aware interventions into AIS produce develop-
agricultural research, as well as deriving common patterns across them. ment impact for systems and their stakeholders.
Complexity-aware, middle-range theories of change transcend the
individual program level and therefore have the potential to apply to Acknowledgements
a wider set of interventions seeking to produce sustainable development
impact through stimulating local innovation and change processes. We would like to acknowledge and thank the team of researchers
As such, the model we develop in this paper responds to calls from and authors who produced the salted-fish (Conrad Muyaule and
within the recent literature to develop (and/or adapt) middle-range Catherine Longley) and abaca (Lily Ann Lando and Maripaz Perez)
theories to describe how interventions operating in contexts character- cases and who allowed us to use and further develop these cases for the
ized by complexity produce system change and impact (Pawson, 2013; purposes of creating the complexity-aware ToC. We are also very
Arkesteijn et al., 2015). Compared to program-level theories of change, grateful for the thoughtful and valuable input and contributions of
middle-range theories better describe contexts characterized by com- two anonymous reviewers, whose detailed suggestions have greatly
plexity because they acknowledge that outcomes result from a broader strengthened this paper.
range of interacting factors, including, but not limited to how stake-
holders engage with program interventions. Programs can—and often References
do—play a role in contributing to these outcomes and program staff,
organizational leaders, and funders have a practical need to be able to [AAS] CGIAR Research Program on Aquatic Agricultural Systems, 2011. CGIAR Research
model, test, and articulate the ways in which their efforts are Program Aquatic Agricultural Systems: Program Proposal. Retrieved on 1 August,
2016 from. http://pubs.iclarm.net/resource_centre/WF_2936.pdf.
contributing to (or designed to contribute towards) broader beneficial [AAS] CGIAR Research Program on Aquatic Agricultural Systems, 2012. Roll-Out
systems change. For this reason, we seek to encourage the development Handbook. CGIAR Research Program on Aquatic Agricultural Systems. Penang,
and formulation of ToC that can better reflect the realities of complex Malaysia. Accessed on 03/01/2017 from. http://pubs.iclarm.net/resource_centre/
WF_3146.pdf.
contexts and complexity-aware interventions, rather than suggesting [AAS] CGIAR Research Program on Aquatic Agricultural Systems, 2013. Improved
that in these contexts ToC should be abandoned altogether in favor of fisheries management in the Barotse Floodplain of Zambia - An urgent call for action.
adaptive, real-time evaluation and learning approaches such as Reflex- CGIAR Research Program on Aquatic Agricultural Systems. Penang, Malaysia.
Evaluation and Learning Series Paper: AAS-2013-24.
ive Monitoring in Action (Arkesteijn et al., 2015) and Episode Studies [PHILFIDA] Philippine Fiber Industry Development Authority, 2013. Abaca Factsheet.
(Carden, 2009). Retrieved on 17 April, 2015 from. http://fida.da.gov.ph/fiber%20profiles/2013/
abaca/abaca%20profile%202013.pdf.
[USAID] U.S. Agency for International Development, Global Development Lab, 2015.
7. Conclusions
2015 the Lab Year in Review: Accelerating Development Through Science,
Technology, Innovation and Partnerships. USAID, Washington.
In recent years, there has been a growing call within academia and Agropolis International, 2015. CGIAR consortium headquarters in Montpellier. Retrieved
evaluation practice for programming and monitoring and evaluation on 11 September from. www.agropolis.org/cooperation/headquarters-cgiar-
consortium.php.
that is both complexity-aware and reflexive, particularly within the Apgar, M., Douthwaite, B., 2013. Participatory action research in the CGIAR research
domain of agricultural innovation systems, which are increasingly program on aquatic agricultural systems. In: CGIAR Research Program on Aquatic
understood to possess the characteristics of complex, adaptive systems Agricultural Systems. Penang, Malaysia. Program Brief: AAS-2013-27.
Arkesteijn, M., van Mierlo, B., Leeuwis, C., 2015. The need for reflexive evaluation
(CAS). The necessity and usefulness of a complexity-aware evaluation approaches in development cooperation. Evaluation 21, 99–115.
approach has been effectively argued, and theoretical frameworks have Axelrod, R., Cohen, M.D., 2000. Harnessing Complexity: Organizational Implications of a
been developed to guide and stimulate the creation of evaluation Scientific Frontier. Basic Books.
Ayele, S., Duncan, A., Larbi, A., Khanh, T.T., 2012. Enhancing innovation in livestock
approaches that take complexity into account. However, this field of value chains through networks: lessons from fodder innovation case studies in
work is still nascent and has remained mostly at the theoretical and developing countries. Sci. Public Policy 39, 333–346.
conceptual level, laying necessary foundations for the development of Béné, C., 2011. CAADP and fisheries policy in Africa: are we aiming for the right reform?
In: Policy Brief 40. University of Sussex, Brighton, UK: Future Agricultures
practice-oriented frameworks and tools to guide implementers and
Consortium.
evaluators in adopting a complexity-aware approach in their program Biggs, S.D., 1990. A multiple source of innovation model of agricultural research and
design and evaluation efforts. While some individual projects and technology promotion. World Dev. 18 (11), 1481–1499.
Britt, H., Patsalides, M., 2013. Complexity-aware monitoring. In: Discussion Note.
programs have begun to develop their own internal complexity-aware
Monitoring and Evaluation Series December USAID, Washington, DC.
evaluation processes, there are few, if any, examples in the literature or Campbell, B.M., Sayer, J. (Eds.), 2003. Integrated Natural Resource Management: Linking
the public domain of evaluation frameworks (e.g. ToC, logframes) that Productivity, the Environment and Development. CABI, Wallingford, UK.
integrate the insights from the emerging field of complexity-aware Carden, F., 2009. Understanding influence: the episode studies approach. In: Tussie, D.
(Ed.), The Politics of Trade: the Role of Research in Trade Policy and Negotiation.
evaluation into their design. 2009. International Development Research Centre, pp. 273–298.
This is the gap in both the literature and practice that this paper CGIAR, 2016. CGIAR strategy and results framework 2016–2030. Retrieved on 1 August,
begins to address, by extending existing theoretical and conceptual 2016 from. https://goo.gl/XgLfI2.
CGIAR Consortium, 2015. Draft 2016 CGIAR financial plan. Retrieved on 31 July, 2016
work to develop an example of what a complexity-aware ToC might from. http://cgiarweb.s3.amazonaws.com/wp-content/uploads/2015/09/Draft-
look like for a particular intervention which was explicitly complexity-

101
B. Douthwaite, E. Hoffecker Agricultural Systems 155 (2017) 88–102

2016-CGIAR-Financial-Plan.pdf. Enhance Innovation and Use. Guilford Press, New York.

Chisonga, N., Maravanyika, T., Paz-Ybarnegaray, R., 2014. Outcome evidencing report Pawson, R., 2013. The Science of Evaluation: A Realist Manifesto. Sage, Thousand Oaks,
2014: substantiating the case of the AAS program and approach – Barotse Hub, California.
Western Province, Zambia. Retrieved on 1 August, 2016 from. https://goo.gl/ Pawson, R., Tilley, N., 1997. Realistic evaluation. Sage, London.
zg71Dy. Paz-Ybarnegaray, R., 2014. Outcome evidencing report 2014: VisMin hub, Philippines.
Conway, D., Mustelin, J., 2014. Strategies for improving adaptation practice in Unpublished AAS program report retrieved on 1 August, 2016 from. https://goo.gl/
developing countries. Nat. Clim. Chang. 4 (5), 339–342. SULYs1.
Dalrymple, D.G., 2008. International agricultural research as a global public good: Paz-Ybarnegaray, R., Douthwaite, B., 2016. Outcome evidencing: a method for enabling
Concepts, the CGIAR experience, and policy issues. J. Int. Dev. 20, 347–379. and evaluating program interventions in complex systems. Am. J. Eval. (November
Douthwaite, B., Gummert, M., 2010. Learning selection revisited: How can agricultural 2016), 1–19. http://dx.doi.org/10.1177/1098214016676573.
researchers make a difference? Agric. Syst. 103 (5), 245–255. Pretty, J.N., Chambers, R., 1993. Towards a Learning Paradigm: New Professionalism and
Douthwaite, B., 2016. Beyond the pipeline model: New paths for agricultural research to Institutions for Agriculture. Discussion Paper-Institute of Development Studies,
enhance capacity to innovate. Welt-Sichten. Dossier 7-2016, 9–10. University of Sussex (United Kingdom).
Douthwaite, B., Apgar, M., Schwarz, A., McDougall, C., Attwood, S., Senaratna Probst, K., Hagmann, J., with contributions from Fernandez M, Ashby JA, 2003.
Sellamuttu, S., Clayton, T., 2015. Research in development: learning from the CGIAR Understanding participatory research in the context of natural resource
research program on aquatic agricultural systems. In: Penang: CGIAR Research management—paradigms, approaches and typologies. Agricultural
Program on Aquatic and Agricultural Systems. Working Paper. AAS-2015-16. Research & Extension Network Network Paper, (130).
Douthwaite, B., Kuby, T., van de Fliert, E., Schulz, S., 2003. Impact pathway evaluation: Reason, P., Bradbury, H. (Eds.), 2001. Handbook of Action Research: Participative
an approach for achieving and attributing impact in complex systems. Agric. Syst. 78 Inquiry and Practice. Sage.
(2), 243–265. Renkow, M., Byerlee, D., 2010. The impacts of CGIAR research: A review of recent
Dugan, P., Apgar, M., Douthwaite, B., 2013. CGIAR reserarch program on aquatic evidence. Food Policy 35, 391–402.
agricultural systems. Penang, Malaysia. In: Working Paper, Retrieved on 3 April Rogers, P.J., 2008. Using programme theory to evaluate complicated and complex aspects
2017 from. http://aquaticcommons.org/11247/1/AAS-RIND-Approach.pdf. of interventions. Evaluation 14 (1), 29–48.
Ekboir, J., 2003. Why impact analysis should not be used for research evaluation and Schut, M., van Paassen, A., Leeuwis, C., Klerkx, L., 2014. Towards dynamic research
what the alternatives are. Agric. Syst. 78, 166–184. configurations: A framework for reflection on the contribution of research to policy
Gaunand, A., Hocdé, A., Lemarié, S., Matt, M., Turckheim, E.d., 2015. How does public and innovation processes. Sci. Public Policy 41, 207–218.
agricultural research impact society? A characterization of various patterns. Res. Schut, M., Klerkx, L., Sartas, M., Lamers, D., Campbell, M.M., Ogbonna, I., Kaushik, P.,
Policy 44, 849–861. Atta-Krah, K., Leeuwis, C., 2016. Innovation platforms: experiences with their
Hall, A., Rasheed Sulaiman, V., Clark, N., Yoganand, B., 2003. From measuring impact to institutional embedding in agricultural research for development. Exp. Agric. 52,
learning institutional lessons: an innovation systems perspective on improving the 537–561.
management of international agricultural research. Agric. Syst. 78, 213–241. Scriven, M., 1976. Maximizing the power of causal investigations: the modus operandi
Hellin, J., Bellon, M.R., Badstue, L., Dixon, J., La Rovere, R., 2008. Increasing the impacts method. In: Evaluation Studies Review Annual. 1. pp. 101–118.
of participatory research. Exp. Agric. 44, 81–95. Senge, P., 1990. The fifth discipline: The art and science of the learning organization.
Hounkonnou, D., Brouwers, J., van Huis, A., Jiggins, J., Kossou, D., Roling, N., Sakyi- Currency Doubleday, New York.
Dawson, O., Traore, M., 2016. Triggering Regime Change: A Comparative Analysis of Snowden, D., 2010. Naturalizing sensemaking. In: Mosier, K.L., Fischer, U.M. (Eds.),
the Performance of Innovation Platforms that Attempted to Change the Institutional Informed by Knowledge: Expert Performance in Complex Situations. Taylor and
Context for nine Agricultural Domains in West Africa. Agric. Syst.. http://dx.doi.org/ Francis, New York, pp. 223–234.
10.1016/j.agsy.2016.08.009. Spielman, D.J., Ekboir, J., Davis, K., 2009. The art and science of innovation systems
ISPC, 2014. Draft ISPC Comments on the revised extension proposal of the CRP Aquatic inquiry: applications to Sub-Saharan African agriculture. Technol. Soc. 31 (4),
Agricultural systems (AAS) for 2015–2016. Retrieved on 31 July, 2016 from. https:// 399–405.
goo.gl/KaBX6A. Stame, N., 2004. Theory-based evaluation and types of complexity. Evaluation 10 (1),
Joly, P.-B., Gaunand, A., Colinet, L., Larédo, P., Lemarié, S., Matt, M., 2015. ASIRPA: A 58–76.
comprehensive theory-based approach to assessing the societal impacts of a research Sterk, B., Kobina, A.C., Gogan, A.C., Sakyi-Dawson, O., Kossou, D., 2013. Five years after:
organization. Res. Eval. 24, 440–453. the impact of a participatory technology development programme as perceived by
Klerkx, L., Aarts, N., Leeuwis, C., 2010. Adaptive management in agricultural innovation smallholders in Benin and Ghana. Int. J. Agric. Ext. 19 (4), 361–379.
systems: the interactions between innovation networks and their environment. Agric. Sumberg, J., Okali, C., Reece, D., 2003. Agricultural research in the face of diversity, local
Syst. 103, 390–400. knowledge and the participation imperative: theoretical considerations. Agric. Syst.
Klerkx, L., Van Mierlo, B., Leeuwis, C., 2012. Evolution of systems approaches to 76, 739–753.
agricultural innovation: concepts, analysis and interventions. In: Danhefer, I., Tabada, M., Abamo, A.P., Ratilla, M.C., Moreno, L.O., Gapasin, R.M., 2013. Rapid
Gibbon, D., Dedieu, B. (Eds.), Farming Systems Research Into the 21st Century: The appraisal on the viability of rehabilitating abaca farming in selected barangays of
New Dynamic. Springer, Netherlands, pp. 457–483. Sogod, Southern Leyte. In: Project Terminal Report. WorldFish, Penang, Malaysia.
Kraaijvanger, R., Veldkamp, T., Almekinders, C., 2016. Considering change: evaluating Team Tip, 2011. Guidelines for drawing causal loop diagrams. In: Systems Thinker.
four years of participatory experimentation with farmers in Tigray (Ethiopia) 22(1).
highlighting both functional and human-socio aspects. Agric. Syst. 147, 38–50. Ton, G., Vellema, S., Ge, L., 2014. The triviality of measuring ultimate outcomes:
Kristjanson, P., Reid, R.S., Dickson, N., Clark, W.C., Romney, D., Puskur, R., MacMillan, Acknowledging the span of direct influence. IDS Bull. 45, 37–48.
S., Grace, D., 2009. Linking international agricultural research knowledge with action Uphoff, N., Wijayaratna, C.M., 2000. Demonstrated benefits from social capital: the
for sustainable development. Proc. Natl. Acad. Sci. 9, 5047–5052. productivity of farmer organizations in Gal Oya, Sri Lanka. World Dev. 28 (11),
Leonard-Barton, D., 1995. Wellspring of Knowledge. Harvard Business School Press, 1875–1890.
Boston, MA. Weiss, C.H., 1995. Nothing as practical as good theory: Exploring theory-based evaluation
Longley, C., Kruijssen, F., Armstrong, J., Phiri, N., Chisule, G., Mundia, N., Amukena, M., for comprehensive community initiatives for children and families. New approaches
Nasilele, F., Jahan, N., Pellegrini, C., Größbrink, J., Muyaule, C., Rajaratnam, S., to evaluating community initiatives: Concepts, methods, and contexts. 1. pp. 65–92.
Madzudzo, E., Chilala, A., 2016. Fish value chain analysis and development in the Weiss, C.H., 1997. How can theory-based evaluation make greater headway? Eval. Rev.
Barotse floodplain of western Zambia: towards a research in development approach. 21 (4), 501–524.
In: Penang, Malaysia: CGIAR Research Program on Aquatic Agricultural Systems. Westhorp, Gill, 2014. Realist Impact Evaluation: An Introduction. Methods Lab, Overseas
Working Paper: AAS-2016-04. Development Institute, London.
Mayne, J., 2015. Useful theory of change models. Canadian J. Program Eval. 2, 119–142. Wigboldus, S., Klerkx, L., Leeuwis, C., Schut, M., Muilerman, S., Jochemsen, H., 2016.
van Mierlo, B., Arkesteijn, M., Leeuwis, C., 2010. Enhancing the reflexivity of system Systemic perspectives on scaling agricultural innovations. A review. Agron. Sustain.
innovation projects with system analyses. Am. J. Eval. 31 (2), 143–161. Dev. 36, 1–20.
Miller, J.H., Page, S.E., 2007. Complex adaptive systems: An introduction to Williams, B.K., 2011. Adaptive management of natural resources—framework and issues.
computational models of social life. Vol. 41 Princeton University Press, J. Environ. Manag. 92 (5), 1346–1353.
Princeton, NJ. Yin, R.K., 1989. Case Study Research: Design and Methods, revised ed. Applied Social
Nederlof, E.S., Röling, N., Van Huis, A., 2007. Pathway for agricultural science impact in Research Methods Series Sage, New York.
West-Africa: lessons from the converge of sciences programme. Int. J. Agric. Sustain.
5, 247–264.
Ojha, H.R., Hall, A., 2013. Adaptive collaborative approaches in natural resource Further reading
governance: Rethinking participation, learning and innovation. Routledge, Chicago. Douthwaite, B., Apgar, M., Crissman, C., 2014. Monitoring and evaluations strategy Brief.
Orlikowski, W., Hoffman, D., 1997. An improvisational model for change management: In: CGIAR Research Program on Aquatic Agricultural Systems. Penang, Malaysia.
the case of groupware technologies. In: Inventing the Organizations of the 21st Program Brief: AAS-2014-04.
Century. MIT, Boston, MA, pp. 265–282.
Patton, M.Q., 2011. Developmental Evaluation: Applying Complexity Concepts to

102