Synthesis

Public Participation in Scientific Research: a Framework for Deliberate Design

• Abstract
• Introduction
• Models of Participation
  • Background
  • The context and construct of participation
  • Five project models
• Common Framework for Deliberate PPSR Design
  • Inputs
  • Activities
  • Outputs
  • Outcomes
  • Impacts
• Relating Inputs to Outcomes of Different Models
• Considerations for Using the Models and Framework
• Conclusion
• Acknowledgments
• Literature Cited

INTRODUCTION

Members of the public are increasingly participating in scientific research and monitoring. Consider the following cases: (a) thousands of birdwatchers across North America collect data that are combined to reveal trends in bird distributions and behaviors, such as advancing first egg dates for nesting tree swallows (Tachycineta bicolor) (Dunn and Winkler 1999); (b) residents of Pennsylvania monitor water turbidity, conductivity, and macroinvertebrate populations in tributaries near active natural gas wells to document impacts of gas extraction (Zerbe and Wilderman 2010); and (c) experienced hunters and anglers living near protected areas in the Philippines monitor and react to changes in resource use related to wildlife populations (Danielsen et al. 2007). In each of these cases, lay people interact with scientists to participate in a scientific research effort. Here, in the context of ecological monitoring and research, we explore how certain outcomes may be associated with different approaches to “public participation in scientific research” (hereafter, PPSR).

PPSR efforts have emerged from a variety of social and academic fields, ranging from participatory action research in the fields of development studies (Chambers 1994) and public health (Cashman et al. 2008) to citizen science projects with a long history of ornithology and astronomy research (Droege 2007, Bonney 2008, Raddick et al. 2009) to water quality monitoring (Firehock and West 1995, Ely 2002, Wilderman 2005) and community-based natural resource management (Guijt 2007, Fernandez-Gimenez et al. 2008, Wilmsen et al. 2008b). As collaborative endeavors between science researchers and public participants—including but not limited to amateur experts, concerned community members, scientists trained in other fields, and/or school students—PPSR projects must address the needs and interests of all parties. In this paper, we focus on projects in the contexts of conservation, ecology, and environmental management, where efforts also deal with complex questions and issues regarding how people relate to their environments (Campbell and Vainio-Mattila 2003).

PPSR projects in environmental contexts have successfully addressed complex issues in science and society. Some have collected and mobilized monitoring information to respond to pollution (Overdevest and Mayer 2008), whereas others have improved communication within and across resource management stakeholder groups (Tudor and Dvornich 2001, Lawrence 2006). Projects have increased political participation and social networking around water resource issues (Overdevest et al. 2004) and compiled large data sets to inform landscape management practices for bird conservation (Rosenberg et al. 1999, 2003). To have an impact on conservation, PPSR projects generally strive for outcomes that fall into one or more of three main categories: outcomes for research (e.g., scientific findings); outcomes for individual participants (e.g., acquiring new skills or knowledge); and/or outcomes for social–ecological systems (e.g., influencing policies, building community capacity for decision making, taking conservation action).

Such a combination of outcomes has the potential to affect robust and integrated resource management decisions (Pahl-Wostl et al. 2008, Armitage et al. 2009). However, individual PPSR projects do not always consider or acknowledge all three categories of outcomes, which can diminish the ability of a project to address complex problems. Additionally, project activities do not always align well with intended outcomes (Nerbonne and Nelson 2008), and some achieve unanticipated outcomes (Cornwall 2008). Even attentively designed projects require compromises regarding outcomes, such as balancing large-scale data collection against opportunities for close interaction between researchers and community members (Berkes 2004, Evans et al. 2005), or deciding between timeliness and precision in data collection (Whitelaw et al. 2003). Additionally, with information on PPSR outcomes both limited and dispersed across fields, little in the way of empirically based guidance has been available to inform strategic decisions about aligning goals, outcomes, and trade-offs in the design and refinement of projects.

This paper outlines how PPSR project design relates to project outcomes, drawing on work from varied fields of practice. We bring together previously conducted, convergent case studies and synthetic work in conservation management, informal science education, community-based forestry, and volunteer monitoring to describe three predominant programmatic models and their potential outcomes. Our new alignment of models across traditions yields two conclusions, supported by case analyses: (1) the degree to which the public participates in the research process, as well as the quality of that participation, are closely related to the range and types of outcomes achieved; and (2) a common framework can inform project design choices across fields of practice. Therefore, we propose such a framework, based on the quality of participation and the management of interests addressed through a project; present examples to support application of the framework and models across contexts; and explore ways that the framework can be used by project designers in any disciplinary field to deliberately align PPSR project design with specific desired outcomes.

MODELS OF PARTICIPATION

Background

Recognizing the burgeoning of citizen science, in 2008, the National Science Foundation’s Center for the Advancement of Informal Science Education (CAISE) sponsored an “Inquiry Group” to help define the field and understand the broad educational impacts of various citizen science models (Bonney et al. 2009a). The group convened practitioners and researchers from diverse fields (all of whom are co-authors on this paper), including volunteer water quality monitoring (Wilderman), participatory action research and community-based forestry (Ballard), science/museum education and public engagement (McCallie), and citizen science related to both ornithology (Bonney, Phillips, and Shirk) and behavioral ecology (Jordan). The group quickly found that confusion over existing terminologies complicated effective communication about different projects.

“Uncertain and contradictory nomenclature” is how Rowe and Frewer (2005) described a similar concern in the broader context of public engagement in science (the context from which this CAISE inquiry emerged). The term “citizen science,” for example, is employed in the United States and in the fields of ecology and conservation primarily to describe large-scale data-collection initiatives (Bonney et al. 2009b). In European contexts and in social studies of science, the same term describes a philosophy of engaging public perspectives and knowledges in science discourse and policy making (Irwin 1995). Also problematic is that many initiatives sharing similar programmatic elements employ different terms, such as volunteer biological monitoring (Lawrence 2006); community science (Carr 2004, Wilderman et al. 2004a); community-based monitoring (Danielsen et al. 2009); and participatory monitoring (Bell et al. 2008), all of which we argue can be considered public participation in scientific research.

To acknowledge the tradition of distinct terms on the one hand and the need to share understandings across fields on the other, the CAISE inquiry team proposed the term “Public Participation in Scientific Research” to collectively describe a range of diverse projects. We define PPSR as intentional collaborations in which members of the public engage in the process of research to generate new science-based knowledge. Depending on context, projects may be driven by such goals as public education or the management of social–ecological systems, but in all cases PPSR projects aim explicitly to contribute to scientific research and/or monitoring. PPSR encompasses hypothesis-driven science, such as citizen science investigations into how weather and urbanization constrain the distributions of wintering bird populations (Zuckerberg et al. 2011), as well as projects that employ local knowledge and observational data to address political and social goals for underrepresented communities, such as participatory mapping exercises that aim to bolster local claims of authority over forest territory (Peluso 2005). The use of the term PPSR as an overarching category allows us to explore similarities and differences that are programmatic, as opposed to differences that are primarily nominal or historical. Specifically, we consider different ways in which opportunities are structured for public participation, and how those opportunities relate to the outcomes that projects achieve.

The Context and Construct of Participation

Over the past few decades, academic discussions in the broad contexts of public engagement in science policy, discourse, and research have taken a “participatory turn” (Jasanoff 2003). Much of the theoretical debate regarding participation comes from the fields of development studies (Whyte 1991, Fishkin 2009) and political science (Fischer 2000). In contexts of natural resource monitoring for management, public participation can be a means of engaging diverse stakeholders and accessing new knowledge, making power relationships transparent, adapting activities to evolving conditions, and encouraging both ownership and accountability of the management process among constituents (Kapoor 2001, Armitage et al. 2007, Arora-Jonsson et al. 2008, Wilmsen 2008, Wulfhorst et al. 2008). Such approaches often emphasize generating “knowledge for action” as opposed to just “knowledge for understanding” (Cornwall and Jewkes 1995).

PPSR and other participatory projects can and have achieved some of these potentials. However, simply invoking the language and ideals of participation is insufficient. The term “participation” is used to describe a wide spectrum of approaches for engaging individuals and communities, with each approach often tied to different intentions and outcomes. A recent summary of participation theory by Cornwall (2008) laid out numerous continua of participation in development studies, revealing important distinctions between participation for the sake of garnering “buy-in” and participation that enables social transformation. Although there is now some degree of consensus in development studies regarding appropriate participation strategies (Chambers 2002), Campbell and Vainio-Mattila (2003) and others raise concerns that these hard-won lessons are not being transferred to what we are calling PPSR work. Scholars in both development and PPSR fields call for moving beyond what has become a “rhetoric of participation” (Cooke and Kothari 2001, Cornwall 2008) to identify what Rowe and Frewer (2004) call “effectiveness,” the features of an intervention that enable intended outcomes (see also Lawrence 2010a).

Such a move demands the careful, intentional, and transparent employment of participation strategies to achieve targeted outcomes, as well as to help reveal relationships between the way that participatory opportunities are designed and structured and the specific outcomes of resulting initiatives (Cooke and Kothari 2001, Cornwall 2008). Scholars tend to focus on two key facets of participation: degree and quality. In order to inform and support deliberate project design for specific outcomes, whether those outcomes are for individuals, science, or social–ecological systems, it is necessary to identify relationships between both degree and quality of participation and the types of outcomes they influence when handled in different ways.

Degree of participation

Degree of participation is a dimension that can be quantified, compared, and/or standardized. By comparing projects that demonstrate different degrees of participation, we can account for and examine the relationships between participation and various outcomes. Degree of participation can be measured in terms of duration of involvement (Ballard et al. 2008); research effort (Dickinson et al. 2010), numbers (Wilmsen and Krishnaswamy 2008) and/or diversity (Cheng et al. 2008) of participants; the depth/intensity of involvement in the process (Wilmsen and Krishnaswamy 2008); or the power that participants have over the processes in which they engage. Relative degrees of power have in fact been the focal point of landmark typologies of participation in development studies (e.g., Arnstein 1969, Pretty 1995, White 1996), as participation in development and resource management contexts can hinge on power issues and bring about complex political relationships (Charvolin et al. 2007). These typologies, however, conflate power as a degree of participation—how much or how little a given individual/group “may” have—with evaluative statements about how much power a group “should” have (Cornwall 2008). Furthermore, Lawrence (2006) suggests that the normative assumptions of these typologies (e.g., that more power is transformative and less power is exploitative) do not necessarily reflect individuals’ experiences in voluntary biological monitoring contexts. In fact, individuals and communities should not be assumed to have an interest in—or be advantaged by—a greater degree of control over a given research process or agenda (Saldivar-Tanaka and Krasny 2004, Cornwall 2008).

For our purposes of relating participation to outcomes of PPSR, we define the degree of participation as the extent to which individuals are involved in the process of scientific research: from asking a research question through analyzing data and disseminating results. We focus on the “process” of scientific research for several reasons. First, as PPSR projects inherently aim to produce knowledge through science, the research process is a common element across all projects. Additionally, the degree of public participation in the research process varies across projects in quantifiable ways—which we illustrate through the models described in this paper—and there appear to be relationships between the degree of participation in the research process and project outcomes. Across the range of ways that degrees of participation are considered in different contexts, scholars agree in a general sense that opportunities for increased degrees of participation can open doors to a wider range of potential outcomes, assuming that the quality of participation is handled thoughtfully (Hickey and Mohan 2004, Wulfhorst et al. 2008).

Quality of participation

Discussions of degree of participation often do not capture important subjective and context-relevant dimensions, such as credibility and trust (Wynne 1992, Wulfhorst et al. 2008), fairness (Rowe and Frewer 2005, Cheng et al. 2008), responsiveness (Gaventa 2004), relevance (Cumming et al. 2008), agency (Cleaver 2004), and due diligence in the development of appropriate research strategies (Cheng et al. 2008). We consider these to be key components of high-quality participation. Many of these dimensions are related to building and negotiating relationships among constituents (Cheng et al. 2008, Wilmsen et al. 2008a). For our purposes of understanding the role of public participation in project design, we use “quality of participation” to describe the extent to which a project’s goals and activities align with, respond to, and are relevant to the needs and interests of public participants. This focus on the public is not at the exclusion of the interests of science researchers, but rather reflects work in development studies (e.g., Arnstein 1969, Wilmsen et al. 2008a) to elevate the needs and interests of public participants in contexts where those interests have historically been marginalized. High-quality participation in the design of a project can be found in projects supporting any degree of participation in the research process, so long as the degree of participation adequately reflects the needs and interests of the public.

By paying explicit attention to the social and interactional dimensions that affect the quality of participation, organizers can directly affect the outcomes of a PPSR project. Sustainable and robust outcomes such as environmental management may be most effectively achieved by attentiveness to the issues of whose interests are being served (Kapoor 2001) and how the balance of those interests is negotiated in designing a project and defining desired outcomes (Bell et al. 2008, Wilmsen et al. 2008a). Attentiveness to the quality of participation can also help yield outcomes, such as social learning, that could be considered as critical for retaining participants and affecting conservation (Fernandez-Gimenez et al. 2008, Tabara and Pahl-Wostl 2007, Pahl-Wostl et al. 2008). Luks (1999) and other scholars of post-normal science even suggest that high-quality relationships between scientists and the public can enhance resulting scientific research. The design framework offered in this paper portrays the relationships between scientific and public interests, and between that balance of interests and likely project outcomes, offering a tool for explicitly considering the quality of participation.

We address the degree and the quality of participation as separate but related elements in examining the relationship between participation and outcomes by presenting, respectively, models of PPSR (based on “degree” of participation) and a framework for project development (considering the “quality” of participation). By so doing, we aim to advance thinking about participation in specific, strategic ways in order to inform project design that deliberately considers the outcomes that different degrees and qualities of participation can achieve.

Five Project Models

Our work to create models of PPSR grew out of a need to explore relationships between project design and project outcomes across the many fields of practice in which these activities take place, as well as across the different approaches to project design employed within a given research field. To construct the models presented here, the CAISE team built on earlier typologies of broad approaches to public engagement in science (e.g., Cornwall and Jewkes 1995 (citing Biggs 1989), Rowe and Frewer 2005) as well as of PPSR activities more specifically (e.g., Wilderman et al. 2004a), all of which converge on the degree of participation in the research process as an indicator of outcomes. Several other concurrent explorations of PPSR outcomes, across varied fields of practice and research, have similarly considered the degree of individuals’ participation in the research process to be closely related to outcomes (Lawrence 2006, Cooper et al. 2007, Wilderman 2007, Fernandez-Gimenez et al. 2008, Danielsen et al. 2009). The models presented here acknowledge the convergence of thinking by scholars working in different fields of practice and research. Therefore, we look at PPSR projects across fields of practice to explore and elaborate specifically on the different degrees to which the public participates in the process of scientific research.

We divide PPSR projects into five models based on degree of participation:

• Contractual projects, where communities ask professional researchers to conduct a specific scientific investigation and report on the results;
• Contributory projects, which are generally designed by scientists and for which members of the public primarily contribute data;
• Collaborative projects, which are generally designed by scientists and for which members of the public contribute data but also help to refine project design, analyze data, and/or disseminate findings;
• Co-Created projects, which are designed by scientists and members of the public working together and for which at least some of the public participants are actively involved in most or all aspects of the research process; and
• Collegial contributions, where non-credentialed individuals conduct research independently with varying degrees of expected recognition by institutionalized science and/or professionals.

Table 1 briefly describes the interactions between public participants and scientists in each model, and Table 2 illustrates, for contributory, collaborative, and co-created models, the aspects of the scientific process in which public participants are involved.

The contractual and collegial models lie at the far boundaries of the PPSR spectrum. In the contractual model, which is exemplified by European Science Shops (Jorgensen et al. 2004, Leydesdorff and Ward 2005), the public participates by raising a question of concern, often a question that researchers would otherwise not consider. This model allows an expansion of traditional science research from being driven solely by the interests of researchers (or the needs of the field) to consider community-relevant questions and interests. As opportunities for public participation are limited throughout the remainder of the research process in this model, however, it can arguably reinforce the traditionally distinct roles of scientists as producers of knowledge and the public as consumers, albeit in this case consumers with enhanced control over the research agenda and the resulting knowledge produced.

At the other end of the spectrum is the collegial model, as exemplified by amateur astronomers, archaeologists, and taxonomists, who often work on their own to make important contributions to science (Stebbins 1980, Hopkins and Freckleton 2002). In this model, professional and amateur researchers may collaborate only when an amateur writes and submits findings for peer review and publication. Although often overlooked or highly critiqued, committed amateurs can make critical contributions that may not otherwise transpire owing to a lack of resources, time, skills, or inclinations in the professional scientific community. As such, their work demands a reconsideration of expertise as exclusive to traditionally credentialed scientists (Taylor 1995, Ellis and Waterton 2005). In these cases, the degree of amateur participation in the research process is so extensive and independent that expert amateurs arguably adopt the traditional role of scientist-as-knowledge-producer.

The other three models, which capture a range of public participation in scientific research, align closely with categories recently or concurrently defined by other scholars (Table 3). Although Wilderman et al. (2004b), Lawrence (2006), Danielsen et al. (2009), and others use different terms to label their models, they differentiate models similarly by degrees of practice. Thus, we suggest that meaningful programmatic differences exist not between fields of practice or research, but between project models based on the degree of participation, regardless of the field of practice. Likewise, we suggest that the range of models is very similar across the different fields of practice and research from which PPSR initiatives have emerged and that these similarities are grounds upon which analytic comparisons can be made regarding degree of participation and its relationship to outcomes. For these reasons, we focus the remainder of this paper on the center three models, while acknowledging that programmatic innovation often occurs at boundaries.

COMMON FRAMEWORK FOR DELIBERATE PPSR DESIGN

At the heart of the design process is the quality of participation. The design and implementation of every project requires decisions to be made about whose interests can and should be addressed, and how the end goals, or desired outcomes, are defined. Resulting choices in project design reflect how those interests are considered and negotiated. In some PPSR fields of practice, design choices are guided by theories of participation, expertise, or democracy. In other traditions, project design is guided primarily by a growing body of practical knowledge, along with implicit assumptions about participation or expertise.

We present an overarching PPSR framework to help project developers—whether community members, researchers, or teams involving each—think deliberately about design choices (Fig. 1). This framework, based on the W. K. Kellogg Foundation’s (2004) format for outcome-oriented logic models, suggests that negotiations and interactions between scientific interests and public interests can influence a range of potential outcomes. Although the three models (contributory, collaborative, and co-created) can be used to explore implications for projects that employ different degrees of participation, the fundamental question the framework asks is, “whose interests are being served?”

Below, we describe considerations for each of the framework elements (inputs, activities, outputs, outcomes, and impacts), and discuss implications of the ways in which different elements may be treated. Each element, considered alone, represents complex processes worthy of future investigation. For the purposes of this paper, we introduce each with just enough depth to illustrate the framework as a whole. To help demonstrate how different models of participation fit within this framework, we provide brief case examples from a contributory project (Project NestWatch) and a co-created project (Shermans Creek Conservation Association) (each of which is described in more depth in Bonney et al. 2009a).

Inputs

PPSR projects are, by design, collaborative endeavors, and thus project design must manage inputs from multiple constituents. We consider Inputs to be the interests (the hopes, desires, goals, and expectations) of both the public and the scientific community as they come together to determine the focus of a project. Although other interests inevitably come into play (e.g., those of funders, management agencies, political entities), we focus here specifically on interests of professional researchers and public participants as the common elements across all PPSR collaborations.

Public volunteers’ interests can include contributing to scientific knowledge (Evans et al. 2005, Raddick et al. 2010), making scientific discoveries (Raddick et al. 2010), collecting and disseminating information on environmental hazards (Overdevest and Mayer 2008), affecting resource stewardship (Wilderman et al. 2004a), protecting livelihoods (Danielsen et al. 2007), or satisfying personal identities and/or learning goals (Weston et al. 2003, McCallie et al. 2009). And, although it is easy to assume that individual scientists are interested primarily in achieving scientific results, some may be just as interested in affecting education (Firehock and West 1995), conservation (Swaisgood and Sheppard 2010), managing their own observational data (Wood et al. 2011), or any of the interests attributed to public volunteers. Interests are also not necessarily homogenous within a group of researchers or a community. Furthermore, the lines between individuals who are “scientists” and those who are of “the public” may be blurred in many cases (Ellis and Waterton 2004).

Nonetheless, the ways that interests are envisioned, articulated, acknowledged, and balanced can be fundamental to the subsequent design steps and, therefore, are likely to influence the outcomes of a project. These interests may in fact be used to define project goals to strive for particular outcomes. We have structured the Inputs category to reflect the interplay of interests between professional science researchers and members of the public considered in the development or enhancement of a research project. Each initiative differently balances these interests (which take into account the motivations, skills, experiences, and available resources of these two groups as well) to identify the focus of the scientific work, which may be a research question, an issue addressed through data collection, or a monitoring protocol.

Cases

Many projects that aim to produce data on a large geographic or temporal scale are contributory in nature, due in part to the necessary spatial distance among participants and project leaders. Often designed almost exclusively by professional scientists, contributory projects address public interests and abilities in part to ensure meaningful participation and data accuracy. For example, projects at the Cornell Lab of Ornithology such as NestWatch take into account the willingness of volunteers to repeatedly monitor bird nests, collect breeding data, and submit their nest records to a central online database, where records can be accessed by scientists and used to detect changes in reproductive timing and fledging success (Phillips and Dickinson 2009). Co-created projects, based extensively on volunteer initiative, may incorporate scientific expertise mainly to ensure that projects are conducted in a scientifically rigorous manner. For example, when residents in Pennsylvania’s Shermans Creek watershed wanted to set up a long-term water quality monitoring initiative for the purpose of targeting critical areas for restoration and protection, they enlisted technical input from researchers at the Alliance for Aquatic Resource Monitoring (ALLARM) to help them design and implement a scientifically sound monitoring program (Wilderman 2005).

Activities

The category of Activities includes the bulk of the work that is necessary to design, establish, and manage all aspects of a project. This work is generally conducted by a lead team, which may include scientists, members of the public, and/or others (educators, technologists, etc.). Importantly, the tasks involved in project design and management differ from the steps of the scientific research process articulated in Table 2, although some tasks, such as protocol development, do overlap. Activities in this context include the tasks necessary for developing project infrastructure, such as designing sampling strategies and protocols, training materials, and data submission/data entry technologies, as well as establishing a network of volunteers and the communication and support mechanisms necessary to maintain their participation. Activities here also include tasks for managing project implementation, such as facilitating training, distributing materials, holding meetings and events, and communicating with all collaborators/participants. Although the focus of this article is not on how to conduct PPSR projects, resources for guiding many of these activities are compiled at http://www.citizenscience.org.

Establishing an infrastructure for data collection and management determines the type and quality of data collected as well as the utility of those data for affecting outcomes (Vaughan et al. 2003, Dickinson et al. 2010). The way that activities are handled will likely reflect how interests have been balanced at the input stage, as the interests represented will influence choices regarding what to measure, how often measurements are taken, and who has control over the resulting data. Engaging scientist partners in these activities can enhance the credibility of data collected (Lathrop and Markowitz 1995, Penrose and Call 1995, Nerbonne and Vondracek 2003). Deep involvement of public participants and communities in these activities can enhance both scientific and local relevance as well as local utility (if not actual use) of findings (Wilderman et al. 2004a, Corburn 2007, Cheng et al. 2008, Nerbonne and Nelson 2008), although actual use of data may require additional design considerations (Nerbonne and Nelson 2008).

Cases

For Project NestWatch, the protocols, data sheets, and a data management infrastructure were iteratively developed by a team of researchers, educators, and technologists, to align with skills of different audiences (Phillips and Dickinson 2009). NestWatch training and support materials (such as a monthly newsletter and video tutorials) are distributed online and by email, and a small staff is available to answer questions by phone. In contrast, members of Shermans Creek Conservation Association (SCCA), after successfully defeating a plan to site a power plant along Shermans Creek, realized that they were going to need baseline data on the condition of the stream in order to participate meaningfully in decisions regarding future development in the watershed. They called upon ALLARM for technical support, and together developed a study design that included the types of data to collect and analyze to provide critical information about the health status of the creek and its tributaries. Project management, including overseeing who was responsible for monitoring and when, was all handled by SCCA membership (Wilderman et al. 2004a).

Outputs

Outputs are the initial products or results of activities. Outputs of PPSR collaborations include observations, recorded as data, and the active experiences of making, facilitating, and/or analyzing those observations or measurements. Outputs are often quantified, for example, in terms of the number of observations in a database, or the numbers of individuals, website visits, volunteer hours, workshops, and trainings (Phillips et al. 2012). Differences in project outputs often hinge on how and why data are gathered, how they are used, and the meaning they are given, as well as the depth and meaning of the lived experience (Lawrence 2010b). Outputs will reflect choices (in the activity phase) regarding such things as trade-offs between the depth and precision of data collected and the need for timely responses to environmental conditions (Vaughan et al. 2003), as well as what observations are considered important by different parties (Long Martello 2004, Lawrence 2010a).

Choices of what data are collected, and how those data are made available and usable for different constituents, also heavily influence outcomes, including publications, education, and decision making. Data analysis, workshops for visualizing data, dissemination via community meetings or publications, influence on policy makers, and personal reflection on these experiences all affect the transition from tangible data and experiences to project outcomes. The priority and the resources given to particular interests at the input stage influence the type of observations and experiences that a given collaboration yields and the way any resulting data are used.

Cases

Project NestWatch is designed to gather data on nesting birds across wide geographic and temporal scales in order to understand environmental influences on breeding behavior. This demands the acquisition of an optimal number of nesting records (and thus participants) geographically distributed across species’ ranges. In collecting these records, participants experience interactions with birds, their local environment, and the process of making and documenting observations for biologically relevant research. Members of SCCA collected 3 years of water quality monitoring data and then asked ALLARM staff to facilitate a data analysis workshop. The usefulness of this analysis depended not only on the data, which revealed problems in the watershed, but also on the experiences of volunteers working in that watershed to help pinpoint the likely causes of those problems. SCCA members leveraged their subsequent experience in data analysis and interpretation to achieve a number of management outcomes.

Outcomes

Outcomes are measureable elements, such as skills, abilities, and knowledge that result from the specific outputs of a project. Remembering that we are focusing on PPSR in the context of conservation and ecology, we address outcomes of PPSR projects in three categories: those for science, those for individual participants, and those for social–ecological systems.

First, we consider outcomes for science (for a comprehensive treatment of outcomes and associated practices in ecological research, see Dickinson et al. 2010). As just a few examples, PPSR projects have advanced scientific understandings about: trends in species ranges, distributions, abundances, and diversity (e.g., Root et al. 1981, Batalden et al. 2007, Crimmins et al. 2008, Senko et al. 2010); the spread of disease (e.g., Hochachka et al. 2004, Lindsey et al. 2009) and of invasive species (e.g., Cooper et al. 2007, Simpson et al. 2009, Bonter et al. 2010); changes in life-cycle events (e.g., Torti and Dunn 2005, Wolfe et al. 2005), as well as implications of such changes for aspects of human health (e.g., van Vliet et al. 2002, Bigham et al. 2009). Projects have also yielded innovative and enhanced techniques for collecting, analyzing, managing, and networking data (e.g., Baker and Oeschger 2009, Crall et al. 2010, Fink et al. 2010). As one metric of scientific success, Dickinson et al. (2010) estimate that over 1000 peer-reviewed publications and technical reports have been produced using data from just eight large-scale projects. At the crux of outcomes for science is the ability of PPSR to access otherwise unavailable knowledge, whether by compiling large-scale data networks (e.g., Sullivan et al. 2009) or depending on very localized insights (e.g., Berkes et al. 2000). Given this range, it is important to note that the types of scientific outcomes that a project can achieve can depend upon assumptions of project designers about what counts as knowledge and whose knowledge and observations are relevant (Ellis and Waterton 2005, Nerbonne and Nelson 2008, Lawrence 2010b).

Outcomes described for individual participants include development of new skill sets (Bell et al. 2008, Ballard and Belsky 2010), an increased understanding of the process of scientific research (Trumbull et al. 2000, Ballard and Belsky 2010), an improved sense of place and/or stewardship (Wilderman et al. 2004a, Evans et al. 2005), and opportunities to deepen relationships with the natural world (Bell et al. 2008) as well as with other people (Overdevest et al. 2004, Bell et al. 2008, Kountoupes and Oberhauser 2008). Some individuals gain new content knowledge (e.g., Brossard et al. 2005, Evans et al. 2005) or increase their scientific literacy (Trumbull et al. 2000, Jordan et al. 2011). Others gain a sense of ownership of their own knowledge and expertise as it relates to their contributions to science (Bell et al. 2008, Lawrence 2009) and to their surroundings and social contexts (Ross et al. 2008). Professional scientists also experience outcomes as individuals; for example, one study documented that resource agency personnel gained an enhanced understanding of local conditions and an appreciation for the knowledge and skills of undocumented salal harvesters (Ballard and Belsky 2010). Other work suggests that engaging in research partnerships can provide professional scientists welcome relief from their desk jobs (Noss 2001–2002) and even foster a sense of hope in the sometimes-bleak profession of conservation (Swaisgood and Sheppard 2010). Stoking such hopes are outcomes for participants such as enhanced self-efficacy and community capacity, social capital, and agency—in short, the skills and social resources to put knowledge into action (e.g., Overdevest et al. 2004, Jones et al. 2006, Ballard and Belsky 2010).

Outcomes identified for social–ecological systems include improved relationships between communities and management agencies (Tudor and Dvornich 2001, Ballard et al. 2008), backyard enhancement of wildlife habitat (Evans et al. 2005), access to and use of data to address environmental degradation (Overdevest and Mayer 2008), and increased likelihood of participant engagement in policy processes to improve their surroundings (Overdevest et al. 2004, Wilderman et al. 2004a). Resource management strategies can be improved, whether through research findings (e.g., Pattengill-Semmens and Semmens 2003, Rosenberg et al. 2003, Hamel et al. 2009, Campbell and Godfrey 2010), responsiveness to stakeholder knowledge and values (e.g., Bird et al. 2003, Cheng et al. 2008), strategically targeted interventions (Danielsen et al. 2010), monitoring for adaptive management on both public and private lands (e.g., Cooper et al. 2007, Fernandez-Gimenez et al. 2008), or rapid detection of and direct response to environmental problems (e.g., Simpson et al. 2009). Many community-based monitoring and environmental justice projects may in fact start with social–ecological outcomes in mind before deciding that a PPSR approach would be effective to these ends. In adaptive co-management contexts, environmental monitoring in collaboration with stakeholders provides information as feedback on management practices, which can be used to adapt the practices (Armitage et al. 2009). Both enhanced adaptive management practices, and the social learning process that is embedded in collaborative and participatory monitoring, can contribute to more resilient social–ecological systems (Berkes 2009, Walker and Salt 2006). As many of these outcomes hinge on deep collaboration and relationships, this category may be most influenced by the quality of participation at the input stage.

Not all projects yield outcomes in all categories, regardless of goals, and some projects will achieve unanticipated outcomes. Success in achieving one category of outcome may influence outcomes in other categories (e.g., science outcomes improve as participants improve their bird identification skills; management of social–ecological systems may change with new science outcomes and increased public knowledge). We also suggest that as projects evolve, outcomes affect the handling of subsequent inputs. For example, achieving science outcomes likely reinforces science interests. However, sustainable projects likely depend on achieving outcomes in all three categories. In well-designed projects, inputs can be understood as goals, and outcomes should reflect those inputs. Attentive projects can modify their design as interests change or new interests are revealed.

Cases

Successful projects may be weighted toward one outcome category and still be able to achieve outcomes in the other two groups. For example, although Project NestWatch is driven by scientific interests to increase understanding of the factors that limit breeding success, it also emphasizes individual learning outcomes with implications for social–ecological systems, such as increased understanding of breeding biology, increased engagement with the scientific community, improved nest monitoring skills, increased appreciation for the natural world, and increased bird-friendly practices (Phillips and Dickinson 2009). And, whereas SCCA research was originally driven by an interest in collecting scientific data to assess the state of Shermans Creek, efforts resulted in improved relationships between formerly disparate community groups, participation in writing a state-funded rivers conservation plan, a strong educational outreach effort to all municipal officials, emergence of new leadership within the organization, increased scope of activities for the organization, and scientific knowledge skills sufficient to revisit and revise the study design for continuing monitoring efforts (Wilderman 2005).

Impacts

Compared with outcomes, impacts are long-term and sustained changes that support improved human well-being or conservation of natural resources. Whereas short-term outcomes are typically measured within 1–3 years of project implementation and long-term outcomes in 4–6 years, noticeable impacts may only occur only 10 years or more after projects have been established (W.K. Kellogg Foundation 2004). Given this lengthy time scale, impacts are rarely measured. Nonetheless, conservation programs can benefit from distinguishing impacts from outcomes to address the interests of stakeholders operating on different time scales, such as land managers and funding agencies (Bottrill et al. 2011).

Desired impacts may include sustained stewardship and conservation (Penrose and Call 1995, Pattengill-Semmens and Semmens 2003), a knowledgeable and empowered citizenry (Middleton 2001), resilient human and natural communities (Fernandez-Gimenez et al. 2008), and responsive science (Penrose and Call 1995). Because of the integrated nature of such impacts, they may best be achieved through combined successful outcomes for research, individual participants, and social–ecological systems (Ballard and Belsky 2010). Because impacts are difficult to measure and confirm, in the remainder of this paper, we focus our attentions on project outcomes.

RELATING INPUTS TO OUTCOMES OF DIFFERENT MODELS

To gauge the usefulness of the framework and models for guiding project design, we turn to several recent syntheses and comparative analyses of case studies. In Table 4, we follow Danielsen et al. (2009) and document—for contributory, collaborative, and co-created projects—the outcomes described through empirical syntheses and case studies. We also consider available information on the costs and benefits of different approaches. From these cases, we can see that outcomes do tend to relate to the degree to which members of the public are engaged in the research process. Similarly, projects do demonstrate outcomes that align with predominant inputs.

These cases suggest that each model has strengths and limitations in terms of expected outcomes. In general, contributory projects are associated with robust scientific research outcomes and content knowledge gains, whereas co-created projects have demonstrated success in affecting timely policy decisions and enhanced resource management capacity of communities (Wilderman and Shirk 2010). These cases also reveal trade-offs regarding the resources and capacity needed to achieve outcomes of interest. For example, although co-created projects are driven and organized to a large degree by communities, they may actually involve as much if not more input, resources, and commitment by scientists than would a contributory project.

We assert, however, that the particular outcomes documented in these synthesis papers are likely more attributable to design choices regarding the quality of participation (whose interests are being served), than they are to the degree of participation. Concluding their programmatic synthesis, Fernandez-Gimenez et al. (2008) note that, “... clear objectives and design, rather than the type or phase of community participation, seem to determine the level of ecological learning,” and we suggest that this is likely true across all three categories of outcomes. For example, developers of a project such as the co-created SCCA water quality monitoring initiative that prioritizes community interests in timely, locally relevant, actionable data may not be concerned with designing their study to yield the kind of precise and generalizable data that are often important for peer-reviewed publication in scientific journals. It is important to distinguish this as a result related to inputs, rather than to the degree of participation itself; involving participants more deeply in the research process does not inherently result in data that are less scientifically interesting or useful. Likewise, engaging thousands of people across large geographic scales does not itself preclude the development of community building efforts or the applicability of data to locally relevant concerns; although building community for a large project may be challenging and certainly requires innovation, it can conceivably be done.

Projects must, therefore, reflect carefully on, and design deliberately for, the interests that sustain participation and yield the full range of desired outcomes for both science and the public in each specific programmatic context. It is worth considering what advantages could be gained by applying lessons from one model to another, particularly with regard to deeper participation by the public. Using the framework and models in tandem, project leaders can choose to enhance outcomes beyond what a particular model might be expected to yield, through (1) consideration of the challenges and opportunities of different models (e.g., Danielsen et al. 2010) and (2) enhanced attentiveness to the interests of the public in participation opportunities and desired outcomes (e.g., Fernandez-Gimenez et al. 2008).

CONSIDERATIONS FOR USING THE MODELS AND FRAMEWORK

Deliberate project design—that is, thoughtfully employing a design strategy that will yield specific and measurable project outcomes—requires project designers to begin with the end in mind. The complexity of conservation-oriented PPSR projects stems from the need to keep multiple ends in mind, considering that achieving significant gains for conservation may likely depend upon affecting related outcomes for science, individual participants, and social–ecological systems. Deliberate project design benefits from understanding the range of interests that need to be addressed, to inform clearly articulated goals (Nerbonne and Nelson 2008, Powell and Colin 2008, Alliance for Aquatic Resource Monitoring 2010).

There are often tensions between interests (Nerbonne and Nelson 2008), and design requires certain compromises among and between project developers and other constituents. As Bradbury and Reason (2008) suggest, “The degree of participation ... must be negotiated among co-researchers in every participatory research project.” Although certain degrees of participation may efficiently achieve particular outcomes (e.g., contributory projects generally result in large-scale data sets), projects should consider both whether a given degree of participation is sufficient to achieve desired outcomes, and if it is within the capacity of all partners to participate or facilitate. New projects have the opportunity to consider whether a hybridized model might address a broader range of outcomes. Likewise, as ongoing projects reflect on their accomplishments and opportunities to address goals in new ways, they may strategically add new, complementary participation activities that more deliberately address specific outcomes.

Bradbury and Reason (2008) also suggest that, “...the quality of participation must be evaluated on an ongoing basis.” Not all interests may be known or recognized at the outset of a project, and new interests may arise as a project evolves. Given that projects need not be locked in to a certain model, project designers and managers who are attentive to changing, or newly revealed, interests can strategically adapt participation approaches.

Projects can also facilitate different degrees of participation by different individuals. In fact, it is likely that individual participants create their own unique experiences, regardless of a project’s predominant model of participation (Lawrence 2006). In co-created projects, it is not uncommon for a core group of individuals to be deeply involved in the entire process of research while others participate in discrete activities such as data collection or analysis (e.g., Farquhar and Wing 2008). Some contributory projects also intentionally facilitate opportunities for individuals or groups to conduct their own research investigations (Tomasek 2006).

No matter how individualized a participant’s experience may be, the social and interactional aspects and outcomes of PPSR participation should not be underestimated. In fact, unintended social outcomes may come to be seen as essential precursors to achieving goals, because increased opportunities for social interactions may sometimes be necessary to sustain or deepen project participation, build relationships for sharing knowledge (e.g., Cohen 2010), and even enhance resulting science or management actions (e.g., Plummer et al. 2007). If designing for the fullest range of potential outcomes, the social aspects of participation should be considered for all partners.

In general, typologies of participation and project design are best considered tools for understanding trends, as practice inevitably “blurs boundaries” (Cornwall 2008). Additionally, every PPSR initiative arises in a unique context, in response to different needs, meaning prescribed approaches are unreasonable (Wiggins and Crowston 2010). In fact, practitioners and theorists in development fields suggest that generalized participation methodologies can result in dogmatic practice, diverting attention away from quality participation, the essential element of building the foundations for trust, credibility, and reciprocity, and other factors critical for achieving desired outcomes (Wilmsen and Krishnaswamy 2008). For this reason, we see the participation models and associated work offered here as descriptive starting points that highlight relationships between inputs and outcomes. We encourage project designers to use these guidelines creatively to address needs specific to their context, and to reflect and report on the results in order to inform the growth of this field.

CONCLUSION

Across fields of research and practice, collaborations involving public participation in scientific research share the common element of explicitly engaging the public in the research process to produce science-based knowledge. Although scientific research is at the heart of these initiatives, we see the PPSR movement as much more than just the gathering of data for science or management. It is precisely the inherent mix of likely outcomes (for science, for individual participants, and for social–ecological systems) that makes PPSR a powerful concept, particularly in fields of conservation and natural resource management where actions must respond to integrated social–ecological needs with diverse understandings and knowledges. Given that any one PPSR project will invariably have some effect on outcomes across all three categories, there is all the more reason to design deliberately so that activities align with, and therefore affect, intended outcomes for sustainability, resilience, and conservation.

The process of studying and understanding the best ways to develop, implement, and evaluate PPSR is just beginning, bridging a number of different social and academic traditions from which these initiatives have emerged. Given the convergence of findings thus far, there are advantages to continued conversations and investigations that span these different fields of research and practice. Research about PPSR is also being conducted in fields not explicitly discussed here; for example, in public health research (e.g., Cashman et al. 2008, Minkler and Wallerstein 2008), astronomy (e.g., Raddick et al. 2009), traditional ecological knowledge (e.g., Berkes 2004), mediated model building (van den Belt 2004, Cockerill et al. 2007), and information sciences (e.g., Wiggins and Crowston 2011). Further collaborative work can help us all broaden and refine definitions and, more importantly, practice. We believe that the field of PPSR will grow in new and compelling directions if project developers and PPSR scholars (of whom there are a growing number) begin a critical analysis of program design using the presented framework as a guide, learning from history in certain fields of practice and innovation in others.

LITERATURE CITED

Alliance for Aquatic Resource Monitoring (ALLARM). 2010. Study design manual. [online] URL: http://www.dickinson.edu/uploadedFiles/about/sustainability/allarm/content/Study%20Design%20Manual(1).pdf

Armitage, D.R., F. Berkes, and N. Doubleday. 2007. Adaptive co-management: collaboration, learning and multi-level governance. University of British Columbia Press, Vancouver, British Columbia.

Armitage, D. R., R. Plummer, F. Berkes, R. I. Arthur, A. T. Charles, I. J., Davidson-Hunt, A. P. Diduck, N. C. Doubleday, D. S. Johnson, M. Marschke, P. McConney, E. W. Pinkerton, and E. K. Wollenberg. 2009. Adaptive co-management for social–ecological complexity. Frontiers in Ecology and the Environment 7(2):95–102. http://dx.doi.org/10.1890/070089

Arnstein, S. R. 1969. A ladder of citizen participation. Journal of the American Institute of Planners 35(4):216–224. http://dx.doi.org/10.1080/01944366908977225

Arora-Jonsson, S., H. L. Ballard, R. Buruchara, J. Casolo, L. Classen, J. DeHose, M. Emretsson, L. Fortmann, A. Lundgren Halvarsson, E. Halvarsson, S. Humphrise, J. W. Long, M. W. Murphree, N. Nemarundwe, A. Olssen, S. Rhee, A. Ryen, C. Wilmsen, and E. Wollenberg. 2008. Conclusions. Pages 245–265 in L. Fortmann, editor. Participatory research in conservation and rural livelihoods: doing science together. Blackwell Publishing Ltd., London, UK.

Baker, M. S. Jr., and I. Oeschger. 2009. Description and initial evaluation of a text message based reporting method for marine recreational anglers. Marine and Coastal Fisheries: Dynamics, Management, and Ecosystem Science 1:143–154. http://dx.doi.org/10.1577/C08-042.1

Ballard, H. L., and J. M. Belsky. 2010. Participatory action research and environmental learning: implications for resilient forests and communities. Environmental Education Research 16(5):611–627. http://dx.doi.org/10.1080/13504622.2010.505440

Ballard, H. L., J. A. Trettevick, and D. Collins. 2008. Comparing participatory ecological research in two contexts: an immigrant community and a Native American community on Olympic Peninsula, Washington. Pages 187–215 in C. Wilmsen, W. Elmendorf, L. Fisher, J. Ross, B. Sararthy, and G. Wells, editors. Partnerships for empowerment: participatory research for community-based natural resource management. Earthscan, London, UK.

Batalden, R. V., K. Oberhauser, and A. T. Peterson. 2007. Ecological niches in sequential generations of eastern North American Monarch butterflies (Lepidoptera: Danaidae): the ecology of migration and likely climate change implications. Environmental Entomology 36(6):1365–1373. http://dx.doi.org/10.1603/0046-225X(2007)36[1365:ENISGO]2.0.CO;2

Bell, S., M. Marzano, J. Cent, H. Kobierska, D. Podjed, D. Vandzinskaite, H. Reinert, A. Armaitiene, M. Grodzińska-Jurczak, and R. Muršič. 2008. What counts? Volunteers and their organisations in the recording and monitoring of biodiversity. Biodiversity and Conservation 17(14):3443–3454. http://dx.doi.org/10.1007/s10531-008-9357-9

Berkes, F. 2004. Rethinking community-based conservation. Conservation Biology 18(3):621–630. http://dx.doi.org/10.1111/j.1523-1739.2004.00077.x

Berkes, F. 2009. Evolution of co-management: role of knowledge generation, bridging organizations and social learning. Journal of Environmental Management 90(5):1692–702. http://dx.doi.org/10.1016/j.jenvman.2008.12.001

Berkes, F., J. Colding, and C. Folke. 2000. Rediscovery of traditional ecological knowledge as adaptive management. Ecological Applications 10(5):1251–1262. http://dx.doi.org/10.1890/1051-0761(2000)010[1251:ROTEKA]2.0.CO;2

Biggs, S. 1989. Resources-poor farmer participation in research: a synthesis of experiences from nine national agricultural research systems. International Service for National Agricultural Research, The Hague, The Netherlands.

Bigham, D. L., M. V. Hoyer, and D.E. Canfield, Jr. 2009. Survey of toxic algal (microcycstin) distribution in Florida lakes. Lake and Reservoir Management 25(3):264–275. http://dx.doi.org/10.1080/07438140903136555

Bird, K. E., W. J. Nichols, and C. R. Tambiah. 2003. The value of local knowledge in sea turtle conservation: a case from Baja California, Mexico. Pages 178–183 in N. C. Haggan and L. Wood, editors. Putting fishers’ knowledge to work: conference proceedings. Fisheries Centre Research Reports 11(1). Fisheries Centre, University of British Columbia, Vancouver, British Columbia, Canada. [online] URL: http://www.seaturtle.org/PDF/author/Bird_2001_FisherKnowledge.pdf

Bonney, R. 2008. Citizen science at the Cornell Lab of Ornithology. Pages 213–230 in R. E. Yager and J. H. Falk, editors. Exemplary science in informal education settings: standards-based success stories. National Science Teachers Association (NSTA) Press, Arlington, Virginia, USA.

Bonney, R., H. Ballard, R. Jordan, E. McCallie, T. Phillips, J. Shirk, and C. Wilderman. 2009a. Public participation in scientific research: defining the field and assessing its potential for informal science education. A CAISE Inquiry Group Report. Center for Advancement of Informal Science Education (CAISE), Washington, D.C., USA.

Bonney, R., C. B. Cooper, J. Dickinson, S. Kelling, T. Phillips, K. V. Rosenberg, and J. Shirk. 2009b. Citizen science: a developing tool for expanding science knowledge and scientific literacy. BioScience 59(11):977–984.

Bonter, D. N., B. Zuckerberg, and J. L. Dickinson. 2010. Invasive birds in a novel landscape: habitat associations and effects on established species. Ecography 33(3):494–502. http://dx.doi.org/10.1111/j.1600-0587.2009.06017.x

Bottrill, M., M. Hockings, and H. P. Possingham. 2011. In pursuit of knowledge: addressing barriers to effective conservation evaluation. Ecology and Society 16(2): 14. [online] URL: http://www.ecologyandsociety.org/vol16/iss2/art14/

Bradbury, H., and P. Reason. 2008. Issues and choice points for improving the quality of action research. Pages 225–242 in M. Minkler, and N. Wallerstein, editors. Community-based participatory research for health. Jossey-Bass Publishers, San Francisco, California, USA.

Brossard, D., B. Lewenstein, and R. Bonney. 2005. Scientific knowledge and attitude change: the impact of a citizen science project. International Journal of Science Education 27(9):1099–1121. http://dx.doi.org/10.1080/09500690500069483

Campbell, L. M., and M. H. Godfrey. 2010. Geo-political genetics: claiming the commons through species mapping. Geoforum 41(6):897–907. http://dx.doi.org/10.1016/j.geoforum.2010.06.003

Campbell, L. M., and A. Vainio-Mattila. 2003. Participatory development and community-based conservation: opportunities missed for lessons learned? Human Ecology 31(3):417–437. http://dx.doi.org/10.1023/A:1025071822388

Carr, A. J. L. 2004. Why do we all need community science? Society and Natural Resources 17(9):841–849. http://dx.doi.org/10.1080/08941920490493846

Cashman, S. B., S. Adeky, A. J. Allen, III, J. Corburn, B. A. Israel, J. Montano, A. Rafelito, S. D. Rhodes, S. Swanston, N. Wallerstein, and E. Eng. 2008. The power and the promise: working with communities to analyze data, interpret findings, and get to outcomes. American Journal of Public Health 98(8):1407–1417. http://dx.doi.org/10.2105/AJPH.2007.113571

Chambers, R. 1994. The origins and practice of participatory rural appraisal. World Development 22(7):953–969. http://dx.doi.org/10.1016/0305-750X(94)90141-4

Chambers, R. 2002. Participatory workshops: 21 sources of ideas and activities. Earthscan Publications, London, UK.

Charvolin, R., A. Micoud, and L. K. Nyhart, editors. 2007. Des sciences citoyennes? La question de l'amateur dans les sciences naturalistes. Éditions de l'Aube, La Tour d'Aigues, France.

Cheng, A. S., K. Bond, C. Lockwood, and S. Hansen. 2008. Calibrating collaboration: monitoring and adaptive management of the Landscape Working Group process on the Grand Mesa, Uncompahgre and Gunnison National Forests in Western Colorado. Pages 147–165 in C. Wilmsen, W. Elmendorf, L. Fisher, J. Ross, B. Sararthy, and G. Wells, editors. Partnerships for empowerment: participatory research for community-based natural resource management. Earthscan, London, UK.

Cleaver, F. 2004. The social embeddedness of agency and decision-making. Pages 217–277 in S. Hickey, and G. Mohan, editors. Participation: from tyranny to transformation? Zed Books Ltd, London, UK and New York, New York, USA.

Cockerill, K., V. C. Tidwell, H. D. Passell, and L. A. Malczynski. 2007. Cooperative modeling lessons for environmental management. Environmental Practice 9(1):28–41. http://dx.doi.org/10.1017/S1466046607070032

Cohen, S. 2010. From observer to extension agent—using research experiences to enable proactive response to climate change. Climatic Change 100(1):131–135. http://dx.doi.org/10.1007/s10584-010-9811-z

Cooke, B., and U. Kothari. 2001. The case for participation as tyranny. Pages 1–15 in B. Cooke and U. Kothari, editors. Participation: the new tyranny? Zed Books, Ltd., New York, New York, USA.

Cooper, C. B., J. Dickinson, T. B. Phillips, and R. Bonney. 2007. Citizen science as a tool for conservation in residential ecosystems. Ecology and Society 12(2): 11. [online] URL: http://www.ecologyandsociety.org/vol12/iss2/art11/

Cooper, C. C., W. Hochachka, and A. A. Dhondt. 2007. Contrasting natural experiments confirm competition between house finches and house sparrows. Ecology 88(4):864–870. http://dx.doi.org/10.1890/06-0855

Corburn, J. 2007. Community knowledge in environmental health science: co-producing policy expertise. Environmental Science and Policy 10(2):150–161. http://dx.doi.org/10.1016/j.envsci.2006.09.004

Cornwall, A. 2008. Unpacking “participation”: models, meanings and practices. Community Development Journal 43(3):269–283. http://dx.doi.org/10.1093/cdj/bsn010

Cornwall, A., and R. Jewkes. 1995. What is participatory research? Social Science and Medicine 41(12):1667–1676.

Crall, A., G. Newman, C. Jarnevich, T. Stohlgren, D. Waller, and J. Graham. 2010. Improving and integrating data on invasive species collected by citizen scientists. Biological Invasions 12(10):3419–3428. http://dx.doi.org/10.1007/s10530-010-9740-9

Crimmins, T. M., M. A. Crimmins, D. Bertelsen, and J. Balmat. 2008. Relationships between alpha diversity of plant species in bloom and climatic variables across an elevation gradient. International Journal of Biometeorology 52(5):353–366. http://dx.doi.org/10.1007/s00484-007-0130-7

Cumming, G., S. J. Guffey, and C. Norwood. 2008. Opportunities and challenges in community capacity-building: lessons from participatory research in Macon County, North Carolina. Pages 127–146 in C. Wilmsen, W. Elmendorf, L. Fisher, J. Ross, B. Sararthy, and G. Wells, editors. Partnerships for empowerment: participatory research for community-based natural resource management. Earthscan, London, UK.

Danielsen, F., N. D. Burgess, A. Balmford, P. F. Donald, M. Funder, J. P. Jones, P. Alviola, D. S. Balete, T. Blomley, and J. Brashares. 2009. Local participation in natural resource monitoring: a characterization of approaches. Conservation Biology 23(1):31–42. http://dx.doi.org/10.1111/j.1523-1739.2008.01063.x

Danielsen, F., N. Burgess, M. Funder, T. Blomley, J. Brashares, A. Akida, A. Jensen, M. Mendoza, G. Stuart-Hill, M. K. Poulsen, H. Ramadhani, M. K. Sam, and E. Topp-Jørgensen. 2010. Taking stock of nature in species-rich but economically poor areas: an emerging discipline of locally based monitoring. Pages 88–112 in A. Lawrence, editor. Taking stock of nature: participatory biodiversity assessment for policy, planning and practice. Cambridge University Press, Cambridge, UK. http://dx.doi.org/10.1017/CBO9780511676482.005

Danielsen, F., M. M. Mendoza, A. Tagtag, P. A. Alviola, D. S. Balete, A. E. Jensen, M. Enghoff, and M. K. Poulsen. 2007. Increasing conservation management action by involving local people in atural resource monitoring. AMBIO: A Journal of the Human Environment 36(7):566–570.

Dickinson, J. L., B. Zuckerberg, and D. N. Bonter. 2010. Citizen science as an ecological research tool: challenges and benefits. Annual Review of Ecology, Evolution, and Systematics 41:149–172. http://dx.doi.org/10.1146/annurev-ecolsys-102209-144636

Droege, S. 2007. Just because you paid them doesn't mean their data are better. Pages 13–26 in C. McEver, R. Bonney, J. Dickinson, S. Kelling, K. Rosenberg, and J. Shirk, editors. Proceedings of the Citizen Science Toolkit Conference, (Ithaca, New York, 20–23 June 2007). Cornell Laboratory of Ornithology, Ithaca, New York, USA. [online] URL: http://www.citizenscience.org/conference/proceeding-pdfs/Droege%202007%20CS%20Conference.pdf

Dunn, P. O., and D. W. Winkler. 1999. Climate change has affected the breeding date of tree swallows throughout North America. Proceedings of the Royal Society of London Series B-Biological Sciences 266(1437):2487–2490. http://dx.doi.org/10.1098/rspb.1999.0950

Ellis, R., and C. Waterton. 2004. Environmental citizenship in the making: the participation of volunteer naturalists in UK biological recording and biodiversity policy. Science and Public Policy 31(2):95–105. http://dx.doi.org/10.3152/147154304781780055

Ellis, R., and C. Waterton. 2005. Caught between the cartographic and the ethnographic imagination: the whereabouts of amateurs, professionals, and nature in knowing biodiversity. Environment and Planning D: Society and Space 23:673–693. http://dx.doi.org/10.1068/d353t

Ely, E., editor. 2002. Success stories. The Volunteer Monitor 14(Summer):24. [online} URL: http://water.epa.gov/type/rsl/monitoring/upload/2004_10_13_monitoring_volunteer_newsletter_volmon14no2.pdf

Evans, C., E. Abrams, R. Reitsma, K. Roux, L. Salmonsen, and P. P. Marra. 2005. The Neighborhood Nestwatch program: participant outcomes of a citizen-science ecological research project. Conservation Biology 19(3):589–594. http://dx.doi.org/10.1111/j.1523-1739.2005.00s01.x

Farquhar, S. A., and S. Wing. 2008. Methodological and ethical considerations in community-driven environmental justice research: two case studies from rural North Carolina. Pages 221–241 in M. Minkler and N. Wallerstein, editors. Community-based participatory research for health. Jossey-Bass Publishers, San Francisco, California, USA.

Fernandez-Gimenez, M. E., H. L. Ballard, and V. E. Sturtevant. 2008. Adaptive management and social learning in collaborative and community-based monitoring: a study of five community-based forestry organizations in the Western USA. Ecology and Society 13(2): 4. [online] URL: http://www.ecologyandsociety.org/vol13/iss2/art4/

Fink, D., W. M. Hochachka, B. Zuckerberg, D. W. Winkler, B. Shaby, M. A. Munson, G. Hooker, M. Riedewald, D. Sheldon, and S. Kelling. 2010. Spatiotemporal exploratory models for broad-scale survey data. Ecological Applications 20(8):2131–2147. http://dx.doi.org/10.1890/09-1340.1

Firehock, K., and J. West. 1995. A brief history of volunteer biological water monitoring using macroinvertebrates. Journal of the North American Benthological Society 14(1):197–202. http://dx.doi.org/10.2307/1467734

Fischer, F. 2000. Citizens, experts, and the environment: the politics of local knowledge. Duke University Press, Durham, North Carolina, USA.

Fishkin, J. S. 2009. When the people speak: deliberative democracy and public consultation. Oxford University Press, Oxford, UK.

Gaventa, J. 2004. Towards participatory governance: assessing the transformative possibilities. Pages 25–41 in S. Hickey and G. Mohan, editors. Participation: from tyranny to transformation? Exploring new approaches to participation in development. Zed Books, London, UK.

Guijt, I. 2007. Negotiated learning : collaborative monitoring in forest resource management. Resources for the Future, Washington, D.C., USA.

Hamel, N. J., A. E. Burger, K. Charleton, P. Davidson, S. Lee, D. F. Bertram, and J. K. Parrish. 2009. Bycatch and beached birds: assessing mortality impacts in coastal net fisheries using marine bird strandings. Marine Ornithology 37(1):41–60.
Hickey, S., and G. Mohan. 2004. Towards participation as transformation: critical themes and challenges. Pages 3–24 in S. Hickey and G. Mohan, editors. Participation: from tyranny to transformation? Zed Books, London, UK.

Hochachka, W. M., A. A. Dhondt, K. J. McGowan, and L. D. Kramer. 2004. Impact of West Nile virus on American crows in the northeastern United States, and its relevance to existing monitoring programs. EcoHealth 1(1):60–68. http://dx.doi.org/10.1007/s10393-004-0015-8

Hopkins, G. W., and R. P. Freckleton. 2002. Declines in the numbers of amateur and professional taxonomists: implications for conservation. Animal Conservation 5(3):245–249. http://dx.doi.org/10.1017/S1367943002002299

Irwin, A. 1995. Citizen science: a study of people, expertise and sustainable development. Routledge, London, UK.

Jasanoff, S. S. 2003. Technologies of humility: citizen participation in governing science. Minerva 41(3):223–244.

Jones, F. C., D. Baird, M. Bowman, G. Cameron, B. Craig, B. Cutler, J. Diamond, N. Dmytrow, M. Nicol, J. Parker, T. Pascoe, H. Vaughan, and G. Whitelaw. 2006. Performance of Ontario’s Benthos Biomonitoring Network: impacts on participants’ social capital, environmental action, and problem-solving ability. Environments 34(1):37–53.

Jordan, R. C., S. A. Gray, D. V. Howe, W. R. Brooks, and J. G. Ehrenfeld. 2011. Knowledge gain and behavioral change in citizen-science programs. Conservation Biology 25(6):1148–1154. http://dx.doi.org/10.1111/j.1523-1739.2011.01745.x

Jorgensen, M. S., I. Hall, D. Hall, A. Gnaiger, G. Schroffenegger, S. Brodersen, K. von der Heiden, R. Reimer, M. Strahle, C. Urban, W. Endler, C. Teodosiu, T. Rojo, and L. Leydesdorff. 2004. Democratic governance through interaction between NGOs, universities and science shops: experiences, expectations, recommendations. D1c, The Science Shop at the Technical University of Denmark, Lyngby, Denmark.

Kapoor, I. 2001. Towards participatory environmental management? Journal of Environmental Management 63(3):269–279. http://dx.doi.org/10.1006/jema.2001.0478

Kountoupes, D. L., and K. S. Oberhauser. 2008. Citizen science and youth audiences: educational outcomes of the Monarch Larva Monitoring Project. Journal of Community Engagement and Scholarship 1(1):10–20.

Lathrop, J. E., and A. Markowitz. 1995. Monitoring water resource quality using volunteers. Pages 303–314 in W. S. Davis and T. P. Simon, editors. Biological assessment and criteria. Lewis Publishers, Boca Raton, Florida, USA.

Lawrence, A. 2006. “No personal motive?” volunteers, biodiversity, and the false dichotomies of participation. Ethics, Place and Environment 9(3):279–298. http://dx.doi.org/10.1080/13668790600893319

Lawrence, A. 2009. The first cuckoo in winter: phenology, recording, credibility and meaning in Britain. Global Environmental Change 19(2):173–179. http://dx.doi.org/10.1016/j.gloenvcha.2009.01.006

Lawrence, A. 2010a. Introduction: learning from experiences of participatory biodiversity assessment. Pages 1–29 in A. Lawrence, editor. Taking stock of nature: participatory biodiversity assessment for policy, planning and practice. Cambridge University Press, Cambridge, UK. http://dx.doi.org/10.1017/CBO9780511676482.001

Lawrence, A. 2010b. The personal and political of volunteers’ data: towards a national biodiversity database for the UK. Pages 251–265 in A. Lawrence, editor. Taking stock of nature: participatory biodiversity assessment for policy, planning and practice. Cambridge University Press, Cambridge, UK. http://dx.doi.org/10.1017/CBO9780511676482.012

Leydesdorff, L., and J. Ward. 2005. Science shops: a kaleidoscope of science–society collaborations in Europe. Public Understanding of Science 14(4):353–372. http://dx.doi.org/10.1177/0963662505056612

Lindsey, E., M. Mehta, V. Dhulipala, K. Oberhauser, and S. Altizer. 2009. Crowding and disease: effects of host density on response to infection in a butterfly–parasite interaction. Ecological Entomology 34(5):551–561. http://dx.doi.org/10.1111/j.1365-2311.2009.01107.x

Long Martello, M. 2004. Global change science and the Arctic citizen. Science and Public Policy 31(2):107–115. http://dx.doi.org/10.3152/147154304781780082

Luks, F. 1999. Post-normal science and the rhetoric of inquiry: deconstructing normal science? Futures 31(7):705–719. http://dx.doi.org/10.1016/S0016-3287(99)00028-2

McCallie, E., L. Bell, T. Lohwater, J. H. Falk, J. L. Lehr, B. V. Lewenstein, C. Needham, and B. Wiehe. 2009. Many experts, many audiences: public engagement with science and informal science education. Center for Advancement of Informal Science Education (CAISE), Washington, D.C., USA.

Middleton, J. V. 2001. The Stream Doctor project: community-driven stream restoration. BioScience 51(4):293–296.

Minkler, M., and N. Wallerstein. 2008. Introduction to community-based participatory research. Pages 5–24 in M. Minkler and N. Wallerstein, editors. Community-based participatory research for health. Jossey-Bass Publishers, San Francisco, California, USA.

Nerbonne, J. F., and K. C. Nelson. 2008. Volunteer macroinvertebrate monitoring: tensions among group goals, data quality, and outcomes. Environmental Management 42(3):470–479. http://dx.doi.org/10.1007/s00267-008-9103-9

Nerbonne, J. F., and B. Vondracek. 2003. Volunteer macroinvertebrate monitoring: assessing training needs through examining error and bias in untrained volunteers. Journal of the North American Benthological Society 22(1):152–163. http://dx.doi.org/10.2307/1467984

Noss, R. 2001–2002. Citizen scientist or amateur naturalist? Wild Earth 11:14–17.

Overdevest, C., C. Huyck Orr, and K. Stepenuck. 2004. Volunteer stream monitoring and local participation in natural resource issues. Human Ecology Review 11(2):177–185.

Overdevest, C., and B. Mayer. 2008. Harnessing the power of information through community monitoring: insights from social science. Texas Law Review 86:1493–1526.

Pahl-Wostl, C., E. Mostert, and D. Tabara. 2008. The growing importance of social learning in water resources management and sustainability science. Ecology and Society 13(1):24. [online] URL: http://www.ecologyandsociety.org/vol13/iss1/art24/

Pattengill-Semmens, C. V., and B. X. Semmens. 2003. Conservation and management applications of the REEF volunteer fish monitoring program. Environmental Monitoring and Assessment 81(1–3):43–50. http://dx.doi.org/10.1023/A:1021300302208

Peluso, N. L. 2005. Seeing property in land use: local territorializations in West Kalimantan, Indonesia. Geografisk Tidskrift, Danish Journal of Geography 105(1):1–16.

Penrose, D., and S. M. Call. 1995. Volunteer monitoring of benthic macroinvertebrates: regulatory biologists’ perspectives. Journal of the North American Benthological Society 14(1):203–209. http://dx.doi.org/10.2307/1467735

Phillips, T., and J. L. Dickinson. 2009. Tracking the nesting success of North America’s breeding birds through public participation in NestWatch. Pages 633–640 in T. D. Rich, C. Arizmendi, D. W. Demarest, and C. Thompson, editors. Proceedings of the Fourth International Partners in Flight conference, 13–16 February 2008, McAllen, Texas. Partners in Flight, McAllen, Texas, USA. [online] URL: http://www.partnersinflight.org/pubs/mcallenproc/toc.cfm

Plummer, R., and J. FitzGibbon. 2007. Connecting adaptive co-management, social learning, and social capital through theory and practice. Pages 38–61 in D. Armitage, F. Berkes, and N. Doubleday, editors. Adaptive co-management: collaboration, learning, and multi-level governance. UBC Press, Vancouver, British Columbia, Canada.

Powell, M. C., and M. Colin. 2008. Meaningful citizen engagement in science and technology: what would it really take? Science Communication 30(1):126–136.

Pretty, J. N. 1995. Participatory learning for sustainable agriculture. World Development 23(8):1247–1263. http://dx.doi.org/10.1016/0305-750X(95)00046-F

Raddick, M. J., G. Bracey, K. Carney, G. Gyuk, K. Borne, J. Wallin, and S. Jacoby. 2009. Citizen science: status and research directions for the coming decade. Astro2010: The Astronomy and Astrophysics Decadal Survey, Position Papers, no. 46. [online] URL: http://adsabs.harvard.edu/abs/2009astro2010P..46R

Raddick, M. J., G. Bracey, P. L. Gay, C. J. Lintott, P. Murray, K. Schawinski, A. S. Szalay, and J. Vandenberg. 2010. Galaxy Zoo: exploring the motivations of citizen science volunteers. Astronomy Education Review 9(1):18. [online] URL: http://aer.aas.org/resource/1/aerscz/v9/i1/p010103_s1 http://dx.doi.org/10.3847/AER2009036

Root, T. L., M. A. Holmgren, and R. W. Andrews. 1981. Winter abundance patterns of some songbirds near the 100th meridian in the southern United States. The Southwestern Naturalist 26(2):95–100. http://dx.doi.org/10.2307/3671104

Rosenberg, K. V., R. W. Hames, R. W. Rohrbaugh, Jr., S. Barker Swarthout, J. D. Lowe, and A. A. Dhondt. 2003. A land manager’s guide to improving habitat for forest thrushes. The Cornell Laboratory of Ornithology, Ithaca, New York, USA.

Rosenberg, K. V., R. W. Rohrbaugh, Jr., S. E. Barker, J. D. Lowe, R. W. Hames, and A. A. Dhondt. 1999. A land manager’s guide to improving habitat for scarlet tanagers and other forest interior birds. The Cornell Laboratory of Ornithology, Ithaca, New York, USA.

Ross, J., S. Brawley, J. Lowrey, and D. L. Hankins. 2008. Creating common ground: a collaborative approach to environmental reclamation and cultural preservation. Pages 105–126 in C. Wilmsen, W. Elmendorf, L. Fisher, J. Ross, B. Sararthy, and G. Wells, editors. Partnerships for empowerment: participatory research for community-based natural resource management. Earthscan, London, UK.

Rowe, G. ,and L. J. Frewer. 2004. Evaluating public-participation exercises: a research agenda. Science, Technology, and Human Values 29(4):512–556. http://dx.doi.org/10.1177/0162243903259197

Rowe, G., and L. J. Frewer. 2005. A typology of public engagement mechanisms. Science, Technology, and Human Values 30(2):251–290. http://dx.doi.org/10.1177/0162243904271724

Saldivar-Tanaka, L., and M. E. Krasny. 2004. Culturing community development, neighborhood open space, and civic agriculture: the case of Latino community gardens in New York City. Agriculture and Human Values 21(4):399–412. http://dx.doi.org/10.1023/B:AHUM.0000047207.57128.a5

Senko, J., M. C. Lopez-Castro, V. Koch, and W. J. Nichols. 2010. Immature East Pacific green turtles (Chelonia mydas) use multiple foraging areas off the Pacific coast of Baja California Sur, Mexico: first evidence from mark–recapture data. Pacific Science 64(1):125–130.

Simpson, A., C. Jarnevich, R. Westbrooks, C. Fournier, E. Sellers, J. Madsen, L. Mehrhoff, M. Browne, and J. Graham. 2009. Invasive species information networks: collaboration at multiple scales for prevention, early detection, and rapid response to invasive alien species. Biodiversity 10(2–3):5–13. http://dx.doi.org/10.1080/14888386.2009.9712839

Stebbins, R. A. 1980. Avocational science: the amateur routine in archaeology and astronomy. International Journal of Comparative Sociology 21(1/2):34–48.

Sullivan, B. L., C. L. Wood, M. J. Iliff, R. E. Bonney, D. Fink, and S. Kelling. 2009. eBird: a citizen-based bird observation network in the biological sciences. Biological Conservation 142(10):2282–2292. http://dx.doi.org/10.1016/j.biocon.2009.05.006

Swaisgood, R. R., and J. K. Sheppard. 2010. The culture of conservation biologists: show me the hope! BioScience 60(8):626–631.

Tabara, J. D., and C. Pahl-Wostl. 2007. Sustainability learning in natural resource use and management. Ecology and Society 12(2): 3. [online] URL: http://www.ecologyandsociety.org/vol12/iss2/art3/

Taylor, B. 1995. Amateurs, professionals and the knowledge of archaeology. The British Journal of Sociology 46(3):499–508. http://dx.doi.org/10.2307/591853

Tomasek, T. M. 2006. Student cognition and motivation during the Classroom BirdWatch citizen science project. Dissertation. University of North Carolina, Greensboro, North Carolina, USA.

Torti, V. M., and P. O. Dunn. 2005. Variable effects of climate change on six species of North American birds. Oecologia 145(3):486–495. http://dx.doi.org/10.1007/s00442-005-0175-4

Trumbull, D. J., R. Bonney, D. Bascom, and A. Cabral. 2000. Thinking scientifically during participation in a citizen-science project. Informal Science 84(2):265–275. http://dx.doi.org/10.1002/(SICI)1098-237X(200003)84:2<265::AID-SCE7>3.3.CO;2-X

Tudor, M. T., and K. M. Dvornich. 2001. The NatureMapping program: resource agency environmental education reform. Journal of Environmental Education 32:8–14. http://dx.doi.org/10.1080/00958960109599132

van den Belt, M. 2004. Mediated modeling: a system dynamics approach to environmental consensus building. Island Press, Washington, D.C., USA.

van Vliet, A. J. H., A. Overeem, R. S. De Groot, A. F. G. Jacobs, and F. T. M. Spieksma. 2002. The influence of temperature and climate change on the timing of pollen release in the Netherlands. International Journal of Climatology 22(14):1757–1767. http://dx.doi.org/10.1002/joc.820

Vaughan, H., G. S. Whitelaw, B. Craig, and C. Stewart. 2003. Linking ecological science to decision-making: delivering environmental monitoring information as societal feedback. Environmental Monitoring and Assessment 88(1–3):399–408.

W.K. Kellogg Foundation. 2004. Using logic models to bring together planning evaluation and action: logic model development guide. W.K. Kellogg Foundation, Battle Creek, Michigan, USA. [online] URL: http://www.wkkf.org/knowledge-center/resources/2006/02/WK-Kellogg-Foundation-Logic-Model-Development-Guide.aspx

Walker, B., and D. Salt. 2006. Resilience thinking: sustaining ecosystems and people in a changing world. Island Press, Washington, D.C., USA.

Weston, M., M. Fendley, R. Jewell, M. Satchell, and C. Tzaros. 2003. Volunteers in bird conservation: insights from the Australian Threatened Bird Network. Ecological Management and Restoration 4(3):205-211.– http://dx.doi.org/10.1046/j.1442-8903.2003.00169.x

White, S. C. 1996. Depoliticising development: the uses and abuses of participation (La dépolitisation du développement: usages et abus de la participation / Despolitisando o desenvolvimento: participação: usos e abusos / Despolitizando el desarrollo: los usos y abusos de la participación). Development in Practice 6(1):6–15.

Whitelaw, G. S., H. Vaughan, B. Craig, and D. Atkinson. 2003. Establishing the Canadian Community Monitoring Network. Environmental Monitoring and Assessment ∫(1–3):409–418.

Whyte, W. F. 1991. Participatory action research. Sage Publications, Newbury Park, California, USA. http://dx.doi.org/10.1007/BF02393343

Wiggins, A., and K. Crowston. 2010. Developing a conceptual model of virtual organisations for citizen science. International Journal of Organisational Design and Engineering 1(1/2):148–162.

Wiggins, A., and K. Crowston. 2011. From conservation to crowdsourcing: a typology of citizen science. Pages 1–10 in Proceedings of the 44th Annual Hawaii International Conference on Systems Sciences, 4–7 January 2011, Koloa, Hawaii. http://dx.doi.org/10.1109/HICSS.2011.207

Wilderman, C. C. 2005. Portrait of a watershed: Shermans Creek. Pennsylvania Department of Environmental Protection, Harrisburg, Pennsylvania, USA. [online] URL: http://www.shermanscreek.org/ShermansCreekPortrait.pdf

Wilderman, C. C. 2007. Models of community science: design lessons from the field. Pages 83–96 in C. McEver, R. Bonney, J. Dickinson, S. Kelling, K. Rosenberg, and J. Shirk, editors. Proceedings of the Citizen Science Toolkit Conference, 20–23 June 2007, Ithaca, New York, USA. Cornell Laboratory of Ornithology, Ithaca, New York, USA. [online] URL: http://www.citizenscience.org/conference/proceeding-pdfs/Wilderman%202007%20CS%20Conference.pdf

Wilderman, C. C., A. Barron, and L. Imgrund. 2004a. From the field: a service provider’s experience with two operational models for community science. Community-Based Collaboratives Research Consortium Journal Spring 2004.

Wilderman, C. C., A. Barron, and L. Imgrund. 2004b. Top down or bottom up? ALLARM’s experience with two operational models for community science. Article 235 in Proceedings of the 4th National Water Quality Monitoring Council Conference, 17–20 May 2004, Chattanooga, Tennessee, USA. [online] URL: http://acwi.gov/monitoring/conference/2004/poster_presentations.html

Wilderman, C. C., and J. L. Shirk. 2010. From citizen science to volunteer monitoring: seeking hybridization of models for community science. (Abstract) Page 140 in Proceedings of the 7th National Monitoring Conference, 25–29 April 2010, Denver, Colorado, USA. [online] URL: http://acwi.gov/monitoring/conference/2010/2010_NMC_Program_2010-04-13.pdf and (powerpoint presentation) http://acwi.gov/monitoring/conference/2010/L4/L4_Wilderman.pdf

Wilmsen, C. 2008. Negotiating community, participation, knowledge and power in participatory research. Pages 1–22 in C. Wilmsen, W. Elmendorf, L. Fisher, J. Ross, B. Sararthy, and G. Wells, editors. Partnerships for empowerment: participatory research for community-based natural resource management. Earthscan, London, UK.

Wilmsen, C., W. Elmendorf, L. Fisher, J. Ross, B. Sarathy, and G. Wells. 2008a. Participation, relationships and empowerment. Pages 259–284 in C. Wilmsen, W. Elmendorf, L. Fisher, J. Ross, B. Sararthy, and G. Wells, editors. Partnerships for empowerment: participatory research for community-based natural resource management. Earthscan, London, UK.

Wilmsen, C., W. Elmendorf, L. Fisher, J. Ross, B. Sararthy, and G. Wells. 2008b. Partnerships for empowerment: participatory research for community-based natural resource management. Earthscan, London, UK.

Wilmsen, C. and A. Krishnaswamy. 2008. Challenges to institutionalizing participatory research in community forestry in the US. Pages 47–67 in C. Wilmsen, W. Elmendorf, L. Fisher, J. Ross, B. Sararthy, and G. Wells, editors. Partnerships for empowerment: participatory research for community-based natural resource management. Earthscan, London, UK.

Wood C., B. Sullivan, M. Iliff, D. Fink, S. Kelling. 2011. eBird: engaging birders in science and conservation. PLoS Biology 9(12):1–5. http://dx.doi.org/10.1371/journal.pbio.1001220

Wolfe, D. W., M. D. Schwartz, A. N. Lakso, Y. Otsuki, R. M. Pool, and N. J. Shaulis. 2005. Climate change and shifts in spring phenology of three horticultural woody perennials in northeastern USA. International Journal of Biometeorology 49(5):303–309. http://dx.doi.org/10.1007/s00484-004-0248-9

Wulfhorst, J. D., B. W. Eisenhauer, S. L. Gripne, and J. M. Ward. 2008. Core criteria and assessment of participatory research. Pages 23–46 in C. Wilmsen, W. Elmendorf, L. Fisher, J. Ross, B. Sararthy, and G. Wells, editors. Partnerships for empowerment: participatory research for community-based natural resource management. Earthscan, London, UK.

Wynne, B. 1992. Misunderstood misunderstanding: social identities and public uptake of science. Public Understanding of Science 1(3):281–304. http://dx.doi.org/10.1017/CBO9780511563737.002

Zerbe, F. and C. Wilderman. 2010. Monitoring IMPACTS of new gas-drilling technologies. Pages 1–4 in E. Ely, editor. The volunteer monitor. US Environmental Protection Agency Office of Water, Washington, D.C., USA.

Zuckerberg, B., D. N. Bonter, W. M. Hochachka, W. D. Koenig, A. T. DeGaetano, and J. L. Dickinson. 2011. Climatic constraints on wintering bird distributions are modified by urbanization and weather. Journal of Animal Ecology 80(2):403–413. http://dx.doi.org/10.1111/j.1365-2656.2010.01780.x