Step 5 — Sustain and Improve Your Project
Successful citizen science projects require sustained participation and funding, as well as sound methods of evaluating progress toward goals. At each stage of your project you’ll need to plan for all the later stages, including documenting and preserving records and data for future use.
The previous “How To” steps (Step 1 Scope Your Problem; Step 2 — Design a Project; Step 3 — Build a Community; and Step 4 — Manage Your Data) cover stages of your project. Here, we give you additional tips for sustaining and improving your project over time.
The following tips will help you get started:
- Adapt to cycles of participation.
- Communicate effectively.
- Solicit feedback from your participants.
- Sustain your project funding.
- Evaluate the quality of your data.
- Evaluate your participants’ engagement.
- Build flexibility into your project.
- Know how to end your project.
- The Enhanced Passive Surveillance System: Crowdsourcing for Early Detection of Animal Disease Outbreaks
- EteRNA: Crowdsourcing New RNA Designs
- The Smithsonian Transcription Center: Crowdsourcing Document Transcription
- The GLOBE/S’COOL Partnership: Citizen Scientists Validate Satellite Data
- The Monarch Larva Monitoring Project: Citizen Scientists Monitor Monarch Butterflies
- CoCoRaHS — Community Collaborative Rain, Hail and Snow Network: Citizen Scientists Track Precipitation
- EyeWire: Crowdsourcing Brain Mapping
- eBird: Crowdsourcing Bird Data
- Did You Feel It?: Crowdsourcing Earthquake Maps
- Disk Detective: Crowdsourcing New Planets
- Citizen Archivist Dashboard: Improving Access to Historical Records Through Crowdsourcing
Adapt to Cycles of Participation
Project participation levels can change over time or vary from person to person. Some people participate just once; others volunteer sporadically; still others are long-term participants. Then there are the “SuperUsers” — the small number who do a lot of the work. Do what you can to motivate volunteers to keep coming back, but recognize that “life happens” and sometimes people discontinue participation for unavoidable reasons.
- Remember that people’s motivations change, as do their life situations. To keep participants motivated, pay attention to their needs, tap into their personal interests and stay on top of any shifts in what they want from your project.
- Analyze user data to improve project participation. Don’t be discouraged if most contributors participate only once; instead, try to understand why so you can adapt management strategies, participation procedures, and analytical tactics.
- Reward your volunteers by giving them certificates, prizes and other recognition to encourage them to keep coming back. Make sure to acknowledge their work; if possible and appropriate, attribute the work to them personally. Be aware that some participants may also prefer to contribute without recognition.
- If it’s suited to your goals, turn your project into something fun and engaging by making it a game! Examples include EyeWire, Fold-It and Old-Weather. Recognize users who do well in the game by awarding badges, using visible leaderboards and giving other creative rewards — but also remember that good games must make the task fun even when success is a struggle. Be careful to avoid incentivizing “gaming the system” in an unproductive fashion, and be aware that some participants may be turned off by competition and game elements. The key lies in identifying what works best for your particular community, and being willing to adapt to meet their needs.
- Let your volunteers know how their work is making a difference. For example, Galaxy Zoo collectively acknowledged its volunteers in a scientific paper. By validating the work of your volunteers, you encourage them to keep coming back.
- Evaluate whether they may be able to contribute in a special role, such as reviewing others’ data submissions or moderating online discussions.Consider special recognition for SuperUsers, and find out what motivates them so you can recruit more.
- Dynamic Changes in Motivation in Collaborative Citizen-Science Projects (Journal Article, 2012)
- EyeWire: A Game to Map the Brain
- FoldIt: A Game to Solve Puzzles for Science
- Galaxy Zoo: A Zooniverse Project Blog
- Gaming for (Citizen) Science (Journal Article, 2011, PDF)
- Most Participants in Citizen Science Give Up (ARS Technica)
- Old Weather: Transcribing Ship Logs for Climate Modeling
Clear and consistent communication is the most crucial component of a successful project from beginning to end. To avoid misunderstandings, use a “pre-nup” agreement to clarify roles and responsibilities. Give your participants news about your project and chances to respond, and keep data accessible rather than locked away.
- Draft and sign a collaborative agreement outlining tasks, roles and responsibilities, including who has access to what data. Regularly review and update the document, particularly when new people join the team.
- Hold regular meetings so everyone can understand how the project is progressing, talk about new developments and raise any concerns.
- Watch for any conflict (such as over how to interpret data) and be prepared to handle it.
- Use a common website, hub, tool or software to keep your project information and data up to date and accessible. Store data in shared spaces that are easy to find (such as the cloud). You may need to sign a terms-of-service agreement.
Solicit Feedback From Your Participants
Participant feedback can help you evaluate your project and make needed changes. Your funding source might also require you to collect participant feedback.
- Collect feedback using tools that allow free text comments, such as collaboration-based websites or forums. If your project uses an app, make sure it lets you collect feedback from your participants, both for resolving technical issues and providing other input.
- Use surveys to encourage your participants to provide feedback. Find tools that make the results more quantifiable, and that that yield consistent results over time. Make sure to evaluate whether to secure human subjects approval based on your intended use of survey results.
- Incorporate useful feedback into your project and let your participants know why you are making those changes.
- Keep participant feedback throughout the life of your project and use it in overall project evaluation.
Sustain Your Project Funding
At different stages of your project, you might need different sources of funding, depending upon your project’s maturity, your partners and other factors. Sustainable funding will depend on your project design, your success in recruiting participants and achieving your goals, and your ability to learn from experience and improve your planning and budgeting.
- Always think ahead about how to fund the next phase of your project. Keep your project aligned with your organization’s mission and with the goals of your citizen science community.
- Review existing case studies of citizen science and crowdsourcing projects for lessons about hidden or expanding costs. Keep careful account of your expenditures. As phases of your project end, review your budget estimates and adjust as needed.
- Use a variety of sources for funding, including your organization’s budget, government grants (local, regional or federal), university/private/public partnerships, foundation grants, crowdfunding, and fundraising or donations. Consider breaking out parts of your project for funding in different ways.
Evaluate the Quality of Your Data
How do you know if your project is accomplishing its goals? You can use project evaluation and metrics to continually monitor the quality of your data and outcomes.
- Evaluate your project early to make sure you’re on track to achieve your goals. Make project evaluation a routine part of your project.
- Measure your project’s outputs or activities, such as the amount of data or number of samples collected and the number of training sessions held or Web visits recorded. Keep records of changes to your methods of monitoring project health and productivity.
- Use metrics to detect problems with the quality of your data. Your metrics should be specific, measurable, attainable, relevant and time-bound.
- Quantify and account for systematic bias in your data.
Evaluate Your Participants’ Engagement
Citizen science and crowdsourcing projects often have multiple goals and stakeholders. Some projects are designed to teach participants new knowledge and skills, or change how they act. In designing your evaluation strategy, take both data quality and participant engagement into account.
- Carefully choose the right instrument for evaluation, taking into account your project goals and all the people and organizations who have a stake in your project.
- Evaluate the motivation levels of your volunteers and take steps to keep them well-motivated.
- Measure how your participants change over time — in their motivation, their level of interest in science and the environment, their self-confidence in participating in citizen science, their knowledge of the nature of science, their skills in scientific inquiry and their stewardship of natural resources. Choose the right instrument. Keep in mind that self-selected volunteers are less likely to show attitudinal changes than other groups, and learning may take many forms. for these measurements.
Build Flexibility Into Your Project
Keep your eye fixed on your endpoint, but be flexible — your needs might change, or early results might show that you aren’t hitting your goals. As part of your ongoing project assessment, be prepared to adjust your project design, then reassess and readjust it in an iterative process.
- Consider using adaptive management As your project goes forward, conditions could change and you might face unforeseen obstacles. You can then adjust your processes to better meet your goals.
- Look into iterative project management. Each time you make adjustments, look at your project as something new, to be reassessed and readjusted in a process of iterative development. Document project changes, particularly if they affect participation or data interpretation.
Know How to End Your Project
When should your project end, in whole or in part? What do you do with the data you have gathered? The answers may lie in who funded you, who the participants are or what you are studying. After your project ends, a smooth transition will preserve what you’ve learned and protect the good reputation of your project’s brand.
- Make sure your volunteers understand why and when your project is ending. From the very beginning, give them a sense of the project timeline, with updates at regular intervals or milestones to remind them of project status. Good communication is key!
- Help your volunteers find other projects that might interest them.
- Keep your data safe and secure. If you do not share all data throughout the project, you can share data without contacting each person individually by uploading to a hub that all participants have access to, and sending a bulk notification.
- If necessary, get permission to continue using your data in the future. If the data includes personally identifiable information, find a way for those who contributed the data to share it with analysts.
- At the completion of the project, create a persistent public-facing presentation of the results rather than leaving a no-longer active project website to look abandoned.