Evaluating a Multi-Component Assessment Framework for Biodiversity Education

Introduction

The world is facing a biodiversity crisis (Wilson 2002).  In response, colleges and universities are being urged to prepare students to face the real-life issues they will routinely encounter in efforts to sustainably manage the biosphere and integrate biodiversity conservation with other societal goals (e.g., Noss 1997, Colker 2004, European Platform for Biodiversity Research Strategy 2006). 

The Network of Conservation Educators and Practitioners (NCEP) was designed to meet this challenge.  The vision for NCEP is a network of highly trained individuals effectively teaching about, managing, and sustaining the world’s biological and cultural diversity. Based at the American Museum of Natural History’s Center for Biodiversity and Conservation, NCEP targets educators working with undergraduate and graduate students as well as trainers working with conservation professionals in a variety of settings. One of the most important tangible products of the NCEP project are the teaching materials or “modules”, which are flexible, multi-component resources that include: 1) a comprehensive Synthesis document summarizing a conservation topic, 2) practical, problem-solving Exercises and Solutions that emphasize active learning strategies, 3) an electronic Presentation with source information and suggestions for active teaching, and 4) interdisciplinary Case Studies that may span the topics of multiple modules. Modules are available free of charge on the NCEP website.

NCEP modules emphasize active learning pedagogies. Active learning and inquiry-based modes of teaching represent more effective pathways for student learning than traditional lecture-based methods (McNeal and D’Avanzo 1997; Olson and Loucks-Horsley, eds. 2000).  These pedagogies have shown promising results in wildlife conservation education when compared to traditional lecture-based approaches (Ryan and Campa 2000). Some laboratory manuals in conservation biology also emphasize active learning strategies (e.g., Gibbs et al. 1998; Shultz et al. 1999).

Conservation practitioners require expertise in a diverse set of process skills such as professional communication, teamwork and cooperation, the ability to integrate diverse sources of information, and problem-solving (Kessler 1995; Ryan and Campa 2000). Because traditional lecture-based teaching does not adequately provide students with opportunities to develop these professional competencies, NCEP modules explicitly seek to integrate process skills development with active teaching techniques throughout the various components. NCEP modules are designed to allow students to be both users and creators of knowledge, develop critical thinking skills, apply knowledge to practical problems and contexts, and develop process skills such as communication, problem solving, and collaboration on group work.

Teaching strategies that emphasize active learning and process skills development also require innovative approaches to assessment.  In biodiversity conservation education, it is essential that learning goals extend beyond factual content to include conceptual understanding, familiarity with the methods and practices of conservation professionals, and changes in attitudes and worldview toward the environment. This study directly responds to recommendations of international groups such as the European Platform for Biodiversity Research (2006), which urges further study on methods that evaluate and measure the effect and effectiveness of biodiversity education.

Several authors have shown that academic coverage of environmental topics and ecological principles increases student awareness, and positively affects attitudes, behaviors, and values regarding conservation issues (Leeming et al. 1993; Zelezny 1999; Rickinson 2001; Humston and Ortiz-Barney 2005; Anderson et al. 2007).  It has been more difficult to create reliable instruments that correlate specific course teaching methods and learning objectives with changes in attitudes and values (Humston and Ortiz-Barney 2005).

In this paper, we report on the results of developing and piloting a multi-component assessment framework that measures and assesses learning gains in biodiversity education. We use this framework to evaluate the effectiveness of three NCEP modules in producing content learning gains, along with changes in students’ interest in biodiversity, student self-perceptions of changes in process skills, and shifts in ecological worldview.

Methods

Faculty Focus Group (FFG)

In spring 2007, NCEP organized a group of educators into a “faculty focus group” (FFG) that developed and tested the assessment framework described here.  Participating institutions ranged from private liberal arts colleges to state universities, and included two minority-serving institutions (MSI) (Table 1). Faculty incorporated NCEP biodiversity modules into courses ranging from introductory biology to a required course for conservation biology majors. 

Assessment Framework
We developed a comprehensive outcomes framework to assess the efficacy of NCEP modules in enhancing teaching and training in biodiversity conservation. The framework measured changes in conceptual understanding, improvements in self-perceptions of process skills, confidence in biodiversity knowledge, interest in biodiversity topics, and changes in environmental orientation and worldview. The methodology adapted and integrated three types of evaluation instruments in a pre-module exposure test/post-module exposure test format:

• Content Knowledge Tests:  Pre- and post- tests specific to each NCEP module assess student learning outcomes;
• Student Assessment of Learning Gains (SALG): A self-reporting instrument measures changes in student confidence, interests, and process skills;
• New Ecological Paradigm Scale (NEP): A widely used self-reporting instrument that quantifies environmental orientation and worldview.

The FFG worked with three NCEP modules: What is Biodiversity?, Why is Biodiversity Important?, and Threats to Biodiversity.  Each module includes an interactive PowerPoint lecture of 25-50 slides with notes and discussion questions, a detailed topical synthesis paper, and a series of hands-on exercises in which students collect, analyze, and synthesize biodiversity data from multiple sources. Each module component contains specific learning objectives to assist faculty teaching the material.

Most FFG participants used both module presentations and exercises, except for one professor who did not have time to adapt the exercises for a large class (see Table 1). Professors used presentations to introduce and discuss topics and applied the exercises as complements to lectures. Three out of the four professors used exercises in two class sessions or more. Two professors introduced the activity and answered questions at the end of the lecture, allowed students to work on the problems and then discussed the results in the following class, while one professor used an entire activity during one class session and another modified an exercise to last the whole semester. Variability in use and adaptation was allowed in this study since we were testing the proposed assessment framework rather than applying a quasi-experimental design.

Assessment instruments were reviewed and ruled exempt from review by the American Museum of Natural History Institutional Review Board (IRB). Student’s names were used for logistical purposes only; professors used names only to match pre and post test scores. Both name of the participant and his/her institution were not associated with each other or with any other responses. All other information was coded into a database and the original information was kept by individual professors so that responses analyzed for this study could not be matched to particular students.

Content Knowledge Tests
Content knowledge assessments measure student learning from the module component used. These assessments include true/false questions, multiple choice, matching, short answer, problem sets, and short essays. In addition to measuring knowledge recall, assessments focus on higher-order learning, including comprehension and application of material and problem solving in new situations. The FFG used a written content knowledge test,consisting of twenty multiple-choice, true/false, and matching questions that were selected from the three modules, to measure changes in students’ knowledge of biodiversity (see Resources). Pre-tests were given prior to classroom use of the modules. The post-test was administered either immediately after teaching the modules, or was embedded in the final exam. 

Student Assessment of Learning Gains (SALG)

The Student Assessment of Learning Gains (SALG) is an on-line self-reporting survey instrument that measures students' perceptions of their knowledge, attitudes, and skills. The SALG was originally created as a pencil/paper assessment scheme to assess changes in students’ confidence, interest and involvement in scientific modes of inquiry (Seymour and Hewitt 1997).  The current on-line version was developed with support from Science Education for New Civil Engagements and Responsibilities (SENCER). The SENCER community constitutes the SALG’s largest user constituency and utilizes the instrument as a common platform to collect cross-program formative and summative data.

For faculty, the SALG represents an evaluative mechanism to determine the influence of content and teaching styles on student learning. NCEP adapted the SENCER SALG instrument to measure the perceptions of students in five areas: 1) confidence in knowledge and understanding of biodiversity conservation; 2) interest in the field of conservation biology; 3) confidence in process skills; 4) environmental orientation and worldview (NEP scale), and 5) preferred learning styles. SALG questions used a standard five-point Likert scale ranging from 1 (not at all confident) to 5 (extremely confident). (see Resources for SALG instruments).

With regard to confidence in process skills, NCEP identified thirteen skills that are important within the conservation biology profession, including: professional oral and written communication; public communication and outreach; problem and question definition; information gathering, critical inquiry, and research skills; sorting and filtering diverse sources of information; predicting potential outcomes and consequences; critical thinking for decision-making; data collection and management; data analysis and interpretation; graphical expression and interpretation; collaborative working skills; and project coordination and management skills. Each NCEP module exercise emphasizes at least one of these process skills.

Because NCEP emphasizes active-learning approaches, we developed the SALG instrument to allow students to rank their preferred learning styles.  Choices ranged from traditional lectures to hands-on activities and outdoor field experiences. The standard Likert scale ratings ranged from Strongly Disagree to Strongly Agree. Demographic information, including gender, ethnicity, class standing and major, as well as reason for enrolling in the course were also collected.

New Ecological Paradigm Scale (NEP)
Conservation biology is an interdisciplinary field of study that is inherently values-based (Soule 1985). Therefore, we are interested in the potential impact of our curriculum on changing the attitudes, behaviors, and beliefs of students.  Since the 1970s, the New Environmental Paradigm Scale has been widely used to measure environmental orientation and worldview of the general public and specific populations, such as students (Dunlap and van Liere 1978; Anderson and Kim 2006).  In the late 1990s, a revised New Ecological Paradigm (NEP) updated the original instrument (Dunlap et al. 2000) and has been validated as a tool for measuring perceived changes in student environmental worldview (Humston and Ortiz-Barney 2007; Anderson et al. 2007). 

The NEP scale consists of 15 relatively straightforward statements; eight of which show strong pro-environment (biocentric) orientation and seven of which show strong anthropocentric orientation. Responses are assessed on a standard five-point Likert scale gradient from strongly agree to strongly disagree.  We administered the NEP scale, in a pre- and post-test format, to measure changes in environmental orientation and worldview of students (NEP survey is Part 5 on SALG instrument (see Resources).   The NEP was incorporated into our assessment framework as a means to determine the influence of NCEP materials on environmental worldview of students.

Statistical Analysis
Paired-sample two-tailed t-tests compared pre- test and post- test means for each question on the content knowledge test and on the SALG survey for all respondents.  SALG questions were grouped into three categories to compare overall reported changes in learning:  Biodiversity Knowledge Confidence, Biodiversity Interest, and Biodiversity Process Skills (Table 2).  A single summary score for Overall Learning Gain in Biodiversity was created by totaling the scores of the three categories.  Paired-sample t-tests assessed differences across pre- and post- tests for each of these measures. We used Pearson product-moment correlation coefficients to assess whether change in biodiversity knowledge was correlated with change in confidence in biodiversity knowledge in individual students.

On the NEP, a positive ecological view was indicated by agreement with the odd-numbered questions (n=8) and disagreement with the even-numbered questions (n=7).  Consistent with previous analyses (Anderson et al. 2007), we subtracted the total of the odd-numbered questions from the total of the even-numbered questions to obtain a single composite measure, Ecological Viewpoint.  A paired sample t-test assessed the difference between pre- and post- test scores for this composite measure.

Multiple regression analysis was performed to determine if differences in pre- and post- scores for content knowledge and various sections of the SALG were predicted by a series of demographic and motivational variables, including school, gender, ethnicity, class standing, reason for enrolling in the class, or academic major.

Results

Demographics of Respondents and Responses
The majority of participants in the study population were first-year (45.4%) and second-year (37.0%) students. Students predominantly self-identified as African-American (56.3%), White (33.6%), or Native American (5.9%). Females represented 72.0% of the population. The majority of students surveyed expected an A grade (52.9%) or B grade (41.2%) for the course in which they enrolled. Nearly one-third of students participating in the study reported majors in the natural sciences: biology (23.5%), conservation biology (5.0%), or environmental science (3.4%). Pharmacy majors comprised the highest percentage of students (29.4%) while “Other” majors made up 22.7% of the population. In addition, 66% of students indicated that the course was required as part of their academic curriculum.

Respondents indicated that interactive, hands-on learning exercises (85.4%) were the preferred learning pedagogy, followed by lectures (74.0%), outdoor field activities (64.7%), “other” (43.7%), class discussions (40.3%), group projects (40.2%), and student presentations (27.1%).

Multiple regression analysis revealed no significant differences with respect to overall change in either the content knowledge tests or the SALG (reflecting changes in confidence in biodiversity knowledge, interest in biodiversity conservation, and confidence in biodiversity process skills), with demographic and motivational variables such as school, gender, ethnicity, class standing, reason for enrollment, or academic major.

Changes in Content Knowledge
Students significantly increased their content knowledge of biodiversity conservation between pre- and post- testing (For the 20 question Pretest mean + SE = 12.85 ± 2.56; Post test mean +  SE = 15.28 ± 3.00, t = -8.69, p<0.001).

SALG: Gains in Confidence of Biodiversity Conservation Knowledge, Interest, and Skills
Students gained significantly in their confidence in biodiversity conservation knowledge (t = 13.20, p < 0.001), showed a significant increase in interest in biodiversity conservation (t = 5.87, p< 0.001), and an increase in overall confidence in their knowledge (t = 3.14, p < 0.01) (Fig. 1). Student confidence in biodiversity conservation knowledge increased uniformly for all questions including those relating to: defining biodiversity, identifying principal threats, providing examples of how biodiversity is important to human society, describing methods and strategies used in conservation, identifying underlying issues in a conservation controversy, analyzing/synthesizing information on an issue, and critically reviewing the content quality of researched material.

There was no correlation between change in content knowledge and change in confidence in content knowledge in individual students (Pearson’s r= 0.03, p>.05, n=103).

While students reported increased interest in taking additional courses (t = 4.62, p < 0.001), exploring a major (t = 5.48, p < 0.001) or career in conservation (t = 4.64, p < 0.001), and considering lifestyle changes to support biodiversity (t = 4.92, p < 0.001), somewhat surprisingly, they did not indicate increased interest in understanding the relevance of biodiversity to real world issues (t = 1.20, p > 0.05). Since the pre-test mean (3.34) indicated better than average interest in the relevance of biodiversity, the change in post-test means (3.48) was small and insignificant relative to the high starting value.

In terms of biodiversity process skills, students reported significant gains in confidence in their skills in identifying conservation issues (t = 4.33, p < 0.001) and evaluating diverse sources of information (t = 10.25, p < 0.001).  However, students showed a significant decline in overall confidence in biodiversity related process skills (t = -2.26, p < 0.01). They reported a significant drop in confidence in oral communication (t = -5.19, p < 0.001), written communication (t = -6.96, p < 0.001), applying critical thinking (t = -2.86, p < 0.005), working in a group  (t = -5.58, p < 0.001) and coordinating a real world project (t = -2.04, p < 0.05).  No significant changes were noted in the other process skills assessed (gathering, managing and analyzing information, predicting potential outcomes, and creating/ interpreting graphs). Additional analysis to examine the potential impacts of site differences, such as the inclusion of exercises, yielded no significant differences for biodiversity confidence, interest or process skills.

NEP: Changes in Ecological World View
Student scores on the NEP pre-test indicated prevalent pro-environment attitudes. No significant difference emerged from a t-test comparison of the pre- (mean + SE = -10.75 ± 6.53) and post-test (-11.59 ± 6.59) NEP scores (t = 1.25, p = 0.22).

Discussion

For the five institutions involved in this study, the use of the three NCEP modules tested here increased both biodiversity content knowledge (based on content knowledge tests) and students’ confidence in their knowledge of biodiversity concepts (as reported in the SALG instrument). Based on this, we conclude that the modules provide an effective introduction to the topical material, and spark students’ interest in biodiversity and conservation issues. These results are consistent with other studies that showed an increase in learning gains as well as concern for the environment after an undergraduate ecology or environmental science course (Leeming 1993, Zelezny 1999, Humston and Ortiz-Barney 2007, Anderson et al. 2007). Despite the small sample size in this study, learning and confidence gains occurred in a wide range of classes, across different types of institutions, and with different amounts of time devoted to the materials, indicating that the modules are an effective tool in a wide range of settings and situations.

Interestingly, knowledge gains were not correlated with gains in confidence for individual students. In other words, individual students in this study did not demonstrate the ability to effectively gauge their own knowledge gains. Given the short amount of time devoted to module use in this study, we cannot make strong inferences from this result. However, this result does suggest that metacognitive abilities (sensu D’Avanzo 2003) are not yet strongly developed in the study population.

In this study, we did not see overall positive changes in either confidence in the development of process skills relevant to biodiversity, or changes to deeply held beliefs such as environmental worldview. One possible explanation for the significant decrease in some process skills confidence scores is that students thought themselves already capable in these areas before module use, and this may have inflated the pre-test scores. For example, the pre-test for skills in which students later reported a decline were initially scored quite high: 3.41 (oral communication), 3.86 (written communication), 3.72 (applying critical thinking), 3.91 (working in a group), and 3.22 (coordinating a real world project) on a five-point Likert-style scale. By the end of the course or unit, student perception of their ability may have declined, but actual skills may have stayed the same or increased. Such information can reflect that awareness may have increased in what these process skills entail with respect to the field of conservation.

With regard to the lack of change seen in the NEP, one probable explanation is that most students took the course in which the modules were taught because they majored in a natural sciences discipline, were interested in the topic, or were broadly exposed to the topic in the past. Thus, it is possible that students generally started the course with a tendency towards a pro-environment worldview. It is also likely that the various approaches to teaching the NCEP modules in the classroom was not sufficient to affect deeply held beliefs such as an environmental worldview. In general terms, we hypothesize that changes in worldview or significant gains in process skills are likely to require more intense or prolonged interventions or activities than those presented here.

A limitation in using an instrument such as the SALG to assess process skills and worldview changes is the need for independent data verification. The SALG and NEP are self-reporting instruments in which students assess changes in their own confidence, interest, and learning. To verify students’ self-reported process skills, NCEP is in the process of incorporating rubrics into our assessment framework to measure actual versus perceived gains in the process skills previously identified.  

In response to the results of this pilot investigation and feedback from participating faculty, NCEP has increased the number and diversity of participants in the FFG, established topical subgroups within the FFG for marine and minority-related issues, and expanded testing to include more modules and institutions.  By establishing virtual collaborative workspaces, we are also attempting to provide more support in active teaching techniques and action research methods. We have also encouraged the FFG to diversify and personalize the kinds of action research questions they undertake in the classroom, as indicated by the diversity of projects taking place in spring and fall semesters 2008.

Practitioner Reflections

Following the presentation of our results in a poster at the 2007 Ecological Society of America meetings in San Jose, NCEP convened the FFG for a debriefing and evaluation session of the project. Faculty lauded the materials in the NCEP modules as effective teaching tools from both pedagogical and practical standpoints. According to the faculty participants in the FFG, the structure of the modules—Synthesis document, modifiable Presentation, Exercises, and Teaching Notes—provided faculty with complete, easy-to-use pedagogical products that could be modified and adapted for individual classrooms. In addition, the richness and diversity of topics for the modules provided faculty with a wide range of topics from which to choose. The flexibility of the modules was seen as an advantage in that faculty could adapt material to their student population. Specific elements of the modules, such as the Presentations and Exercises, saved faculty considerable preparation time and enabled them to approach teaching topics in novel ways that further engaged students in the learning process, according to participating faculty.

The FFG indicated that the NCEP modules and assessments represented “a change in how most of (us) teach.” Because of limited familiarity with NCEP’s novel approach to teaching and assessment, many faculty new to the project mentioned the need for more time to prepare and implement the modules into their classroom. As a whole, the participating faculty experience with active learning strategies appeared to vary widely. Although most faculty members were encouraged by their home institutions to embrace active learning in their classes, it was recognized that most faculty lecture more than anything else. As one participant indicated, “switching a 50-minute period to active learning instead of lecturing takes faculty development in pedagogy, practice with a mentor, and trial and error. If the goal is active learning then we need to build faculty skills and support to do so.” However, members of the FFG felt that it was important to enhance active learning skills in current pedagogies, as “students have a very hard time taking classroom material and actually applying it in a meaningful way."

In other reflections, the on-line SALG assessment tool appeared to work well with students although there were concerns about whether all students were carefully reading the questions. Most students finished survey in about 10 minutes (some as little as 2 minutes). Surveys completed in less than three minutes should probably be considered suspect in terms of whether students read and understood the questions.

As discussed above, in this study the use of three NCEP biodiversity modules did not result in positive change in student worldview toward the environment. One faculty member registered disappointment with this by saying, “I don’t want to just teach them facts, I want them to care about the environment.”  But worldview changes may require additional elements- one suggestion from the FFG was for NCEP to develop more Exercises that personalize conservation biology and make it hit closer to home with students. Examples of such exercises include the various online ecological footprint calculators, or the excellent personal footprint calculator by P. Camill.  Another possibility is for faculty members to adapt existing NCEP Exercises to local issues and conditions. Another suggestion was the use of journals for reflective thinking about the issues and how they tie in personally for the students.

All members of the 2006-2007 FFG remained excited about the prospects for NCEP and planned to continue their involvement in the NCEP project.  For many faculty, NCEP has introduced them to pedagogical research – as one participant said, “I’m just getting into pedagogical research so I don’t really know what’s out there… But I imagine that NCEP has a very unique opportunity to do some really interesting research because of the nature of the focus group and the institutions and the variability of the courses across institutions.”

In the years ahead, the project will strive to continue to offer a service to faculty who use the modules, and to develop stronger collaborative ties with other organizations and projects committed to improving teaching of biodiversity and ecology. Having established the basic efficacy of the NCEP core biodiversity modules in a variety of contexts, we now plan more controlled application and evalu­ation of the modules using experimental and semi-experimental design. Our focus will continue to be on research that benefits and guides teachers in classroom instruction in conservation biology. We are also committed to further faculty development workshops on specific topics, such as integrating formative evaluation and addressing misconceptions in the classroom.

We are also committed to working beyond the borders of the US; in fact, NCEP is currently active in Bolivia, Laos, Madagascar, Mexico, Myanmar, Peru, Rwanda, the Solomon Islands, the United States and Vietnam, involving thousands of individuals from hundreds of institutions. In each case, modules are written or adapted to local languages and contexts. Our vision is a network of highly trained individuals effectively managing and sustaining the world’s biological and cultural diversity, and we hope to move toward this goal by working collaboratively with faculty members all over the world to improve teaching in conservation biology.

References

• Anderson, E. and D. Kim. 2006. Increasing the success of minority students in science and technology. The Unfinished Agenda: Ensuring Success for Students of Color. American Council on Education, Washington, D.C.
• Anderson, M.W., M.F. Teisl, G.K. Criner, S. Tisher, S. Smith, M.L. Hunter, S.A. Norton, J.
• Jellison, A. Alyokhin, E. Gallandt, S. Haggard, and E. Bicknell. 2007. Attitude changes of undergraduate university students in general education courses. Journal of General Education  56:149-168.
• Colker, R. M., and R. D. Day. 2004. Issues and recommendations. A conference summary: Conference on personnel trends, education policy and evolving roles of federal and state natural resources agencies. Renewable Resources Journal: 21: 6-32.
• D'Avanzo, C. 2003. Application of Research on Learning to College Teaching: Ecological Examples. BioScience 53:1121-1128.
• Dunlap, R.E. and K.D. van Liere. 1978. The new environmental paradigm. Journal of Environmental Education 9:10-19.
• Dunlap, R.E., K.D. van Liere, A.G. Mertig, and R.E. Jones. 2000. Measuring endorsement of the New Ecological Paradigm: A revised NEP scale. Journal of Social Issues 56:425-442.
• European Platform for Biodiversity Research Strategy (EPBRS.) 2006. Recommendations of the meeting of the European Platform for Biodiversity Research Strategy. http://www.epbrs.org/PDF/EPBRS-FI2006-Education(final).pdf
• Gibbs, J.P., M.L. Hunter, Jr., and E.J. Sterling. 1998. Problem-solving in  conservation biology and wildlife management. Blackwell Science, Cambridge, Massachusetts, USA.
• Humston, R and E. Ortiz-Barney. 2007. Evaluating course impact on student environmental values in undergraduate ecology with a novel survey instrument. Teaching Issues and Experiments in Ecology, Vol. 5. http://tiee.ecoed.net/vol5/research/humston/article.html.
• Kessler, W.B. 1995. Wanted: A new generation of environmental problem-solvers. Wildlife Society Bulletin 23:594-597.
• Leeming, F.C., W.O. Dwyer, B.E. Porter, and M.K. Cobern. 1993. Outcome research in environmental education: A critical review. Journal of Environmental Education 29:28-34.
• McNeal, A. and C. D’Avanzo. 1997. Student-active science: Models of innovation in college science teaching. Saunders, Sunderland, MA.
• Noss, R. F. 1997. The failure of universities to produce conservation biologists. Conservation Biology 11:1267-1269.
• Olson, S. and S. Loucks-Horsley, eds. 2000. Inquiry and the national science education standards: A guide for teaching and learning. Committee on the Development of an Addendum to the National Science Education Standards on Scientific Inquiry, National Research Council.
• Rickinson, M. 2001. Learners and learning in environmental education: A critical review of the evidence. Environmental Education Research 7:207-230.
• Ryan, M.R., and H. Campa. 2000. Application of learner-based teaching innovations to enhance education in wildlife conservation. Wildlife Society Bulletin 28:168-179.
• Shultz, S.M., A.E. Dunham, K.V. Root, S.L. Soucy, S.D. Carroll, and L.R. Ginzburg. 1999. Conservation biology with RAMAS EcoLab. Sinauer, Associates, Sunderland, Massachusetts.
• Seymour, E. and N.M. Hewitt. 1997. Talking about leaving: why undergraduates leave the sciences. Westview Press, Boulder, Colorado.
• Soulé, M.E. 1985. What is conservation biology. BioScience 35:727-734.
• Wilson, E.O. 2002. The Future of Life.  Alfred A. Knopf, New York, NY.
• Zelezny, L.C. 1999. Educational interventions that improve environmental behaviors: A meta- analysis. The Journal of Environmental Education 31:5-14.