A good service-learning course or program is shaped by data and the good administrator or faculty member cannot ever have enough data. Evaluation should be done on the course itself, the student both as a worker in the community and as an experiential learner, and on the community sites as effective or less effective placements for students from a particular course. All of this needs to be done with a careful eye on the time and effort it takes to collect data and the animosity many people have toward data collecting devices.
- Evaluating the Student as Engaged Citizen
- Evaluating the Student as Learner
- Evaluating Service-Learning Sites
- Evaluating the Course/Program
Evaluating the Student as Engaged Citizen - While some argue that students shouldn't be graded on their service work (see, e.g., Howard, 2001), we believe it is critical for all students working in communities to be evaluated in part by those to whom they report in the community. Many students will be reliable and responsible only when they feel they have a vested interest in responding to the needs of the community. This evaluation of work in the community should enter into the evaluation and grading of a student's overall performance in the course (i.e. his or her grade).
Evaluating the Student as Learner - It is hard to evaluate students on self-reports, on the degree to which they complied with the basic stipulations of their service-learning contract, or to the degree to which they engaged in the reflective process. Some reflections can be more easily graded than others in that they call upon the students to integrate observations and classroom material. But we believe that the most effective way to evaluate student learning is a summative paper at the conclusion of the term or in response to an item on the final exam. If service-learning is thought of as another course text, it is appropriate that this material be tested just as other material might be tested. The goal is for students to demonstrate their ability integrate the readings, lectures and other course material with their service-learning experiences in the community. To see our evaluation forms, please see a sample JEP Student Evaluations.
Evaluating Service-Learning Sites - The community sites at which students are placed are partners in the educational process. It is therefore important that their effectiveness be evaluated periodically so that faculty members can, with some assurance, send students out knowing that students are likely to have a positive experience. The best way to evaluate sites is by closely examining the testimony of students at that site. A certain amount of confusion, ambiguity, and broken agreements is part of the messy process we call service-learning, but patterns of problems should indicate that the site is inappropriate for student placement. We recommend that this information be combined with an annual site visit and periodic phone calls to assure that the site is an acceptable place to send students.
Some of the more common problems at sites include:
- Poor or inappropriate use of students (e.g., using students for menial tasks, rather than assignments that contribute to the student's learning process, while also benefiting the site in some way),
- Unrealistic expectations (e.g., too many hours, too much work, too complex tasks),
- Unreliability (e.g., frequent closures, absences of supervisors, changing of directions midway through the project),
- Unacceptable working conditions (e.g., demanding physical tasks, sexual harassment, Unsafe conditions).
It is important to note that these are extremely subjective matters and what may seem "inappropriate" to students may be a normal part of community life and an important part of the learning experience. Conditions that are less than ideal often serve as important points for reflection and contribute to a deeper understanding of the problems agencies and people face on a daily basis.
Evaluating the Course/Program - While we think it is important to survey students at the end of the semester to see whether or not they thought the class to be beneficial, an evaluation of a service-learning class should involve much more than that. Students generally enjoy engaging in service-learning and generally give the courses high marks even when there is a disconnect between the service and the learning. As educators, we should try to determine to what extent our original objective for the course has been accomplished (See Stated Course Objectives). If the objective in our environmental education course was to have students propose at least two new water conservation initiatives to the town council, did they do that and did their proposal indicate that they knew what they were talking about both from an environmental standpoint and a community standpoint? If the objective in our Language Development in Children course was to offer students a chance to observe and document the language acquisition of 3 to 5 year olds in a non-English speaking neighborhood, could those students, in their end-of-term papers, offer adequate documentation of the issues citing personal observations, readings and lecture material? If objectives are not met, then it is important to carefully review the structure of the course, the quality of the placements and the interplay between course material and community experience, and see what needs modification.
When conducting such an analysis, we have found that the more people who are engaged in the process, the better. A typical debriefing session might involve a faculty member, TAs, community members, any college staff members that are involved in the placement process and some representative students. These do not all need to be present at the same time but the debriefing will be richer if it involves a variety of voices and perspectives. Service-learning can be a powerful tool but requires constant analysis and tinkering to reach its potential. Our experience is that it takes about three tries to get the major bugs worked out.