Indirect Assessment Methods

Updated March 2015

Curriculum Mapping

This is a tool that we requested that all departments use because it is so important for knowing in which courses a department should make changes if it finds that a learning goal is not being successfully met. It also helps departments to identify courses in which to conduct the direct assessment of student work that meets a particular learning goal.  Additionally, it can be informative in leading to the realization that something that a department deems important is not being addressed adequately in the curriculum.  Biology used this process early on, in conjunction with transcript analysis (see below) to understand whether students were moving through the major in a way that exposed them adequately to courses that addressed each learning goal. They will use the mapping that they have done in conjunction with transcript analysis to track what specific skills and content each student in the major is exposed to and analyze this in conjunction with their direct assessment. Anthropology has combined mapping to identify where in their curriculum they meet particular goals.  They have combined this analysis with results from their Senior Survey to look at the extent to which students feel they have mastered content and skills offered in different level courses.  Performance and Communication Arts, Sociology, and Religious Studies have also used mapping as part of their assessment efforts. 

Surveys

Departments have used surveys of either current students or alumni in order to gather indirect evidence of student learning, as well as to understand how students view and what they have learned in their major after they graduate.  The Anthropology Department has used surveys of sophomores, juniors, and seniors to ask whether they feel that they have met each of the department’s ten learning goals.  Similarly, Environmental Studies and Sociology do a survey each year with seniors.  The Anthropology experience suggests that such survey data is most useful if it is done by all majors and if the department has also mapped its learning goals onto the curriculum, since this combination allows the department to know what goals students should meet given their progress through the major and provides a more precise way of making adjustments if students do not feel that they are mastering goals.  

Additionally, students’ perception of their learning can be looked at in conjunction with direct assessment of student learning.   Psychology has, in fact, combined the two in a single survey.  Last year, they added a pilot direct assessment section to the survey enabling them now to look at both indirect and direct measures for the same student.  Psychology is in particular interested in the extent to which successfully answering these questions correlates with the number and types of courses taken; one particular area of focus is the role of the Research Methods lab versus other upperclass lab courses. 

In departments that have learning goals such as “Students should be encouraged to take an active interest in political life and to develop the habits of intellectual curiosity, self-reflection, and open-mindedness that are the hallmarks of lifelong learning” (Government) or “Students will learn skills that lead to success in their professional life” (Mathematics, Computer Science, and Statistics) or “Students will develop a habit of continuing to explore the questions, ‘Who am I?’ ‘Who are we?’  ‘Who do I/we want to become?’” (Philosophy), indirect assessment from alumni is critical for evaluating outcomes.  Given the importance of peer relationships and student-faculty relationships for student engagement and learning in college, surveys can also assess the extent to which the departmental fostered these relationships, as the Government survey will do.

The First-Year Program uses a supplemental survey instrument in addition to the University-wide course evaluations.  Questions on the supplemental survey have all been organized around assessing the program’s major learning goals (such as students’ confidence in their writing ability, oral communication skills, research skills, peer learning, etc.).

There is also survey data from national surveys that may provide significant insights into student learning.  For example, we have used the Global Perspectives Inventory to assess diversity learning in abroad programs.  Though we were not trying to use it as part of departmental assessment, we believe that it is useful for MLL, Global Studies, Anthropology, and other majors that have defined as goals increased understanding of human diversity and cultures.  As we move forward, we will need to see whether changes to this survey make it as useful as it has been in the past.  The Research Practices Survey combines direct and indirect assessment.  Additionally, for large departments, we may be able to use HERI and NSSE in disaggregated ways that will allow us to understand how students in large programs understand their learning. 

Transcript Analysis

Transcript analysis can very useful for majors where students have a fair amount of choice in how they complete their major.  When combined with mapping, as it was by Biology, it can be a useful tool for identifying the frequency with which students take courses that meet particular learning goals. Again, a follow-up study could then explore how effectively courses that are taken by large numbers of students to meet a particular goal or goals are actually teaching the content or skills necessary to achieve the goal.  Biology will be using this analysis with direct assessment results. 

Program Review

The Physics Department used an American Association of Physics Teachers document as a guide for assessing their program.  This sort of internal self-study can be beneficial.  Though we do not mandate regular program reviews, this is an option that departments might consider when planning significant curricular changes.

Assessment of non-course parts of a curriculum

The Music Department was concerned by the fact that many of their students participate in ensembles that were not evaluated.  They developed an evaluation instrument that work for these experiences.  Though a significant portion of this instrument evaluates instruction, there is a question that asks students to reflect in how the experience helped them to develop as a musician. The Biology Department uses a survey instrument to evaluate the general biology lab courses and uses the results to improve the labs as well as the peer pod component.  The Sociology Department senior survey also asks students to evaluate how the experiential parts of the curriculum contributed to their major.