Self-Study and External Review Process
A department or program self-study and external review usually occurs every ten years. It includes a self-study by the department, followed by a visit from three external reviewers, and the process of selecting the outside reviewers begins as early as one year before the external review in order to be able to schedule qualified reviewers. Approximately one month after the external review, the reviewers submit their report to the Dean, who then forwards it to the department chair/program coordinator and meets with the department/program to discuss the contents of the report. One year later the Department Chair/Program Coordinator makes a brief follow-up report to the Dean, outlining how the department/program has responded to the external reviewers’ recommendations, and discusses with the Dean any further actions.
The budget for each self-study is $7000. Most of this amount will be used to cover the travel expenses and honoraria for the outside reviewers, whose standard fee is $1000 apiece. The rest can cover other costs, such as photocopying, mailing, and meals or space rental for a retreat. Although the Dean’s Office will arrange for travel and lodging, and issue the three honoraria directly, individual departments/programs manage the other expenses as they see fit.
Selecting the outside reviewers
The chair/coordinator sends a list to the Associate Dean with the names, titles, institutions and email addresses of six to eight potential outside reviewers, ranked in order of preference and grouped according to their specific niches. These names need to be discussed and approved by all continuing members of the department before being submitted. The Associate Dean, in consultation with the Dean, selects three external reviewers from this list. The Dean and Associate Dean may decide to select an individual not on the list, but will discuss this with the department or program before doing so. The three external reviewers will normally include at least one person from a comparable liberal arts college and at least one person from a research university that offers PhDs in the pertinent discipline. The latter is able to provide feedback on how well we are preparing our students for further study; the former to appreciate the unique character of liberal arts colleges. The three reviewers should also include at least one person who is or has been a department chair.
The department/program will indicate the dates it would prefer the outside reviewers to visit campus. The Associate Dean will ascertain the availability of the Dean and the President on the proposed dates and contact the reviewers to determine their willingness to serve on these dates. Once the reviewers have accepted, the Dean’s Office is responsible for arranging their travel and housing. The department/program arranges the on-campus scheduling for the two-day visit.
The Self-Study Document
The department/program creates a report that it will give to the President, the Dean, and the Associate Dean, and the external reviewers no later than two weeks before the external review. Generally the chair/coordinator is responsible for overall organization, but is not the sole author of the self-study; the writing of the self-study is a shared enterprise among department members. Student liaisons may be enlisted to help gather data from students and alumni. Early in the process the department/program will set aside time for intensive discussion of the self-study issues, either as a one- or two-day retreat or over a series of meetings. To facilitate discussion, the chair may want to distribute a written set of possible topics or issues for the self-study to all faculty before the initial meeting.
Core information in a department/program self-study document
- A history of the department/program, including changes in faculty, facilities, etc;
- A statement of the department/program learning objectives;
- An assessment of student learning in the major based on actual student work, not student opinion, in light of the department/program learning objectives (could be a review of the department’s recent annual assessments of student learning in the major);
- Data on enrollments and majors, disaggregated by gender and ethnicity (the Registrar will supply data for the department/program to analyze);
- Feedback from and profiles of current students and alumni obtained from surveys taken early in the self-study process;
- Comparative data from departments or programs at similar colleges;
- Faculty profiles, such as CVs or biographies, including teaching, research, and service;
- The department's tenure and promotion standards;
- Catalog copy and course syllabi;
- A history of recent financial support for faculty and students, including grants received for research, travel, and senior projects.
Sample issues or questions for self-studies
The following questions may spark further discussion among colleagues in your department/program. They are only suggestions, not requirements.
Teaching and learning
- How do the learning goals of the department/program relate to the institutional learning goals of the College?
- How does the department/program determine the learning goals are being met with its current curriculum? What kinds of evidence are used?
- How does the department/program use this evidence to evaluate and improve its curriculum?
- What sorts of active learning opportunities does the department/program offer students?
- What sorts of faculty/student collaboration does the department/program support? How effective is this collaboration?
- How appropriate is the senior exercise, given the department/program goals? How does it help students integrate the information, concepts, and skills that they have learned?
- Are there striking ethnic, racial, and/or gender disparities among majors and non-majors taking courses in the department/program? What can be done to address them?
- How coherent are the major and minor requirements?
- How well-prepared are majors for graduate study? How successful are majors in getting into graduate school? What other post-graduate alternatives do the majors pursue?
- How do department/program offerings help all students lead satisfying, productive lives? Does the curriculum offer courses that provide useful skills and knowledge for students who will take only one or two courses in the concentration?
- What courses outside of the department/program do faculty advisors encourage students to take?
- How fully does the major take advantage of courses at the other Claremont College Colleges?
- To what extent do professors find their teaching at Pomona College satisfying? How can the department/program help faculty members continue to improve their pedagogy or develop new areas of teaching expertise?
- How appropriate is the faculty staffing given the curricular goals and enrollments? To what degree are students taught by regular and continuing members of the faculty? Are resources available at the other Claremont Colleges that can help the department/program achieve its curricular goals?
- How are the department/program goals hampered or facilitated by its physical facilities?
- In what ways does the department/program facilitate research productivity among faculty members?
- How well is the department/program functioning? Is there shared governance, or do a minority of faculty or even just the chair make most decisions? Are there written guidelines for department/program governance?
- How does the department distribute or delegate responsibilities among individual faculty members? How is leadership encouraged and developed across the department/program?
- How does the department distribute resources among individual faculty members?
- How are junior faculty members mentored with respect to their teaching, scholarship, and service? Are information and expectations communicated effectively?
- Are there sufficient opportunities for the department/program faculty to interact with one another and share experiences (through both formal meetings and informal interactions)? Is there potential for better interdepartmental and intercollegiate cooperation and complementarity?
- What are administrative or technical staff needs within the department/program and how well are they being met?
Resources for self-studies
- National studies of general curricular issues for liberal arts colleges in general.
- Studies by national organizations in your discipline or field.
- Self-studies conducted at Pomona by other departments (permission is required from the department to view the self-study document).
- The Director of Institutional Research, Jennifer Rachford, who can offer advice on strategies for data collection, provide technical support for web-based surveys (of alumni and students) and supply existing data from institutional surveys to inform the review where appropriate. The Senior Survey and the Enrolled Student Survey cover many topics relevant to departmental self-studies.
The External Review
Reviewers will spend a minimum of two days here, normally either a Monday-Tuesday or a Thursday-Friday, arriving the evening before the first day of the review. The review team meets with the Dean and the Associate Dean at the beginning of the first day of their visit and meets with the President and Dean for an exit interview in the late afternoon on the second day. After this exit interview, the reviewers should have time to discuss their written report (and no further contact with department members). Over the two-day visit the reviewers should meet with all continuing department/program faculty, support staff, as well as individuals in joint or related departments or programs – including those at the other Colleges – so that they can get the fullest possible picture of the department/program being reviewed. Division II departments may wish to have the reviewers meet with the Associate Dean who oversees research support. The reviewers should meet with students, and they may visit one or more classes and see any significant resources, such as laboratories or libraries. Reviewers should have time to read or view student work, such as senior exercises or class projects. Many departments/programs find it helpful to have a reception or dinner for the reviewers and the entire department/program on the evening of the first day of the visit. The chair/coordinator is responsible for putting together the two-day schedule of the reviewers’ visit, in consultation with the Associate Dean.
After digesting the reviewers’ report, the department/program meets with the Dean and Associate Dean to discuss it. The chair/coordinator should contact the Dean’s Office to arrange this meeting, normally within a month of receiving the report.
A year after the external review, the department/program submits a brief follow-up report to the Dean, with a copy to the Associate Dean. The report outlines which recommendations have been implemented and why, which have not been implemented and why, what impact the changes have had, and where the department/program intends to move in the future. The report can also provide an opportunity to remind the administration of any steps it should be taking. The chair/coordinator meets with Dean and Associate Dean to discuss the follow-up report, helping ensure a productive outcome to the self-study.