Qualitative Reviews of Graduate Programs

I. Mandate for Graduate Program Reviews

At an interval not to exceed ten years, each department or other academic organizational unit offering one or more graduate degree program(s) will undergo a review of its graduate degree program(s).  The sequence of reviews is established by the Graduate Program Review Committee (GPRC) in consultation with the Division of Academic Affairs.

Programs may request to have these periodic reviews coincide with professional accreditation reviews.  The extent to which accreditation self-study reports and the subsequent review reports and recommendations will be accepted by the GPRC will depend on their appropriateness to the purposes of the GPRC and UWM administration.  Otherwise, all qualitative reviews of continuing programs require the submission of a Program Self-Study conforming to the requirements specified herein (see Appendix A).

Where applicable, and at the discretion of the program, the Graduate School, and the Academic Program and Curriculum Committee (APCC), concurrent graduate and undergraduate program reviews will be conducted.  However, separate Self-Study reports will be prepared by the program, and separate review reports will be prepared by the external reviewers.

Based on the recommendations resulting from a review of a graduate program, the program may be subject to a follow-up review two or five years later.

The Graduate School will notify a graduate program faculty of an upcoming program review in ample time to allow for preparation of the program Self-Study document and the selection of external reviewers. 

II. Review Cycle

1. Full-scale Reviews
   1. New Graduate Programs:
      A new graduate program will undergo a full-scale review, involving site visits by external reviewers, five years after its inception.  The Self-Study for such a review will follow the format described in Appendix A, with two small differences: the Authorization to Implement document should be included as an appendix to the Self-Study; and in section I.A. of the Self-Study, where asked to address developments in the program since the previous GPRC review, the authors should instead address any differences between the current form of the program and that which was laid out in the program’s Authorization to Implement document.
   2. Continuing Graduate Programs:
      Continuing graduate programs are subject to reviews using external reviewers every ten years after initial review.  When circumstances warrant, full-scale reviews of continuing programs may be authorized by the GPRC at intervals of less than ten years.  Special reviews may be initiated at the request of the program, the GPRC, the Dean of the Graduate School, the Dean of the appropriate college or school, or the Provost.  Any special review will be formally authorized by vote of the GPRC.  The Provost’s office shall be informed of all special reviews by the Dean of the Graduate School. 
   3. Combined Graduate Program and Certificate Reviews: 
      Certificate programs are subject to review every 10 years.  When circumstances warrant, graduate certificate reviews may be combined with full graduate program reviews or follow-up reviews of the home department. 
      
      Graduate programs which are professionally accredited may opt to synchronize the submission of the report with the re-accreditation report.  Some re-accreditation cycles occur every 5 years; others are on a 7-year cycle. Programs may not regularly extend the time between UWM Graduate Program Reviews to align with accreditation cycles. For instance, a program cannot extend the time between graduate program reviews from 10 to 14 years in order to align with every other instance of a 7-year accreditation cycle. 

III. Purpose of Reviews

Graduate Program Reviews have the Following Purposes:

1. To assess the essential quality of each graduate program in terms of its faculty, students, curriculum, mechanisms for ongoing assessment, and support, and to insure the continuity of program quality. 
2. To provide the concerned governance bodies with a basis for the evaluation of proposals to expand, modify, or discontinue programs. 
3. To guide deans and the Provost in administrative decision–making and reporting related to graduate programs. 

IV. Role of Graduate Program Review Committee

The GPRC supervises a systematic and continuing review of existing graduate programs and makes recommendations about the continuance of graduate programs.  The GPRC assures that reviews and subsequent responses are presented in a timely schedule for action. 

V. Internal Review Team (IRT) 

A. Selection/Membership

An Internal Review Team (IRT) is appointed by the Chair of the GPRC and is composed of two members of the GPRC.

For concurrent undergraduate and graduate program reviews, one or more faculty members appointed by the APCC may join the IRT during the site visit, but discussion of the graduate and undergraduate programs will be separate.

B. Responsibilities

The IRT is responsible for: 

1. Being fully acquainted with the Program Self-Study, the report submitted by the external reviewers, and the program response to the report. 
2. Meeting with the external reviewers at the beginning of the site visit, attending as many meetings as schedules permit, and participating in the exit interview, when possible. 
3. Presenting the report of the external reviewers to the GPRC, identifying any inconsistencies or inaccuracies therein, advising the GPRC concerning the merits of the program response to the report, and, when necessary, reconciling the external reviewers’ report and the program’s response. The IRT may also make recommendations for modifications to the report. 

(Guidelines for the preparation of the report and the presentation are attached as Appendix C). 

VI. External Reviewers

A. Selection

At least two external reviewers who are experts from the appropriate discipline are selected by the Dean of the Graduate School in consultation with the program being reviewed.  Additional external reviewers may be appointed depending on the number of concentrations or degrees being reviewed. 

B. Responsibilities

The external reviewers examine the documents from section VII.A., below, and any other information they may request, conduct the site visit, and jointly prepare a site visit report conforming to the format outlined in Section VIII, below.  This report should be submitted to the Graduate School within six weeks of the site visit. 

VII. Site Visit Procedures

A. Information Base

At least two weeks before the site visit, the Graduate School will distribute an electronic copy of the Graduate Program Self-Study to members of the IRT, the Provost, the Dean and Academic Associate Dean of the program being reviewed, and the external reviewers.

During the site visit, the external reviewers will be given an opportunity to review course syllabi, teaching evaluations, assessment practices and data, and theses. 

B. Agenda

The site visit agenda is developed by the program being reviewed in consultation with the Graduate School.

The agenda should include meetings between the external reviewers and the following persons and groups: the Provost and/or Vice Provost in charge of communicating program reviews to the UW System, the Dean and/or Associate Dean of the Graduate School, the Dean and/or Associate Dean of the appropriate college or school with responsibility for the program budget, Graduate Program Director, Department Chair (for programs hosted in departments), tenured program faculty, untenured program faculty, students, and recent graduates.

The IRT members attend as many of these meetings as possible.

Information exchanged in these meetings is of a confidential nature.  It is important that non-tenured faculty and students be able to speak freely about the program(s) being reviewed.

The agenda concludes with an exit interview among the external reviewers, the IRT, the Dean of the Graduate School, the chair and graduate program representative of the program being reviewed.  Invitations may be extended to the Provost’s Office, and the Dean and Academic Associate Dean of the appropriate School or College. 

VIII. External Reviewers’ Report and Model Format [formerly GFC Doc. 45]

A. Report

The external reviewers jointly prepare a report to be submitted to the Dean and the Graduate School within six weeks of the site visit.

An executive summary will be written jointly by the external reviewers (approximately 1/2 page, single spaced) that includes a letter grade rating (selected from the following five categories). 

1. Continuance without Conditions: This program meets or exceeds the external reviewers’ expectations of quality standards.  The program will continue operations taking into account recommendations from reviewers.  The next full review will be scheduled in ten years. 
2. Continuance with Considerations: This program meets the external reviewers’ expectations of quality standards but with minor concerns.  The program’s response will address the reviewer’s concerns and recommendations per the guidelines in IX. Section A.  The next full review would be scheduled in ten years. 
3. Continuance with Show Cause: This program does not meet the external reviewers’ expectations of quality standards at this time.  The program has one year to submit an action plan to the Unit Dean showing cause for continuing as described in Section X.  The next review will be scheduled in five years unless the Unit Dean certifies progress.  If progress is unacceptable, program will be moved to Provisional Status. 
4. Provisional Status: Due to critical issues identified during the review process, the program is asked to suspend admissions for 1 – 2 years.  During this period, the program will be the subject of an extended internal review involving senior administrators (selected by Provost and Dean of Grad School).  To reinstate admission the program needs to show credible progress in rectification of critical issues.  Two-year and five-year internal reviews will be conducted. 
5. Discontinuance: The program should be discontinued.

In the case of a concurrent graduate/undergraduate review, the external reviewers will prepare a separate report.

For convenience, the analysis section may follow the rubrics of the program Self-Study, but the external reviewers should use their best judgment about the necessary material to be included in the body of their report. 

B. Model Format

Conclusions

The external reviewers’ report should provide general conclusions about the state of the graduate program: the need for the program, the number and quality of the faculty and their productivity, the number and quality of its students relative to its capacity, the appropriateness of the curriculum, the quality of program assessment practices, and so forth. 

Recommendations

The external reviewers are asked to provide specific recommendations for action by the faculty, school or college, Graduate School, and UWM administration, along with a statement of rationale for each recommendation.

A statement of rationale for each recommendation will be helpful to the GPRC and the UWM faculty and administration.

Analysis

Where appropriate, the external reviewers may wish to organize this section according to the rubrics of the program Self-Study:

1. The Program
   1. Description and Evaluation 
   2. Administrative Structure 
2. Faculty 
3. Students 
4. Curriculum 
5. Outcomes and Assessment 
6. Research/Scholarship Environment & Productivity
7. Resources 
8. Appendices 
9. Supplementary Information

Attention to the major strengths of the program and issues which need to be addressed are preferable to a point-by-point response to the items of the Self-Study. 

C. Distribution

Upon receipt, the Graduate School forwards the report to the Provost, the School/College Dean, and Academic Associate Dean, the Department Chair, and the Graduate Program Representative. 

IX. Preparation of the Final Review Report

A. Program Response 

The Department Chair and Graduate Program Representative are charged with sharing the external reviewers’ report with the graduate program faculty.

Within six weeks of receiving the report, the graduate program will submit a response to the Graduate School.

The purpose of this response is to correct errors of fact in the external reviewers’ report and to make any necessary clarifications.

The Graduate School will then forward the external reviewers’ report and the program’s response to the IRT.

Though the program may wish to initiate discussion of the conclusions and recommendations of the report, a detailed and specific status report in response to the charges of the review is not expected until the next program review. 

B. GPRC Actions 

The IRT presents the external reviewers report at a meeting of the GPRC.  The school/college dean or their designee and representatives of the graduate program will be invited to this meeting.  Based on the IRT’s observations and the program’s response, the GPRC may amend the report.  The GPRC will clearly document changes made to the report submitted by the external reviewers and the justification for such changes. The GPRC also will append the program response if so requested.

If the current review was conducted early because of problems cited in the prior review and the current review cites continuing problems with the program, the GPRC will make a recommendation regarding continuation of the program. 

C. GFC Actions in Case of a D or E Rating

If an Executive Summary recommends Provisional Status or Discontinuance GPRC will refer the report and its recommendations to the GFC for full discussion.

The Graduate Faculty Committee (GFC) will consider the report and recommendations of the GPRC and refer its recommendations on the graduate program(s) to the Dean of the Graduate School.

The Dean then transmits the document to the Provost (with copies to unit dean, chair, grad director, GFC and GPRC chairs), which will be based on the site visit, executive summary and recommendations, GPRC report and GFC approval. 

X. Subsequent Utilization of the External Report

Develop action plan and assess implementation.

1. Dean of individual unit meets department chair and graduate program director, and other key personnel at discretion of program; they prepare proposed action plan. 
2. Provost discusses newly reviewed programs and their action plans in their annual evaluations of individual Deans and their school/college performance.  The Provost approves the final action and resource plan for each review. 
3. The Dean and Associate Dean of the Graduate School meet annually with the Provost to understand patterns in reviewer recommendations for all newly reviewed programs and discuss program/school/college action plans including resources. 

XI. Accreditation Reviews

The Dean of the Graduate School will be apprised of all accreditation reviews of graduate programs.  If the program review cannot be coordinated with an accreditation review, the Dean will read program accreditation self-study reports, meet with accreditation teams when appropriate, receive final accreditation reports and report any concerns to the GPRC. 

• Appendix A - Format for Graduate Program Self-Study
• Appendix B - Summary of Data Tables Supplied by the Graduate School

March 17, 2022

Faculty Document No. 3197R1

September 20, 2018

No. 3197