Instituional Research & Assessment

An electronic publication for communication to the campus community about assessment and institutional effectiveness, the Newsletter is published by the Provost's Office.

Assessment During the Year 2000. The University's ad hoc committee on assessment that met during the 1998-99 academic year, recommended a number of approaches for systematic assessment of learning and evaluation of institutional effectiveness. During 2000 much has been done to put in place the committee's recommendations:

• To better understand attitudes of freshmen, the CIRP survey has been expanded to nearly 700 entering freshmen.
• To assess learning in academic programs, each department has prepared a departmental assessment plan that will be implemented during the 2000-2001 academic year. Reports on the plans will be submitted at the end of the academic year. The plans are accessible on the SUU web page.
• To understand how students are engaged in learning processes, the University is participating in the 2001 national survey of student engagement.
• To keep the campus community informed about assessment a newsletter is published each semester with news, notes and comments on assessment issues. The current newsletter and past issues are accessible on the SUU web page.
• To guide systematic assessment on campus, the ad hoc committee on assessment issued an assessment outline that has been posted on the SUU web page.
• To understand attitudes of alumni, an ACT survey of graduates was conducted.
• To understand why students leave SUU, a survey of students who have withdrawn was conducted and will be conducted again in 2001.
• To assist faculty in reviewing academic programs for prioritization processes, extensive data on demand, enrollments, graduation, SCH production, and costs was provided.
• To better compile and analyze data, a comprehensive data warehouse was designed and implemented.
• To assess graduates more thoroughly, the annual survey of graduates was changed from a local instrument to a nationally normed, ACT instrument.
• To understand public perceptions of SUU, a state polling firm surveyed a sample of citizens about SUU and other institutions of higher education in Utah.
• To more accurately understand issues related to faculty salaries, a new and more comprehensive survey design was implemented, demonstrating inequities and need.

In addition, responses to national surveys, external reports, performance indicator studies, retention analyses, accreditation reports, and other measures of institutional effectiveness were part of the University's systematic assessment effort.

Principles of Good Practice for Assessing Student Learning.

The essence of assessing learning rests with faculty in the classroom. To help individual faculty and institutions, the Association for Higher Education Assessment prepared principles of good practice for assessing student learning.

1. The assessment of student learning begins with educational values. What kinds of learning are most valued for students? Where questions about education mission and values are skipped over, assessment threatens to be an exercise in measuring what's easy, rather than a process of improving what we really care about.

2. Assessment is most effective when it reflects an understanding of learning as multidimensional, integrated, and revealed in performance over time. Assessment must employ an array of methods, and employ them over time.

3. Assessment works best when the programs it seeks to improve have clear, explicitly stated purposes. Assessment is a goal-oriented process. It entails comparing educational performance with educational purposes and expectations.

4. Assessment requires attention to outcomes but also and equally to the experiences that lead to those outcomes. Where students "end-up" matters greatly. Assessment can help us understand which students learn best under what conditions.

5. Assessment works best when it is ongoing, not episodic. Assessment is a process, not "one shot." It means track through a semester, through a program, and beyond into careers. Along the way, the assessment process itself should be evaluated and refined as needed.

6. Assessment fosters wider improvement when representatives from across the educational community are involved. Student learning is a campus-wide responsibility, and assessment is a way of enacting that responsibility. Assessment is a collaborative activity engaging the whole campus.

7. Assessment is most likely to lead to improvement when it is part of a larger set of conditions that promote change. The contribution of assessment comes where the quality of teaching and learning is visibly valued and worked at. Academic rigor and the push to improve educational performance is a goal of assessment and of the effectiveness of the institution.

8. Through assessment, educators meet responsibilities to students and to the public. The public and our students depend on the institution to meet our goals and published expectations. But there is a deeper obligation to continuously improve.

Fact Book Published

Each year, information for campus leaders is published in a Fact Book of statistics on enrollments, students, finances, faculty and staff, and other institutional information. More and more improvements to data accuracy have resulted from the data warehouse project that greatly enhances the reliability and access to information. Thanks to Jeanette Ormand and Bonny Rayburn for their tireless work on this project, and to Bonny, again, for preparation of the Fact Book for 2000-2001.

The Fact Book for 2000-2001 will be published in March 2001. Two key reasons have caused the publication date to move from January to March. First, the Integrated Postsecondary Education Data System (IPEDS) of the U.S. Department of Education has changed its reporting methodology and deadlines. The methodology is now web based with two collection points, one in the fall and one in the spring. We must have data from both collections to complete the Fact Book. Second, the Commissioner's office has changed the date that the system data book is issued. We now receive it in early February. Elements of the data book are necessary for our own local Fact Book to be as up-to-date as possible.

Policy on Educational Assessment

Commission on Colleges

Northwest Association of Schools and Colleges

Ten years ago, the Commission issued a policy statement on educational assessment. The policy's questions are extracted here and worth re-examining in departments, schools and colleges.

A list of outcome measures used in appropriate combinations and informed by the institutional mission could yield an efficacious program of outcomes assessment. This list is intended to be illustrative and exemplary as opposed to prescriptive and exhaustive.

A. Student Information

From what sources does the institution acquire its students? What percentage directly from high school? Community college transfers? Transfers from other institutions? What blend of gender, age group, and ethnicity has the institution attracted over time? Retained over time? Graduated over time? What is the mean measured aptitude, over time, of entering students? What are the local grade distribution trends? What changes have appeared over time?

B. Mid-Program Assessments

If the institution has some kind of required writing course or an emphasis on writing across the curriculum, what evidence is there that students are better writers after having been exposed to the course or curriculum? How are these judgments rendered? If student writing improves, do students appear to retain this newly acquired proficiency? If so, why, and if not, why not? What changes are planned as a result of the assessment exercise?

A required course, program or sequence in mathematics can be assessed in a similar fashion. What evidence is there that the skills improved or declined as a result of the program? How are these judgments rendered? Does the improvement appear permanent or transitory? How has the program been changed as a result of the assessment program?

A required course, program or sequence in any subject matter can be addressed in a similar fashion, as can nearly any part of the program in general education or the program as a whole.

C. End of Program Assessment

What percentage of those students who enter an institution graduate? Is the percentage increasing or decreasing? Why? What is the mean number of years in which students graduate? Is that mean increasing or decreasing? Why? What are the criteria for these judgments? What is the several-year retention pattern from one class to the next, such as freshman to sophomore? If patterns reflect significant losses between one level and another, what are the reasons? Similar questions may be asked by gender and/or ethnic background. If the institution or program requires a capstone experience at the end of the curriculum, are present students performing better or worse then their predecessors? What are the reasons? What are the bases for the judgments? (E.g. "The cumulative judgment of the faculty is that the quality of the senior theses in art has improved during the past five years. This judgment is based upon the following evidence. . ." or "The Psychology Department requires the advanced test on the Graduate Record Examination of all graduates. These scores have declined by an average of 2% each year for the past five years. The faculty is of the opinion that the reasons for this decline are. . . .")

D. Program Review and Specialized Accreditation.

Some institutions require periodic program review of each academic program, either through an institutionally approved internal process and/or through seeking and achieving specialized accreditation, or by utilizing external experts. Either or both of these activities can provide a wealth of outcomes assessment data, particularly if the methodology remains somewhat standardized over time.

E. Alumni Satisfaction and Loyalty.

A number of institutions engage in a variety of alumni surveys which elicit, over time, the judgments of alumni of the efficacy of their educational experience in a program or at an institution. Use of such a mechanism can assist an institution in understanding whether alumni satisfaction with various aspects of the educational program, particularly those facets which the institution stresses, appears to be growing or diminishing over time. If satisfaction is increasing, why? If decreasing, why? What are the bases for the judgments? What curricular implications do these findings have?

F. Dropouts/Non-completers.

What methods has the institutional utilized to determine the reasons why students drop out or otherwise do not complete a program once they have enrolled in it? What is the attrition rate over the past five years? Is it increasing or decreasing? What are the reasons? What programs or efforts does the institution engage to enhance student retention? Which tactics have proved to be efficacious?

G. Employment and/or Employer Satisfaction Measures.

One relatively straightforward outcomes measure used by some institutions concerns that number and/or percentage of former students who have sought and found employment. Are they happy with what they have found? Do they think the program prepared them well for their chosen occupations? If trained in a particular area, teacher education, for example, have they found a teaching position?

Other institutions have found qualitative comments of frequent employers to be particularly helpful in assessing educational outcomes. Do the employers regularly recruit program graduates? Why or why not? How well do program graduates perform in comparison with graduates from other similar programs? Are there areas of the curriculum in which program graduates are particularly well prepared? Which areas? Why is preparation judged to be particularly good? Where are the weaknesses? Why? What is being done to provide remedial activity?