Instituional Research & Assessment

An electronic publication for communication to the campus community about assessment and institutional effectiveness, the Newsletter is published by the Provost's Office.

Special Issue: SUU and the National Study of
First Year Seminars

A University goal is to improve fall-to-fall retention of first-year students. To structure itself to fulfill this goal, SUU has implemented a number of changes effective this fall semester. This issue examines the changes within the context of the National Study of First Year Seminars.

Effective July 1, 2003, SUU organized under direction of the Provost, a Student Success Center and took multiple steps to improve retention:

• Each college/school has a full-time staff adviser as a primary point of contact for advising declared majors
• Staff in the Student Success Center will deliver advising to non-declared majors
• The Student Success Center will be responsible for UNIV courses
• A Student Success course (1 credit hour) was developed as part of general education
• Dr. Brian Heuett has been asked to coordinate the course
• Faculty and academic advisers, specially trained to deliver the course, will teach classes of 25-30 students.
• University orientation, also a responsibility of the Student Success staff, was revised for an improved experience for new students
• Policies and procedures regarding declaring a major (by the end of earning 30 semester hours), “mandatory” advising, and 5th week progress reporting are being implemented
• The Noel-Levitz College Student Inventory will become part of the Student Success Course to assess initial SUU experiences of students
• The National Survey of Student Engagement will continue to assess the extent to which freshman are “engaged” in SUU’s collegiate experience

What SUU is doing is common nationally...

First-year seminars are a key feature of the first college year at over
70% of American institutions of higher education (USC National Resource
Center research).

. . . and being studied anew

While it is commonly reported that first-year seminars are among the most frequently assessed structures in American higher education, there are two serious deficiencies in the research literature on these programs. First, very few attempts at cross-institutional research have been reported – the kind of meta-analyses that could provide answers to important questions about the unique impact of various course structures. Second, it is not unusual to find assessment of grade point averages and retention, but far fewer studies have investigated broader learning outcomes of first-year seminars such as gains in skills and knowledge (writing, speaking, critical thinking, and campus specific information).

With grants from The Pew Charitable Trusts and The Atlantic Philanthropies,
The Policy Center on the First Year of College, in partnership with Educational Benchmarking, Inc., developed a benchmarking survey.

The FYI survey investigates fourteen aspects of first-year seminars. Ten factors are learning outcomes, two factors assessed the course delivery and course effectiveness, and two factors assessed the overall satisfaction with the institution and “student/institution fit”:

For those who would like to read more about the national findings from the
First Year Initiative project go to the sites below.

Engaging Pedagogy
http://www.brevard.edu/fyc/fyi/essays/essay2.pdf
How Many Weekly Contacts Hours is Enough
http://www.brevard.edu/fyc/fyi/essays/hours.pdf
What Type of Seminar is Best
http://www.brevard.edu/fyc/fyi/essays/essay4.pdf

FYI Factor Development

The FYI survey investigates fourteen aspects of first-year seminars.
Ten factors are learning outcomes:
1. Course improved study strategies (Alpha* = .91)
2. Course improved academic/cognitive skills (Alpha = .88)
3. Course improved critical thinking skills (Alpha = .91)
4. Course improved connections with faculty (Alpha = .77)
5. Course improved connections with peers/others (Alpha = .83)
6. Course increased out-of-class engagement (Alpha = .86)
7. Course improved knowledge of campus policies/procedures (Alpha = .82)
8. Course improved knowledge of campus services (Alpha = . 82)
9. Course improved managing time/priorities (Alpha = .89)
10. Course improved knowledge of wellness/ spirituality issues (Alpha = .92)

Two factors assessed the course delivery and course effectiveness:
11. Course included engaging pedagogy (Alpha = .91)
12. Overall course effectiveness (Alpha = .89)

Two factors assessed the overall satisfaction with the institution and “student/institution fit”:
13. Overall satisfaction with the institution (Alpha = .77)
14. Sense of belonging/acceptance (Alpha = .89)

*Factors are formed by averaging responses from a group of questions which are designed to explore a single construct. For example, five questions about study skill practices might be combined into a single score to create a factor on study skills. Factors provide more stable and richer information than individual questions because they are composed of multiple views of the core construct.

Alpha is a measure of internal consistency where 1.0 would be considered to be the best possible form of a factor. Generally, internal consistency at or above 0.70 is considered acceptable in social science research. The FYI factors were formed around constructs found in the research literature to matter in first-year programs and then statistically evaluated to form the final set of questions which comprise each factor.

Results have been compared to Carnegie comparison groups for inter-institutional findings. The following factors (see list of factors above) were statistically significant predictors (in order of importance) of overall satisfaction.

1. Usefulness of course readings
2. Course included engaging pedagogy
3. Course improved managing time and priorities
4. Satisfaction with college
5. Course improved knowledge of campus policies

Other factors (not predictors of overall satisfaction) were, course improved...

• study strategies
• academic/cognitive skills
• critical thinking
• connections with faculty
• connections with peers

As the National Center for the First-Year Experience repeats this study, SUU is prepared to participate to assess its own success compared to national peer institutions.