Portland State University Research Update

Office of Institutional Research and Planning

Volume 6, Issue 3, Fall 2002

The National Survey of Student Engagement (NSSE) assesses the institutional learning environment for undergraduates. In order to represent students at the early and late stages of their college careers, the survey sample at each participating school includes both first-year students and seniors. Portland State University (PSU) participated in the NSSE during winter and spring of 2002.

The overall response rate across institutions participating in the NSSE was 41%. At PSU, 241 of the 700 students included in the sample responded to the survey, with a response rate of 36%¹. This report summarizes the PSU survey results for freshmen and seniors and compares them to results at other urban universities² (UUs) and at other doctoral/research intensive universities³ (DRIUs). All reported differences are statistically significant (p < .05). The Executive Summary includes a brief overview of the findings organized in relation to major initiatives currently underway at PSU. A detailed report of the findings follows the Executive Summary.

Executive Summary:

Satisfaction. PSU students were generally satisfied with their educational experience, but seniors at PSU rated the quality of their education somewhat higher than other DRIU students did. PSU students were as likely as other UU and DRIU students to indicate that they would attend the same institution if they had the chance to start over again. PSU students rated their relationships with instructors, students, and administrative personnel as somewhat friendly, helpful and supportive.

Advising: PSU students rated the quality of advising at PSU as fair to good, however, the average ratings of PSU seniors were lower than those of other DRIU seniors. In addition, PSU students reported that they talked with faculty members or other advisors about career plans less frequently than DRIU students did.

PSU freshmen felt they received less support than other DRIU freshmen received for succeeding academically and socially. In contrast, PSU seniors felt they received more support than other DRIU and UU seniors did for coping with non-academic responsibilities.

Diversity. In general, PSU students, especially freshmen, reported having frequent discussion with students who hold diverse religious beliefs, political opinions, or personal values, or who are from diverse racial or ethnic backgrounds. PSU seniors engaged in such conversation about as frequently as other UU or DRIU seniors did, but PSU freshmen were generally more likely than other freshmen to converse with students who hold diverse opinions or who come from diverse backgrounds. PSU students reported frequently incorporating diverse perspectives (different races, religions, genders, political beliefs) into class discussion or assignments. In particular, PSU seniors incorporated diverse perspectives more often than other seniors did.

Few PSU, other UU, or DRIU students planned to participate or participated in study abroad or foreign language courses. PSU students were more likely than UU or DRIU students to take a foreign language course. In addition, more PSU freshmen than other UU or DRIU freshmen studied or planned to study abroad. PSU students felt that their education had contributed to their understanding of persons from diverse racial or ethnic backgrounds "quite a bit". In general, PSU students were more likely than other students to attribute their understanding of diversity to their education.

PSU students also indicated that PSU offers "some" to "quite a bit" of encouragement for contact among students from different economic, social, and racial or ethnic backgrounds. PSU students were more likely than other students to report that their institution encouraged contact among diverse students.

Community-based experiences. PSU seniors participated in community-based projects more often than other UU or DRIU seniors did. More PSU students than UU students participated or planned to participate in community services or volunteer work. In addition, PSU students reported participating or planning to participate in culminating senior experiences at higher rates than either DRIU students or UU students. The culminating senior experience at PSU is Capstone, which includes a community-based learning component. Please note that, although PSU students had more community-based experiences than students at peer institutions did, PSU students' participation was occasional relative to other learning activities in which they participated more frequently (e.g., integrating ideas from many sources for assignments, contributing to class discussions, and preparing multiple drafts of papers).

General education outcomes. The most frequent learning activity of PSU students was integrating ideas and information from many sources in papers and other projects. In general, freshmen at PSU, other UUs and DRIUs read books and wrote papers with similar frequency, but PSU seniors read and wrote more than UU and DRIU seniors did.

PSU students indicated that their college education had contributed most to their broad-based general education and their ability to think critically and analytically. Students at PSU, other UUs and DRIUs reported that their coursework emphasized critical thinking at similar levels; however, PSU freshmen were less likely than UU and DRIU freshmen to report that their coursework emphasized applying theories or concepts to practical problems or in new situations.

Respondents used a 4-point scale (1 = poor, 2 = fair, 3 = good, and 4 = excellent) to rate their overall satisfaction with the entire educational experience at the institution. On average, PSU students rated their experiences as good (M = 3.02). Students also answered the question, "If you could start over again, would you go to the same institution you are now attending?" Using a 4-point scale (1 = definitely no, 2 = probably no, 3 = probably yes, and 4 = definitely yes), PSU students responded that they probably would attend PSU again (M = 2.96).

For both of these items, the responses of PSU freshmen and seniors were similar to each other and to those of other UU students. Although PSU and DRIU seniors did not differ in their ratings of their educational experiences, PSU freshmen rated their overall experience lower than DRIU freshmen (see Figure 1). PSU and DRIU students were similar in their certainty that they would attend the same institution if they could start over (see Figure 2).

Quality of Advising

Respondents rated the overall quality of academic advising they received at PSU on a 4-point scale (1 = poor, 2 = fair, 3 = good, and 4 = excellent). In general, students indicated that PSU advising was fair to good (M = 2.60). PSU seniors and freshmen were similar in their ratings. As shown in Figure 3, freshmen rated the quality of academic advising at PSU lower than at other DRIUs and other UUs. Seniors at PSU were similar with those at other DRIUs and higher than those at other UUs in their ratings of academic advising.

Frequency of Active Learning Experiences

Respondents indicated how often they participated in active learning experiences using a 4-point scale (1 = never, 2 = sometimes, 3 = often, and 4 = very often). PSU students worked most often on papers or projects that required the integration of ideas and information from various sources (M = 3.26). Another frequent classroom activity was incorporating diverse perspectives (different races, religions, gender, political beliefs) into class discussion or assignments (M = 3.04). Other frequent learning activities that involved interaction with faculty included discussing grades or assignments with instructors (M = 2.48), working harder than students thought possible to meet instructor standards or expectations (M = 2.52), and receiving prompt feedback from faculty on academic performance (M = 2.60). Frequent forms of interaction with other students included conversing with students from diverse ethnic backgrounds (M = 2.70) or with

students who hold diverse religious or political beliefs and opinions or values (M = 2.74). In addition, PSU students often used e-mail to communicate with instructors (M = 2.84) and various forms of electronic media to discuss or complete course assignments (M = 2.72). Activities in which PSU students reported engaging never or occasionally included working with faculty members on activities other than coursework (M = 1.36) and tutoring or teaching other students (M = 1.51). As shown in Figure 4, PSU seniors indicated that they engaged in a variety of active learning experiences more frequently than freshmen did. In particular, seniors more frequently participated in community-based projects as part of their regular courses, talked about career plans with a faculty member or advisor, and integrated ideas from other courses into class discussions or projects. Freshmen prepared multiple drafts of assignments and papers more frequently than seniors did.

PSU seniors differed from other DRIU seniors on nine active learning items. PSU seniors engaged more frequently than DRIU seniors in community-based learning projects as part of regular coursework and in activities involving the integration of diverse ideas or information from a variety of sources. In addition, PSU seniors were more likely than DRIU seniors to include diverse perspectives in class discussion and writing assignments and to discuss readings with others outside of classes (see Figure 5). PSU seniors were less likely than DRIU seniors to interact with faculty outside of class time as shown in Figure 6. In addition, PSU seniors (M = 1.49) were less likely than DRIU seniors (M = 1.78) to tutor or teach other students.

PSU seniors differed from other UU seniors on four active learning items. PSU seniors were more likely than UU seniors to include diverse perspectives in class discussion or writing assignments (M = 3.02 and 2.67, respectively), participate in community-based projects as part of regular courses (M = 1.87 and 1.46, respectively), and discuss ideas from readings or classes with others outside of class (M = 3.00 and 2.77, respectively). PSU seniors were less likely than UU seniors to tutor or teach other students voluntarily or for pay (M = 1.49 and 1.70, respectively). As shown in Figure 7, PSU freshmen differed from both DRIU and UU freshmen in the frequency of eight of their learning activities. In particular, PSU freshmen were more likely than

other freshmen to include diverse perspectives in their class discussion or projects, but they were less likely to talk about career plans with an instructor or advisor. For a few active learning experiences, PSU freshmen differed from only DRIU or UU freshmen. PSU freshmen were more likely than DRIU freshmen to prepare multiple drafts of papers or assignments (M = 2.68 and 2.90, respectively), but less likely to discuss grades or projects with instructors (M = 2.53 and 2.38 , respectively) or work with them on activities other than coursework (M = 1.47 and 1.26 , respectively). Compared to UU freshmen, PSU freshmen participated in community-based projects as part of regular coursework (M = 1.28 and 1.46, respectively) and conversed with students holding diverse beliefs and opinions more often (M = 2.61 and 2.79, respectively).

Reading and Writing

Respondents rated the amount of reading and writing they accomplished during the previous academic year using a 5-point scale (1 = none), 2 = fewer than 5, 3 = between 5 and 10, 4 = between 11 and 20, and 5 = more than 20). On average, PSU students read between five and ten assigned textbooks, books, or book-length packets of course readings (M = 3.52), whereas they read fewer than five unassigned books on their own (M = 2.34). PSU students wrote brief papers (i.e., fewer than 5 pages in length) most frequently (M = 3.20). They wrote between five and ten short papers or reports (i.e., between 5 and 19 pages in length) (M = 2.61) and fewer than five long papers or reports (i.e., 20 or more pages) (M = 1.47). Seniors (M = 2.84) wrote more short papers than freshmen (M = 2.37). They also read more assigned books (M = 3.64 and 3.40, respectively) and books for personal enjoyment (M = 2.50 and 2.16, respectively) than freshmen did.

Freshmen at PSU did not differ from freshmen at other UUs in the amount of reading and writing they accomplished, however, PSU freshmen (M = 1.39) wrote more long papers (i.e., 20 or more pages) than freshmen at DRIUs (M = 1.22) wrote. Otherwise, freshmen at PSU and other DRIUs were similar in their amounts of reading and writing. As shown in Figure 8, PSU seniors read more assigned and unassigned books than DRIU or UU seniors. PSU seniors also wrote more papers fewer than 20 pages in length than other UU or DRIU seniors. Although it appears in Figure 8

that PSU seniors write fewer papers of 20 pages or more in length when compared to UU or DRIU seniors, the difference is not statistically significant. PSU students also described how much their exams challenged them to do their best work using a 7-point scale (1 =Very little) to 7=Very much). PSU seniors (M = 5.58) were more likely than PSU freshmen (M = 5.18) to find their exams challenging. PSU seniors were similar to DRIU and UU seniors in their ratings of exams. PSU freshmen (M = 5.18) found their exams less challenging than either UU (M = 5.44) or DRIU freshmen (M = 5.51) did.

Critical Thinking

>Respondents used a 4-point scale (1 = very little, 2 = some, 3 = quite a bit, and 4 = very much) to estimate how much their coursework emphasized five mental activities corresponding to levels of critical thinking skill. As shown in Figure 9, PSU students indicated that "quite a bit" of their coursework emphasized analyzing the basic elements of an idea, experience, or theory. The coursework of PSU seniors and freshmen differed in how much it emphasized critical thinking skills. Senior coursework was more likely than freshman coursework to emphasize analyzing the basic elements ideas (M = 3.31 and 3.12, respectively), synthesizing and organizing ideas into more complex interpretations (M = 3.02 and 2.80, respectively), and applying theories or concepts to practical problems (M = 3.03 and 2.68, respectively).

The emphasis of critical thinking was generally similar for PSU and other freshmen; however, PSU freshmen (M = 2.68) were less likely than either UU (M = 2.91) or DRIU (M = 2.96) freshmen to apply theories or concepts to practical problems or in new situations. The critical thinking activities of PSU seniors were similar those of seniors at other UUs or DRIUs.

Learning Outcomes

Respondents rated the extent to which their college education contributed to their knowledge, skills, and personal development on a 4-point scale (1 = very little, 2 = some, 3 = quite a bit, and 4 = very much). PSU students indicated that their college education contributed the most to their ability to think critically and analytically (M = 3.11) and to their broad, general education (M = 2.97). Students also rated PSU education highly for contributing to their ability to write clearly and effectively (M = 2.89), learn effectively on their own (M = 2.87), and work effectively with others (M = 2.86). PSU seniors more than freshmen felt that their college education had contributed to their writing ability (M = 3.02 and 2.75, respectively), work-related knowledge and skills (M = 2.77 and 2.27, respectively), understanding of themselves (M = 2.94 and 2.70, respectively), and ability to solve complex real-world problems (M =2.60 and 2.34, respectively). PSU students indicated that their college education had contributed the least to helping them fulfill their civic duty by voting in elections (M = 1.58) and improving the welfare of their community (M = 2.21). These results do not mean that PSU students lack a sense of volunteerism or civic duty; instead, these results indicate that students did not feel that PSU contributed to these aspects of their personal development, regardless of individual levels of growth in these areas.

The ratings of PSU freshmen were generally similar to those of other UU and DRIU freshmen, but a few differences were present. PSU freshmen (M = 2.27) were less likely than DRIU freshmen (M = 2.48) to attribute gains in work-related knowledge and skills to their education, but they (M = 2.77) were more likely than DRIU freshmen (M = 2.59) to indicate that their education had contributed to their understanding of people of other racial and ethnic backgrounds. PSU freshmen (M = 2.14) were also more likely than UU freshmen (M = 1.92) to attribute their contributions to the welfare of their communities to their education.

More differences between PSU, UU, and DRIU students were present in the senior data than in the freshman data. As shown in Figure 10, both UU and DRIU seniors were more likely than PSU seniors to attribute gains in analyzing quantitative problems to their college education. In contrast, they were less likely than PSU seniors to attribute their understanding of people of other racial and ethnic backgrounds to their education. In addition, DRIU seniors were more likely than PSU seniors to attribute gains in work-related knowledge or skills to their education. In comparison to UU seniors, PSU seniors were more likely to indicate that their college education had contributed to an understanding of themselves and increased their contributions to the welfare of their communities.

Enriching Educational Experiences

The survey items pertaining to enriching educational experiences asked students to report past or expected future participation. Therefore, seniors were more likely to report past actual behavior and freshmen were more likely to report a mix of past actual behavior and predicted future behavior on these items. The majority of PSU students reported that they participated or expected to participate in culminating senior experiences (69%); practica, internships, or field experiences (61%); and community service or volunteer work (65%) before graduating. Fewer PSU students participated or expected to participate in independent study or self-designed majors (20%), research projects with faculty members outside of course or program requirements (17%), or study abroad programs (26%). PSU freshmen were more likely than seniors to be undecided about whether they would undertake special coursework.

More PSU students than other UU students or DRIU students planned to participate or participated in a senior culminating experience (see Figure 11). A similar pattern held true for foreign language coursework: more PSU students than UU or DRIU students planned to study or studied a foreign language (see Figure 12). More PSU students (71% of freshmen and 59% of seniors) than UU students (58% of freshmen and 48% of seniors) participated or planned to participate in community service or volunteer work. More PSU freshmen (40%) than other UU (24%) or DRIU freshmen (29%) studied or planned to study abroad. More PSU freshmen (35%) than UU freshmen (27%) participated or planned to participate in a learning community. Fewer PSU seniors (55%) than DRIU seniors (70%) enrolled or expected to enroll in practica, internships, field experiences, or clinical assignments.

Student use of time is likely to be related to the number of classes in which they enroll; the NSSE samples included both full- and part-time students. Possible differences in student load within groups, between class levels, and among PSU, other UUs, and DRIUs were not factored into the reported responses, but should be kept in mind when interpreting the results.

Students used an 8-point scale (1 = 0 hrs, 2 = 5 or fewer hrs, 3 = 6 - 10 hrs, 4 = 11 - 15 hrs, 5 = 16 - 20 hrs, 6 = 21 - 25 hrs, 7 = 26 - 30 hrs, and 8 = more than 30 hrs) to estimate the hours they spend during a typical week engaging in a variety of activities. On average, PSU students spent the most hours working for pay off campus (M = 4.19) and preparing for class (M = 4.15). They spent the fewest hours working for pay on campus (M = 1.35) and participating in co-curricular activities (M = 1.44). PSU seniors were more likely than freshmen to spend time working for pay on campus (M = 1.49 and 1.21, respectively) and off campus (M = 4.66 and 3.70, respectively), preparing for class (M = 4.47 and 3.82, respectively), and providing care for dependents living with them (M = 2.59 and 1.83, respectively).

As shown in Figure 13, PSU seniors spent less time than DRIU seniors participating in co-curricular activities and working for pay on campus and more time commuting to class. In addition, PSU seniors spent more time than either DRUI or UU seniors (M = 3.86) preparing for class.

Figure 14 shows that PSU freshmen spent more time commuting to class and working for pay off campus when compared to DRIU freshmen. PSU freshmen worked for pay on campus less frequently than either DRIU or UU (M = 1.41) freshmen did. PSU freshmen also spent less time than DRIU freshmen did on co-curricular activities.

College Support Environment

PSU students used a 4-point scale (1 = very little, 2 = some, 3 = quite a bit, and 4 = very much) to rate the extent to which PSU emphasized a supportive college environment. PSU students indicated that PSU emphasized spending significant amounts of time studying and producing academic work, and provided support to help students succeed academically. They reported that PSU placed less emphasis on helping students cope with non-academic responsibilities such as work and family life or providing support for students to thrive socially (see Figure 15). PSU seniors and freshmen did not differ in their ratings of the college support environment.

As shown in Figure 16, PSU freshmen differed from UU and DRIU freshmen on two items. PSU freshmen were more likely than other freshmen to indicate that their institutions encouraged contact among students from different economic, social, and racial backgrounds. PSU freshmen were less likely than other freshmen to indicate that their institution emphasized attending campus events and activities (e.g., special speakers, cultural

Similar to the freshmen, PSU seniors were more likely than other seniors to indicate that their institution encouraged contact with diverse persons. PSU seniors were more likely than other seniors to report that their institution helped them to cope with non-academic responsibilities (e.g., work and family) (see Figure 17). In addition, seniors felt that PSU (M = 2.03) emphasized attending campus events and activities less than DRIUs (M = 2.27) did.

Quality of Relationships

Respondents rated the quality of their relationships with people at their school using 7-point scales (1= least favorable to 7 = most favorable rating). On average, PSU students were favorable in their ratings of other students, faculty, and administrative personnel. PSU students indicated that their relationships with other students were somewhat friendly and supportive, and that they experienced a sense of belonging (M = 5.26). They described faculty members as somewhat available, helpful and sympathetic (M = 5.22). PSU student ratings of administrative personnel and offices were more neutral (M = 4.74); some students found personnel to be helpful, considerate, and flexible, whereas other students found personnel to be unhelpful, inconsiderate, and rigid. The ratings of PSU freshmen and seniors did not differ from each other.

PSU students were similar to those at UUs and DRIUs in their ratings of faculty members. Although both PSU and DRIU students' ratings of relationships with other students were somewhat favorable, PSU students rated their relationships lower than DRIU students did, as shown in Figure 18. Seniors characterized their relationships with administrative personnel as more positive at PSU (M = 4.76) than at either UUs (M = 4.21) or DRIUs (M = 4.36).

Conclusions

The results of the NSSE indicate the extent to which students are engaged in good educational practices and what they gain from their college experience. Items on the NSSE ask about student behaviors that correlate with positive learning and personal development outcomes of attending university. PSU students were generally satisfied with their education. Overall, their experiences and activities were on par with those of other UU and DRIU students. More PSU seniors than other UU or DRIU seniors participated in community-based projects as part of their coursework. In addition, more PSU students than UU or DRIU students participated in culminating senior experiences.

The NSSE results suggest that PSU has helped students achieve desirable learning outcomes, but they also draw attention to areas where the institution could improve. One notable example is advising. PSU students rated the quality of their advising lower than other UU and DRIU students did. PSU students also reported discussing career plans with faculty members and advisors less often than DRIU students did. As PSU continues to improve the advising services provided to students, results of future administration of the NSSE should reflect higher levels of satisfaction on par with our peers.

Portland State University plans to participate in the NSSE again in winter 2003. For more information about these results and future administrations of NSSE, please contact Juliette Stoering, Office of Institutional Research and Planning, 503-725-3427, stoeringj@pdx.edu.

1. The response rate was adjusted to account for bad addresses.

2. The UUs that participated in the NSSE along with Portland State University are Boise State University, Cleveland State University, DePaul University, Indiana University-Purdue University - Indianapolis, The Metropolitan State College of Denver, Northeastern Illinois University, Oakland University, Pace University, Purdue University - Calumet, Southern Illinois University at Edwardsville, Towson University, University of Cincinnati, University of Colorado at Colorado Springs, University of Massachusetts - Boston, University of Missouri - Kansas City, University of Missouri - Saint Louis, The University of Toledo, University of Wisconsin - Milwaukee, and Virginia Commonwealth University.

3. "Doctoral/Research Universities - Intensive" is a designation of the Carnegie Classifications System. 32 institutions that belong in this classification participated in the survey along with Portland State University. Seven of these also participated in the Urban University Consortium: Cleveland State University, DePaul University, Indiana University-Purdue University - Indianapolis, Oakland University, Pace University, University of Massachusetts - Boston, and University of Missouri - Kansas City. Contact the Office of Institutional Research and Planning for a complete list of participating universities included in this classification.