Portland State University is engaged in a process of identifying and defining learning markers. Learning markers are an expected set of student learning outcomes resulting from a PSU education.  The process has resulted in a preliminary list of markers.  The markers include:  oral communication; written communication; quantitative reasoning and representation; use of technology; critical inquiry and dialogue; understanding the variety of human experience; ethical reasoning; social and civic responsibility; problem solving; analytical thinking; creative synthesis; collaboration and working in teams; importance of a sustainable environment; engaging with important areas of investigation.  No attempt has been made yet to evaluate student learning in light of the markers. This report uses the National Survey of Student Engagement (NSSE) data collected from PSU seniors as a starting point for providing information about students’ achievement in terms of the learning markers. 

The National Survey of Student Engagement (NSSE) evaluates a university's learning environment. During the winter and spring terms of 2003, the survey was administered to PSU seniors and freshmen.  Only the data from the 699 seniors who responded to the survey are included in this report because seniors have nearly completed their PSU education.  Their responses serve as evidence of the extent to which PSU encourages the activities and characteristics captured by the markers.  Because the NSSE was not designed specifically to assess PSU learning markers, nor were the markers designed around the survey items, not all items are relevant to PSU learning markers and not all the markers are represented among the survey items. 

The first step in the analysis was to map NSSE items onto the markers.  During this process, it became clear that some of the markers need further clarification and differentiation.  In particular, analytical thinking, creative synthesis, and engaging with important areas of investigation were difficult to distinguish. Several survey items seemed to relate to all of them.  None of the survey items related to the markers regarding the importance of a sustainable environment and engaging in important areas of inquiry.

A factor analysis was conducted to explore the relationships among the NSSE items that appeared to be related to the learning markers.  The results of this analysis mirrored the results of the analysis conducted by the national survey administrators and tended to fall into categories related to the benchmarks they developed; however, the national benchmarks do not pertain directly to the discussion of PSU learning markers.  The present examination of learning markers will report student scores on the survey items related to each marker.

 

Oral Communication

 

Oral communication is defined as " the ability to articulate meaning to others through a verbal medium." [1]   Two survey items provided information about PSU seniors' oral communication skills. Students were asked how often they made a class presentation during the current school year and the extent to which their experiences at PSU had contributed to their ability to speak clearly and effectively.  As reported in Figure 1, PSU seniors "sometimes" or often made class presentations.  They also felt that PSU had contributed "quite a bit" to their ability to speak clearly and effectively.

Written Communication

 

The written communication marker indicates whether students have "developed the ability to articulate meaning to others through written medium."  Five NSSE items were related to this marker. Figure 2 illustrates that PSU seniors "sometimes" prepared two or more drafts of a paper before turning it in.  They also felt that PSU had contributed "quite a bit" to their ability to write clearly and effectively. 

In addition, students reported the number of papers they had written of various lengths over the course of the last year.  Figure 3 indicates that PSU seniors reported writing between 5 and 10 short papers (i.e., fewer than 5 pages), between 1 and 4 medium-length reports (i.e., 5 to 19 pages), and none or one long report (i.e., longer than 20 pages). 

Quantitative Reasoning and Representation

As a result of their experience at PSU, students should "deepen their understanding of the value and need for quantitative reasoning."  They should "exhibit the ability to understand graphical presentation of data and to transform information into quantitative and graphical representations."  On the only NSSE item related to quantitative reasoning, seniors reported that their experiences at PSU had contributed "some" or "quite a bit" to their ability to analyze quantitative problems. On a scale of 1 (very little) to 4 (very much), seniors' mean rating of PSU's contribution was 2.58.

Use of Technology

PSU expects its students to "become proficient with computers and other technologies and use them effectively in pursuit of academics."   PSU seniors reported technology use as illustrated in Figure 4. PSU seniors had "often" used e-mail to communicate with an instructor.  In contrast, they only "sometimes" used electronic media (list-serv, chat group, internet, etc.) to complete assignments.  Figure 5 reveals that they felt that PSU had contributed "quite a bit" to their use of computer and information technology and they felt that PSU had emphasized the use of computers in their academic work "quite a bit."

Critical Inquiry and Dialogue

The NSSE items related to critical inquiry and dialogue reflect the frequency of PSU seniors' conversations regarding course ideas and readings that occurred outside of class as well as whether they asked question or engaged in class discussion. Seniors reported that they only "sometimes" discussed course ideas and readings with faculty members outside of class.  As Figure 6 indicates however, they were much more likely to discuss course-related topics with others such as students, family members, or co-workers.  Seniors "often" asked questions or engaged in class discussion.

Understanding the Variety of Human Experience

Seven NSSE items measured the extent to which seniors had "the ability to know and interact with people of diverse backgrounds as part of a global community."  Figure 7 reveals that PSU seniors were very consistent in their ratings of the frequency of activities that involved diverse perspectives or interactions with people different than themselves and Figure 8 shows that they were equally consistent in their ratings of the institution's commitment to diversity.  PSU seniors "often" had serious conversations with students of different racial or ethnic backgrounds or who had different religious, political or personal beliefs.  They were also likely to include diverse perspectives in class discussions or writing assignments.  Seniors believe that PSU encourages contact among students of diverse backgrounds and has contributed to their understanding of people from diverse backgrounds.  Additionally, 32% of the seniors had taken or planned to take foreign language coursework and 13% had studied abroad or planned to study abroad, activities that provide an opportunity to have contact with diverse people and perspectives.

Social and Civic Responsibility

Four NSSE items ask students about their engagement in "actions that strengthen democracy and the community."  PSU seniors felt that PSU had some influence on their contributions to their community and their voting behavior.  PSU seniors had engaged only "sometimes" in community based class projects in the past year (see Figure 9).  In contrast however, 72% reported that they had participated or would participate in community or volunteer work prior to graduation. 

Ethical Reasoning

 

PSU expects its graduates to be able to make decisions based on values and ethical principles.  The only NSSE item related to ethical reasoning asked students to estimate the contribution PSU has made to the development of a personal code of values and ethics.  Seniors felt PSU had made "some" contribution to the development of their personal ethics (M = 2.38, on a scale of 1 = very little to 4 = very much).

Problem Solving

Problem solving requires that students have the "the ability to use information and reason to discover solutions to problems."  Seniors "often" engaged in activities that required them to apply theories or concepts to practical problems or in new situations (M = 2.90, on a scale of 1 = never to 4 = very often). However, seniors felt that PSU had made only "some" contribution to their ability to solve complex real-world problems (M = 2.48, on a scale of 1 = very little to 4 = very much). 

Analytical Thinking and Creative Synthesis

When trying to determine which NSSE items corresponded to analytical thinking and creative synthesis, it made sense to look at these two learning markers together because the items related to them seemed to overlap.  Overall, PSU seniors engaged frequently in activities requiring analysis and synthesis.  They also felt that PSU had contributed "quite a bit" to their ability to think critically and analytically.  Students felt that they had "often" synthesized ideas into more complex interpretations and integrated ideas from different sources.  They were "often" required to analyze the basic elements of an idea, experience or theory (see Figure10). 

Collaboration and Working in Teams

"The ability to reach desired outcomes through cooperation with others" is an important learning outcome for PSU students.  Although the NSSE items do not address whether students reached desired outcomes through group work, there are three items that address a student's ability to work with others.  PSU seniors felt that PSU had contributed "quite a bit" to their ability to work effectively with others.  This despite the fact that in the last year, respondents had only sometimes worked with classmates on projects during or outside of class (see Figure 11).

Conclusion

The NSSE data provide a way for PSU to begin to assess students' experiences and skills related to the learning markers.  The NSSE provides the most information about the variety of human experience, analytical and critical thinking, and technology use.   Students are the most positive about the contribution PSU has made to their analytical and critical thinking skills.  Their responses reveal that PSU provides seniors with a number of opportunities to evaluate and synthesize information and ideas. Students were very consistent in their ratings on the items related to diversity.  PSU's campus environment encourages students to interact with diverse people and ideas.  However, with students' ratings in the range of 2.4 to 2.7 on a 4-point scale, there is room for improvement.  In general seniors used technology and attributed improvement in technology use to PSU's emphasis on technology.

On other markers, the NSSE data provides a starting point, but more information is needed. For written communication, oral communication and quantitative reasoning markers, the self-report nature of the NSSE survey may be problematic.  While student ratings of their own writing, speaking, and quantitative reasoning skills are interesting, objective assessment of these skills may provide better information about the marker.

Finally, the NSSE data provide no information about two of the learning markers, "importance of a sustainable environment" and "engaging with important areas of inquiry."  To get a more complete understanding of PSU students learning in relation to the learning markers, other sources of data need to be investigated. University Studies collects and evaluates student portfolios.  That could be a source of information about student writing.  The Civic Capacity Initiative in the College of Urban and Public Affairs collects data that are related to the social and civic engagement markers.  These are just two examples of other sources of information regarding PSU's learning markers.  With a combination of NSSE data providing student perspectives of the PSU experience and other, more objective sources of data, a fuller picture of PSU students' learning will emerge.