ELPS 431

Running Head: SOPHOMORE SUPPER EVALUATION 1 1

Second Year Experience Sophomore Supper Evaluation

Katherine Glesser and Roy Rodriguez

Loyola University Chicago

SOPHOMORE SUPPER EVALUATION 2

Table of Contents Introduction..3 Review of Literature........3 Setting the Context...4

University Context...4 Department Overview. 6 Program History...7 Program Description ...9 Program Stakeholders........10 Logic Model Description ..........11 Previous Evaluation...13

Evaluation Approach.....................13 Quantitative Approach...16 Sampling and Population...................16

Survey Design........17 Data Collection..19 Analysis..20

Qualitative Approach....22 Qualitative Design.............22 Participant Demographics..23

Qualitative Instrument...24 Implementation Procedure.....25 Analysis..26 Limitations.....27 Results....28

Next Steps..28 References..............30 Appendices A. Logic Model......32 B. Prior Evaluation Results/Instruments....34 C. Survey................36 D. Survey Construct Map......40 E. Interview Protocol.....41 F. Consent Forms.......44 G. Coding Rubric.......45 H.Budget........47 I. Timeline......48 J. 2012-2013 Sophomore Supper Summary..49 K. SYE Background Information..52 L. SYE Outcomes and Goals.55 M. PowerPoint Slides57


Introduction

The Sophomore Supper, an event hosted by the office of Second Year Experience (SYE) at

Loyola University Chicago, serves as instrumental piece in carrying the mission of the office. In the

broadest sense, SYE hopes that Sophomore Suppers encourage second year and transfer students to

determine their passion areas as it relates to finding a vocation. Additionally, the event aims to assist in

the development of faculty and staff mentoring relationships with those students.

As a new office and a new program, the Sophomore Supper event is lacking a formal evaluation.

Though we were provided with the results of a short, 10-question survey administered to previous

participants of the Sophomore Supper, the results provided little information about the actual process

and outcomes of the event. It is our hope that a formal evaluation plan for this event will provide the

stakeholders with tangible instruments to assess both the achievement of learning outcomes set forth by

SYE, as well as overall student satisfaction.

Review of the Literature

The sophomore year of college is oftentimes viewed as the lost time or slump of the traditional,

4-year college experience (Hunter, Tobolowsky, Gardner, Evenbeck, Pattengale, Schaller, Schreiner, &

Associates, 2010). During the second year of college, students tend to face multiple, significant changes

in the academic setting, professional setting, and in the social setting (Hunter et al., 2010). These

changes can range from transition to major-based courses, to searching for an internship, to maintaining

friendships from the previous year. Colleges and universities put forth a number of resources for the

first year student experience for events such as welcome week, convocation, and transition assistance.

Additionally, these colleges and universities often spend significant resources on the celebration of the

successful completion of the college experience like commencement and university service awards.

Therefore, second year students are often placed on the outer rings of the university environment and

culture (Hunter et al., 2010).


It would be quite the reductionist act to make the claim that colleges and universities simply do

not care about second year students or the struggles they face during this time. Instead, it appears as

though much of the attention and resources at the disposal of the college or university are heavily

directed towards first year students with a strong commitment placed on ensuring that these new

students are welcomed, transitioned into their new environments smoothly and successfully, and that

they persist for a second year. Ironically, with so much attention placed on the retention of first year

students, or rising sophomores, few colleges and universities across the nation have specific program

areas or offices that aim to support second year students (Hunter et al., 2010). Although some would

argue that many colleges and universities have programs such as second year advising or second year

living communities within departments of residence life and academic advising, these programs focus

only on a portion of the second year experience and fail to assess and address the holistic needs of

second year students (Hunter et al., 2010).

The struggles second year students face are not new, but there are fewer resources and services

made available to them by the college or university than is the case during the first year or senior year of

the college experience. For this reason, the purpose of this assessment will be to evaluate the

Sophomore Supper event. This evaluation will seek to determine how effective the Sophomore Suppers

are in achieving the learning outcomes presented by SYE and how they impact the developmental needs

of second year students at Loyola.

Setting the Context

Loyola University Chicago

Founded in 1870, Loyola University Chicago is the nations largest Jesuit institution with

approximately 16,000 students enrolled during the 2012-2013 academic year (Association of Jesuit

Colleges and Universities, n.d.). Nestled in the community of Rogers Park, Loyola has the unique

advantage of being an urban institution in a bustling metropolis and prides itself as being Chicagos

SOPHOMORE SUPPER EVALUATION 5 Jesuit University (Association of Jesuit Colleges and Universities, n.d.; Loyola University Chicago,

n.d.). Aside from its main Lakeshore Campus, Loyola also has several other campuses in the

Chicagoland area and abroad that offer courses and a Jesuit education away from the main Lakeshore

Campus (Association of Jesuit Colleges and Universities, n.d.; Loyola University Chicago, n.d.). Of the

16,000 students that study at Loyola, some 9,723 are undergraduate students enrolled one or more of the

80 majors and minors that the university has to offer (Howes, 2013; Loyola University Chicago, n.d.

b). Loyola also prides itself on being an institution that values diversity, which can be seen in some of

the diversity the students bring into each incoming freshmen class. For example, the 2012 freshmen

class entered with 2,003 new students representing 44 states in the U.S. and 24 foreign countries (Loyola

University Chicago, n.d.). Of the 2,003 new students, 34% identified as students of color, 45% of

Jewish, Muslim, Hindi, Protestant, or other religiously affiliated faith tradition, and 96% qualified for

need-based financial assistance (Loyola University Chicago, n.d.). Such a diverse university contributes

significantly to the overall college experience for students, but for student affairs practitioners and other

university stakeholders, such diversity can bring with it significant challenges to providing effective

programs and resources for students.

Loyola University Chicagos mission drives its academic and co-curricular experience for its

students: We are Chicago's Jesuit Catholic University - a diverse community seeking God in all things

and working to expand knowledge in the service of humanity through learning, justice and faith

(Association of Jesuit Colleges and Universities, n.d.; Howes, 2013, p. 4); Loyola University Chicago,

n.d.). Loyolas strong commitment to its values and social justice mission has earned it the title of being

one of the nations best values in higher education by U.S. News and World Report (Association of

Jesuit Colleges and Universities, n.d.). To follow and remain aligned with these espoused values,

Loyola strongly identifies with the five characteristics of a Jesuit education in order to help members of

SOPHOMORE SUPPER EVALUATION 6 the Loyola community remain aware and cognizant of their actions and to live out the values and

mission of the institution (Loyola University Chicago, n.d.; Howes, 2013):

x A Commitment to excellence.

x Faith in God and the religious experience.

x Service that promotes justice.

x Values-based leadership.

x Global awareness.

These characteristics are ingrained into the fabric of the university and can be seen in the

universitys curriculum, student development departments, campus buildings, and everything else in

between. These characteristics, along with the universitys mission, have contributed to new initiatives

and programs such as Second Year Experience that are designed to help students get the most out of

their four (or less) years at Loyola (Loyola University Chicago, n.d.).

The Department

Second Year Experience (SYE) at Loyola University Chicago (formerly Second Year Initiatives)

exists to support undergraduate students throughout their second year at Loyola (Loyola University

Chicago, n.d.; Second Year Experience Planning Document, 2012). In understanding that students are

often left to find their own path after their first year of college, the program area offers a number of

events and resources in order to better assist second year students as they return to Loyola to continue on

with their undergraduate experience. More specifically, Second Year Experience strives to provide

second year students with a plethora of opportunities to make commitments, both internally to values,

identities, and vocations, and externally to majors, careers and community (Second Year Experience

Planning Document, 2012). Second Year Experience works with students as they develop a better

understanding of themselves, their directions, and how their stories contribute to the Loyola University

Chicago Experience. Second Year Experience, in collaboration with campus partners, provides

SOPHOMORE SUPPER EVALUATION 7 programs, services, and opportunities to assist students in their discernment processes, moving through

exploration toward making commitments (Second Year Experience Planning Document,

2012). Second Year Experience at Loyola University Chicago aims to encourage students to make

commitmentsto their major, their community, and themselves while also helping them through other

developmental milestones.

Second Year Experience has developed five outcomes for students who engage with the program

throughout their second year. First, the office hopes students feel connected to their community and

develop meaningful mentor relationships with members of the university community. Additionally,

students are expected to demonstrate commitment to their academics by declaring a major, and

commitment to their passion areas by attempting to discern a vocation. Finally, the office strives for

students to actively participate in university clubs and organizations (Second Year Experience Planning

Document, 2012).

Program History

The Sophomore Suppers program were inspired from a previously existing program at Loyola

known as the Discernment Dinners, a programmatic series put on by EVOKE, a program area designed

to assist in the vocational discernment process for Loyolas undergraduate student population (Loyola

University Chicago, n.d.). Established in 2001 by grant funding from the Lily Endowment, EVOKE

provided a number of opportunities for students to engage with faculty, staff, community members, and

peers, all with the intention of assisting students in finding their calling, or vocational discernment (Lily

Fund, Inc., n.d.; J. Cot & S. Howes, personal communication, September 19, 2013). The program

hosted speakers to share personal stories about their callings, facilitated activities to bring together

faculty and staff members with students to share knowledge and their personal vocational journeys, and

co-sponsored opportunities for students in the form of retreats, leadership development workshops, and

immersion trips (Loyola University Chicago, n.d.; J. Cot, personal communication, September 19,

SOPHOMORE SUPPER EVALUATION 8 2013). Though EVOKE as a program area itself remained consistent for many years, the activities

hosted by EVOKE changed often largely due to high employee turn-over rates in the office (S. Howes,

personal communication, September 19, 2013). New employees brought in new ideas and new

perspectives on what would be the best way for students to engage with the vocational discernment

process.

One EVOKE activity which remained consistent and which has evolved with the changing

Loyola population, climate, and culture is the Discernment Dinners program hosted by the office (J.

Cot & S. Howes, personal communication, September 19, 2013). These dinners provided students who

were struggling with determining who they were and what they felt called to do in life with the

opportunity to eat a meal with a faculty, staff, or administrator at the university while discussing their

hopes and aspirations for their future endeavors (J. Cot & S. Howes, personal communication,

September 19, 2013).

However, the 2012-2013 academic year brought a plethora of change to the EVOKE

program. Since EVOKE was an area funded primarily through the Lily Endowment, the future of the

office was uncertain once the grant was no longer available. Another factor as to why the EVOKE

program would no longer continue was the creation and implementation of the Loyola Experience by the

university. The creation of this new university-wide initiative led Loyola and the Division of Student

Development to reevaluate the EVOKE program in order to determine whether the services provided to

students were still effective (J. Cot, personal communication, September 19, 2013).

The Loyola Experience is a four-year plan focused on traditional undergraduate students to help

guide them in reaching certain developmental milestones during their four years on campus while also

ensuring that they complete their degrees within four years or less (Loyola University Chicago,

n.d.). The milestone that is specific to the second year of the Loyola Experience is to make

commitments (Loyola University Chicago; n.d.; Second Year Experience Planning Document,

SOPHOMORE SUPPER EVALUATION 9 2012). In order to assist second year students in making commitments and with other developmental

needs that they face during this time, Loyola and the Division of Student Development created the

Second Year Initiatives program, now called Second Year Experience, to join the already existing

Student Leadership Development program at university (S. Howes, personal communication,

2013). The creation of this new program area was a leap forward for the university in the enhancement

of the student experience while also being a step forward in the area of the Sophomore Experience

nationwide.

Program Description

Each semester, Second Year Experience hosts several events known as Sophomore

Suppers. This program aims to reach outcomes 2 and 3 listed in the Second Year Experience Planning

Document (2012) by assisting students in developing meaningful mentoring relationships with Loyola

faculty, staff, and administrators, and articulating their abilities in hopes of vocational discernment. The

program is designed to encourage networking and resource sharing between second-year students and

members of the Loyola community, as well as allow students the opportunity to share their talents and

passions with one another.

Students become aware of the event through the Second Year Email Blasts, sent through the

Office of First & Second Year Advising, personal emails from signing up on the SYE listserv, on

various social media outlets such as Facebook or Twitter, and, perhaps the most effective form of

advertising at the disposal of the office, by word of mouth. In order to ensure attendance, students are

asked to pre-register for the event through a Google Document registration form. The semi-formal event

has a business casual dress code and involves both a Mocktail mingling hour and sit-down plated

dinner or buffet. Both portions of the evening work together to provide students with the opportunity to

learn important networking skills and learn to make meaningful connections with different prominent

members of the Loyola community and with fellow second year students with discussing their

SOPHOMORE SUPPER EVALUATION 10 distinctive abilities and talents. The two Sophomore Suppers that have previously taken place (one in

the spring of 2013 and the one at the start of the fall 2013 semester) attracted and served approximately

30 students for each event.

Program Stakeholders

The primary stakeholders, or the individuals and organization that will be affected by the

evaluation process and its findings, identified for the Sophomore Suppers, those who have the most

invested into this program and who get the most out of it, are second year undergraduate students at

Loyola University Chicago (Bryson & Patton, 2010). Secondarily, the Office of Leadership

Development & Second Year Experience (namely the program director for Leadership Development &

Second Year Experience, the program coordinator for Second Year Experience, and the graduate

assistant for Second Year Experience), the Division of Student Development, and the Loyola Experience

Committee have also been identified as secondary stakeholders, or groups within the organization that

directly work with the program and office, but that are not directly affected by the evaluation process or

findings, in this program (Bryson & Patton, 2010). Being able to adequately understand the overall

satisfaction of the second year students in attendance for the event coupled with the effectiveness of

reaching the intended learning outcomes set forth by Second Year Experience, are the focal point of the

evaluation and assessment of the Sophomore Suppers (J. Cot & S. Howes, personal communication,

September 19, 2013). This focus on the assessment is important because the overall learning and

development of the students in attendance may be a direct reflection SYE activities and

initiatives. Indirectly, families, friends, and the immediate community of second year students can also

be considered stakeholders in the process as well through their interactions with their students taking

part in the Sophomore Suppers.

Other key stakeholders include the staff members within the SYE office. Shannon Howes,

Director of Leadership Development & Second Year Experience, John Cot, Program Coordinator for

SOPHOMORE SUPPER EVALUATION 11 Second Year Experience, and Roy Rodriguez, Graduate Assistant for Leadership Development &

Second Year Experience, all play a role in the implementation and success of the Sophomore

Suppers. The graduate assistant for the area was the person directly responsible for the development,

logistics, and execution of the past two, larger Sophomore Suppers. Additional staff members who are

invested in the program in the event that additional human resources are the Program coordinator for

Leadership Development, Assistant Vice President for Student Development, the Dean of Students, the

Vice President for Student Development, and the new Director for Student Academic Services (Howes,

2013).

In addition to students and SYE staff members, the faculty, staff, and administrator facilitators of

the Sophomore Suppers also play a crucial role in the experience of the second year student participants

and the overall success of the program. SYEs relationship with these individuals also plays an

important role in the facilitation of the Sophomore Suppers and ensuring that quality members of the

Loyola community are eager and excited to participate.

Logic Model

McLaughlin and Jordan (2010) define a logic model as a plausible and sensible model of how a

program will work under certain environmental conditions to solve identified problems (p. 56). As the

Logic Model (Appendix A) created for the Sophomore Supper indicates, a number of resources make up

the Sophomore Supper program. For the larger dinners hosted by SYE, approximately $1,400 and two

staff members are required to run the event and to accommodate the number of individuals in attendance

(J. Cot, personal communication, September 19, 2013). Additionally, the social capital SYE invests to

recruit faculty, staff, and administrators at Loyola in order to participate in the event is an input that

must be accounted for, although, fiscally, it costs the office nothing. Though a $1,400 budget for each

large supper has been the resulting expense for the two suppers in the past, the director of and

coordinator of the area both expressed deep concern with the sustainability of the program if each dinner

SOPHOMORE SUPPER EVALUATION 12 continues to cost that much (J. Cot & S. Howes, personal communication, September 19, 2013). With

this concern voiced and noted the stakeholders, the importance of a thorough assessment of the

Sophomore Suppers became blindingly apparent.

Through a networking event and sit-down dinner, students are given the opportunity interact with

faculty, staff, and administrators to address and explore their future goals, especially with regards to

vocational discernment. Not only is there the potential to significantly influence the second year

students who attend the Sophomore Suppers, but the faculty, staff, administrators, and other upper-class

facilitators also have the potential to be affected by the event as well. For the second year students

involved, a variety of intended outcomes exist. First, students have the opportunity to connect with a

faculty, staff, or administrator at Loyola in order to learn about and explore the resources available to

them on campus and to learn about different options for majors and future careers (Second Year

Experience Planning Document, 2012). Within one year after attending a Sophomore Supper, it is

expected that students commit to a major, develop a genuine mentor/mentee relationship with a Loyola

faculty, staff, or administrator, and that they begin to make plans to further engage with Loyola for the

remainder of their undergraduate experience. This may include planning for an internship or to study

abroad during the third year. Finally, longer-term outcomes for students include vocational discernment,

sustaining mentor/mentee relationships, and developing mentoring relationships with less-experienced

students.

In terms of the Sophomore Suppers, there are a few assumptions made of the students involved

with the program that must be addressed. The first assumption is that all or most second year students

are in need of some type of assistance in vocational discernment and major selection. The second

assumption made is that the Sophomore Suppers operate under the belief that most, if not all, students

who attend these events will find their discussions centered on major selection and vocational

discernment with the faculty, staff, and administrators in the room helpful. For students who have

SOPHOMORE SUPPER EVALUATION 13 already determined and committed to a career path, the opportunity to engage in meaningful

conversations with prominent members of the Loyola community on key developmental talking points

may be lost altogether. Additionally, it is assumed that a dinner in a large group setting is most

conducive for having these meaningful conversations with second year students. If a student is unable

to attend a dinnertime event due to family commitments, class, work, or other obligations, or if a student

does not feel comfortable engaging in a group setting, a Sophomore Supper-type event may not be the

best way to reach and serve these students. Finally, the hosts of the Sophomore Suppers have always

been faculty, staff, and administrators of the university. This relies on the assumption that these

individuals are the best community members, rather than alumni, local business owners, or university

donors, to speak with second years about vocational discernment and major selection.

Previous Evaluation

Second Year Experience has only administered and received substantial responses for one

assessment that was sent out for the Sophomore Suppers. The survey was administered

between September 9, 2013 and September 20, 2013 through Campus Labs at Loyola University

Chicago and was completed by 16-second year students. Though the survey gathered some quantitative

feedback (students felt strongly that the event should be publicized more), the majority of the multiple-

choice questions focused on whether students were satisfied with the logistics of the event: the location,

ability to interact with fellow students and faculty/staff members, and gaining an understanding of the

SYE office (Appendix B). At this point, the evaluators plan to develop an entirely new survey to

administer to second year students, as the current survey does not appropriately address the process and

outcomes we hope to examine as a result of this evaluation.

Evaluation Approach

To best meet the needs of the stakeholders for this evaluation, the researchers will utilize a

hybrid evaluation approach in which components of both formative and summative evaluation

SOPHOMORE SUPPER EVALUATION 14 approaches will be used to evaluate the Sophomore Suppers. Formative evaluation approaches utilize

evaluation methods to improve the ways a program is delivered whereas summative evaluation

approaches measure program outcomes and how program impacts can be linked to program activities

(Newcomer, Hatry, & Wholey, 2010). A process-based evaluation will be used in an attempt to gain an

understanding of the extent to which second year students connect with faculty, engage in intentional

conversations with faculty, staff, and peers, and build mentoring relationships with faculty members

(Newcomer et. al, 2010). As the Logic Model indicates, SYE invests a large portion of their funds,

social capital, and time into Sophomore Suppers. Therefore, we also plan to implement an outcomes-

based evaluation of the Sophomore Supper in order to determine if the intent behind the suppers matches

the students experience (Newcomer et. al, 2010).

We plan to examine general student demographics, along with the student experience of

interacting with a faculty, staff, or administrator both during and following the event. To achieve this,

our overarching evaluation questions are [1] to what extent are Sophomore Suppers effective in

achieving the learning outcomes developed by SYE? [2] How do Sophomore Suppers address the

developmental needs of second year students at Loyola? And [3] what is the overall level of student

satisfaction at Sophomore Suppers?

We will also attempt to evaluate the intentionality and authenticity of the conversations students

and members share during Sophomore Suppers. A long-term goal of the program is to serve as a spring-

board in developing faculty-student mentoring relationships. The final part of our process evaluation

will look at the long-term relationships formed as a result of attending a Sophomore Supper. In terms of

our outcome evaluation, we plan to evaluate if students believe they have easier access to faculty

members as a result of their attendance at a Sophomore Supper.

This formative evaluation, focused primarily on program improvement, will assess the program

itself, in addition to the student-centered outcomes (Newcomer et. al, 2010). Staff members in SYE

SOPHOMORE SUPPER EVALUATION 15 indicated a desire to more fully understand the student experience of Sophomore Suppers, along with the

students understanding of the purpose of Sophomore Suppers. By using a formative approach, we hope

to offer SYE with tangible feedback to improve the Sophomore Supper event in the future. The benefit

of using both a process and outcomes-based evaluation approach is that both the students experience

and the effectiveness of the manner in which the program outcomes are delivered through the

Sophomore Suppers will be assessed as a result of this evaluation.

Our approach offers a variety of strengths. First, the evaluation is focused on the overall student

experience. This very much aligns with the goal of the SYE office. By focusing on what students gain,

our evaluation will be most useful to the office. Additionally, our focus on the evaluation of learning

outcomes will be helpful to SYE as they determine the best ways to improve the Sophomore Supper

experience. The outcome-based evaluation will provide direct feedback in terms of the purpose of the

event.

Both evaluators work with second year students, and one evaluator works in SYE. While this

presents a strength because of our background knowledge of this particular population of students, there

is a potential for bias in the evaluation results. We will attempt to avoid bias by working closely with

our stakeholders to ensure the information we use for the evaluation is valid and accurate. An additional

weakness that this assessment cannot address is whether or not the needs of the second year were

appropriately addressed. Though there is research available on second year students, we cant be sure

that external factors dont affect the needs of our students in a different way. Finally, the faculty

component of Sophomore Suppers accounts for a significant portion of the students experience. This

assessment will not address the faculty experience of Sophomore Suppers, which is necessary to

understanding the holistic picture of the event.


Quantitative Approach Introduction There are several unsupported assumptions held by the office of Second Year Experience with

regards to the Sophomore Suppers and the way in which they impact the holistic development of second

year students at the university (Hunter et. al, 2010; J. Cot & S. Howes, personal communication,

September 19, 2013; Second Year Experience Planning Document, 2012). Assumptions that a formal,

sit down dinner are effective in engaging students and facilitators in meaningful conversations,

assumptions that second year students are not already engaging in conversations around vocational

discernment, major selection, and mentorship, and assumptions that second year students are fully

participating in the process of the Sophomore Suppers are ones that the primary stakeholders

(Leadership Development & Second Year Experience) would like to see assessed and quantified through

data. In order to determine if the format of the Sophomore Suppers is effective, will be create and

utilize a survey to gather quantitative data on the overall student experience (Newcomer & Triplett,

2010).

Sampling and Population

The population of interest, that is, the group or individuals that will be assessed, will be the

second year student participants of the Sophomore Supper (Newcomer & Triplett, 2010). Focusing

solely on the second year students in attendance at the supper allows the researchers to gain information

pertinent to help improve the program (Newcomer & Triplett, 2010). The researchers are also aware

that by excluding members of the second year student population at Loyola, we will not have sufficient

information on whether this treatment, the Sophomore Supper, is meeting the outcomes set forth by

Second Year Experience (Newcomer & Triplett, 2010). Suggestions for future research and assessment

will be addressed later in this assignment. Supplemental information may be gathered from the Loyola

SOPHOMORE SUPPER EVALUATION 17 faculty, staff, and administrators who assist in the roundtable facilitation during the Sophomore Suppers

to gather information about faculty and staff satisfaction to further improve the event.

We will utilize census sampling, a sampling option that interviews the entire population, to

gather survey responses from all of the participants of the Sophomore Supper (Newcomer & Triplett,

2010). We decided to go with census sampling because of the relatively low number of participants at

the supper. Inherent to its design, the Sophomore Supper is intended to be a smaller, more intimate

event in which students can have the unique ability to connect with prominent individuals of the Loyola

community over a meal (S. Howes, personal communication, September 19, 2013). The larger suppers

(held at the beginning of the semester) accommodate approximately 30 to 40 second year participants

whereas the smaller Sophomore Suppers accommodate no more than fifteen second year students from

specific academic and curricular populations (J. Cot, personal communication, September 19, 2013).

Some of the concerns for analysis are (1) the risk of some participants not responding and what

that will do to the data and (2) the small sample size may not be representative of the overall second year

student population of Loyola (Newcomer & Triplett, 2010). These issues are of concern to the

researchers because in order to develop an effective means of communicating the importance of

vocational discernment and networking to second year students, we must better understand the

population of students who attend Sophomore Suppers while also understanding which students are not

being represented. This will enable the researchers to make suggestions to Second Year Experience as

to which populations of the Loyola community they might be able to better and effectively reach out to

(J. Cot & S. Howes, personal communication, 2013).

Survey Design

In thinking about the survey design for this assignment, we wanted to create an instrument that

examined the process second year students went through while participating in the Sophomore Supper

(Appendix C). This process-based approach will assist us in gaining a more clear understanding of the

SOPHOMORE SUPPER EVALUATION 18 extent to which second year students connect with and engage in intentional conversations with Loyola

faculty, staff, administrators, and their peers to help them build mentoring relationships with these

individuals. To gather information about the process, some questions of the survey will be satisfaction-

based questions that will help us gauge the overall student experience during the event (Appendix D).

This will give us the opportunity to examine and focus on the level of satisfaction students have with the

facilitators, the flow and design of the event, the number of participants, the day/time, the time of

year/semester the supper is held, and the general topics of discussion. Aside from creating satisfaction-

based questions, we will also create impact-based questions. Impact-based questions will help us

preliminarily gauge some of the outcomes of the suppers that Second Year Experience develops and

hopes to reach through events such as this (Second Year Experience Planning Document, 2012). These

impact-based questions will prompt the participants to select the level in which they agree with a

statement around confidence and self-efficacy in engaging with faculty, staff, and administrators. In

addition, these questions will assist us in understanding how participants engage with the vocational and

major discernment processes. Aside from creating and using satisfaction-based and impact-based

questions, we will also ask for participants university identification number so we can track majors,

race/ethnicity, and other demographic questions to get a clearer picture of the students participating in

the event so as to better promote the event and identify which populations of second year students we

need to be more intentional about reaching out to. This question will be asked of our participants at the

end of the survey.

Our quasi-experimental cross-sectional research design will give us the opportunity to collect

data that will indicate characteristics of our sample population time of the Sophomore Supper. We plan

to use a cross-sectional design because it focuses on the patterns of data after a single collection (Schuh,

2009). Utilizing a cross-sectional design can help us identify key characteristics, such as overall

satisfaction, of students who participate in the supper to be able to identify key areas of development for

SOPHOMORE SUPPER EVALUATION 19 future events. Our quasi-experimental design will help us observe the impact of the Sophomore Supper

on second year student participants. By design, the Sophomore Supper (the treatment) does not

randomly select participants, instead, students self-select which Sophomore Supper they would like to

attend and participate on their own accord.

We will pretest our survey instrument (Appendix C) to see if the questions being asked are

understandable, non-leading, and if they will assist us in gathering the information we are hoping to

receive (Newcomer & Triplett, 2010). We will pilot test our survey instrument on the student workers

and members of the student leadership teams in the Leadership & Second Year Experience office. Some

of these students may have attended one of the Sophomore Suppers during their second year and will

provide some substantial information on ways in which the survey can be improved to better meet the

needs of our students.

Data Collection

The method in which we will collect the data for our survey will be using a pen-pencil-and-

paper survey (Newcomer & Triplett, 2010, p. 271). This method of data collection will be effective

and useful for our purposes because of our accessibility to this audience, the second year students

participating in the supper. For the Sophomore Supper that occurred at the beginning of the Fall 2013

semester at Loyola, Second Year Experience sent out a web-based survey, created through Campus

Labs, to gain knowledge about the overall satisfaction and experience of the supper participants (J. Cot,

personal communication, September 19, 2013, Appendix B). This 10-question assessment tool focused

on questions around overall student satisfaction and areas of improvement for future suppers, and it was

sent to the email addresses that the participants left on an event sign-in sheet. Responses from sixteen

participants were collected (Appendix B). We have decided to not utilize a web survey primarily

because of the unpredictability of response rates. Although Newcomer & Triplett (2010) advocate that

using web surveys along with other means of data collection can improve response rates from

SOPHOMORE SUPPER EVALUATION 20 participants, the offices past experience using such a method of data collection has proven to yield an

undesirable response. It is our hope that by using a pen-pencil-and-paper survey instrument

administered by a Second Year Experience staff member directly after the supper, we will improve the

response rates from participants. Although we hope for a perfect 100% response rate from participants,

realistically, a healthy response rate is anywhere from 70% to 80% of attendees (J. Cot, personal

communication, September 19, 2013).

An issue with utilizing a pen-pencil-and-paper survey for data collection that we must be mindful of

is the fact that some students leave the event early and, therefore, do not receive a survey to fill out and

contribute. One way in which Second Year Experience attempted to resolve these types of situations in

the past was by sending a web-generated survey to the email address of the student that left the event

early. Several issues also arose when this approach was taken. First, it was difficult to identify which

students left early if no personal communication or contact was made with the student. Second, there

was no way to ensure that students who received the web survey would actually fill it out and return it to

the office.

Analysis

In considering the best course of action to undertake the analysis of our survey, we may utilize a

combination of inferential and descriptive analyses. We will utilize descriptive statistics, statistics used

to describe a group of items such as averages, to present the data collected (Newcomer & Conger, 2010).

Descriptive statistics will enable us to collect information on our sample, not the entire second year

student population, that we do not already have as it pertains to academic major, race-ethnicity, and

other demographic factors. Although we ask for the information in the survey and although it may help

us further understand which students are not participating in the event, we do not have to further

disaggregate the results across particular demographic segments for this assessment. Inferential

statistics allow the researchers to take the statistics that are computed from the responses of the sample

SOPHOMORE SUPPER EVALUATION 21 population to make generalizations of that particular population (Newcomer & Conger, 2010). For the

purpose of this evaluation, the researchers have decided to utilize a multiple regression analysis as a

means of gathering inferential statistics on the sample population. A multiple regression analysis can be

used to predict the values of a dependent variable from a linear combination of independent variables

(Newcomer & Conger, 2010). With regards to the Sophomore Supper, a multiple regression analysis

would allow the researchers to disaggregate the data collected by various dependent variables gathered

from the demographic information we obtain from the supper participants. A multiple regression

analysis would allow us to further understand how the program is or is not achieving the outcomes set

forth by SYE and which student populations may be gaining the most from the program. It is the hope

of Second Year Experience that students who attend the Sophomore Supper are meeting the outcomes

set forth by the office because they attend a supper. The researchers also want to understand what

process students are going through during the Sophomore Supper by gauging the overall experience and

satisfaction of the participants.

In order to best display the results of the evaluation, the researchers will utilize a confidence

interval to display and convey the results of the evaluation (Newcomer & Conger, 2010). The

confidence level is the amount of, or range of, evidence the evaluators want to be sure to have in order to

be certain that they are correct in concluding that the program does indeed produce the observed effect

(Newcomer & Conger, 2010). The researchers believe that a chi-square test would be the means most

suitable for displaying the results of a confidence interval. A chi-square test would allow the researchers

to test the statistical significance of any of the relationships between the Sophomore Supper and the

students that participate in the program achieving the desired learning outcomes presented by SYE

(Newcomer & Conger, 2010).


Qualitative Approach Introduction

In the quantitative approach to our assessment, we spent some time digging deeper into the

process of the Sophomore Supper. We hoped to determine if our method for engaging students was

effective, and we wanted to know the extent to which students fully participated in the event. For our

qualitative approach, we plan to look even further into the student experience that occurs at the

Sophomore Supper. Our primary stakeholders asked that we get at the root of the student experience.

Specifically, they want to know if students are achieving the outcomes set for them by Second Year

Experience (SYE) as a result of attending a Sophomore Supper (J. Cot & S. Howes, personal

communication, 2013). In order to determine if the specific outcomes of the Sophomore Supper

developing mentoring relationships with the Loyola community and discovering passions in attempt to

discern a vocationare being met, we plan to gather data on the student experience using a qualitative

method.

Qualitative Design

In addition to our survey, we will also implement a qualitative approach in our assessment of the

Sophomore Supper. Our quantitative assessment is focused on the specific processes of the suppers.

The qualitative aspect of our assessment, on the other hand, will focus on the overall outcomes of the

program set by SYE. Specifically, we plan to examine outcomes two and three of the SYE office.

Outcome two aims to achieve that students will, demonstrate deeper and more meaningful relationships

with mentors, faculty, staff and/or alumni (Second Year Experience Planning Document, 2012).

Outcome three asserts that students will be able to, articulate their unique passions, values and talents

in connection to relevant possible vocational choices and academic majors (Second Year Experience

Planning Document, 2012). The stakeholders in this evaluation are interested in learning about the

extent to which the SYE learning outcomes are achieved by students as a result of participating in the

SOPHOMORE SUPPER EVALUATION 23 Sophomore Suppers (J. Cot & S. Howes, personal communication, 2013). Our qualitative approach

will attempt to address this need.

We have chosen a semi-structured interview approach for our qualitative assessment. Schuh

(2009) describes this particular type of interview as a way to guide the interviewer from point A to point

B. However, in a semi-structured interview, the interviewer is also given creative license to briefly stray

from the prescribed questions in order to make the interview flow (Schuh, 2009). We chose a semi-

structured interview approach instead of a focus group for our assessment because of the individual

nature of student goals and relationships with others. We would not have enough time in a focus group

setting to talk to each student about their growth as a result of attending a Sophomore Supper.

Therefore, a semi-structured interview, though it will take more time for the evaluators, will likely

provide the most fruitful information regarding the learning out comes of Sophomore Suppers.

Participant Demographics

In order to determine the number of participants needed for interviews, we will use Seidmans

(2006) measure: sufficiency and saturation. Sufficiency suggests that there are enough participants

to reflect the stories of those not interviewed, while saturation occurs when the evaluator begins to hear

the same information from the interviewees (Seidman, 2006). We will recruit participants based on

those who provide us with contact information on the survey which will be distributed at the event.

Those who are interested will then be chosen to participate in an interview. Approximately 30 students

attend each Sophomore Supper. We plan to begin by interviewing approximately eight students after the

conclusion of the next supper. If more than eight students are interested in interviewing, we plan to use

maximum variation sampling. Maximum variation sampling attempts to understand general themes as

they relate to the interviewees (Schuh, 2009). We hope to gather a wide range of opinions from students

of different social identities, college majors, and with varying plans for their future. Additionally, as

Schuh (2009) states, sampling for qualitative assessments is ongoing. Therefore, depending on the

SOPHOMORE SUPPER EVALUATION 24 number of volunteers we get from our survey results, and depending on the information we get from the

first several interviews, we may need to recruit more participants to interview. We will do this by

reaching out individually via email to students who attended a Sophomore Supper, even if the student

did not volunteer for an interview. In attempt to meet our goal of maximum variation sampling, we will

reach out to the students who are the least represented in our interviews, specifically in terms of race,

gender, and academic major. It is our hope that a personal invitation to participate in an interview will

get students to participate.

As a result of our recruitment efforts, it is likely that the validity of our results may be skewed.

We will attempt to address this concern by personally inviting every student attendee of a Sophomore

Supper to interview, not just those students who are known by the SYE staff. In doing this, we hope to

recruit a more even distribution of participants. This recruitment technique also addresses the ethics of

our work. Depending on the individual, some might see this particular recruitment technique as

problematic because it requires specifically reaching out to individual students to participate in an

interview. Schuh (2009) suggests that evaluators be faithful in the presentation of the data (p. 197).

To ensure the presentation of accurate information, we plan to include our recruitment process in our

final report.

Qualitative Instrument

Our semi-structured interview guide and consent form (Appendix E and F) focuses on the

outcomes students might achieve as a result of attending a Sophomore Supper. Questions 1-6 focus on

the students overall experience as a participant of the Sophomore Supper. We hope to learn about

students expectations coming into the supper, and whether or not those expectations were met.

Additionally, the questions focus on positive and negative experiences of the supper. Questions 7-10

begin to gauge the desired impact of the Sophomore Supper: opportunities to discuss major selection

and vocational discernment. We want to know if students felt comfortable engaging with their

SOPHOMORE SUPPER EVALUATION 25 facilitator on the topics of their passion areas and vocational discernment. The questions in this section

are intentionally very broad to allow students to share what they wanted to learn and what they actually

learned as a result of attending the supper. The last section, Questions 11-13, focus on another desired

outcome: ability to connect and develop relationships with other individuals on campus, namely those

that have the potential to serve as mentors for second year students. The final question of the interview

is open-ended and allows students to share any additional information they thing is important to share

with the interviewer.

In addition to the 13 guided questions, the interview protocol also provides a script for

introductions and instructions, as well as probing questions for less verbal interviewees (Appendix E).

The interview should take no more than 60 minutes. Our questions will be pilot tested on the student

employees and leaders in SYE. Many of these students have attended a Sophomore Supper in the past,

and will be invaluable resources in sharing how students might perceive the questions for the interview

and ways to improve it. We plan to sit down with at least three different students to pilot test the

questions. Additionally, the evaluators will meet with Shannon Howes, Director of SYE, and John Cot,

Program Coordinator of SYE, to ensure the interview questions ask for the information Shannon and

John hoped to receive as a result of this assessment. This will help to address the face validity of the

interview questions. Schuh (2009) describes face validity as a review by experts to decide if the

questions formulated will gauge what they are intended to gauge (p. 123). As key stakeholders in our

assessment, Shannon and John can serve as additional resources to ensure the validity of this study.

Implementation Procedure

Interviews will be implemented as soon as possible upon the completion of the next Sophomore

Supper. Because we plan to recruit participants through the survey instrument administered at the

conclusion of a supper, it is likely that individual interviews will take place one to three weeks after the

supper. In order to prevent the need for historical recall among the interviewees, we plan to move as

SOPHOMORE SUPPER EVALUATION 26 quickly as possible after the supper to set up and execute the interviews. Roy will serve as the primary

leader of the interviews. His familiarity with SYE, in addition to his role as the Graduate Assistant for

the office will be a great fit and allow him to ask probing questions to have participants reflect deeper on

their experience Furthermore, one of Roys primary responsibilities is to plan and implement the

Sophomore Suppers. Since Roy is one of the key stakeholders in this assessment, it may be helpful for

him to hear student feedback directly. At this same time, having Roy serve as the interviewer leads to

potential bias. In order to prevent bias, our note taker will be a student employee in SYE. The student

taking notes will be instructed to be objective in recording the interview and record only what they hear.

At the conclusion of the interview, Roy and the student note taker will work together to sort out any

misunderstanding in the notes.

Interviews will take place in the SYE offices in Damen Student Center Suite 127. The purpose

for this is two-fold. First, the office serves as a convenience factor for SYE staff facilitating the

interviews. Second, inviting students into the office for an informal interview will allow them to feel

more comfortable visiting the office in the future if they are seeking guidance. At this time, we do not

plan to offer an incentive for students who participate in the interview. There are a variety of reasons for

this. First, the budget for the year did not allocate for the purchase of gift cards or food for assessment

purposes this year. Additionally, all students who are asked to participate in an interview will have

recently attended a Sophomore Supper. It is our hope that the meal provided at the Sophomore Supper

is enough for students to be willing to give us a few minutes of their time for an individual interview. In

the event that we have extreme difficulty recruiting participants, we may revisit the idea of incentives

for students.

Analysis

To prepare our data gathered from the interviews for analysis, each interview will be transcribed

by one of the evaluators. Both evaluators will read over the interview transcripts, as well as the notes

SOPHOMORE SUPPER EVALUATION 27 written by the note taker. We have already established a priori codes and a coding rubric in order to

categorize the topics discussed throughout the session (Appendix G). In doing this, we also understand

that, as a fairly new program that has lacked formal assessment in the past, additional codes may be

generated while reviewing the transcripts. For now, our codes reflect the intended outcomes of the

Sophomore Supper program: to develop mentoring relationships with other members of the Loyola

community and to explore their passion areas to choose a major and vocation (Second Year Experience

Planning Document, 2012).

To analyze our data by codes, we plan to use the cutting/sorting technique. This will allow us to

observe the major themes present in the data, which is what we hope to get from our interviews with

students. In order to address concerns of validity, we plan to use triangulation. Triangulation means

multiple ways of analyzing and understanding information (Schuh, 2009). In order to practice

triangulation, both evaluators will read through all transcripts, as well as code the transcripts separately.

We also plan to triangulate by determining any connections and similarities between our survey results

and interview data. Additionally, we will practice member checks. Member checking requires the

involvement of interview participants in order to make sure we correctly understood the information

they shared (Schuh, 2009). We will use email communication with our student interviewees as our

primary form of member checking.

Limitations

There are a number of limitations to the qualitative aspect of our assessment. First and most

importantly, Roy works in the SYE office, and he is also directly connected to the Sophomore Supper

program. We hope to address this bias by using Kathy, an individual who is not a stakeholder in the

program, as a key component of the data analysis process. Additionally, student staff members and

professional staff members in SYE will be asked to serve in a face validity capacity, because of their

strong understanding of the needs of second year students and the role of the SYE office.


Both evaluators must also be prepared for the possibility for negative information to surface

throughout the data gathering and analysis process. Though this may make the data collection process

difficult, it is vital for both evaluators to understand this possibility from the beginning. In doing this,

the evaluators will be better prepared to view the data in the best interest of the students involved, and

not in the best interest of their position. It is also important to note that Roys position in SYE has the

potential to change for the next year. So, although Roy is invested in the program, there is an

understanding that his replacement for the 2014-2015 academic year will likely be able to view these

evaluation results from a much different lens than Roy. In the long term, both positive and constructive

findings from the data will be helpful for SYE to determine the next steps for the Sophomore Supper

program.

Results

For the final report, we will communicate results of the interviews in a table format. The table will

include each of the a priori codes that have been predetermined (Appendix G), along with a short

description of what each code means and how we developed that code. Beyond the table, we will also

provide direct quotes from students. These quotes will range from extremely positive to very

constructive to allow the best opportunity for improvement to the program. We see it as a necessary

component of the final report to include direct quotations, as these will likely help paint a more

comprehensive picture of the Sophomore Supper program for SYE staff.

Next Steps

Moving forward, there are number of next steps the researchers recommend in order to begin the

implementation of this assessment. As the next Sophomore Supper is scheduled for January 23, the

most pressing item is to reserve spaces necessary to conduct the semi-structured interviews with the

student volunteers. We also need to move quickly in imputing our survey into an online version with

Campus Labs for the students who leave the supper early and are unable to take the paper-and-pencil

SOPHOMORE SUPPER EVALUATION 29 version. While both the reservations and Campus Labs subscriptions would normally require funding,

our previously purchased subscription through the Division of Student Development, as well as our role

as an office with the division, allow us both of these amenities, free of an additional charge. Our budget

(Appendix H) lists additional resources required to implement the evaluation. The timeline (Appendix

I) lists a number of additional next steps including pilot testing both the survey instrument and the

interview protocol.

Conclusion

Looking ahead, we hope to provide the SYE office with a mixed-method evaluation approach to

determine if learning outcomes are being met, as well as to address the overall satisfaction of students.

Our quantitative and qualitative approaches provide concrete and tangible instruments to collect the data

with suggestions on the best method for analysis. It is our hope that SYE finds these useful as they

move forward in the development of the Loyola Experience for second year students.


References

Adams, W. C. (2010). Conducting semi-structured interviews in Wholey, J.S., Hatry,

H.P., & Newcomer, K.E. (Eds.) (2010). Handbook of practical program evaluation (Third

Edition). Jossey-Bass: San Francisco.

Association of Jesuit Colleges and Universities. (2013). Jesuit Institutions [online website].

Retrieved from http://www.ajcu/institutions

Bryson, J. M. & Patton, M. Q. (2010) Analyzing and engaging stakeholders in Wholey, J.S., Hatry, H.P.,

& Newcomer, K.E. (Eds.) (2010). Handbook of practical program evaluation (Third Edition).

Jossey-Bass: San Francisco.

Gardner, J. N., Pattenagle, J. A., Tobolowsky, B. F., & Hunter, M. S. (2010). Introduction. In M.S.

Hunter, B.F. Tobolowsky, J. N. Gardner, et al (Eds.), Helping sophomores succeed:

Understanding and improving the second-year experience (pp. 1-11). San Francisco: Jossey-

Bass.

Howes, S. D. (2013). Assessing and Meeting the Needs of Second Year Students. (Unpublished

evaluation assignment). Loyola University Chicago, Chicago, IL.

Hunter, M. S., Tobolowsky, B. F., & Gardner, J. N. (2010). Helping sophomores succeed:

Understanding and improving the second-year experience. Jossey-Bass: San Francisco

Lily Endowment, Inc. (2013). The Endowment [online website].

Retrieved from http://www.lillyendowment.org/theendowment.html McLaughlin, J. A. & Jordan, G. B. (2010). Using logic models in in Wholey, J.S., Hatry,



Newcomer, K. E. & Conger, D. (2010). Using statistics in evaluation in Wholey, J.S., Hatry,




Newcomer, K. E., Hatry, H. P., & Wholey, J. S. (2010). Planning and designing useful evaluations in

Wholey, J.S., Hatry, H.P., & Newcomer, K.E. (Eds.) (2010). Handbook of practical program

evaluation (Third Edition). Jossey-Bass: San Francisco.

Newcomer, K. E. & Triplett, T. (2010). Using surveys in Wholey, J.S., Hatry,



Schaller, M. A. (2010). Understanding the impact of the second year of college. In M.S. Hunter,

B.F. Tobolowsky, J. N. Gardner, et al (Eds.), Helping sophomores succeed: Understanding and

improving the second-year experience (pp. 13-29). San Francisco: Jossey-Bass.

Schuh, J.H. & Associates (2009). Assessment methods for student affairs. San Francisco, CA:

Jossey-Bass.

Second Year Experience (2012). Second year experience planning document. Retrieved September

2013.

Seidman, I. (200). Interviewing as qualitative research: A guide for researchers in education and the

social sciences (3rd ed.). Williston, VT: Teachers College Press.

SOPHOMORE SUPPER EVALUATION 32 Appendix ALogic Model

INPUTS

OUTPUTS Activities Participation

OUTCOMES IMPACT Short Medium Longer term

What we invest x Approximately

$1400.00 (per 40 people).1

x 1 professional

SYE staff member, 1 graduate assistant for SYE.2

x Social capital (in

the way faculty & administrators are chosen).

x Time (planning for

larger suppers can take 4-5 months, for smaller, 1 month).

x Room reservations

(space preparation contingent on past attendance).

What we do x Networking

activities (Mocktail hour, ice breakers in tables/introductions).

x Staff/faculty

facilitated conversation (questions will be in appendix and focus on vocational discernment, major selection, interpersonal connections).

x A meal. x Large group

debrief and information sharing.

Who is reached x Second year

students. x In the future

(specific populations of second years: pre-med, business, etc.).

x Faculty/staff/admi

nistrators. x Upper-class

student leaders.

Short-term changes we expect x Connect with

faculty/staff member.

x Learn about

different resources.

x Learn about

different major options, research opportunities, & leadership positions on campus.

Medium term changes we expect x Vocational

discernment. x Make

commitments (to major, study abroad, etc.)

x Develop

mentor/mentee relationships.

x Begin to

determine next steps at LUC (study abroad prep, internship planning).

Long-term changes we expect x Continue vocational

discernment. x Sustain

mentor/mentee relationships.

x Developing

mentoring relationships with less-experienced students.

x Determine how your

calling can impact the world.

x Go forth and set the

world on fire.3

1 Stakeholders urged researchers to take into consideration that a new, smaller format of the Sophomore Suppers was to be implemented in September 2013 and that the financial investment would be significantly smaller than that reported in this document. 2 The LDSYE office houses 4 staff members (1 director, 2 coordinators, 1 graduate assistant). The coordinator and graduate assistant for SYE work directly with the Sophomore Supper with the director and coordinator for LD assisting when needed. 3 One of Loyola University Chicagos key goals for its graduates.

SOPHOMORE SUPPER EVALUATION 33 ASSUMPTIONS 1) Most second year students need assistance in vocational discernment and major selection. 2) A facilitated dinner (in a group setting) is a comfortable and conducive place for these conversations for most students. 3) Faculty/staff/administrators are the best members of the Loyola community to talk to second years about these topics. 4) The time frame of the dinner (approximately 2 hours) is long enough to have these meaningful conversations around vocation, life, and school. 5) Evening is best time of day the event. EXTERNAL FACTORS 1) Insufficient advertisement 2) Introversion versus extraversion (how students engage) 3) Demographics of faculty/staff/admin (is there a lack of diversity in our facilitators?) 4) Time of daywhat are student commitments? Would breakfast/lunch be better? 5) Diverse student population (students with family commitments or commuter students might not have the opportunity to participate)


Appendix BOriginal Sophomore Supper Survey Instrument


Appendix CSophomore Supper Survey Instrument


Appendix DConstruct Map


Appendix ESemi-Structured Interview Protocol

Semi-Structured Interview Questions (Script) Semi-structured interviews (SSI) are viewed to be an intermediate method of interviewing that utilizes elements from both structured and unstructured interviews (Adams, 2010). SSIs are typically conducted in a conversation-style setting with one respondent at a time and blends together both closed- and open-ended questions. These questions are often accompanied by follow-up questions that enable the interviewers to probe deeper to gain further clarification for a response or for the respondent to reflect deeper (Adams, 2010). This interview is scheduled to last approximately one hour so as to minimize the risk of interviewer/respondent fatigue. Welcome and Interviewer Introductions (Brief) Hello everyone and thank you all for joining us today! My name is (NAME) and I am (AFFILIATION/POSITION AT LOYOLA). On behalf of ourselves and Second Year Experience we are extremely grateful for your willingness to participate in this interview to provide feedback about the Sophomore Supper program so that we can improve it for the future. Framing So just to give you some context as to what we are doing here today, we will spend the next hour talking about your experience as a participant of Second Year Experiences Sophomore Supper. Of course, there is no right or wrong answer to the questions we will be asking you, so do not hesitate or feel as though you have to provide the right answer. Your honest response will be very much valued and appreciated. Recording and consent forms We just want to let you know that what is shared today will be recorded by (NAME OF SCRIBE) and a digital tape recorder so we can be sure to accurately capture your responses to our questions. The information you share will be used to make changes and improvements to the Sophomore Suppers so as to be able to provide the best and most impactful experience for future participants. Any information shared here today will not be released to anyone beyond the Second Year Experience staff. We will now give you a consent form outlining your rights as a participant in this study and our promise to ensure your comfort, confidentiality, and safety--- please read it over and sign it if you agree to the terms stated. Do you have any questions before we begin? (Please see attached consent form). Purpose and Intent of the Program

The Sophomore Supper brings together sophomore students, like you, with faculty and staff to discuss the hopes, fears, and transitions that may come with your sophomore year. We know the second year of college can bring some anxiety, as you look to declaring your major, studying abroad, or taking on a leadership role on campus. Our faculty and staff are here to help with good food and good conversation. Our hope is that the one of the faculty or staff members you meet at the dinner will become a mentor for you as you move forward during your time at Loyola and beyond. The Sophomore Supper has several outcomes for the program participants as outlined by the Second Year Experience Planning Document (2012):

Outcome 2 [Second year students will be able to] articulate their unique passions, values and talents in connection to relevant possible vocational choices and academic majors.


Outcome 3 [Second Year students will be able to] demonstrate deeper and more meaningful relationships with mentors, faculty, staff and/or alumni.

Get started- introductions Well begin by getting to know one another more and hearing about our connections to this program and office. We will go around and please share your name. General experience questions: Q1: How prepared did you feel for the Sophomore Supper?

Probes: Did you feel like you knew enough about the event to feel prepared and confident about attending? Did anything surprise you?

Q2: How prepared did you feel to engage in personal conversations with Loyola faculty, staff, and administrators?

Probe: Was there adequate information given to you prior to the event that let you know who the Loyola faculty, staff, and administrators were?

Q3: Tell us a little about what the most memorable part of the Sophomore Supper was for you.

Probes: What was your favorite part, the most important thing you learned, the most meaningful part of your experience?

Q4: Can you share what part of the Sophomore Supper surprised you the most? Probes: What was something you didnt expect, something you learned that you didnt think you

would?

Q5: How were you able to connect with any of the faculty, staff, or administrators during the Mocktail hour of the event?

Probes: Was it difficult to approach these people? Did you know what to say?

Q6: What, if anything, would you change about the Sophomore Supper? Probes: What didnt have value, or had a negative impact on your experience?

Process related questions: Q7: What, if any, part of this Sophomore Supper was helpful in getting you to connect with members of the Loyola community? In talking about your own passions and calling in life? In talking about your major?

Probe: What, if anything, was not helpful? Too complicated? Boring? Uncomfortable?

Q8: What things, if any, did your facilitators (faculty, staff, or administrators) do that were helpful? Probe: How did the facilitators create an environment that was comfortable for you to open up?

To engage in reflection about your passions?

Q9: How comfortable and confident were you in being able to open up and share about your passions, your major, your desired career path?


Probe: Did you feel comfortable speaking and sharing personal information about yourself at a table of 10 other people?

Q10: Did being able to communicate with a Loyola faculty, staff, or administrator in a public setting like this allow you to reflect on your passions, what you feel called to do, your major, and/or life in general?

Probe: How deep were you able to go with your conversation? Did you feel like you were being heard and engaged in the dialogue?

Wrap-up: Q11: What impact, if any, has your participation in the Sophomore Supper had on your involvement on campus?

Probe: With other organizations/offices? Off-campus? Classes you are thinking about taking? Did it at least get you thinking about getting involved?

Q12: After attending the Sophomore Supper, how comfortable do you feel approaching Loyola faculty, staff, and administrators to talk over your goals, ambitions, and life?

Probe: Do you feel as though you genuinely connected with a Loyola faculty, staff, or administrator?

Q13: Is there anything else you want to add before we wrap up our time together?

Probe: Anything we did not ask you about that you think we should? Conclusion and Thank you We truly appreciate and value you sharing your feedback with us today. The success of the Sophomore Supper program is deeply connected with your participation and feedback for this event. It is important to us and Second Year Experience that we receive your input and seek to understand how the experience was for you, so that we can improve the program for the future. Thank you so much for taking the time today to have this conversation with us! We look forward to working with you and seeing you at our events.


Appendix FConsent Form Consent Form for the Sophomore Supper Interview

You have been invited to participate in an interview for Loyola University Chicagos Sophomore Supper program, hosted by Second Year Experience. Purpose of the Interview

This interview will collect information about your experience with the Sophomore Supper program you recently attended. Please read this form carefully, as it contains important information about your consent to participate in this study and interview. If you have any questions, feel free to take a moment to ask the facilitators in front of you.

Once you have agreed to participate in this interview, you will be asked a number of questions related to your overall experience of the Sophomore Supper. While we hope that all participants will respond to all questions of the questions we have, if at any moment in time you feel uncomfortable responding, you are not required to do so. There are no correct or incorrect responses to these questions-- all we ask for is for your honest response. Your participation in this interview is entirely voluntary and you are free to withdraw your participation in this study at any point in time without penalty.

Your name will be omitted as part of your participation in this focus group and a pseudonym will be provided to you if you do not wish to select your own. The information that you choose to share with us today will only be shared with the Second Year Experience staff. We ask that participants respect the confidentiality of fellow participants and not discuss what is shared during the interview outside of the space. Interview Procedure

This session will be audio recorded for the purposes of this evaluation process; the Second Year Experience staff will be authorized to use the contents of this recording for purposes related to the evaluation. Risks & Benefits There are limited risks involved in participating in this interview. Participants will agree to keep what is shared during the interview within the physical space; that they do not break confidentiality; do not disregard this agreement; and do not discuss what is shared or disclosed during the interview outside the space. Benefits to participation may include increased understanding of your experiences as a result of participating in the Sophomore Supper and how it connects to your experience at Loyola and beyond. Other benefits include providing Second Year Experience with information that can help improve the program for the future.

If you have any questions about the interview, you may contact John Cot, Coordinator for Second Year Experience at [email protected]. A copy of this document will be provided for your records if you wish. If you agree to the terms above, please sign below: __________________________ Date:___/___/_____ Participant Name __________________________ Participants Signature


Appendix G- A Priori Coding Rubric

Coding Rubric: A Priori Codes

Construct Meaning Definitions

Code

VOCA Vocational Discernment

Discovering one's passions and using those passions to discern what one is called to do is a key milestone for second year students. Due to participation in the Sophomore Supper, second year students engage in conversation around vocational discernment and begin to articulate what they feel called to do. Students' perceptions of engaging in dialogue over vocation.

MAJOR Major Selection & Involvement

Students engage in conversation around major selection and how to further their involvement within their academic discipline. Students' perceptions on role of major selection and involvement.

RELATE Build Relationships

Second year students often find it difficult to build and maintain relationships with authority figures. Students who attend the Sophomore Supper try to build relationships with other attendees. Any comments made about building relationships during the supper.

CONNECT Make Connections

Connecting with peers and professionals at a university helps retain students. Second year students attempt to make connections with other attendees to add to their personal and professional networks. Any comments made on connecting with others at the supper.


NETWORK Develop Networking Skills

The Sophomore Supper provides second year students with the opportunity to practice essential networking skills. General perceptions of networking process or opportunities displayed.


Appendix HSophomore Supper Evaluation Budget

Activity Item Cost Quantity Total

Survey Instrument Creation (Qualtrics) $0.00 1 $0.00

Survey Administration

Campus Labs (via Leadership Development & Second Year Experience account) $0.00 1 $0.00

Interviews

Room reservation (Damen RM 128 $0.00 1 $0.00

Digital audio recorder (Apple iPad from LDSYE) $0.00 1 $0.00

Print outs of interview questions $0.00 15 $0.00

Print outs of consent form $0.00 15 $0.00

Statistical Analysis

SPSS software (available in Loyola computer labs) $0.00 1 $0.00

Data Analyst (LDSYE Graduate Assistant) $0.00 1 $0.00

Interview Decoder (LDSYE Graduate Assistant) $0.00 1 $0.00

Report Dissemination

Paper Dissemination (reports to LDSYE and Division of Student Development) $0.00 3 $0.00

Online Dissemination $0 1 $0.00 Total $0.00


Appendix ISophomore Supper Evaluation Timeline

Timeline for Sophomore Supper Assessment November December January February March

Design Tool Pilot Test Instrument (SYE student workers) Submit to Campus Labs Finalize online survey (for participants that leave early) Schedule Interview based on staff availability

On-going

On-going

On-going

Reserve conference room On-

going On-

going On-

going

Spring Sophomore Supper #1 1/23/14

Handout survey to participants 1/23/14

Close survey 1/23/14 Email interview request to participants 1/24/14

Email reminder request 1/27/14 Analyze Survey results, SPSS entry 1/27/14 Send confirmation to participants 1/29/14

Conduct Interviews On-

going On-

going On-

going

Analyze interviews, code data On-

going On-

going On-

going

Prepare Finalized report 3/24/14 Present findings and recommendations 3/31/14


Appendix J2012-2013 Sophomore Supper Summary 2012-2013 School Year

Updated December 6, 2013

www.luc.edu/secondyear

LOYOLA UNIVERSITY CHICAGO

Second Year Experience Planning Document

SOPHOMORE SUPPER EVALUATION 50 2012-2013 Programming and Outreach Summary Sophomore Suppers

Description The Sophomore Supper program brings together sophomore students, like you, with faculty and staff to discuss the hopes, fears, and transitions that may come with your sophomore year. We know the second year of college can bring some anxiety, as you look to declaring your major, studying abroad, or taking on a leadership role on campus. Our faculty and staff are here to help with good food and good conversation. Our hope is that the one of the faculty or staff members you meet at the dinner will become a mentor for you as you move forward during your time at Loyola and beyond. 2012-2013 Implementation We hosted one supper this year, which was less than originally planned. There was concern with being able to recruit enough students for the smaller dinners so the large format was attempted. Students enjoyed the supper very much, which allowed them to connect to staff and faculty whom they may not have connected with previously. Students expressed a strong desire to connect with more faculty from their major/school. The cocktail hour prior to the dinner also provided a chance for students to practice networking skills, which is an important skill to learn for their futures. The faculty seemed to be excited to host the tables and enjoyed the conversations, from discussion that evening.

Budget $1,339.89 Attendance 27

2013-2014 Vision Next year, SYE plans on having one large format supper, similar to the 2012-2013 supper, as well as several smaller suppers themed around a particular school/college, major, or vocation. This will allow the flexibility to introduce a large audience to the suppers initially during Welcome Back while giving an opportunity to connect specifically with people in their interest areas. We would still recruit hosts at a ratio of 1 host per 4-6 students. The large scale supper will be part of welcome back week and can feature the cocktail hour before hand. The small scale suppers can take place either on campus or in faculty members homes. It is recommended that the hosts be recruited from a variety of disciplines and outside the normal student development group. Dinners at hosts homes would provide a unique program for students while also decreasing costs significantly from Aramark. There are two confirmed dates for next year: Wednesday August 28th at 6pm for the large scale supper, and October 22, at 6pm for the Catholics on Call Supper. The other themed suppers will take place with a goal of one every one-two months. SYE has established an understanding with Catholics on Call that a special supper will take place on October 22 to explore the vocation of ministry, both lay and ordained. Catholics on Call will work to get hosts for this event, while SYE will work to get students.

Budget Requested $2,000.00 total

SOPHOMORE SUPPER EVALUATION 51 $500 for large supper (in addition to $500 from WB Week budget) $100-$200 for each small sup

ELPS 431

Documents

Transcript of ELPS 431