ROBIN G. ISSERLES – PROFESSOR, SOCIOLOGY, BMCC, CUNY DANIEL DOUGLAS – SENIOR RESEARCHER, EDUCATION AND EMPLOYMENT RESEARCH CENTER,

RUTGERS UNIVERSITY

PEDAGOGY, RESEARCH, AND THE HIDDEN LIVES OF STUDENTS

OUTLINE

•  Data, Methods and Assumptions in Community College Research

•  Evidence from Experimental and Quasi-Experimental Research

•  Discussion of Pedagogical Implications

•  Research on Academic Momentum

•  Student Sensibility

•  Discussion of Pedagogical Implications

DATA, METHODS, AND ASSUMPTIONS IN COMMUNITY COLLEGE RESEARCH

•  RCTs and QEDs are two ways to assess the causal impacts of policy changes or new interventions.

•  Randomized Controlled Trials viewed as the “Gold Standard” of Educational Research •  Quasi-experiments accepted as a provisional alternative

•  “What Works” as the dominant theoretical framework •  Narrows the field of eligible questions •  Elides large structural forces •  Typically assumes that there are deficits in the ways

community colleges and their students operate

NOT SO GOLD: DISTINCTIONS AND LIMITATIONS

•  Interventions at CC’s evaluated by RCT/QED exist on a spectrum: •  “Invasive” interventions seek to fundamentally change the

student population – particularly hard to evaluate •  “Adaptive” interventions seek to alter institutional structures

– some have found strong effects

•  All RCT/QED evaluations are subject to important limitations: •  RCTs: Attrition, Defiance, Labor-intensive implementation,

External Validity •  QEDs: Complicated analysis, Narrow analytic range •  Both: Uncontrolled settings, The problem of “how?”

EVIDENCE I: MATH MAINSTREAMING

•  Funded by Spencer Foundation and CUNY; implemented and analyzed by CUNY researchers

•  Addresses high rate of math remediation, which is seen as a hurdle to degree completion, by placing students directly into College level math

•  In Fall 2013, 907 first-time students assessed as needing elementary algebra were randomly assigned to: •  Elementary Algebra (EA) – CONTROL GROUP •  Elementary Algebra + Workshop (EA-WS) •  College-level Statistics + Workshop (Stat-WS)*

EVIDENCE I: MATH MAINSTREAMING

Proportion Attrition from Assigned Group

Proportion Passed Assigned Course

College Credits earned after one year

Elementary Algebra (Control Group)

18 percent 39 percent 15.8 Credits

Elementary Algebra + Workshop

27 percent* 45 percent 14.4 Credits

College Level Statistics + Workshop

17 percent 56 percent** 20.5 Credits**

EVIDENCE II: OTHER RCT STUDIES

Student Retention

Credit Earning

Degree Completion

Published Source

First-Year Learning Communities1

Positive effect

No effect No effect MDRC (2014)

Student Coaching2

Positive effect

Not assessed Not assessed Bettinger & Baker (2014)

CUNY ASAP Program3

Positive effect

Positive effect

Positive effect

MDRC (2015)

- Computer-based accelerated developmental math in a summer bridge program has shown strong early impacts.

PEDAGOGICAL IMPLICATIONS 1: QUESTIONS TO THINK ABOUT

•  How can what we know from these studies be communicated to our students and colleagues?

•  How might these findings impact the way you advise students?

•  How might you think about using these findings in your classroom?

PEDAGOGICAL IMPLICATIONS 1

Math Mainstreaming

Other RCT/QED Studies

CUNY RESEARCH PROJECT ON ACADEMIC MOMENTUM

§  Funded by Bill & Melinda Gates Foundation § Momentum: Net of demographics and pre-college

prep, students’ academic engagement is predictive of retention and graduation.

§ Operationalized momentum as credits attempted;

looked at early momentum among first year students using three ‘treatments.’

§  Intervention targeted so-called “Reluctant” students

ACADEMIC MOMENTUM – QUASI-EXPERIMENTAL FINDINGS

Student Retention

Credit Earning

Degree Completion

Summer Bridge Program1

Positive effect (5 pct pts)

Positive effect (3.5 credits)

Not Assessed

Increased Enrollment Intensity2

Positive effect (24 pct pts)

Not assessed Positive effect (16 pct pts)

Summer Course after First Year3

Positive effect (13 pct pts)

Not assessed Positive effect (8 pct pts)

ACADEMIC MOMENTUM – RCT FINDINGS

Student Retention

Credit Earning

Degree Completion

Summer Bridge Program

Negative effect (-8 pct pts)

No effect No effect

Increased Enrollment Intensity*

Positive Effect (+20 pct pts)

Positive Effect (+3 credits)

No effect

Summer Course after First Year

Positive Effect (+ 9 pct pts)

Positive Effect (+3 credits)

Positive effect (+6 pct pts)

*Very small sample size, results should be interpreted with caution

•  90-minute interviews w/30 study participants (Summer and Fall 2013)

•  60% female •  60% Black and/or Hispanic

•  24 years old (average)

•  By the Fall 2014 semester (based on 25 students from cohorts F10, F11, F12): •  2 transferred to 4yr CUNY, still enrolled •  10 graduated or earned at least 60 credits •  4 still at community college •  9 stopped out, dropped out, or had earned less than 20 credits

INTEGRATING QUALITATIVE RESEARCH: GETTING AT THE HIDDEN LIVES OF STUDENTS

THE HIDDEN LIVES OF STUDENTS

•  Student vulnerabilities

•  Institutional assumptions

•  Enculturation into student life

•  Student Sensibility

STUDENT SENSIBILITY

•  Attunement to the norms and expectations of college life •  A priori knowledge of navigating through institutions

•  Comfort with asking questions/advocating for themselves •  Central identity as college student

•  When students have a low degree of this sensibility, it often appears as reluctance, passivity, apathy– in other words, as a problem with individual motivation. But it is far more complex

and requires institutional and pedagogical responses.

•  Important early college experiences where a low level of student sensibility hinders their persistence

and progress

•  Choosing a college •  The Entrance/Placement Exam •  Financial Aid & Paying tuition

•  Advisement, Course scheduling

STUDENT SENSIBILITY

Different socialization around higher education: •  Most know college is important… BUT

•  Often lack parental guidance and knowledge about college •  Don’t have the tool set to research, visit, develop questions to

ask. •  Decisions of where to attend based primarily on location and

cost.

“Nowadays you need a degree for everything, even to work at a McDonald's you need a degree for that. You can’t just go straight out of high school.” [Lorenzo] “Q. But before you went you’d literally never been there? A. I had no idea about it. Nothing at all.” [Sabino]

CHOOSING A COLLEGE

•  Student identity: Most students don’t think of themselves first and foremost as college students in the way that traditional college students do.

•  “It wasn’t like a big deal for me to say I’m going to college. It was just something that had to be done, I guess” [Cherise]

•  Not knowing what questions to ask/feeling entitled to ask: “Actually for some reason, they put me in an associate degree. So when I was trying to [do] [specific courseA] 2 and [specific courseB] 2 it didn’t let me, I had a block because it was only for the bachelor’s degree… I have no idea I just spoke to the chair of my department and she said I was in an associate’s degree. And I’m like, I don’t know why” [Alicia]

CHOOSING A COLLEGE

•  “I’m kind of disappointed myself because I kind of went through the campus test, I didn’t take my time… I actually had to work that day, I had to be at work for 1:00 and I was there for 12:00 or 11:00 and I had to rush back to Brooklyn to get to work and it was just a Saturday, I just wanted to get my day started so I wasn’t thinking until it was time to register for class and I’m like, damn, why did I do that, I should have taken my time.” [Naritza]

•  “So, I told her and she said, okay. So, she pulled up my records and was looking at my SAT scores and stuff, and then she asked me, why are you in remedial English? And I said, because that's where I was placed. They told me I had to go. Then she said, not according to your SAT scores. Yeah. So, I pretty much did that entire intensive course for nothing…. Q. So, did you question the fact that you didn't think you belonged there or-- I mean, it was hard. … It was hard to believe but I was like what am I going to do about this? The school says this based on my test, that’s what I thought.” [Kayla]

THE ENTRANCE EXAM- CUNY ASSESSMENT TEST (CAT)

FINANCIAL AID/TUITION

•  Throughout the CUNY system, FA offices are oversubscribed and understaffed. Several students spoke of very long lines. •  “Mm-hmm. I wait. I remember the first time I went to the

financial aid in XXXX, there’s like a really long hall, maybe the length of a subway train, that’s how long the line was.” [Darius]

FINANCIAL AID/TUITION

•  Fairly draconian FA policies in place •  In our small study, 3 of the 30 students we interviewed encountered

major problems with inflexible FA policies that led them to leave school.

•  Many more experienced major disruptions: •  It was a situation, my second semester …I didn't know you had

to pay a little part of it, because they just give you the rest of the loan. So, I just went to my second session class and it dropped me. I didn't understand. I went to room after room trying to answer my question. One person said you didn't pay $63.00…. That same person said don't worry about it, pay it, I'll put you back in the class. I said, fine, let me pay it now… I go back to the person, the person was busy. Went to the next person, the person said, well, it doesn't matter. You could pay it. You're not going back in the class… This fall I have to take it over. [Celestine]

•  Advisement and scheduling are two distinct processes.

•  Students are advised what classes to take by faculty or non-faculty advisors, but students are responsible to do the actual scheduling of their classes.

•  The double sword of DegreeWorks

ADVISEMENT & SCHEDULING

•  “I guess because…they didn't advise me on the hours. So, the first year I was taking basically, a total of 10 hours for four days of the week. And no one actually sat down and said, okay, look at the time…it was a 10-hour day for four days, I had six hour breaks between classes.” [Celestine]

•  “[I]t was a hard semester just because of the scheduling. It was a horrible scheduling, because I know Mondays I had a 13-hour day in school. That was a hard day. I had classes 8:00 in the morning and I did not get out till like 9:00 at night. [Travis]

•  “…they'll put a stop on your classes until you see an advisor. But I only saw

that person one time, and the second time-- basically, there are so many students waiting….It was so confusing I just went online and just did my classes myself…there was another advisor there, I'm not sure of his name. He told me if I send him an email, basically, he'll just see my information and he will just take the stop off.”[Saul]

ADVISEMENT & SCHEDULING

“Everywhere you go people say, go there, go there. It's a goose chase in there, that's why I didn't like that school at all.” [Travis] “Other than the transfers, the situation where if you had a problem with anything, like teachers, tuition, loans, whatever it's a long process. And the longer the process because you jump from one person to another person to answering one question, and each person gives you a different answer. [Celestine]

HOW STUDENTS RESPOND: CAUGHT IN THE IRON CAGE

“Q. And so, you switched to biotechnology. Had you met with a Biotech faculty or advisor about the major? A: No. I made that decision on my own.” [Kayla] “I did go to some advisors a few times just to make sure…everything was good. But for the most part, if I had a question about a class…I'd call my sister's friend and ask her whatever.” [Belle] “Basically, my friend was still going to XXX at the time. So, she was telling me how to get to places… I had a cellphone number so I called her… and she just helped me out.” [Saul]

HOW STUDENTS RESPOND: DIY SCHOOLING

“When the semester came I didn’t have any financial aid because I don’t live with my parents so it’s hard for me to get financial aid again. So I never got to do my second semester... [Asked about returning]… Yes, when I become independent. That’s my only choice… I can’t do anything to put myself back into school because I’m not independent…If it was meant to happen, it would have happened.” [Grayson] “Okay, what happened was that it was the first week when class actually started…Then there was, I guess, a mistake on my financial aid because my guidance counselor in high school filled out my application... I guess she was overwhelmed with so much people and I guess she made a big mistake, the 20XX’s tax return I did $9,000 and she put that I made $39,000…So from there I received a letter from the financial aid stating that there was an error so they thought I was trying to commit fraud…So I only went to school for three days and I got kicked out. [Derrick]

HOW STUDENTS RESPOND: DISENGAGEMENT

“I went to that school 10 times before it was fixed. So, I just got my sister, because she is very articulate with things…she is like I'm going to come with you… and they're just going to have you talk to the people who are at the front…no, you're going to talk to the dean. Because she has went with me like three times…”[Travis] “…it all comes from you kind of taking charge also a little bit. Like if there is something you don't understand, tell the professor. Don't just walk away and be like, all right. You have to be pushy. For me I was a little pushy like also in terms of the class that I wanted to get into… And I guess that's how I got my way through.” [Belle]

HOW STUDENTS RESPOND: PERSEVERANCE WITH SUPPORT

“The professor actually helped me during the advisement period. So they came and said, we recommend that you take these classes first and then you start your credits…the professor took us to to the main campus to get registered for the full class.. He took us from the xxxx building to the main campus to the basement to the registration, he did us that favor because everybody was like, ‘we don’t know where it’s at’. So he was like, ‘all right, I’ll show you guys’.” [Derrick]

WHEN THINGS WORKED: INCIDENTAL INSTITUTIONAL SUPPORT

PEDAGOGICAL IMPLICATIONS 2: QUESTIONS TO THINK ABOUT

•  How does thinking about the Student Sensibility influence what/how we teach, advise, interact with

our students?

•  How do we help students develop a sense of themselves as college students?

•  How do we help students develop a sense that they can and should advocate for themselves?

•  How do we ensure that this doesn’t become too overwhelming for us?