The Interplay of Teaching, Learning, and Research: Graduate Student Collaborations in Law and...

The Interplay of Teaching, Learning, and Research:

Graduate Student Collaborations in Law and

PsychologyA. Holtzworth-Munroe, Ph.D. & A.G. Applegate, J.D.

Annakarina BortnerB.M. D’Onofrio, Ph.D. & J.E. Bates, Ph.D.

Indiana University BloomingtonMarch 30, 2012

OutlineI. Introduction

A. Interdisciplinary TeamB. Child Informed Mediation Study (CIMS) – Interdisciplinary

Research Program (improving outcomes for families with children)

C. CIMS – Opportunity for Interdisciplinary Training (in law, psychology research, and program evaluation)

II. SOTL Study – Interdisciplinary Training Program (improving student learning outcomes)

A. Methods B. Knowledge Tests: Methods and ResultsC. Focus Group Discussions: Methods and Results

1. Focus Group Coding2. Focus Group Student Quotes

D. Coding Student Behavior During Mediations E. Student Involvement in Program Evaluation Research

III. Discussion

I. Introduction

A. Interdisciplinary Team

Interdisciplinary Collaborations Grow out of Overlapping Interests and Goals:

Our Team • Amy G. Applegate (IU School of Law):

• Family and Children Mediation Clinic • Training clinic for law students• Interested in effectiveness of her training and program

• Amy Holtzworth-Munroe (IU Psychology)• Relationship distress• Couples therapy• Intimate Partner Violence

• Brian D’Onofrio (IU Psychology) • Behavioral genetics; divorce and mediation

• Jack Bates (IU Psychology) • Developmental psychopathology; Longitudinal research with children

• Graduate and Law Students

Goals

• Does interdisciplinary training improve student learning outcomes (SOTL)?

• Does interdisciplinary training improve outcomes for families and children in mediation (CIMS)?

Interdisciplinary Training• Law and Psychology intersect in real world• Family law: area of extensive overlap

• Psychologists conduct child custody evaluations • Family mediators may be lawyers or psychologists

• Family Law Education Reform Project (FLER) urged more interdisciplinary training in family law

• But little social science research on family law issues or interventions

Mediation in Parental Divorce or Separation Cases

• Litigated cases – Adversarial process– Many parents without lawyers– Burden on the court system

• Mediation– Confidential settlement process– Party self-determination – voluntary, informed decision-making– Impartial, neutral mediator assists parties in

• identifying the issues• reducing misunderstanding• clarifying priorities• exploring areas of compromise• finding points of agreement as well as legitimate points of disagreement

Mediation in Parental Divorce or Separation Cases

• Mediation as an alternative to litigation– Mediation assumed better:

• Parents know what is best for their own children• Parents invested in the process

– One study (Emery et al, 2001) found mediation:• Increased involvement of nonresidential parent• Not associated with increases in co-parenting conflict• Increased cooperation and flexibility

Setting for Our Interdisciplinary Training Effort:IU Maurer School of Law Viola J. Taliaferro

Family and Child Mediation Clinic

• Existing law course/clinic, before SOTL study of interdisciplinary training:

• Intensive Course in Domestic Relations Mediation (40 hours)– Law students become registered (licensed) mediators

– Course had state-required interdisciplinary component

• Transition into Clinic– Law students co-mediate 5-8 cases during semester

– Weekly classes

– Supervision, reflective journals

– Law student mentors

– Clinic did not have interdisciplinary component

IU Clinical Science Program • Existing psychology clinics and practica

before SOTL study of interdisciplinary training:

• Evidence Based Practice– Basic Research on Families and Children – Research on Interventions– Evidence Based Therapy Practica

• Became interested in possibility of interdisciplinary approaches to mediation

I. Introduction

B. Child Informed Mediation Study (CIMS) – Interdisciplinary Research Program(improving outcomes for families with children)

Interdisciplinary Mediation Approaches

• McIntosh: two new interdisciplinary approaches – Both involve a “child consultant”

• psychology or mental health professional

– 1) Child Focused Mediation (CF)– 2) Child Inclusive Mediation (CI)

• In both CF and CI, legal (mediator) and psychological (child consultant) professionals work together to prepare case and work with parents.

• Goal:– Bring child’s “voice” or perspective into the mediation– Make agreements that best meet needs of the children

Child Information Mediation Study (CIMS)

• Use CF and CI – interdisciplinary approaches to mediation

• Mediators = law students• Child consultants = psychology graduate

students

• Working together for better outcomes for families with children.

Child Focused Mediation (CF)• For parents with children of any age • Intake: Mediators (law students) meet and screen

parents• Child consultant (psych student) does not meet

child or interview parents but gets basic information about children

• Parent feedback session (at start of mediation): Child consultant provides parents with developmental information about children and impact of parental separation and conflict on children; tries to individualize to their children

• Mediation negotiations: After parent feedback session, parties enter negotiations with mediators

CF: Parent Feedback Session• Child Consultants (psych students):• Ask Questions about Children

– Can you tell me about your child? Picture of them? – What activities do you enjoy with your child?– In 20 years, what do you want your child to say about how you

handled this situation?• Give research-based information on issues to be

considered in mediation:– development – impact of parental separation – impact of inter-parental conflict– other issues (e.g., parenting time)

• Often use educational materials to make points

Child Inclusive Mediation (CI)• For parents of children ages 5 - 17 • Intake: Mediators (law students) meet and screen

parents• CI Assessment: Child consultant (psych student):

– developmental history interview (regarding children) with each parent

– Interviews child• Parent feedback session (at start of mediation):

Child consultant provides parents with feedback about their child

• Mediation negotiations: After parent feedback session, parties enter negotiations with mediators.

Child Interview in CI Mediation• Explanation

– Understand what its been like for you– Not asked to make decisions– Will share messages with parents (confidentiality)

• Age appropriate assessment of:– Experience of separation and conflict– Attachment to each parent and others– Needs and options

Child Inclusive Example

Altered to Maintain Confidentiality

Case 1: Will • Will was 10 years old • Parents recently separated• Interview with Will

– Quiet, shy, reserved, careful (detail re picture)– Cooperative– Bright, doing well in school, activities, friends

Will

• Magic Wand Question:– Want them to stop fighting and get along

Case of Will

• When child consultant met with parents:• Focus on inter-parental conflict

– Impact on Will – Need to decrease conflict

• in general and in front of Will

– Need for him to maintain good relationships with both parents (get re-connected), but their conflict was interfering with this

Why CF and CI May Work• Wakeup Call to Parents

– Emotional information changes their behaviors– Focus away from “winning” against each other to

doing what is best for their children• Developmentally Appropriate Agreements

– Favors stability and stronger parent-child relationships

• Interdisciplinary Consideration of Issues**– Helps change mediation agenda:

• During mediation negotiations, law student mediators refer to messages from the psychology student child consultants

McIntosh , 2007

McIntosh’s (2008) Research: Comparing CF and CI Mediation

• Compared CF and CI mediation:– In Australia– ~150 families, children ages 5 - 17– No random assignment: CF first, then CI– No Mediation As Usual (MAU) comparison group

• 2 year follow-up data:– Both lowered inter-parental conflict– Both improved child mental health

• CI fared better:– Father satisfaction with parenting arrangement– Father involvement with children’s lives– Less re-litigation

• Improved outcomes for families important to us.McIntosh & Long, 2006; McIntosh, 2007

Child Informed Mediation Study (CIMS)• Our study of new, interdisciplinary mediation

methods• In U.S. • Random assignment to types of mediation• MAU as a comparison condition

• Children ages 5 – 17: – Randomly assigned to 1 of 3 conditions:

• MAU• CF• CI

• Children all under age 5: – Randomly assigned to 1 of 2 conditions:

• MAU• CF

Child Informed Mediation Study (CIMS)• Most cases referred to Law School Clinic by court• Mediation intake

– law student mediators• Parties invited to participate in study

– psych student child consultants• Random assignment to CI, CF, or MAU• Research assessment conducted

– Parents and children age 5 or older• If CI: child interview conducted

– psych student child consultants • Mediation

– IF CI or CF: parent feedback session • child consultants and mediators present

– CI/CF/MAU: negotiation and try to reach agreement • mediators

• Post-mediation research forms – parents and students

• One and two year follow-up research assessments– Parents and children age 5 or older

I. Introduction

C. CIMS– Opportunity for Interdisciplinary Training (in law, psychology research, and program evaluation)

Child Informed Mediation Study (CIMS)

• Interdisciplinary teaching/training opportunity • New practicum opportunity for psychology

students:– No previously existing practicum on mediation or

child consultation • Law school mediation course and clinic:

– Significant change to curriculum and practice• Separate mediation intake and negotiation sessions• Have child consultants present in CI and CF • Affect on mediation agenda

Interdisciplinary Training in CIMS Study: Learning Family Law and

Psychology Research Findings• Law students learn about psychology research findings

on effects of divorce and child development:– Guest lectures (and readings) from psychology professors– Involved in CI and CF parent feedback sessions conducted by

psychology child consultants (hear information presented to parents)

• Psych students learn about law and mediation:– Guest lecture (and readings) from law professor– Feedback from law professor on parent feedback sessions– Planning cases with mediators, and mediators in room for

the parent feedback sessions• Both: Joint case rounds

Interdisciplinary Training in CIMS Study:Learning Program Evaluation Research Methods

• CIMS is a program evaluation study:– uses psychology research methodology to

compare 3 types of mediation • Evidence Based Practice movement:

– Established in psychology– New but gaining recognition in law and family law

• Opportunity for interdisciplinary training of students in program evaluation research

Interdisciplinary Training in CIMS Study:Learning Program Evaluation Research Methods

• Law Students:– Learn importance of evidence based practice and learn program

evaluation methods – Guest lecture from psych professors– See research being conducted in mediation clinic

• Psych students:– Learn about program evaluation in real world setting– Learn to disseminate program evaluation to other disciplines– Apply knowledge from other classes to this study

• Both involved as:– Research participants:

• Complete Informed Consent Form • Student outcome form (their perceptions of the case)• Had behavior during mediation observed and coded (discussed later)

– Researchers:• Planning and implementing study • Interpreting findings and generating questions for the future

Interdisciplinary Work: Potential Misunderstandings

in Program Evaluation Research

• Terminology:– “Experiment” – “Issues”– “Psycho-”

• Study participant recruitment– Coercion and Consent (legal and research)

• Confidentiality – Legal concerns

• Privileged communications • Self-incrimination • Prevent possible use in court cases

– Research frustration• Changing/impeding the process

Interdisciplinary Work: Potential Misunderstandings

in Program Evaluation Research

• Random assignment– Lack of equal access vs “Do No Harm”

• Treatment manuals – Manuals vs individualized plans for each case

• Ease of implementation and affordability– Added work for students/more professionals/higher cost

• Support for involvement in research – Need approval and support from centralized authority (judges)

• Students were involved in discussion of these issues, when designing and implementing the CIMS study – Interdisciplinary training re program evaluation research methods

II. SOTL Study – Interdisciplinary Training

Program

SOTL Study ofInterdisciplinary Training of Students

• CIMS study designed to:– Examine whether interdisciplinary training and

practice make divorce mediation more effective• Better for families and children

• CIMS study provided opportunity for SOTL study: – Evaluate interdisciplinary training

• Effect on students’ learning and attitudes

SOTL Study Hypotheses• Effects of the interdisciplinary training will:

– Increase students’ knowledge of family law,

psychology research, and program evaluation

– Increase students’ positive attitudes towards

interdisciplinary work and research

– Students will become involved in program

evaluation research

II. SOTL Study

A. Methods

Measures: Overview• Student Knowledge Tests

• Knowledge acquired • Student Focus Group Discussions

• Experience and attitudes• Attempt to Code Student Behavior During

Mediation – discuss later

• Student Involvement in the CIMS Study – Information on whether they are doing program

evaluation research?

Timing of Measures: Knowledge Tests and Focus Group Discussions

• To examine change during semesters– Measures given at:

• Start of semester (pre-training)• End of semester (post-training)

• To examine effects of interdisciplinary training relative to a “no-interdisciplinary training” control condition– Measures given during:

• Baseline (pre- interdisciplinary training) semester• Interdisciplinary training semesters

Baseline semester

• Pre-interdisciplinary training comparison group (Spring 2009)

• 8 law students – Mediation As Usual (MAU) training only

• no CI or CF training• no systematic interdisciplinary training

– Note: Psych professors did give other guest lectures that semester (horse already out of the barn?)

• 11 psychology students – No relevant training that semester – No post-semester baseline group data for

psychology

Interdisciplinary Training Semesters

• Interdisciplinary semesters (Fall 2009, Spring 2010, Fall 2010)– MAU, CF, and CI mediations conducted– CIMS program evaluation study conducted – Guest lectures (psych and law)

• Law Students:– ~8 new law students each semester (N = 25)

• Psychology Students:– Same 8 psychology students in all three semesters of

training (N = 8) – Only examined psychology students’ data for the first

semester (Fall 2009) • Data comparable to that of law students (one semester)

Students’ Previous Relevant Training • No random assignment of students to baseline

versus interdisciplinary training semesters– so don’t know if they are comparable or not

• Assessed their previous training – background questionnaire

• Baseline semester > interdisciplinary semesters students in:

• Relevant previous courses • Other relevant experiences (e.g., research)

– Doesn’t end up being a problem for data analyses• Potential Problem: Ideally would have more

baseline semesters – But faculty and students eager to get going on

interdisciplinary training

Outcome Measures

KnowledgeTests

Focus Groups

LegalQuestions

PsychologyQuestions

ProgramEvaluationResearchMethods

PsychologyResearch onDivorce

Law Students Only

Psychology Students OnlyBoth

Outcome Measures

II. SOTL Study

B. Knowledge Tests: Methods and Results

Knowledge Test Questions

Law: • 31 Questions• Relevant laws and legal statutes about divorce & mediation

Program Evaluation Research Methods (PERM): • 12 Questions• Program evaluation methodology

Psychology Research on Divorce (PRD): • 17 Questions• Research findings on relevant issues

– psychological consequences of divorce for children– effectiveness of interventions for divorcing families

60 Questions Total

Knowledge Test Example Questions • Law

– In Indiana, the court may not grant the divorce of a couple until 60 days after one spouse files for divorce.

• True

• Program Evaluation Research Methods (PERM)– In an attempt to keep subjects in your experimental and control

groups comparable on the variables you think might affect the study findings, you can use:

• Random assignment

• Psychology Research on Divorce (PRD)– Reviews of the role of noncustodial fathers after divorce

suggest:• The quality of a relationship between a noncustodial father and his

children is an important predictor of child adjustment after separation.

Knowledge Test:Methods and Results

• Students completed tests anonymously– Not worry about evaluation by professors– Not used in course grade

• Issues:– Students didn’t study for test

• Not a peak performance measure

– Can’t match pre- and post-semester tests of individual students, to see how individual students changed

• Limits statistics • Limits ability to look for predictors of outcome among

students

Pre-Training Knowledge Test Scores of Law vs Psychology Students

• Both Baseline and Interdisciplinary Semesters, at start of semesters: – Law students had more knowledge of law– Psych students had more knowledge or program

evaluation research– No difference in knowledge of psychology

research findings on divorce

Changes in Law Student Knowledge Tests: Baseline Semester

Question Type(total points possible)

Pre or Post Mean t(df)p

Law Knowledge(31)

Pre Training 18.13 -3.95(14).001*

Post Training 22.69

PERM(12)

Pre Training 4.75 .000(14)n.s.

Post Training 4.75

PRD(17)

Pre Training 11.06 .286(14)n.s.

Post Training 10.75

PERM (Program Evaluation Research Methods); PRD (Psychology Research on Divorce); *p<.05

Baseline Semester: -Law Students increased knowledge of law, but not psychology

Changes in Law Student Knowledge Tests: Interdisciplinary Training Semesters



Law Knowledge(31)

Pre Training 16.12 -7.15(48).000*

Post Training 21.9

PERM(12)

Pre Training 4.76 -1.80(48).08

Post Training 5.52

PRD(17)

Pre Training 9.80 -2.31(48).025*

Post Training 11.02


Interdisciplinary Semesters: -Law Students increased knowledge of law and psychology

Changes in Psych Student Knowledge Tests: Interdisciplinary Training Semester



Law Knowledge(31)

Pre Training 13.25 -1.72(13).10

Post Training 16.14

PERM(12)

Pre Training 8.25 -1.262(13)n.s.

Post Training 9.57

PRD(17)

Pre Training 10.5 -.627(13)n.s.

Post Training 11.07


Interdisciplinary Semester: -Psych Students increased knowledge of law and psychology, but not statistically significant

(small sample size)

Law vs Psychology Students: End of Interdisciplinary Semester Knowledge Tests

• At end of interdisciplinary semesters:– Law students still know more law– Psychology students still know more about

program evaluation research– Still no difference in knowledge of psychology

research findings on divorce

Summary of Knowledge Test ResultsBaseline Semester:

-Law Students increased knowledge of law, not psychology

Interdisciplinary Semesters:

-Law Students increased knowledge of law and psychology (statistically significant)

-Psych Students increased knowledge of law and psychology (not statistically significant)

II. SOTL Study

C. Focus Group Discussions: Methods and Results

Focus Group Discussions

• Law students interviewed by psych professors• Psych students interviewed by law professor• Combined group interviewed by both

– Not presenting combined group data

1. Focus Group Coding

Coding Focus Group Discussions

• Listened to all discussions • Developed coding system

– Capture major themes being discussed• Coding manual

– 47 codes– 4 point scale

Focus Group Discussion Coding Manual

4 point scale• 0: did not engage in

the coded behavior• 1: slightly/briefly• 2: to a noticeable or

moderate degree, but not extensively.

• 3: a lot or extensively.

Coding Focus Group Discussions • Trained undergraduate coders• Independent coding• Weekly coding meetings

– discuss questions, disagreements – calculate inter-coder reliability

• Selected best coders; averaged their ratings

• Made 5 summary codes

Example of Summary Code• Confidence/Enjoy Code

– Students appeared to have confidence in their understanding of:

• divorce (D1), • mediation (M1)• research (R1)• Enjoyed

interdisciplinary training (IT2)

• Five such summary codes

CONFIDENCE/ENJOY CODES

Alpha 0.611

Code

Alpha if item deleted

D1 0.816M1 0.615R1 0.285IT2 0.265

Baseline Semester

(N=1)

Interdisciplinary Semester(N=1)

Subscale

Pre Training Pre Training Post Training

Training Goals 2.18 0.30 ---

Confidence/Enjoy 2.00 2.33 2.55

Information from Current Course

0.50 0.60 2.05

Positives 1.87 1.53 1.50

Info/Enjoy IT 1.00 --- 2.20

Note: Baseline semester-- No End of Semester Data (Psychology students hadn’t received any relevant training that semester). Only one group (or “subject”) in interdisciplinary semester, so couldn’t do statistical analyses.

Psychology Students: Changes on Focus Group Summary Codes

Baseline Semester

(N=1)Interdisciplinary Semesters

(N=3)

Subscale

Pre TrainingPost

TrainingPre Training

Post Training

t(df)p

Training Goals 2.10 0.60 2.10 1.25 17.00(1).037*

Confidence/Enjoy 1.73 2.80 1.35 2.57 -10.12(2).010*

Information from Current Course

0.55 2.65 0.67 2.30 -9.63(2).011*

Positives 1.53 2.40 1.30 1.95 -2.17(2).163

Info/Enjoy IT 0.60 2.40 1.25 2.65 -7.00(1).090

Note: Only one group (or “study subject”) in baseline semester, so couldn’t do statistical analyses.

Law Students: Changes on Focus Group Summary Codes

Summary of Focus Group Discussion Coding Findings

• Law students:– Fewer negative attitudes towards research– More confidence in knowledge of research on divorce

and mediation– Learned a lot from, and enjoyed, interdisciplinary

training– Ideas for more interdisciplinary training in the future

• Psychology students:– More confidence in their understanding of mediation– Increased knowledge of relevant research– Enjoyed the interdisciplinary training

2. Focus Group Student Quotes

Law Students:Similarities Across Baseline and

Interdisciplinary Training Semesters

Law Students, Start of Baseline Semester

• Want experiential learning– “The thought of being able to be so hands on

while a law student was really appealing. . .”

– “I’m excited to deal with putting two people who are having problems in a room and learning how to deal with this and remain neutral in this situation . . .”

Law Students, End of Baseline Semester

• Got experiential learning– “When you learn how to listen to people the way

you do in mediation . . . the skill applies to other settings [like] when you are talking to your clients or when you are preparing for trial.”

– “The clinic is so experiential . . .”

Law Students, Start of Interdisciplinary Training Semester

• Want experiential learning– “Practically, this experience will be helpful for

dealing with clients in the future. . .”

– “I would really love to be able to listen to people and understand where they’re coming from. . .”

Law Students, End of Interdisciplinary Training Semester

• Got experiential learning– “I feel like this definitely made me think of my

future career profession differently.”

– “I learned how to transition from advocate to neutral. It’s a different approach where you shift your mindset to stay neutral. It was very helpful.”


• Concerned about the court system making custody decisions– “I think there are a lot of cases where it’s clear,

but in middle road cases I’m not really confident how the court system can accurately decide which parent should have the child or primary custody.”


• Continued concerns about the court system making custody decisions (time available)– “I’m more skeptical of decisions made by judges

after the twenty minute hearing and reading the case file… There have been mediations where I was thinking one thing for the first two or even four hours, but after several hours of conversation you discover these game changing facts. . .”

Law Students, Start of Interdisciplinary Semester

• Concerns about court system making custody decisions– “It seems like mediation would be better than

having some judge say what to do. You’re working together and you feel like you have more control over the outcome. It seems to me that they would be happier. They’re divorced but it’s better than walking into a combative court proceeding.”

Law Students, End of Interdisciplinary Semester

• Continued concerns about court system making custody decisions (mediation better)– “…When mediation works for a couple, it really

works and the outcomes are really good and you can tell that’s really going to work for them.”

– “Every couple I had seemed to really like the process a lot. It seemed really empowering to them to have some control over decision making.”

Law Students:Differences Across Baseline and

Interdisciplinary Training Semester


• Research: Desire for information (possible interest in research)– “I’d like to know how it plays out over time if a

couple who divorces has high or low conflict . . . How does conflict change over time?”


• Research: Continued desire for information (possible interest in research)– “I want to know more about the child inclusive

mediation process . . . .”

– “I’d like to see a study that compares the long term durability of agreements reached through a very directive mediation style versus a more facilitative mediation style . . . .”


• Research: Desire for information (interest in research)– “I don’t know anything about research. I only

know stereotypes probably because I’ve heard it on the news or Maury.”

– “I know almost nothing about research…”

– “Learning how to evaluate interventions based on psychological research would be helpful...”


• Frustrations of conducting research– “I think the most frustrating thing for me was the

random assignment. I just kept on getting divorce mediation as usual . . .”

– “I never realized how reluctant people are to participate in research. I was surprised by parents’ resistance.”


• Positive experiences with research– “I liked that the psychology department came in and

told us . . . the way the study works and how it is designed . . . and this is what we’re hoping to accomplish and yes there are flaws. . . I like that transparency and that honesty in what we were doing.”

– “I learned a lot about research on divorce and mediation. . I find myself referring to information given by the psychology faculty when I’m talking to my parents. I tell people about mediation and random assignment. . .”

– “I learned all my research knowledge from the child feedback session.”


• No discussion of interdisciplinary training at start of semester

• Want interdisciplinary training (didn’t get much)– “I’d like to hear from psychology faculty a little bit more

. . . .”– “I think interdisciplinary training was a really good

thing. The day we just sat down with psychology students (focus group discussion) it was good to see the difference in views between people.”

– “I’d like to see psychology students mediating with law students.. . . ”


• Excited about upcoming interdisciplinary training– “I’m psyched to be working with the

psychology department. It’s very different from what we do on a day to day basis. We do custody battles. . .”

– “I’m excited to work together. . .”


• Liked interdisciplinary training– “Interdisciplinary training was really helpful. Sometimes

we couldn’t tell what it was because it went hand in hand so well. . . It was so intertwined that it just flowed. I’m appreciative to have had it. It definitely helped.”

– “It’s nice to have the child feedback session so that what you’re thinking in your head comes out through the child consultants.”

– “Child consultants could get parents focused on the best interests of [the child]. Recently we did a child inclusive mediation and the parents . . . were very emotional when [they learned about] their daughter’s reaction to the divorce and how their actions [were impacting] their daughter . . .”

Psychology Students:Interdisciplinary Training Semesters

(no comparable baseline semester data)

Psychology Students, Start of Interdisciplinary Training Semester

• Desire to understand the divorce process– “I want to have a better grasp of the whole

divorce process and how it could affect children and families. I’m coming in with little knowledge of this process.”

Psychology Students, End of Interdisciplinary Training Semester

• Understand divorce process (it is an extended and complicated process) – “Something I realized over the course of the

semester… is how complicated it is, how much parents have on their minds, how much they have to think about, decisions that have to be made, and how many things they have to take into account.”

– “ . . . This can be an extended process. It’s not just that you do mediation and you’re divorced; it can be months or years. It’s an ongoing developmental process that changes as the needs of the children and parents and their willingness or ability to manage all of the complexities change over that time.”

Psychology Students, Start of Interdisciplinary Training Semester

• Want to incorporate psychological information into legal process– “I want to know practically, what it looks like to

incorporate psychological information into this process. This is a different approach and different rules than what we’re used to working with.”


• Want to incorporate psychological information into legal process continued (want to do even more and see impact of it)– “It would be nice to have a sense of our impact.

You present this information with the idea of motivating them to make good decisions for their children… but you don’t quite know how that’s going to play out later . . .”

– “It would be nice to know or see how the parents are talking about it after having heard the feedback from the child consultants.”

– “I’m wondering how much the parties will actually adhere to those agreements...”


• Liked Interdisciplinary training– “It’s been fun working with people with different

strengths and I think it’s pretty complimentary honestly. It’s been a nice change.”

– “It’s been really interesting to be involved in thinking about the practical concerns of how to implement this intervention. . .”

Focus Group Discussion:Summary of Student Quotes

• Some similarities in law student experience in baseline and interdisciplinary training semesters

• During interdisciplinary training semesters, law students learned about research and enjoyed interdisciplinary work

• And the psychology students enjoyed interdisciplinary work and wanted even more involvement

II. SOTL Study

D. Coding Student Behavior During Mediations

Another Attempted Measure: Student Behavior During Mediations

• In psychology program evaluation research:– Code what happens during therapy sessions

• Measure of:– Adherence (fidelity) to treatment manual

• Are therapists doing what they are supposed to do?• Do different treatment approaches (being compared)

actually differ as they are supposed to?

– Competence• How well are therapists implementing the intervention?

– For us, also a measure of the effects of teaching and training

• What are the students actually doing in mediation?


• Thus, we planned to code law and psych student behaviors during:– CI– CF– MAU – During baseline and interdisciplinary semesters

• Developed coding system of mediator and child consultant behavior– Mix of adherence and competence coding


• Coding system examples:– General skills

• listened empathically• provided examples

– Skills specific to each type of mediation• CI: Brought in materials completed by child• CF: Used general developmental research findings to

discuss the children

• Coders:– Law and psych students

• Didn’t work


• Methodological problem:– In psychology research: record sessions, for later

coding:• complex human interactions• can watch and re-watch tape• can stop tape and discuss codes assigned

• Not able to record mediation sessions – confidential process

• Tried to code from online, live observation• Coding system: too complex, too many codes• Future plans:

– Micro-analytic coding: Fewer codes coded frequently

Student Behavior During Mediation• Coder problem:• Failure of interdisciplinary training• Interdisciplinary issue: Judging competence• Law students did not like coding

– Didn’t understand the importance– Time consuming and not part of what they expected

to do for the course– Weren’t comfortable judging each other

• Future plans:– Use psychology student coders and/or train law

students better

II. SOTL Study

E. Student Involvement in Program Evaluation Research

Outcome: Student Involvement in Program Evaluation Research

• Psychology students:– Helping to run all aspects of the study– Co-authors on multiple conference presentations

and publications• Law students:

– Involved in parts of the study • Attempt to code student behavior during mediation• Coding content of mediation agreements

– Co-authors on conference presentations and publications (3 law students to date)

Recap of Results Presented • Various measures/outcomes: • Student Knowledge Tests

– Law– Psychology research– Program evaluation research methods

• Student Focus Group Discussions:– Coded– Quotes

• Attempt to Code Student Behavior During Mediation• Student Involvement in Program Evaluation Research

• Across these measures: – Convergence of findings– Generally positive impact of interdisciplinary training

III. Discussion

Discussion: Study Limitations

• Baseline Semester Problems:– Only one baseline semester– Baseline and interdisciplinary semester students differed in

previous experience– Had already begun some interdisciplinary work and teaching

in baseline semester:• IU conference on law and social science in divorce• Psychology professors gave guest lectures in law

student course (not on CI and CF, but other topics)• Started interdisciplinary research on the detection of

violence in mediation cases• Dilemma: Must establish interdisciplinary team and

collaborative working relationship to plan project but doing so meant we’d already begun before SOTL project began


• “Horse already out of the barn” problem:– Baseline versus interdisciplinary training

semesters: • Fewer statistically significant differences than expected

– On knowledge test scores– Focus groups: Even baseline semester students discussed how

much they learned from interdisciplinary training!

• Dilemma:– Developing interdisciplinary efforts and including

student in exciting new developments versus conducting a methodologically stronger study


• Related problem: Small number of psychology students– Baseline semester:

• No post-semester data from psych students • Inability to directly compare baseline and interdisciplinary

semesters

– Interdisciplinary training semesters: • Only one group of psychology students (versus 3 groups of law

students) • N = 8 psychology students in interdisciplinary semester• Lack of statistical power to detect changes in psych students

• Due to psychology training program (small size)• Future– Need more psych students:

– Continue study in future, with new psych students– Conduct study at other universities – Conduct study with other types of social service students

(e.g., social work or counseling psychology graduate students)

Discussion: Findings • Despite these problems…• On knowledge test:

– Law students:• Baseline semester: increased only scores on law section• Interdisciplinary semesters: also increased scores on both

psychology sections

– Psych students: • Interdisciplinary semesters: Increased scores on both law and

psychology sections of knowledge test

• During Focus Group Discussions:– Both groups reported enjoying interdisciplinary training, etc.

• Coding Data• Student Quotes

• Both groups involved in CIMS program evaluation research study and resulting publications

Discussion: Law vs Psych Students

• Overall, appears to have been bigger impact on law students than psych students.

• Possible reasons:– Small sample size of psych students (no stat power) – Law students more motivated

• More likely to go into practice (more directly relevant)

– One law professor (less time in psych class) vs two psych professors (more time in law class)

• But law professor on-site and provided feedback to psych student child consultants

– Most interactions at law school clinic (not psych building), so may have seemed more integrated into law training

Discussion: Law vs Psych Students

• Overall, appears to have been bigger impact on law students than psych students. Possible reasons (cont’d): – Law students attended CI and CF parent feedback

sessions (held by psych students)• Continued exposure to discussion of psychology research

on effects of divorce

– Psych students did not attend mediation negotiations and agreements

• Did not have continued exposure to discussion of legal issues

• Not integrated into entire process

Challenges of Interdisciplinary Training and Research

• General interdisciplinary issues (discussed before):– Speaking different languages

• Interdisciplinary issues with conducting program evaluation research methods (discussed before):– Protect confidentiality of mediation process

• Law students’ perception of burden on mediation process:– Research process slows down mediation process– Extra time and effort for law students

• Psychology students perceptions:– Didn’t always feel equal “ownership” in process (conducted

at law school) – Carried more of the “burden” of conducting the research

Benefits of Interdisciplinary Training and Research for

Both Groups of Students• Changes in clinical practice at the IU Mediation Clinic –

substance and process• Increased focus on process for parents and outcomes for

parents and children • Teacher/student/practitioner development – experience

through another lens• Research informing the teaching and practice• Teaching and practice informing the research• Opportunity to do research in the real world• Overall: The Interplay of Teaching, Learning, and

Research: Graduate Student Collaborations in Law and Psychology– Worth it – Continuing with more new interdisciplinary projects

Thank you!

• Questions?• More information or future questions:

– [email protected] (Amy Holtzworth-Munroe)– [email protected] (Amy G. Applegate)

mailto:[email protected]

mailto:[email protected]

Additional tables, if needed

Question Type (total points possible)

Baseline Law

Baseline Psych

Inter-disciplinary

Law

Inter-disciplinary Psych

PreTraining

PreTraining

t(df)p

PreTraining

PreTraining

t(df)p

Law (31) 18.13 10.82 4.19(17).001*

16.12 13.25 2.14(31).040*

PERM (12) 4.75 8.36 -3.21(17).005*

4.76 8.25 -3.85(31).004*

PRD (17) 11.06 8.86 1.69(17).109

9.80 10.50 -.998(31).326

PRE-TRAINING (START OF SEMESTER): Knowledge Test Scores of Law vs Psychology Students

PERM (Program Evaluation Research Methods); PRD (Psychology Research on Divorce); *p < 0.05

Both Baseline and Interdisciplinary Semesters, at start of semesters: -Law Students had more knowledge of Law -Psych Students had more knowledge of Program Evaluation Research Methods

-No statistically significant difference between Baseline and Interdisciplinary Semesters (despite differences in previous experiences)


Student Type Mean t(df)p

Law Knowledge(31)

Law 21.9 .532(30).000*

Psychology 16.14

PERM(12)

Law 5.52 -6.20(30).000*

Psychology 9.57

PRD(17)

Law 11.02 -.062(30).951

Psychology 11.07


Law Students still know more law.Psych Students still know more program evaluation. No differences in psychology research on divorce.

Interdisciplinary Semesters Knowledge Tests: End of Semester

Code Pre or Post Mean t(df)p

M1- Confidence in understanding of mediation

Pre 1.27 -11.50(2).007*

Post 2.80

M5- Current course as a source of knowledge about mediation

Pre 0.20 -13.23(2).006*

Post 2.53

M14- Expressed negative feelings about mediation

Pre 2.47 7.56(2).017*

Post 1.13

R1- Confidence in understanding of research

Pre 0.80 -10.40(2).009*

Post 2.00

R5- Current course as a source of knowledge about research

Pre 0.60 -6.43(2).023*

Post 1.73

IT2- Enjoyed interdisciplinary training

Pre 0.70 -19.00(1).033*

Post 2.60

Individual Codes: Law Student Changes (3 Interdisciplinary Training Semesters)

6 codes significant; Could not calculate for psychology (n=1 group)

The Interplay of Teaching, Learning, and Research: Graduate Student Collaborations in Law and...

Documents

Transcript of The Interplay of Teaching, Learning, and Research: Graduate Student Collaborations in Law and...