Evaluating Programs to Assist Self Represented Litigants John Greacen, Greacen Associates LLC Deana...
-
Upload
imogen-williams -
Category
Documents
-
view
217 -
download
0
description
Transcript of Evaluating Programs to Assist Self Represented Litigants John Greacen, Greacen Associates LLC Deana...
Evaluating Programs to Assist Evaluating Programs to Assist Self Represented LitigantsSelf Represented Litigants
John Greacen, Greacen Associates LLCJohn Greacen, Greacen Associates LLCDeana Piazza, California AOC, Center for Deana Piazza, California AOC, Center for
Children, Families and the CourtsChildren, Families and the CourtsJune 21, 2005June 21, 2005
ReferencesReferences
2002 meta analysis of 35 program 2002 meta analysis of 35 program evaluationsevaluations2004 TCRIC assessment of programs in 2004 TCRIC assessment of programs in nine jurisdictionsnine jurisdictions2004 California AOC evaluation of 5 pilot 2004 California AOC evaluation of 5 pilot programsprograms
Evaluation FindingsEvaluation Findings
Demographics of Self Represented Demographics of Self Represented LitigantsLitigants
Income and education level vary by locationMajority are poorMajority have a high school educationMajority are womenMajority are petitioners, not respondentsHigh percentage have Internet access
TCRIC JurisdictionsTCRIC Jurisdictions
Baltimore City
Harford
Montgomery
Prince Geor
ges
Worcester
Alaska Dade Hennepin
Maricopa
Male 32% 41% 44% 47% 48% 33% 47% 65% 30%
Female 68% 59% 56% 53% 52% 67% 53% 35% 70%
Medianage 35-44 35-44 35-44 35-44 25-34 25-34 35-44 35-44 35-44
MedianMonthlyincome
$500 -$1,000
$2,000-$2,500
$2,000-$2,500
$2,000-$2,500 $8,000 $1,500-
$2,000$1,000-$1,500
$1,000-$1,500
$2,000-$2,500
Medianeducation
high school high school somecollege
some college high school some
collegesome
collegesome
college high school
White 10% 73% 46% 13% 70% 59% 64% 31% 66%
Non-white 90% 27% 54% 87% 30% 41% 36% 69% 34%
Hispanic 0% 7% 46% 9% 4% 6% 61% 11% 43%
Why don’t they have a lawyer?
Believe they can’t afford oneBelieve they can’t afford oneBelieve the case is simple enough to Believe the case is simple enough to handle on their ownhandle on their ownDon’t want to pay a lawyerDon’t want to pay a lawyerLawyer will slow everything downLawyer will slow everything downDon’t trust lawyersDon’t trust lawyers
Impact of self-representedImpact of self-representedlitigants on the courtlitigants on the court
Minutes required for hearingMinutes required for hearingBoth represented v. at least one notBoth represented v. at least one not
(California 2001)(California 2001)
Probate 3.4 17.2 +400%Felony/personFelony/person 14.0 14.0 37.737.7 +169%+169%M V tortsM V torts 16.1 16.1 22.322.3 +45% +45%Family 15.8 12.2 -30%Small claims 15.5 10.4 -49%Drug 6.8 4.3 -58%Unlwfl detainer 13.0 5.7 -128%Felony/property 8.8Felony/property 8.8 3.7 3.7 -138% -138%
Likelihood of eventLikelihood of eventBoth represented v. at least one notBoth represented v. at least one not
(Washington 2001)
Dissolutions with children Dissolutions with children Non-jury trialNon-jury trial 2.1% 2.1% 41.9% 41.9% Motion hearingMotion hearing 37.3%37.3% 74.7%74.7%ContinuanceContinuance 1.5% 1.5% 35.6%35.6%
Dissolutions without children Dissolutions without children Non-jury trialNon-jury trial 1.0% 1.0% 40.1% 40.1% Motion hearingMotion hearing 23.4%23.4% 57.7%57.7%ContinuanceContinuance 0.1% 0.1% 24.3%24.3%
Time from filing to dispositionTime from filing to disposition
Washington state 2001Washington state 2001Dissolution with childrenDissolution with children
345 days v. 136 days 345 days v. 136 days Dissolution without childrenDissolution without children
283 days v. 111 days283 days v. 111 daysNational Center for State CourtsNational Center for State Courts
similar findings, with one exceptionsimilar findings, with one exception
User Satisfaction with Self User Satisfaction with Self Help ProgramsHelp Programs
High at the Time of UseHigh at the Time of Use
4.14.24.34.44.54.64.74.84.9
5
Ratings of Specific Services Ratings of Specific Services ProvidedProvided
So high that the data does not allow us to So high that the data does not allow us to differentiate among them.differentiate among them.
High at the Time of Court High at the Time of Court AppearanceAppearance
0
0.5
1
1.52
2.5
3
3.5
4
Litigant Rating of “Did the judge Litigant Rating of “Did the judge treat you with respect?”treat you with respect?”
44.24.44.64.8
55.2
Litigant Rating of “Was the Judge’s Litigant Rating of “Was the Judge’s ruling fair?”ruling fair?”
0123456
Litigant Rating of “Did you feel Litigant Rating of “Did you feel prepared?”prepared?”
00.5
11.5
22.5
33.5
44.5
5
Litigant Rating of “Can you explain Litigant Rating of “Can you explain the Judge’s ruling?”the Judge’s ruling?”
3.63.8
44.24.44.64.8
5
Use of ProgramsUse of Programs
Percentage of Self Represented Percentage of Self Represented Litigants Using the Services of Litigants Using the Services of
Programs Provided by the CourtPrograms Provided by the Court
0102030405060708090
Stakeholder Satisfaction with Stakeholder Satisfaction with Self Help ProgramsSelf Help Programs
Stakeholder SatisfactionStakeholder Satisfaction
0%
20%
40%
60%
80%
100%
120%
Judges Staff Lawyers
Process ObservationsProcess Observations
Process ObservationsProcess Observations
Multiple successful program delivery Multiple successful program delivery strategiesstrategies– One on one in person assistanceOne on one in person assistance– Telephone assistanceTelephone assistance– Web-based information and formsWeb-based information and forms– WorkshopsWorkshops
In personIn personVideo-conference supportedVideo-conference supportedVideotaped presentationsVideotaped presentations
Process ObservationsProcess Observations
Participation in self help programs can be Participation in self help programs can be mandatedmandated– Hennepin County – motions Hennepin County – motions – Miami/Dade – all filings Miami/Dade – all filings – Anchorage – participation in workshops by all Anchorage – participation in workshops by all
self represented litigants in contested divorce self represented litigants in contested divorce and child custody casesand child custody cases
Process ObservationsProcess Observations
Multiple successful staffing approachesMultiple successful staffing approaches– Court-employed lawyersCourt-employed lawyers– Court-employed paralegalsCourt-employed paralegals– Court clerksCourt clerks– Contract servicesContract services
Self help program personnel need Self help program personnel need supervision from, or access to, a lawyersupervision from, or access to, a lawyer
Process ObservationsProcess Observations
Court self help programs need to be Court self help programs need to be supplemented with programs that provide supplemented with programs that provide legal assistance to some litigantslegal assistance to some litigants– Legal services programsLegal services programs– Pro bono lawyer programsPro bono lawyer programs– Contracted legal advice programsContracted legal advice programs
Process ObservationsProcess Observations
Self represented litigant assistance must Self represented litigant assistance must go beyond a self help programgo beyond a self help program– Clerk’s office staffClerk’s office staff– Chambers staffChambers staff– Courtroom assistanceCourtroom assistance– Assistance in understanding judicial decisions Assistance in understanding judicial decisions
and ordersand orders– Assistance in enforcing court orders and Assistance in enforcing court orders and
judgmentsjudgments
California Evaluation California Evaluation FindingsFindings
Five Pilot Self-Help CentersFive Pilot Self-Help Centers
Characteristics of Self-Help Center Characteristics of Self-Help Center CustomersCustomers
Income and education levels lower than Income and education levels lower than those of general county populationsthose of general county populationsMonthly household incomes $2,000 or less Monthly household incomes $2,000 or less (66%-96%)(66%-96%)More than 60% in each program lacked a More than 60% in each program lacked a college degreecollege degreeSubstantial proportion unemployed (43%-Substantial proportion unemployed (43%-50%)50%)
Characteristics of Self-Help Center Characteristics of Self-Help Center Customers Customers (cont.)(cont.)
Majority gender varied by case types Majority gender varied by case types served at each centerserved at each centerVast majority were moving party, except Vast majority were moving party, except UD defendants at one centerUD defendants at one center
Experience Seeking Legal Experience Seeking Legal AssistanceAssistance
At least 60% had not sought help before visiting At least 60% had not sought help before visiting the self-help centerthe self-help center– Most who did seek help sought it from legal aid, Most who did seek help sought it from legal aid,
private attorney, or friends/relativesprivate attorney, or friends/relativesAt least 70% had not considered hiring a lawyerAt least 70% had not considered hiring a lawyerMost common reason for self-representing was Most common reason for self-representing was inability to afford a lawyerinability to afford a lawyer– Other common reasons included not knowing whether Other common reasons included not knowing whether
a lawyer was needed or conscious choice to self-a lawyer was needed or conscious choice to self-representrepresent
Customers most commonly heard of self-help Customers most commonly heard of self-help center through clerk’s office or friends/family center through clerk’s office or friends/family
Impact of Programs on the CourtsImpact of Programs on the Courts
Self-help centers improve court efficiencySelf-help centers improve court efficiency– Cases delayed due to procedural problems Cases delayed due to procedural problems
were corrected and completedwere corrected and completed– Paperwork correct the first time, eliminating Paperwork correct the first time, eliminating
repeated trips to clerk’s windowrepeated trips to clerk’s window– Litigants appeared at hearing with papers Litigants appeared at hearing with papers
properly served, eliminating need for properly served, eliminating need for continuancescontinuances
– Clerks and judges spent less time answering Clerks and judges spent less time answering questionsquestions
Impact of Programs on the Courts Impact of Programs on the Courts (cont.)(cont.)
By identifying issues faced by SRLs, By identifying issues faced by SRLs, programs helped courts develop creative programs helped courts develop creative ways to process cases more efficientlyways to process cases more efficiently– Pro per day includes review of files prior to Pro per day includes review of files prior to
hearings and in-court assistancehearings and in-court assistance– Consolidation of UD settlement conferences Consolidation of UD settlement conferences
to allow workshops to be held before calendarto allow workshops to be held before calendar– Referral slip for judges to indicate problems/ Referral slip for judges to indicate problems/
additional needs when directing litigants to additional needs when directing litigants to self-help centerself-help center
Impact of Programs on LitigantsImpact of Programs on Litigants
Facilitate litigants’ effective participation in legal Facilitate litigants’ effective participation in legal processprocess– Court file review shows center customers more likely Court file review shows center customers more likely
to raise relevant issues and defenses, file proper to raise relevant issues and defenses, file proper accompanying paperwork, and reach settlementsaccompanying paperwork, and reach settlements
Promote public trust and confidence in court Promote public trust and confidence in court system system – High levels of customer satisfactionHigh levels of customer satisfaction– Post-hearing interviews show center customers more Post-hearing interviews show center customers more
prepared for court, have more reasonable prepared for court, have more reasonable expectations of hearingexpectations of hearing
0% 20% 40% 60% 80% 100%
Know more abouthow laws work
(N=223)
Less confused abouthow court works
(N=224)
Less worried aboutmy situation
(N=225)
Know what I needto do next (N=224)
Understand mysituation better
(N=227)
Strongly agree Agree Disagree Strongly Disagree
63%
87%80%
53%
69%64%
0%
20%
40%
60%
80%
100%
Surprised by whathappened: Not at all/ not
very
Able to tell judgeeverything needed to
make decision:Extremely/ mostly
J udge would have ruleddifferently if had lawyer:
No
Users of self-help center (N=30-32) Nonusers of self-help center (N=55-58)
Findings and Recommendations Findings and Recommendations from Process Evaluationfrom Process Evaluation
Videoconferencing and coordination between courts is Videoconferencing and coordination between courts is an effective way to address limited fundingan effective way to address limited fundingWorkshops are an effective way to serve larger numbers Workshops are an effective way to serve larger numbers without adding staffwithout adding staff– Wide range of services provided: legal and procedural Wide range of services provided: legal and procedural
information, forms and hearing preparation, assistance with information, forms and hearing preparation, assistance with motions, referrals to other servicesmotions, referrals to other services
– Levels of satisfaction similar among workshop participants and Levels of satisfaction similar among workshop participants and drop-in customersdrop-in customers
Court-based self-help programs should be integrated Court-based self-help programs should be integrated with one another to increase efficiency and quality of with one another to increase efficiency and quality of serviceservice
Findings and Recommendations Findings and Recommendations from Process Evaluation from Process Evaluation (cont.)(cont.)
The day-to-day availability of a managing attorney is The day-to-day availability of a managing attorney is criticalcritical– Understand procedural complexitiesUnderstand procedural complexities– Able to do legal research if necessaryAble to do legal research if necessary– Familiar with legal terminology and professional ethicsFamiliar with legal terminology and professional ethics– Can identify when it is not feasible to self-represent Can identify when it is not feasible to self-represent
Volunteers should not be relied on to perform core daily Volunteers should not be relied on to perform core daily operationsoperations– Short-term commitments for internships; school out of session or Short-term commitments for internships; school out of session or
final exam timefinal exam time– Quality training makes more enticing candidates for paid Quality training makes more enticing candidates for paid
positionspositions
Findings and Recommendations Findings and Recommendations from Process Evaluation from Process Evaluation (cont.)(cont.)
Self-help centers should be designed to assist Self-help centers should be designed to assist litigants at all stages of case processinglitigants at all stages of case processing– Case file review shows cases unresolved or not Case file review shows cases unresolved or not
reaching timely judgmentreaching timely judgment– Service tracking data show at least one-quarter of Service tracking data show at least one-quarter of
customers returning for helpcustomers returning for help– Need for assistance with orders and complianceNeed for assistance with orders and complianceSelf-help centers should be located at the Self-help centers should be located at the courthousecourthouseTriage of cases is an important self-help center Triage of cases is an important self-help center functionfunction
Findings and Recommendations Findings and Recommendations from Process Evaluation from Process Evaluation (cont.)(cont.)
Interpreters are needed in family and other civil Interpreters are needed in family and other civil cases (currently mandated only in criminal and cases (currently mandated only in criminal and DV cases in CA)DV cases in CA)Bilingual and bicultural staff are needed to Bilingual and bicultural staff are needed to provide efficient services in communities with a provide efficient services in communities with a high proportion of non-English speakers high proportion of non-English speakers Coordination with community programs is helpful Coordination with community programs is helpful in serving non-English-speaking populationsin serving non-English-speaking populations
Program Use of Evaluation ResultsProgram Use of Evaluation Results
Court file review results used to retool Court file review results used to retool workshops to address problems identifiedworkshops to address problems identifiedWeb site user testing identified need to Web site user testing identified need to add local form to siteadd local form to siteIntake data showed low referrals from Intake data showed low referrals from clerk’s office; director changed outreach clerk’s office; director changed outreach strategy, resulting in more clerk referrals strategy, resulting in more clerk referrals
Evaluation MethodologiesEvaluation Methodologies
The TCRIC Executive Program The TCRIC Executive Program Assessment ProcessAssessment Process
““Quick and Clean” AssessmentQuick and Clean” AssessmentCourt staff administers five survey Court staff administers five survey instruments, obtaining 250 completed instruments, obtaining 250 completed surveyssurveysElectronic scanner used to automatically Electronic scanner used to automatically score and compile data from the surveysscore and compile data from the surveysConsultant and volunteer from another Consultant and volunteer from another court conduct on site observations and court conduct on site observations and interviewsinterviewsConsultant prepares report, based on Consultant prepares report, based on comparative datacomparative data
Findings from SJI-sponsored TestFindings from SJI-sponsored Test
Time -- 11 to 12 week processTime -- 11 to 12 week processCost -- $8,000 plus an equal amount of Cost -- $8,000 plus an equal amount of court staff timecourt staff timeResults -- Valid and valuable to the court Results -- Valid and valuable to the court assessedassessedBenchmark data – available for nine Benchmark data – available for nine jurisdictionsjurisdictionsSignificant effort required of court to be Significant effort required of court to be assessedassessed
Self AssessmentSelf Assessment
PossiblePossibleNot demonstratedNot demonstratedDrawbacks are likely to be lack of Drawbacks are likely to be lack of objectivity and corresponding lack of objectivity and corresponding lack of credibilitycredibility
Statewide AssessmentStatewide Assessment
The five Maryland site assessments were The five Maryland site assessments were used to produce a statewide assessment used to produce a statewide assessment of programs to assist self represented of programs to assist self represented litigants in the state of Maryland.litigants in the state of Maryland.
California Evaluation California Evaluation MethodologyMethodology
Background on EvaluationBackground on Evaluation
Two-and-a-half year processTwo-and-a-half year processCost in excess of $400,000Cost in excess of $400,000Hired consultants to perform most of the Hired consultants to perform most of the evaluations, additional work performed by evaluations, additional work performed by AOC staffAOC staffCollaborative evaluation design, started Collaborative evaluation design, started with development of logic modelswith development of logic models
Evaluation ComponentsEvaluation ComponentsSite visitsSite visits– One during early implementation, one toward end of One during early implementation, one toward end of
projectproject– In-depth interviews with program and court staff and In-depth interviews with program and court staff and
other stakeholdersother stakeholders– Observation of program operations and gathering of Observation of program operations and gathering of
program materialsprogram materialsIntake and service tracking formsIntake and service tracking forms– Characteristics of customersCharacteristics of customers– Case types and issuesCase types and issues– Services provided, including method of service Services provided, including method of service
deliverydelivery
Evaluation Components Evaluation Components (cont.)(cont.)Court file reviewCourt file review– Focused on procedural issues common to SRLs, not Focused on procedural issues common to SRLs, not
case outcomescase outcomes– Compared sample of SRLs who visited center to Compared sample of SRLs who visited center to
sample who had notsample who had notPost-hearing interviewsPost-hearing interviews– Intercepted litigants leaving courtroomIntercepted litigants leaving courtroom– Assessed preparation for case, use of services, and Assessed preparation for case, use of services, and
courtroom experiencecourtroom experience– Compared sample of SRLs who visited center to Compared sample of SRLs who visited center to
sample who had notsample who had not
Evaluation Components Evaluation Components (cont.)(cont.)Customer satisfaction surveyCustomer satisfaction survey– Assessed general levels of satisfaction on different Assessed general levels of satisfaction on different
dimensions, as well as helpfulness of specific dimensions, as well as helpfulness of specific servicesservices
– Separate versions for workshop and drop-in Separate versions for workshop and drop-in customerscustomers
Supplemental data sourcesSupplemental data sources– Quarterly progress reports and structured writing Quarterly progress reports and structured writing
exercises completed by center directorsexercises completed by center directors– Project proposalsProject proposals– Financial information from projectsFinancial information from projects
Common Methodology Common Methodology ProblemsProblems
Methodology ProblemsMethodology Problems
Difficulty of follow up studies due to Difficulty of follow up studies due to mobility of case participantsmobility of case participants– Low response rate for mail surveysLow response rate for mail surveys– Lack of telephone numbersLack of telephone numbers– Low response rates to telephone follow up Low response rates to telephone follow up
surveyssurveys
Gathering too much demographic dataGathering too much demographic data– Importance of samplingImportance of sampling
Methodology Problems Methodology Problems (cont.)(cont.)Difficulty of file reviewsDifficulty of file reviews– Time consuming: design and piloting of forms, training, data Time consuming: design and piloting of forms, training, data
collection and analysiscollection and analysis– Sample selection and identification of appropriate comparison Sample selection and identification of appropriate comparison
groupsgroups– Interpretation of missing informationInterpretation of missing information
Difficulty of courtroom observationDifficulty of courtroom observation– Establishing objective measuresEstablishing objective measures– Consistency in coding the same eventConsistency in coding the same event
Difficulty of post-hearing interviewsDifficulty of post-hearing interviews– Targeting users of particular programsTargeting users of particular programs– Obtaining adequate sample size may require a long time in the Obtaining adequate sample size may require a long time in the
fieldfield
Methodology Problems Methodology Problems (cont.)(cont.)
Intake forms not consistently filled out; therefore, Intake forms not consistently filled out; therefore, not adequate for capturing volumenot adequate for capturing volumeService tracking forms difficult to use for Service tracking forms difficult to use for services provided by phone and group activities services provided by phone and group activities like workshops or educational outreachlike workshops or educational outreachIssues with customer satisfaction surveysIssues with customer satisfaction surveys– Who is not responding?Who is not responding?– How meaningful are the results when satisfaction is How meaningful are the results when satisfaction is
so high and there’s little variation in responses?so high and there’s little variation in responses?
Some Unanswered Some Unanswered QuestionsQuestions
Unanswered QuestionsUnanswered Questions
Why do many self help program users not Why do many self help program users not pursue their cases in court?pursue their cases in court?Why do self represented litigants not use Why do self represented litigants not use self help programs that are available to self help programs that are available to them?them?Why do courtroom observers have Why do courtroom observers have different perceptions of case outcome from different perceptions of case outcome from those of the judges?those of the judges?
Unanswered QuestionsUnanswered Questions
Cost/benefit analyses of self help Cost/benefit analyses of self help programsprogramsComparative outcomes for represented Comparative outcomes for represented and unrepresented litigants – are the and unrepresented litigants – are the courts truly providing self represented courts truly providing self represented litigants with just outcomes?litigants with just outcomes?How effective are different methods of How effective are different methods of service delivery?service delivery?
Unanswered QuestionsUnanswered Questions
How do self-help centers interplay with the court How do self-help centers interplay with the court and with other available services? Need more of and with other available services? Need more of a system-wide look at SRLs.a system-wide look at SRLs.Why aren’t litigants finishing their cases?Why aren’t litigants finishing their cases?To what extent do self-help centers help to keep To what extent do self-help centers help to keep people/actions out of the court?people/actions out of the court?Do self-help centers help to increase compliance Do self-help centers help to increase compliance with court orders?with court orders?Do self-help centers create real time savings for Do self-help centers create real time savings for the court?the court?
Link to CA Evaluation ReportLink to CA Evaluation Report
http://www.courtinfo.ca.gov/programs/equalahttp://www.courtinfo.ca.gov/programs/equalaccess/modelsh.htmccess/modelsh.htm