Making Social Work Count Lecture 9
description
Transcript of Making Social Work Count Lecture 9
![Page 1: Making Social Work Count Lecture 9](https://reader035.fdocuments.in/reader035/viewer/2022062520/56816226550346895dd25915/html5/thumbnails/1.jpg)
Making Social Work Count Lecture 9
An ESRC Curriculum Innovation and Researcher Development Initiative
![Page 2: Making Social Work Count Lecture 9](https://reader035.fdocuments.in/reader035/viewer/2022062520/56816226550346895dd25915/html5/thumbnails/2.jpg)
Testing for statistical significance
Is there a real difference?
![Page 3: Making Social Work Count Lecture 9](https://reader035.fdocuments.in/reader035/viewer/2022062520/56816226550346895dd25915/html5/thumbnails/3.jpg)
Learning outcomes
• At the end of this session, you should be able to:– Define research question, research hypothesis, null
hypothesis and statistically significant;– Discuss the basic requirements for testing the
difference between two means;– Define and describe the difference between the alpha
value and P value, and Type I and Type II errors;• Advanced Study:– Calculate the difference between the means (t-ratio)
using example data through advanced study
![Page 4: Making Social Work Count Lecture 9](https://reader035.fdocuments.in/reader035/viewer/2022062520/56816226550346895dd25915/html5/thumbnails/4.jpg)
1. theorise and hypothesise
2. collect sample data 3. test 4. determine the likelihood
the hypothesis is true
1. theorise and hypothesize
2. test3. decide and predict
Taking decision-making a step further
![Page 5: Making Social Work Count Lecture 9](https://reader035.fdocuments.in/reader035/viewer/2022062520/56816226550346895dd25915/html5/thumbnails/5.jpg)
Research Questions
![Page 6: Making Social Work Count Lecture 9](https://reader035.fdocuments.in/reader035/viewer/2022062520/56816226550346895dd25915/html5/thumbnails/6.jpg)
What kinds of research questions might we want to ask in social work?
![Page 7: Making Social Work Count Lecture 9](https://reader035.fdocuments.in/reader035/viewer/2022062520/56816226550346895dd25915/html5/thumbnails/7.jpg)
Research questions
1. Do children exposed to domestic violence experience more mental ill health than children who have not been exposed?
2. Is smacking an effective behavioural management tool?3. Are young offenders on non custodial sentences less
likely to offend than young offenders on custodial sentences?
4. Is parenting capacity reduced when parents misuse drugs or alcohol?
5. Do children in kinship care have better outcomes than children with unrelated foster carers?
![Page 8: Making Social Work Count Lecture 9](https://reader035.fdocuments.in/reader035/viewer/2022062520/56816226550346895dd25915/html5/thumbnails/8.jpg)
Hypotheses: Research and Null
![Page 9: Making Social Work Count Lecture 9](https://reader035.fdocuments.in/reader035/viewer/2022062520/56816226550346895dd25915/html5/thumbnails/9.jpg)
Research hypotheses
1. Children exposed to domestic violence experience more mental ill health than children who have not been exposed
2. Smacking is an effective behavioural management tool3. Young offenders on non custodial sentences are less likely
to offend than young offenders on custodial sentences 4.Parenting capacity is reduced when parents misuse drugs
or alcohol5. Children in kinship care have better outcomes than
children with unrelated foster carers
![Page 10: Making Social Work Count Lecture 9](https://reader035.fdocuments.in/reader035/viewer/2022062520/56816226550346895dd25915/html5/thumbnails/10.jpg)
Making statements for testing
Research hypothesis• A proposed explanation for a
phenomenon that can be tested• There is a relationship between
two measured variables• A particular intervention makes
a difference/has an effect
Example: Smacking is an effective behavioural management tool
Null hypothesis • The opposite position of the
hypothesis • There is no relationship
between two measured variables
• The particular intervention does not make a difference/has no effect
Example: Smacking is not an effective behavioural management tool
![Page 11: Making Social Work Count Lecture 9](https://reader035.fdocuments.in/reader035/viewer/2022062520/56816226550346895dd25915/html5/thumbnails/11.jpg)
Accepting the research hypothesis
• Accepting the research hypothesis says that the difference between sample means is too large to be accounted for by sampling error, and, therefore, reflects differences within or between populations.
![Page 12: Making Social Work Count Lecture 9](https://reader035.fdocuments.in/reader035/viewer/2022062520/56816226550346895dd25915/html5/thumbnails/12.jpg)
Research questions and hypotheses in Social Work
• Does a particular intervention work? – Have negative symptoms reduced after the
intervention? (i.e. depression; anxiety; numbers of hours of care)
– Have positive attributes of service users increased after the intervention? (i.e. self-esteem; optimism; hope; less number of hours of care)
• Are there differences in treatment outcomes based on gender, ethnicity, age?
![Page 13: Making Social Work Count Lecture 9](https://reader035.fdocuments.in/reader035/viewer/2022062520/56816226550346895dd25915/html5/thumbnails/13.jpg)
Singing and Mental Health: An example of hypothesis testing
![Page 14: Making Social Work Count Lecture 9](https://reader035.fdocuments.in/reader035/viewer/2022062520/56816226550346895dd25915/html5/thumbnails/14.jpg)
Does singing have an effect on mental health? • Study looking at the effect of choral signing on mental health
(N=218)• Research hypothesis – Participation in choral singing will result in
changes in the participant’s emotional state. • Null hypothesis – Participation in choral singing will result in no
changes in the participant’s emotional state. • Measured aspects of one’s emotional state before and after the
choral singing intervention• Found that positive emotions increased (excited, alert, hopeful,
relaxed, harmonic, to be present) and negative emotions decreased (pain, headache). Also found that participants felt more together and less alone.
![Page 15: Making Social Work Count Lecture 9](https://reader035.fdocuments.in/reader035/viewer/2022062520/56816226550346895dd25915/html5/thumbnails/15.jpg)
Does singing have an effect on mental health? • Are these findings true
or did the differences in emotional states just happen by chance?
• How confident are we that there was a statistically significant difference in emotional state pre and post the choral singing?
![Page 16: Making Social Work Count Lecture 9](https://reader035.fdocuments.in/reader035/viewer/2022062520/56816226550346895dd25915/html5/thumbnails/16.jpg)
Statistically Significant
• The researchers would have employed statistical tests to determine the extent to which they are confident that the results accurately reflect what occurs in the population (a real population difference) versus merely occurring by chance (or sampling error):– t – test
• Based on the outcome of the test, the researchers can determine whether the results are statistically significant. That is, the outcome is unlikely to have occurred by chance
The basic requirements for testing the difference between two means
![Page 17: Making Social Work Count Lecture 9](https://reader035.fdocuments.in/reader035/viewer/2022062520/56816226550346895dd25915/html5/thumbnails/17.jpg)
Basic requirements in hypothesis testing/ answering research questions1. Develop a research hypothesis to test or a research question to answer2. Select or determine the main variable of interest to measure (data)
• This could involve using a standardised measure (i.e. Strengths and Difficulties questionnaire for children; parenting capacity measure) or any other numerical value (i.e. number of units of alcohol or drugs used; number of weeks in foster care)
3. Select your sample(s), gather the data, and calculate the descriptive statistics of the sample(s)
4. Conduct a statistical test of difference between the means (or other descriptive statistics, such as percentages) (t-test; chi-square; ANOVA; Fisher’s exact test)
• This involves pre-determining the alpha value5. Interpret whether the results are statistically significant
![Page 18: Making Social Work Count Lecture 9](https://reader035.fdocuments.in/reader035/viewer/2022062520/56816226550346895dd25915/html5/thumbnails/18.jpg)
An example of determining differences between hope
amongst social work students
![Page 19: Making Social Work Count Lecture 9](https://reader035.fdocuments.in/reader035/viewer/2022062520/56816226550346895dd25915/html5/thumbnails/19.jpg)
A “hopeful” example of how to test for differences
1. Develop a research question to answer or a research hypothesis to test:• Research Question - Is there a difference in level of
hope between first year and final year social work students?
• Research Hypothesis - First year social work students will have higher levels of hope than final year social work students.
• Null Hypothesis – There is no difference in level of hope between first year and final year social work students.
![Page 20: Making Social Work Count Lecture 9](https://reader035.fdocuments.in/reader035/viewer/2022062520/56816226550346895dd25915/html5/thumbnails/20.jpg)
A “hopeful” example of how to test for differences
2. Select an instrument or measure of hope:• The Trait Hope Scale (Snyder et al., 1991)
![Page 21: Making Social Work Count Lecture 9](https://reader035.fdocuments.in/reader035/viewer/2022062520/56816226550346895dd25915/html5/thumbnails/21.jpg)
Question DefinitelyFalse
MostlyFalse
MostlyTrue
DefinitelyTrue
I can think of many ways to get out of a difficult situation.
1 2 3 4
I energetically pursue my goals. 1 2 3 4
I feel tired most of the time. 1 2 3 4
There are lots of ways around any problem. 1 2 3 4
I am easily downed in an argument. 1 2 3 4
I can think of many ways to get the things in life that are most important to me.
1 2 3 4
I worry about my health. 1 2 3 4
Even when others get discouraged, I know I can find a way to solve the problem.
1 2 3 4
My past experiences have prepared me well for my future.
1 2 3 4
I’ve been pretty successful in life. 1 2 3 4
I usually find myself worrying about something. 1 2 3 4
I meet the goals that I set for myself. 1 2 3 4
![Page 22: Making Social Work Count Lecture 9](https://reader035.fdocuments.in/reader035/viewer/2022062520/56816226550346895dd25915/html5/thumbnails/22.jpg)
The Trait Hope Scale
• Sums the scores of items: 1, 2, 4, 6, 8, 9, 10, and 12 (the other items are fillers and do not contribute to the overall score).
• Score can range from 8 – 32 with higher scores equally higher levels of hope.
![Page 23: Making Social Work Count Lecture 9](https://reader035.fdocuments.in/reader035/viewer/2022062520/56816226550346895dd25915/html5/thumbnails/23.jpg)
A “hopeful” example of how to test for differences
3. Select a random sample of students from the first year cohort and final year cohort (N = 30 for each cohort):• Administer the the Trait Hope Scale to the sample
of first and final year students. • Calculate the mean scores for first and final year
students.
![Page 24: Making Social Work Count Lecture 9](https://reader035.fdocuments.in/reader035/viewer/2022062520/56816226550346895dd25915/html5/thumbnails/24.jpg)
First Year Social Work Students (N=30)
Final Year Social Work Students (N=30)
30 25 18 20
25 31 20 28
31 30 25 24
24 32 19 23
30 29 23 22
25 26 24 28
24 29 25 30
22 30 26 25
18 27 30 23
20 28 18 24
31 23 15 27
32 32 16 18
28 29 14 19
30 30 18 20
29 31 30 15
![Page 25: Making Social Work Count Lecture 9](https://reader035.fdocuments.in/reader035/viewer/2022062520/56816226550346895dd25915/html5/thumbnails/25.jpg)
Is there a difference in level of hope between first and final year students?• Calculate the mean scores between the two groups of
students:– First Years = 27.7– Final Years = 22.2
• There is a mean difference of 5.5 • Can we conclude that first years are actually more
hopeful than final years?• Could the difference be a result of sampling error and,
therefore, the difference is due to chance and chance alone?
![Page 26: Making Social Work Count Lecture 9](https://reader035.fdocuments.in/reader035/viewer/2022062520/56816226550346895dd25915/html5/thumbnails/26.jpg)
Descriptive Statistics
N Min. Max. Mean Std. Deviation
Overall hope score for first year students 30 18 32 27.70 3.706
Overall hope score for final year students 30 14 30 22.23 4.688
Valid N (listwise) 30
Output #1: The mean hope score for the first and final year students
![Page 27: Making Social Work Count Lecture 9](https://reader035.fdocuments.in/reader035/viewer/2022062520/56816226550346895dd25915/html5/thumbnails/27.jpg)
The distribution of hope scores for first and final year students
Mean Mean
![Page 28: Making Social Work Count Lecture 9](https://reader035.fdocuments.in/reader035/viewer/2022062520/56816226550346895dd25915/html5/thumbnails/28.jpg)
Standard Deviation: A measure of variability
• The standard deviation (sd) reflects the typical deviation from the mean
• Describes the variability of the scores– how close are the scores or how far apart are the
scores?• The sd for the hope scores were as follows:– sd (first years ) = 3.706– sd (final years) = 4.688
![Page 29: Making Social Work Count Lecture 9](https://reader035.fdocuments.in/reader035/viewer/2022062520/56816226550346895dd25915/html5/thumbnails/29.jpg)
Standard Deviation: An example of great variability and no variability
N=30; Min = 30; Max = 30; sd = .000 N=30; Min = 15; Max = 30; sd = 6.226
![Page 30: Making Social Work Count Lecture 9](https://reader035.fdocuments.in/reader035/viewer/2022062520/56816226550346895dd25915/html5/thumbnails/30.jpg)
A “hopeful” example of how to test for differences
4. Conduct a statistical test of significance (t-test) between the first year hope score
mean and the final year hope score mean.– Determine the level of significance or α (alpha
value)
![Page 31: Making Social Work Count Lecture 9](https://reader035.fdocuments.in/reader035/viewer/2022062520/56816226550346895dd25915/html5/thumbnails/31.jpg)
What level of significance?
• Statistically significant difference – The differences observed in a sample reflect a real population difference and is not a result of sampling error.
• In order to determine if something is statistically significant, you must establish a level of significance (represented by the Greek letter α (alpha)).
![Page 32: Making Social Work Count Lecture 9](https://reader035.fdocuments.in/reader035/viewer/2022062520/56816226550346895dd25915/html5/thumbnails/32.jpg)
Probability (P) and alpha value (α)
• α = the level of probability where the null hypothesis can be rejected with confidence and the research hypothesis accepted with confidence
• We symoblise the probability as P < .05 • It is convention to use the α = .05 level of significance,
but significance levels can be set up for any degree of probability – α = .01 level of significance; P < .01
• We reject the null hypothesis if the P value is less than the alpha value and otherwise retain it.
![Page 33: Making Social Work Count Lecture 9](https://reader035.fdocuments.in/reader035/viewer/2022062520/56816226550346895dd25915/html5/thumbnails/33.jpg)
Output #2: The t-test for differences between two means (Independent samples test)
Levene’s Test for
Equality of Variances
t-test for Equality of Means
Hope Score F Sig. t df Sig. (2-
tailed)
Mean Differenc
e
Std. Error Difference
95% Confidence
Interval of the Difference
Lower UpperEqual variances assumed
Equal variances not assumed
2.464 .122 5.010
5.010
58
55
.000
.000
5.467
5.467
1.091
1.091
3.283
3.280
7.651
7.653
![Page 34: Making Social Work Count Lecture 9](https://reader035.fdocuments.in/reader035/viewer/2022062520/56816226550346895dd25915/html5/thumbnails/34.jpg)
A “hopeful” example of how to test for differences
5. Interpret whether the results are statistically significant – Is there a statistically significant difference
between the two means? – Can we reject the null hypothesis of no difference?
![Page 35: Making Social Work Count Lecture 9](https://reader035.fdocuments.in/reader035/viewer/2022062520/56816226550346895dd25915/html5/thumbnails/35.jpg)
Output #2: The t-test for differences between two means (Independent samples test)
Levene’s Test for
Equality of Variances
t-test for Equality of Means
Hope Score F Sig. t df Sig. (2-tailed)
Mean Difference
Std. Error Difference
95% Confidence Interval of the
DifferenceLower Upper
Equal variances assumed
Equal variances not assumed
2.464 .122 5.010
5.010
58
55
.000
.000
5.467
5.467
1.091
1.091
3.283
3.280
7.651
7.653
![Page 36: Making Social Work Count Lecture 9](https://reader035.fdocuments.in/reader035/viewer/2022062520/56816226550346895dd25915/html5/thumbnails/36.jpg)
The results
• ‘a t of 5.010 with 58 degrees of freedom indicates that the difference between first and final year students in their mean hope scores is statistically significant at the .001 level.’
![Page 37: Making Social Work Count Lecture 9](https://reader035.fdocuments.in/reader035/viewer/2022062520/56816226550346895dd25915/html5/thumbnails/37.jpg)
99% Confident (1 chance out of 100 of a Type I error)
Mean Mean
![Page 38: Making Social Work Count Lecture 9](https://reader035.fdocuments.in/reader035/viewer/2022062520/56816226550346895dd25915/html5/thumbnails/38.jpg)
Statistically Significant
• The researchers would have employed statistical tests to determine the extent to which they are confident that the results accurately reflect what occurs in the population (a real population difference) versus merely occurring by chance (or sampling error):– t – test
• Based on the outcome of the test, the researchers can determine whether the results are statistically significant. That is, the outcome is unlikely to have occurred by chance
Have we made the right decision: Type I and Type II errors
![Page 39: Making Social Work Count Lecture 9](https://reader035.fdocuments.in/reader035/viewer/2022062520/56816226550346895dd25915/html5/thumbnails/39.jpg)
Type I and Type II errors
• Setting a level of significance does not mean we will always be able to say with 100% confidence that we are correct in accepting or rejecting the null hypothesis. – There is a 5 in 100 chance (P = .05) with α = .05
level of significance of being wrong and 1 in 100 (P = .01) chance with α = .01 level of significance of being wrong.
![Page 40: Making Social Work Count Lecture 9](https://reader035.fdocuments.in/reader035/viewer/2022062520/56816226550346895dd25915/html5/thumbnails/40.jpg)
Type I and Type II errors
• Type I error (α) – Reject the null hypothesis (stating there IS a difference between means) when we should have accepted (stating there IS NOT a difference between means).– The more stringent our level of confidence, the less likely we
will make a Type I error• Type II error (β)– Retaining the null hypothesis (stating
there IS NOT a difference between means) when we should have accepted (stating there IS a difference between means)– Increase the size of samples
![Page 41: Making Social Work Count Lecture 9](https://reader035.fdocuments.in/reader035/viewer/2022062520/56816226550346895dd25915/html5/thumbnails/41.jpg)
Type I and Type II errors
Correct Decision
Type I errorP (Type I error) = α
Type II error P (Type II error) = β
Correct Decision
Retain null Reject nullHypothesishypothesis
DECISION
REALITY
Null hypothesisis true
Null hypothesisis false
* Levin, J., & Fox, J.A. (2003). Elementary statistics in social research, (9th ed.). Boston: Allyn and Bacon.
![Page 42: Making Social Work Count Lecture 9](https://reader035.fdocuments.in/reader035/viewer/2022062520/56816226550346895dd25915/html5/thumbnails/42.jpg)
Testing the difference between means
Type of Test When to use (examples)
Independent samples t-test To test whether the means of two unrelated samples differ• Testing whether hope scores differ by year in the social work programme• Testing whether IQ scores differ by sex
Paired-samples t-test Is used with matched pair data where the research question calls for the repeated measurement of responses from the same individual • Testing whether hope scores increase for individuals from before a motivation course to after the course has taken place• Testing whether depression scores decrease for individuals from before to after therapy
![Page 43: Making Social Work Count Lecture 9](https://reader035.fdocuments.in/reader035/viewer/2022062520/56816226550346895dd25915/html5/thumbnails/43.jpg)
Example 2: How do placements in kinship care compare with those in non-kin foster care? (Farmer, 2009)
![Page 44: Making Social Work Count Lecture 9](https://reader035.fdocuments.in/reader035/viewer/2022062520/56816226550346895dd25915/html5/thumbnails/44.jpg)
1. Develop a research hypothesis (or question) to test (or answer)
• Are there differences in parent-related adversities and child-related adversities of children placed with kin versus children placed with unrelated foster carers?
(Farmer, 2009)
![Page 45: Making Social Work Count Lecture 9](https://reader035.fdocuments.in/reader035/viewer/2022062520/56816226550346895dd25915/html5/thumbnails/45.jpg)
2. Select or determine the main variable of interest to measure (data)
• Parent-related adversities (independent variable) – death of a parent; drugs misuse; mental health problems
• Child-related adversities (independent variable) – neglected; sexual abuse; domestic violence
• Placement of child (either with kin or with non-related foster carers)
![Page 46: Making Social Work Count Lecture 9](https://reader035.fdocuments.in/reader035/viewer/2022062520/56816226550346895dd25915/html5/thumbnails/46.jpg)
3. Select your sample, gather the data and calculate descriptive statistics of the sample • List of 2240 children – A sample of 270 children were selected, just over
half of whom (53%) were placed with family or friends and just under half (47%) with unrelated foster carers.
• Reviewed case files, interviews using semi-structured interview format, standardized measures
![Page 47: Making Social Work Count Lecture 9](https://reader035.fdocuments.in/reader035/viewer/2022062520/56816226550346895dd25915/html5/thumbnails/47.jpg)
4. Conduct a statistical test
![Page 48: Making Social Work Count Lecture 9](https://reader035.fdocuments.in/reader035/viewer/2022062520/56816226550346895dd25915/html5/thumbnails/48.jpg)
4. Conduct a statistical test
![Page 49: Making Social Work Count Lecture 9](https://reader035.fdocuments.in/reader035/viewer/2022062520/56816226550346895dd25915/html5/thumbnails/49.jpg)
5. Interpret the results of the study
![Page 50: Making Social Work Count Lecture 9](https://reader035.fdocuments.in/reader035/viewer/2022062520/56816226550346895dd25915/html5/thumbnails/50.jpg)
5. Interpret the results of the study
![Page 51: Making Social Work Count Lecture 9](https://reader035.fdocuments.in/reader035/viewer/2022062520/56816226550346895dd25915/html5/thumbnails/51.jpg)
Learning outcomes
• Are you able to:– Define research question, research
hypothesis, null hypothesis and statistically significant;
– Discuss the basic requirements for testing the difference between two means;
– Define and describe the difference between the alpha value and p value, and Type I and Type II errors;
• Advanced Study:– Calculate the difference between
the means (t-ratio) using example data through advanced study
![Page 52: Making Social Work Count Lecture 9](https://reader035.fdocuments.in/reader035/viewer/2022062520/56816226550346895dd25915/html5/thumbnails/52.jpg)
Activity
![Page 53: Making Social Work Count Lecture 9](https://reader035.fdocuments.in/reader035/viewer/2022062520/56816226550346895dd25915/html5/thumbnails/53.jpg)
Activity
Ask the students to read the following case scenario and tasks:
Andrew (7-years-old) was referred by his social worker to a domestic abuse intervention programme run by a local organisation. The social workerreferred Andrew to the program because he had witnessed domestic abusebetween his mother and father and was presenting with behavioural andemotional difficulties. Andrew was described as “withdrawn, easily tearful, and reluctant to engage with this fellow classmates at school”. The intervention program consisted of 10 weeks of arts-based group work where 5-10 children (ages 6 – 11) met for 1 ½ hours per week. The programme uses the Strengths & Difficulties Questionnaire (SDQ) to assess the participants on behavioural and emotional aspects. The social worker wishes to evaluate whether the programme is effective to determine whether to continue to use this service in the future.
![Page 54: Making Social Work Count Lecture 9](https://reader035.fdocuments.in/reader035/viewer/2022062520/56816226550346895dd25915/html5/thumbnails/54.jpg)
…Continued
Based on the above scenario, you are tasked with designing an evaluation of the domestic abuse intervention programme. Start by exploring the Strengths & Difficulties Questionnaire by accessing the forms and scoring sheets on the website: http://www.sdqinfo.com/ and then answer the following questions:
• What is your hypothesis and null hypothesis?• Which form(s) will you use for the evaluation and why? • Who will fill out the form(s) and when?• How will you determine whether the intervention was successful? In
particular, consider the type of statistical test(s) you will use and how you will interpret the results.
![Page 55: Making Social Work Count Lecture 9](https://reader035.fdocuments.in/reader035/viewer/2022062520/56816226550346895dd25915/html5/thumbnails/55.jpg)
References
• Carpenter, J., Shardlow, S.M., Patsios, D., & Wood, M. (2013). Developing the confidence and competence of Newly Qualified Child and Family Social Workers in England: Outcomes of a National Programme, British Journal of Social Work, 1-24. – Lecturers could use this article as an example of hypothesis testing and comparing the mean between two samples.
• Farmer, E. (2009). How do placements in kinship care compare with those in non-kin foster care: Placement patterns, progress and outcomes? Child & Family Social Work, 14, 331-342.
• Strengths and Difficulties Questionnaire (SDQ) webpage. (nd). Information for researchers and professionals about the Strengths & Difficulties Questionnaires. Retrieved 11 January 2014, from http://www.sdqinfo.com/.
• Webber, M. (2011). Evidence-Based Policy and Practice in Mental Health Social • Work, 2nd edn. Exeter: Learning Matters. – Lecturers should refer to Chapter 11:
Demystifying p-values: a user-friendly introduction to statistics used in mental health research.