Information Resources Evaluation - Headline final results

for Macmillan Cancer Support July2013

• Introduction• Who responded to the survey /

feedback forms?• Who has fed back on which format

of information?• How do information needs change

over time?• Who benefits the most from which

information?• What happens after reading the

information?• How do benefits change over time?• Content-specific findings• Summary of findings

• Next steps– Areas for particular

investigation– Follow-up offers

received– Project elements

ContentsIntroduction

• The Information Resources Evaluation focuses on the following booklets and web pages:

– Cancer-specific• Breast cancer• Colon cancer• Lung cancer• (Early localised) prostate cancer

– Generic• Benefits and financial help / Help

with the cost of cancer• Chemotherapy• Fatigue

• This report provides the findings from:– 672 survey respondents (452 online survey and

220 paper feedback forms), collected between 8 Jan and 6 Jun 2013

– XX telephone interviewees, conducted between XX and XX 2013

– XX in-depth interviewees (two face to face, and XX telephone), conducted between 29 May and XX 2013

• For this headline report, outcome ratings are aggregated in most cases

• The number of respondents (n) is shown where appropriate.

• Although the demographic spread of responses is good, some questions have relatively low responses rates and this should be taken into account when interpreting the results

MethodologyIntroduction

Who has been involved in this evaluation?

• Breast cancer information has generated the most responses overall (almost a quarter)

• There has been a good spread of respondent’s status: 53% of respondents are living with/beyond cancer; 35% are family, friends and carers; 5% are professionals

• Women make up over two thirds of all respondents

• Almost 80% of all respondents are over 45

• 90% of all respondents areWhite. There is too little variation for detailed analysis by ethnic group at this stage

FindingsWho has been involved in this evaluation?

Update

Respondents’ statusWho has been involved in this evaluation?

n = 653 (respondents were able to choose more than one situation) and 59

Just over half of respondents (59%) are people who are living with/beyond cancer

Telephone interviewsSurvey

TopicWho has been involved in this evaluation?

n = 672

Breast cancer information has generated the most survey responses (23%)

Survey

Telephone interviews

Topic and formatWho has been involved in this evaluation?

Responses are broadly representative of distribution and unique online visit monitoring

Type

Booklet distribution

rank

Booklet respondents

rank Unique

visits rank

Online respondents

rank

Benefits and financial help Generic 1 2 5 7

Fatigue Generic 2 1 7 6

Chemotherapy Generic 3 3 1 5

Lung cancer Cancer-specific 4 6 4 3

Colon cancer Cancer-specific 5 6 3 4

Breast cancer Cancer-specific 6 4 2 1

Prostate cancer Cancer-specific 7 5 6 2

Age and genderWho has been involved in this evaluation?

79% of respondents are over 45; 67% are female

Age group Male Female TOTALUnder 16 1 5 6

16 to 25 5 17 22

26 to 45 14 90 104

46 to 65 87 255 342

Over 65 104 67 171

Prefer not to say

3 3

TOTAL 234 434 648

n = 648 and 59

Telephone interviewsSurvey

Age group Male Female TOTALUnder 16 0 0 0

16 to 25 0 1 1

26 to 45 1 7 8

46 to 65 8 22 30

Over 65 12 7 19

Prefer not to say

0 0 0

TOTAL 21 37 58

n = 653

Ethnic groupsWho has been involved in this evaluation?

84% of survey respondents were White British

Survey

• 10 interviews:

In-depth interviewsWho has been involved in this evaluation?

Update

Who responded to which format of information?

• Booklets responses were mostly about generic information (rather than cancer-specific)

• Respondents who look after someone with cancer have been more likely to provide feedback on booklets (as opposed to online information) than other respondents

• Carers and people undergoing treatment have been more likely to feedback on generic information

• Respondents from older age groups have been more likely to feedback on online information*

• Respondents from older age groups have been more likely to respond to cancer-specific information*

• There are only slight differences between genders in terms of format and topics fed back on

* This may be reflecting preferred method of providing feedback rather than preferred means of accessing information, and so requires further investigation

FindingsWho has fed back on which format of information?

Update

Format, by topicWho responded to which format of information?

n = 208

Responses relating to booklets were primarily about generic information (77%) rather than cancer-specific (23%)

Survey

• The mix of booklet and online responses was similar across genders

• The mix of generic and cancer-specific information was similar across genders

• There was no indication of older respondents being put off by answering an online survey (over half of responses from Over 65’s were via the online survey)

Other findingsWho responded to which format of information?

What happens after reading the information? – Next steps

• One out of five respondents apply for benefits soon after reading information on benefits and financial help, and 40% intend to do so in future

• Females are more likely to recommend information than men (8% more)

• The 26 to 45 age group are most likely to recommend information

• Early indications are that respondents are more likely to recommend booklets than online information

FindingsWhat happens after reading the information?

Update

Actions taken / intended (Benefits & financial only)What happens after reading the information?

n = 70 survey (only five of these were health professionals); 7 telephone interviews. * Low base number

26% of respondents apply for benefits soon after reading information on benefits and financial help; 40% intend to

ActionAlready taken this

actionNow intend to take this action TOTAL

35 to 96 days after survey*

Contact a welfare rights adviser/benefits adviser

41% 40% 81% 71%

Contact the Macmillan Support Line 44% 26% 70% 57%

Apply for a Macmillan Grant 19% 29% 47% 14%

See a health or social care professional for more information

21% 37% 59% n/a

Apply for benefits and/or financial support

26% 40% 66% 57%

Likeliness to recommend, by age and genderWhat happens after reading the information?

n shown in brackets

Older males are less likely to recommend information, especially online information

Survey

n = 612 (showing ethnic groups with 10 or more respondents)

Likeliness to recommend, by ethnic groupWhat happens after reading the information?

White European and White Other respondents are less likely to recommend informationSurvey

What happens after reading the information? - Outcomes

• In general, the information (booklet and online combined) has the biggest immediate impact on enabling PABC to have a better understanding of the topic or condition.

• Booklets generally have a bigger immediate impact on PABC than online information (4% to 39% higher)

• Level of immediate impact is similar across respondent types (e.g. living with cancer, friends/family/carers)

• Female respondents are more likely to say they know what to do next after reading the information compared to male respondents

• Level of immediate impact are broadly similar across age groups*

* The under 16s age group has too few responses to be robust, so requires further investigation

FindingsWho benefits the most from which type of information?

Update

By topic – GenericWho benefits the most from which information?

n varies

Online info provides less benefit relative to booklets (Fatigue online info in particular)

Survey

By topic – Cancer-specificWho benefits the most from which information?

n varies

Online info provides less benefit relative to booklets (Lung cancer online info in particular)

Survey

By format of informationWho benefits the most from which information?

n varies

‘Better understanding’ is the biggest immediate impact; feeling less anxious is lowest

Outcome Booklet (paper)

Online info (online)

Difference (booklet –v-

online)Have information needed 4.4 3.9 +13%Have better understanding 4.5 4.2 +8%Feel less anxious 3.9 3.4 +14%Feel more in control 4.0 3.5 +14%Feel better able to make decisions 4.3 3.9 +10%Know what to do next 4.3 3.9 +11%Average of averages (aggregated) 4.3 3.8

Survey

Outcomes measured in surveyWho benefits the most from which information?

Information provides relatively lower benefits in terms of feelings

Survey

Outcomes measured in telephone interviewsWho benefits the most from which information?

n shown in brackets. * These were only asked of respondents based on situation

Mental and physical wellbeing are less influenced by information resources, particularly latter

Telephone interviews

All outcomes, over timeWho benefits the most from which information?

Category Outcome At time of survey

Two to three months after survey

Next steps Know what to do next 4.0 4.4Feelings Feel less anxious 3.6 4.3Knowledge & understanding Have better understanding 4.3Feelings Feel more confident   4.3Communication Better able to communicate with PABC* 4.3Next steps Feel better able to make decisions 4.1 4.3Knowledge & understanding Have information needed 4.1Communication Better able to communicate with your

health / social care prof'l(s)*4.1

Feelings Feel more in control 3.7 4.0Feelings Feel less alone   3.9Wellbeing Mental / emotional wellbeing improved 3.9Wellbeing Physical wellbeing has improved 3.0

All outcomes, over timeWho benefits the most from which information?

In order of highest average rating, descending

Benefit over timeWho benefits the most from which information?

Only showing the 93% of interviews conducted 30 to 90 days after survey response. n shown in brackets

Whilst understanding continues to improve over time, other benefits deteriorate

By respondent statusWho benefits the most from which information?

n = 653

Benefits experienced are broadly similar across respondent status

Survey

People living with / beyond cancerWho benefits the most from which information?

n = 653

Some benefits peak early during treatmentSurvey

People living with / beyond cancerWho benefits the most from which information?

n = 653

Other benefits peak during later treatmentSurvey

By genderWho benefits the most from which information?

n = 648

Outcome ratings are broadly similar across gender

Outcome Males (average outcome

rating out of 5)

Females(average outcome

rating out of 5)

Difference

Have better understanding 4.3 4.3 -0.1

Have information needed 4.0 4.1 -0.1

Feel less anxious 3.7 3.6 0.1

Feel more in control 3.7 3.7 0.0

Know what to do next 3.9 4.0 -0.1

Feel better able to make decisions

4.0 4.1 -0.1

Average of averages 3.9 4.0 0.0

By GenderWho benefits the most from which information?

Only showing the 93% of interviews conducted 30 to 90 days after survey response.

Whilst understanding continues to improve over time for males, other benefits deteriorate faster

By age groupWho benefits the most from which information?

n is shown in brackets

Outcome ratings are broadly similar across age groupsSurvey

By age groupWho benefits the most from which information?

n is shown in brackets

Outcome ratings are broadly similar across age groupsTelephone

n shown in brackets (only showing ethnic groups with 10 or more respondents)

Ethnic groups - outcomesWho benefits the most from which information?

White European and White Other respondents benefit less across all outcomes

Outcome

White British (547)

White European

(34)White

Other (12)

Asian or Asian

British (19)…I have all the information about the topic that I need at this time

4.2 3.7 3.1 4.3

…I have a better understanding of the topic/condition

4.4 3.9 4.2 4.1

…I feel better able to make the right decisions for me

4.1 3.4 3.4 4.1

…I feel less anxious 3.7 3.1 2.8 3.5

…I now know what to do next (e.g. who to speak to, what services to access)

4.1 3.6 3.4 3.9

…I feel more in control 3.8 3.3 3.0 3.6

Only showing ethnic groups with 10 or more respondents

• White European and White Other mainly responded about online information (88% and 83%, compared to 67% across the whole sample)

• Asian and Asian British respondents were overrepresented by females (89%, compared to 67% across the sample)

• Asian and Asian British respondents were overrepresented by ages 16 to 25 (23% aged, compared to 3% across the sample)

Ethnic groups – causal analysisWho benefits the most from which information?

By format of information, over timeWho benefits the most from which information?

Online info Booklet Category Outcome At time of

surveyTwo to three months after survey

At time of survey

Two to three months after survey

Next steps Know what to do next 3.9 4.6 4.3 4.3Feelings Feel less anxious 3.4 4.3 3.9 4.3Knowledge & understanding

Have better understanding 4.2 4.5

Feelings Feel more confident   4.3  4.2Communication Better able to communicate with

PABC*4.3 4.3

Next steps Feel better able to make decisions 3.9 4.5 4.3 4.1Knowledge & understanding

Have information needed 3.9 4.4

Communication Better able to communicate with your health / social care prof'l(s)*

4.0 4.1

Feelings Feel more in control 3.5 3.7 4.0 4.2Feelings Feel less alone   3.8  4.0Wellbeing Mental / emotional wellbeing

improved4.0 3.8

Wellbeing Physical wellbeing has improved 3.0 3.1

By type of information, over timeWho benefits the most from which information?

Generic Cancer-specificCategory Outcome At time of

surveyTwo to three months after survey

At time of survey

Two to three months after survey

Feelings Feel more confident   4.4  4.2Communication Better able to communicate with

PABC*  4.3  4.2

Knowledge & understanding

Have better understanding 4.3 4.3

Next steps Know what to do next 4.1 4.3 4.0 4.6Feelings Feel less anxious 3.7 4.2 3.5 4.4Knowledge & understanding

Have information needed 4.2 4.0

Next steps Feel better able to make decisions 4.1 4.1 4.0 4.5Feelings Feel more in control 3.8 4.1 3.7 3.9Feelings Feel less alone   4.1  3.7Communication Better able to communicate with

your health / social care prof'l(s)*  4.0  4.3

Wellbeing Mental / emotional wellbeing improved

3.9 3.8

Wellbeing Physical wellbeing has improved 2.9 3.2

Summary and conclusions

• In terms of who has responded to date, there was a good spread across respondents’ status but almost 80% of respondents were over 45 and 90% of respondents were White.

• Female respondents feel more able to take the next steps after reading the information compared to male respondents.

• Respondents who look after someone with cancer were more likely to feedback on the booklets (as opposed to online information) than other types of respondents

• Carers and people undergoing treatment were more likely to feedback on generic information

• The information has the biggest immediate impact on PABC having a better understanding of the topic or condition after reading

• The emerging findings indicate that booklets have a bigger immediate impact than online information (4 to 39% higher)

• Outcome ratings from those who have / had cancer are broadly similar for respondents with other status’ (e.g. carers, friends/family)

• One sixth of respondents apply for benefits soon after reading information on benefits and financial help, plus almost half intend to do so in future

Summary of findingsUpdate

• How do typical distribution numbers compare for booklets, to assess how representative the respondent mix is

• This will help differentiate between those preferring to access information in particular ways, and those more likely to provide feedback in particular ways

• Do women experience greater benefits, or are there are other factors (e.g. starting point earlier than males, bias in the way they have responded)?

• Are the experiences of under 16s significantly different?

• Are there barriers to applying for Macmillan grant or seeing a health or social care professional?

• Why does the information have a comparatively smaller immediate impact on reducing anxiousness / feeling more in control?

• Explore differences between ethnic groups further (response numbers too low at present)

• Finer grain analysis of each outcome, and each topic of information across booklets and online information

• How do outcomes change over time?

Areas for further investigationNext steps

Update

Follow-up offers receivedNext steps

n = 546

DELETE?

Ethnic groupSurvey respondents

Telephone interview volunteers

Discussion group volunteers

White British 476 39% 15%White European 30 33% 13%White Other 12 8% 0%Black or Black British 6 33% 17%Asian or Asian British 15 40% 27%Chinese 1 0% 0%Mixed 6 50% 0%

• A written report will be drafted and sent to Macmillan for comment

• The written report will be revised based on Macmillan’s comment, and issued as a final version

Project elementsNext steps

1 Naoroji Street, LONDON, WC1X 0GB020 7756 7600

www.sharedintelligence.net

solutions@sharedintelligence.net@sharedintel