Post on 22-Dec-2015
•CIVITAS PLUS
Evaluation of mobility and social inclusionDr Laurie Pickup
Regional Director for Europe
WYG Environment, Planning and transport
•Venue : Coimbra, Portugal
Date: 22nd/23° March 2012
The scope of this session
1. The nature of mobility and social disadvantage – defining the scope of our evaluation?
2. Setting out your evaluation - the baseline expectations, objectives, target groups, success criteria
3. Understanding the mobility behaviour of different social groups and attitudes – what are the key variables to evaluate?
4. Measuring and analysing mobility and inclusion situations and response to change - Qualitative assessment - Quantitative surveys
5. Conclusions
Coimbra, 23/24 March, 2012
Part 1
The nature of mobility and social disadvantage –
defining the scope of our evaluation?
Coimbra, 23/24 March, 2012
Mobility is not just a transport phenomenon
Spatial/temporal mobility - horizontal
Social mobility - vertical
(Virtual mobility)
Coimbra, 23/24 March, 2012
Ability - Physical access to transport facilities and services in terms of vehicle and system design, waiting and interchange facilities and information services;
Opportunity - Spatial and temporal distribution of transport services and their connectivity with key activities;
Affordability - cost of transport services and its relation to incomes;
Barriers to mobility 1/2
Coimbra, 23/24 March, 2012
Barriers to mobility 2/2
Cognitive and psychological factors - such as ability of users to understand and navigate the system, perceptions of personal safety and fear of crime;
Cultural and social factors - regarding acceptability of the services which are provided, including safety, comfort but also attitudes and social norms.
Coimbra, 23/24 March, 2012
Social Exclusion: What are we actually measuring?
The concept of ‘social exclusion’ is contested, and
has multiple meanings. These meanings are being continually redefined over
time and have different policy implications.
(World Health Organisation review 2008)
Coimbra, 23/24 March, 2012
Social exclusion is a myth?
‘’Exclusion” is not a concept rooted in the social sciences, but an empty box given by the French state to the social sciences in the late 1980s as a subject to study… The empty box has since been filled with a huge number of pages, treatises and pictures, varying degrees academic, popular, original and valuable’. (Murard, 2002)
‘The excluded made up one tenth of the French population: the mentally and the physically handicapped, suicidal people, aged invalids, abused children, drug addicts, delinquents, single parents, multi-problem households, marginal, asocial persons, and other social misfits’ (Lenoir)
Coimbra, 23/24 March, 2012
Definition of social exclusion
‘A shorthand for what can happen when people or areas suffer from a combination of linked problems such as unemployment, poor skills, low incomes, poor housing, high crime environments, bad health and family breakdown’(UK, Social Exclusion Unit, 1997)
So what are we evaluating?
Coimbra, 23/24 March, 2012
Parameters of mobility disadvantage
Frailty
Dependency (young and very old)
Disability (physical, sensory, learning capacity)
Low income
Gender role
Ethnicity
In most cases of exclusion, these aspects arise in combination
Coimbra, 23/24 March, 2012
Children Young people Older people Disabled people
Personal Need to be escorted or driven
Semi-independent but with some restrictions
Very mixed picture in terms of abilities.Can be frail and confused in later life
Can be frail and may experienceMobility and cognitive difficulties
GeographicUrban/ Peripheral/ Rural
Greater exposure to traffic in urban areas
More excluded from activities in rural areas
Problem of retirees moving to remote locations.Longer term resident of social housing estates.
More likely to be located in urban centres.
Table of mobility disadvantage
Coimbra, 23/24 March, 2012
Part 2
Setting out your evaluation - the baseline expectations, objectives, target groups, success
criteria
Coimbra, 23/24 March, 2012
What is the focus of your evaluation?
A specific initiative dedicated to addressing mobility and exclusion/inclusion issues (e.g. Mobility and poverty in Liverpool)
An initiative primarily with a sustainable mobility focus – but which may have social impacts (e.g Mobility policy in Perugia)
An initiative where the social dimension of the issue is to be evaluated, but not specifically for its inclusion components (Most CIVITAS CITIES)
Coimbra, 23/24 March, 2012
LIVERPOOL – October 1986
Study of impact of bus privatisation on the poor
Bus fare increase of 300% on privatisation
The start of ‘Community links’
Followed by ‘Travel safe’
Coimbra, 23/24 March, 2012Coimbra, 23/24 March, 2012
Evaluation life-cycle
What stage of the process are you addressing?
Identification of needs and requirements of marginalised groups
Identification of behavioural responses to a change in provision for marginalised groups
Both
Coimbra, 23/24 March, 2012
An example evaluation life cycle
Systematically assess the accessibility needs; Generate accessibility strategies that are fully inclusive to all
social groups, based on both state of the art experience and local needs;
Generate individual measurement options that integrate to meet the strategy objectives;
Conduct pre-implementation assessment of options on the benefits to different social groups, including economic assessment;
Generate a chosen strategy; Implementation guidelines; Scheme monitoring and quality evaluation (social and
economic evaluation, but also mobility efficiency and sustainability criteria);
Feedback and strategy revision, input to standards and protocols; and
Reporting and intelligence sharing.
Coimbra, 23/24 March, 2012
Defining your target group(s)
Mobility exclusion among:
- specific age groups or gender e.g. Elderly/frail
- income groups e.g poverty definitions
- specific personal mobility disadvantaged groups e.g sensory impaired people
- poor access to specific activities (e.g employment)
- overall mobility in specific types of regions (city centres, peripheral suburbs, rural areas)
Coimbra, 23/24 March, 2012
Taking a cross sector approach
Social exclusion is a cross-sector issue about restricted life-style
The evaluation has to include this perspective
In 1998, the Government’s Social Exclusion Unit found that mobility services in the UK covered the decisions of 11 ministries.
Assess the link between mobility policy and social policy (including employment etc.)
Work with other sectors in the evaluation
The best solution may not be a transport one
Coimbra, 23/24 March, 2012
What indicators will you measure?
Hard indicators to measure needs and/or impacts (quantitative)
Soft indicators to measure needs and/or impacts (qualitative)
Only mobility indicators?
Indicators of social inclusion?
What level of aggregation are you addressing?
What indicators will define success?
Coimbra, 23/24 March, 2012
Indicators (1)
Mobility indicators
Use the wide interpretation of accessibility
Accessibility models of varying complexity
Reviews of accessibility indicators
- increased number of activity opportunities (e.g. Jobs)
- increased number of public transport services (routes, frequency)
- increased patronage of public transport services by the target group
- increased satisfaction scores with access to different facilities
Coimbra, 23/24 March, 2012
Inclusion indicators
National Action Plans11 primary indicators, 3 secondary
indicators and 11 context indicators; these are known as the streamlined indicators
ExamplesPrimary - Early school leavers not in
education or training, total Secondary - Persons with low educational
attainment Aged 25-64 Context - Self-reported limitations in daily
activities by income quintiles (Eurostat)
Coimbra, 23/24 March, 2012
A simple approach to planning your evaluation
4 dimensions:Target group(s) – e.g. Young unemployedType of measure – e.g.
- Policy: Achieving equal access to employment
- Strategy: Improve access to job interviews
- Method: Ticket subsidyType of disadvantage - e.g. Low income,
restricted residential mobilityType of mobility – e.g. public transport
modes
Coimbra, 23/24 March, 2012
Part 3
Understanding mobility behaviour and
attitudes as part of lifestyle – what are the
key variables?
Coimbra, 23/24 March, 2012
“To change travel behaviour, you first
have to fully understand what motivates it”
Coimbra, 23/24 March, 2012
Transport Planning
Follow-the-leader model / optimal velocity model : Newell
2*
0
),(1
)(
s
vvs
v
va
dt
tdv
; relaxation time,
; Optimal velocity function
Intelligent driver model : Treiber et al. (PRE 2000)
; headway
)())((')( tvtdv
dt
tdv e
)()()( 1 txtxtd
cce dddvdv tanh)tanh()2/()(' 0
relative velocity
Coimbra, 23/24 March, 2012
Time and Cost
Cost to driver made up of time, distance and other costs (such as tolls)
Cost = (A x t) + (60 x B x d) + (C x p) whereA = time coefficientst = travel time in minutesB = distance coefficient in minutes per miled = link length in milesC = toll coefficient in minutes per monetary costp = price of the toll in monetary cost units
Coimbra, 23/24 March, 2012
Disaggregated Models
Could you put a value on a person’s time?
Are travel decisions simply a matter of trading
time and cost?
the travel decisions of people were in fact
much more complex
disaggregating models even further
into more varied types of household,
and personal/household
characteristics
Coimbra, 23/24 March, 2012
Disadvantaged Groups
growing mobility problems of the poor, of
women in traditional households, of the
disabled, of children and of the old and frail
Policies began to greatly emphasize the need
to encourage public transport use
car pooling schemes emerged
Coimbra, 23/24 March, 2012
Time-geographic/activity approaches
• the chances to travel are dictated by the periods of time and the spatial opportunities available
• Emphasis on ‘accessibility’ rather than ‘mobility’
• policies should be targeted at travel reduction and access improvement
Coimbra, 23/24 March, 2012
Life-cycle and habit
• travel patterns showed similarities within household types at different life-cycle stages
• ‘Life shocks’ experience
• Short term travel behaviour is dominated by habit
• the actual numbers of trips have not changed greatly over the years
• the distances people travel changed
Coimbra, 23/24 March, 2012
Realistic Choices
• one third of households’ trips could not be changed for various reasons
• ‘realistic choice theory’
• ‘individualised marketing’
• ‘Personalised Travel Planning’
Coimbra, 23/24 March, 2012
Evaluating travel behaviour
• people make travel decisions based on time and cost
• travel behaviour is influenced by habitual behaviour
• providing information and support to produce the changes in behaviour
• expenditure on travel remains stable except in lower income households
• More focus on mobility mind-sets
Coimbra, 23/24 March, 2012
Part 4
Measuring impacts and understanding why
they occurred: impact and process evaluation
Coimbra, 23/24 March, 2012
Measurement methods
Qualitative research
Quantitative research
- Non-statistical surveys
- Statistical surveys of samples to predict population parameters
- Sample selection (random/non-random)
- Survey type
- Statistical error and sample disaggregation
Analysis
- Single variable, bi-variate, multi-variate
Coimbra, 23/24 March, 2012
Qualitative evaluation methods (1)
To understand the types of issues arising and the nature of processes and change
Good for social exclusion/inclusion evaluation
Small samples, semi-structured using topic guides, recording, skilled moderator
Focus groups
One to one interviews
Use of mentoring intermediaries for ‘hard to reach groups’
Coimbra, 23/24 March, 2012
Qualitative research (2)
Qualitative analysis
Record and transcribe the interviews word for word (ensure good microphone positioning)
Unless intermediaries are necessary, try to interview yourself
Analyse ranges and avoid numbers.
Quote ranges of opinions and arguments
Use the results for design of quantitative survey
Coimbra, 23/24 March, 2012
Survey objectives (1)
• Identify features of (and changes to) behaviour and attitudes - in the ‘population’ - in a statistical sense – through measuring the ‘variables’ we define.
• Qualitative research and quantitative surveys
• Identify through variable measurement, the factors that shape attitudes – how this relates to behaviour patterns
• Importance – satisfaction – expectations – prejudice
- measurement of the ‘scale’ of the variable
- measurement of the relative imoportance or ‘ranking’ between variables
Southampton (UK), 20/03/2009
Coimbra, 23/24 March, 2012
Survey approaches
• Randomly sample ‘a defined population’
• Survey the sample in the ‘change’ area and in potential ‘transfer’ sites to infer results
• Use of ‘experimental and control samples’
• Single random sample surveys at a point in time (retrospective – e.g short term after survey)
• Repeat surveys of random samples from the same population (Before and after, periodic monitoring)
• Repeat interviews with the same sample subjects (Panels)
Coimbra, 23/24 March, 2012
Statistical error – designing your sample size
2000
22
1899
1196.1
yxyx
Coimbra, 23/24 March, 2012
Percentile error of sample percentages
Female
18-29
30-39
40-49
50-60
over 60
Male
18-29
30-39
40-49
50-60
over 60
0,52650,4735 M
F
Ex-post car Driver Survey
Sample 2000
Survey methods
Household surveys
Doorstep surveys
In-street/at stop/roadside/on vehicle surveys
Hall tests
Postal surveys
Telephone surveys
Internet surveys
Coimbra, 23/24 March, 2012
Household surveys (1)
• Pre-arranged interviews (initial screening)
• All household members? (Use of incentives)
• Interviewer administered – 30 to 40 minutes
• Fully structured questionnaire? (show cards)
• Possible use of travel/activity diaries
• Piloting essential
• Interviewer training and monitoring important
• Random sampling – assess non-response bias
• Expensive
Coimbra, 23/24 March, 2012
Household surveys (2)
• Achieve a full picture of household mobility
• A picture of trip generation and distribution by
mode/ time of day/ day of week etc.
• The attitudes of household members to their
mobility issues
• The primary variables of the mode choices of
household members
Coimbra, 23/24 March, 2012
Doorstep surveys
• Shorter interviews – maximum 10 minutes
• Screening for target group
• Only one person (spokesperson) from the
household
• Diaries are sometimes left for collection/posting
• Careful piloting and interviewer training
• Structured questionnaires (show cards)
• Random address/walk /random person (repeat
visits – sampling with/without replacement)
• Assess pattern of non-response
Coimbra, 23/24 March, 2012
In-street/at stop/roadside/in vehicle surveys
• Random sampling at chosen/random sampling points/ passing vehicles/ on PT vehicles (specify the population to be sampled)
• Possible screening questions for target groups
• Interview individuals/maximum 4/5 minutes
• 10 key questions – design well to avoid uncompleted forms
• Careful piloting and interviewer training
• Structured questionnaires (show cards)
• Common use of automatic data recorders
• Assess pattern of non-response
• Less cost
Coimbra, 23/24 March, 2012
Hall tests
• Random sampling at chosen sampling points (specify the population to be sampled)
• Usual in-street screening questions for certain target groups
• Respondents invited into a ‘hall’ to complete a more detailed questionnaire.
• Halls provide the opportunity to show plans, demonstrations, stated preference exercises etc.
• Careful piloting and interviewer training
• Structured questionnaires (show cards)
• Possible use of automatic data recorders or computers
• Assess pattern of refusals to attend the hall test
Coimbra, 23/24 March, 2012
Telephone surveys
• Specialist Computer Assisted Telephone Interviewing sysems (CATI)
• Structured questionnaire (piloted)
• Screening questions for certain target groups?
• Telephone panel surveys
• Low respondent tolerance with telephone interviewing
• Only useful for short interviews/well defined issue
• Possible bias in sampling
• Used as an initial screening stage for household interviews
• Used for ‘follow-up’ interviews
• Less cost
Coimbra, 23/24 March, 2012
Postal/self-completion surveys
• Lower cost
• Structured, short, simple questionnaire
• Large differences in response rates across
Europe
• Bias in sampling (check non-response by postal
area?)
• Generally not reliable for mobility information
Coimbra, 23/24 March, 2012
Internet surveys
• Lower cost
• Structured questionnaire
• Possibility for obtaining qualitative
information – chats
• More tolerance than for telephone
surveys
• Self-selecting sample or pre-selection
Coimbra, 23/24 March, 2012
Target Groups (1) – defining variables
• Mobility categories
- car ownership/licence holding/access to use (driver and passenger)
- public transport (service level available/ level of use)
- cycle ownership and use
- walk trip frequency/ length etc.
- mobility impairment (physical, sensory, learning capacity)
- realistic mode choice groups etc.
- perception of balance of mode use
- ‘I consider myself mainly as.....’
Coimbra, 23/24 March, 2012
Target Groups (2)
• Socio-economic categories
- Age and gender
- Occupational group
- Individual/household income
- Educational attainment
- Ethnic origin
- Household structure
Coimbra, 23/24 March, 2012
Target Groups (3)
• Traveller categories
- Commuters
- Business travellers
- Tourists
- Students
• Personality
- Use of psychological variables
- Control theory
- Personal construct theory
Coimbra, 23/24 March, 2012
What questions (1)?
Questionnaires
• Printed (administered
self-completion)
• Data recorders
• Topic guides
Types of questions
• Closed questn.
• Open questns.
• Single responses
• Choices – SP
Quality/data protection standards
Coimbra, 23/24 March, 2012
What questions (2)?
• Ranking
- orders
- groups
- allocation of points
• Scales
- size of the scale
- middle scale?
• Defining frequency categories
• Multiple response questions
• Do not know/DNA
Coimbra, 23/24 March, 2012
The European Dimension
• Common definition of behavioural
indicators
• Common survey designs
• Common types of surveys
• Common definitions of target groups
• Minimum standard errors for analysis and
reporting (95% confidence intervals)
Southampton (UK), 20/03/2009
Coimbra, 23/24 March, 2012
The scope of this session
1. The nature of mobility and social disadvantage – defining the scope of our evaluation?
2. Setting out your evaluation - the baseline expectations, objectives, target groups, success criteria
3. Understanding the mobility behaviour of different social groups and attitudes – what are the key variables to evaluate?
4. Measuring and analysing mobility and inclusion situations and response to change - Qualitative assessment - Quantitative surveys
Coimbra, 23/24 March, 2012